TF-A: Add support for ARMv8.3-PAuth in BL1 SMC calls and BL2U ...

This patch adds support for ARMv8.3-PAuth in BL1 SMC calls and
BL2U image for firmware updates by programming APIAKey_EL1 registers
and enabling Pointer Authentication in EL3 and EL1 respectively.

Change-Id: I875d952aba8242caf74fb5f4f2d2af6f0c768c08
Signed-off-by: Alexei Fedorov <Alexei.Fedorov@arm.com>

Refactor ARMv8.3 Pointer Authentication support code ...

This patch provides the following features and makes modifications
listed below:
- Individual APIAKey key generation for each CPU.
- New key generation on every BL31 warm boot and TSP CPU On event.
- Per-CPU storage of APIAKey added in percpu_data[]
  of cpu_data structure.
- `plat_init_apiakey()` function replaced with `plat_init_apkey()`
  which returns 128-bit value and uses Generic timer physical counter
  value to increase the randomness of the generated key.
  The new function can be used for generation of all ARMv8.3-PAuth keys
- ARMv8.3-PAuth specific code placed in `lib\extensions\pauth`.
- New `pauth_init_enable_el1()` and `pauth_init_enable_el3()` functions
  generate, program and enable APIAKey_EL1 for EL1 and EL3 respectively;
  pauth_disable_el1()` and `pauth_disable_el3()` functions disable
  PAuth for EL1 and EL3 respectively;
  `pauth_load_bl31_apiakey()` loads saved per-CPU APIAKey_EL1 from
  cpu-data structure.
- Combined `save_gp_pauth_registers()` function replaces calls to
  `save_gp_registers()` and `pauth_context_save()`;
  `restore_gp_pauth_registers()` replaces `pauth_context_restore()`
  and `restore_gp_registers()` calls.
- `restore_gp_registers_eret()` function removed with corresponding
  code placed in `el3_exit()`.
- Fixed the issue when `pauth_t pauth_ctx` structure allocated space
  for 12 uint64_t PAuth registers instead of 10 by removal of macro
  CTX_PACGAKEY_END from `include/lib/el3_runtime/aarch64/context.h`
  and assigning its value to CTX_PAUTH_REGS_END.
- Use of MODE_SP_ELX and MODE_SP_EL0 macro definitions
  in `msr	spsel`  instruction instead of hard-coded values.
- Changes in documentation related to ARMv8.3-PAuth and ARMv8.5-BTI.

Change-Id: Id18b81cc46f52a783a7e6a09b9f149b6ce803211
Signed-off-by: Alexei Fedorov <Alexei.Fedorov@arm.com>

AArch64: Disable Secure Cycle Counter ...

This patch fixes an issue when secure world timing information
can be leaked because Secure Cycle Counter is not disabled.
For ARMv8.5 the counter gets disabled by setting MDCR_El3.SCCD
bit on CPU cold/warm boot.
For the earlier architectures PMCR_EL0 register is saved/restored
on secure world entry/exit from/to Non-secure state, and cycle
counting gets disabled by setting PMCR_EL0.DP bit.
'include\aarch64\arch.h' header file was tided up and new
ARMv8.5-PMU related definitions were added.

Change-Id: I6f56db6bc77504634a352388990ad925a69ebbfa
Signed-off-by: Alexei Fedorov <Alexei.Fedorov@arm.com>

Sanitise includes across codebase ...

Enforce full include path for includes. Deprecate old paths.

The following folders inside include/lib have been left unchanged:

- include/lib/cpus/${ARCH}
- include/lib/el3_runtime/${ARCH}

The reason for this change is that having a global namespace for
includes isn't a good idea. It defeats one of the advantages of having
folders and it introduces problems that are sometimes subtle (because
you may not know the header you are actually including if there are two
of them).

For example, this patch had to be created because two headers were
called the same way: e0ea0928d5b7 ("Fix gpio includes of mt8173 platform
to avoid collision."). More recently, this patch has had similar
problems: 46f9b2c3a282 ("drivers: add tzc380 support").

This problem was introduced in commit 4ecca33988b9 ("Move include and
source files to logical locations"). At that time, there weren't too
many headers so it wasn't a real issue. However, time has shown that
this creates problems.

Platforms that want to preserve the way they include headers may add the
removed paths to PLAT_INCLUDES, but this is discouraged.

Change-Id: I39dc53ed98f9e297a5966e723d1936d6ccf2fc8f
Signed-off-by: Antonio Nino Diaz <antonio.ninodiaz@arm.com>

Add end_vector_entry assembler macro ...

Check_vector_size checks if the size of the vector fits
in the size reserved for it. This check creates problems in
the Clang assembler. A new macro, end_vector_entry, is added
and check_vector_size is deprecated.

This new macro fills the current exception vector until the next
exception vector. If the size of the current vector is bigger
than 32 instructions then it gives an error.

Change-Id: Ie8545cf1003a1e31656a1018dd6b4c28a4eaf671
Signed-off-by: Roberto Vargas <roberto.vargas@arm.com>

Rename 'smcc' to 'smccc' ...

When the source code says 'SMCC' it is talking about the SMC Calling
Convention. The correct acronym is SMCCC. This affects a few definitions
and file names.

Some files have been renamed (smcc.h, smcc_helpers.h and smcc_macros.S)
but the old files have been kept for compatibility, they include the
new ones with an ERROR_DEPRECATED guard.

Change-Id: I78f94052a502436fdd97ca32c0fe86bd58173f2f
Signed-off-by: Antonio Nino Diaz <antonio.ninodiaz@arm.com>

Ensure the correct execution of TLBI instructions ...

After executing a TLBI a DSB is needed to ensure completion of the
TLBI.

rk3328: The MMU is allowed to load TLB entries for as long as it is
enabled. Because of this, the correct place to execute a TLBI is right
after disabling the MMU.

Change-Id: I8280f248d10b49a8c354a4ccbdc8f8345ac4c170
Signed-off-by: Antonio Nino Diaz <antonio.ninodiaz@arm.com>

Fix order of #includes ...

This fix modifies the order of system includes to meet the ARM TF coding
standard. There are some exceptions in order to retain header groupings,
minimise changes to imported headers, and where there are headers within
the #if and #ifndef statements.

Change-Id: I65085a142ba6a83792b26efb47df1329153f1624
Signed-off-by: Isla Mitchell <isla.mitchell@arm.com>

Use SPDX license identifiers ...

To make software license auditing simpler, use SPDX[0] license
identifiers instead of duplicating the license text in every file.

NOTE: Files that have been imported by FreeBSD have not been modified.

[0]: https://spdx.org/

Change-Id: I80a00e1f641b8cc075ca5a95b10607ed9ed8761a
Signed-off-by: dp-arm <dimitris.papastamos@arm.com>

Define and use no_ret macro where no return is expected ...

There are many instances in ARM Trusted Firmware where control is
transferred to functions from which return isn't expected. Such jumps
are made using 'bl' instruction to provide the callee with the location
from which it was jumped to. Additionally, debuggers infer the caller by
examining where 'lr' register points to. If a 'bl' of the nature
described above falls at the end of an assembly function, 'lr' will be
left pointing to a location outside of the function range. This misleads
the debugger back trace.

This patch defines a 'no_ret' macro to be used when jumping to functions
from which return isn't expected. The macro ensures to use 'bl'
instruction for the jump, and also, for debug builds, places a 'nop'
instruction immediately thereafter (unless instructed otherwise) so as
to leave 'lr' pointing within the function range.

Change-Id: Ib34c69fc09197cfd57bc06e147cc8252910e01b0
Co-authored-by: Douglas Raillard <douglas.raillard@arm.com>
Signed-off-by: Jeenu Viswambharan <jeenu.viswambharan@arm.com>

AArch32: Add generic changes in BL1 ...

This patch adds generic changes in BL1 to support AArch32 state.
New AArch32 specific assembly/C files are introduced and
some files are moved to AArch32/64 specific folders.
BL1 for AArch64 is refactored but functionally identical.
BL1 executes in Secure Monitor mode in AArch32 state.

NOTE: BL1 in AArch32 state ONLY handles BL1_RUN_IMAGE SMC.

Change-Id: I6e2296374c7efbf3cf2aa1a0ce8de0732d8c98a5

Remove looping around `plat_report_exception` ...

This patch removes the tight loop that calls `plat_report_exception`
in unhandled exceptions in AArch64 state.
The new behaviour is to call the `plat_report_exception` only
once followed by call to `plat_panic_handler`.
This allows platforms to take platform-specific action when
there is an unhandled exception, instead of always spinning
in a tight loop.

Note: This is a subtle break in behaviour for platforms that
      expect `plat_report_exception` to be continuously executed
      when there is an unhandled exception.

Change-Id: Ie2453804b9b7caf9b010ee73e1a90eeb8384e4e8

Introduce some helper macros for exception vectors ...

This patch introduces some assembler macros to simplify the
declaration of the exception vectors. It abstracts the section
the exception code is put into as well as the alignments
constraints mandated by the ARMv8 architecture. For all TF images,
the exception code has been updated to make use of these macros.

This patch also updates some invalid comments in the exception
vector code.

Change-Id: I35737b8f1c8c24b6da89b0a954c8152a4096fa95

FWU: Add Generic Firmware Update framework support in BL1 ...

Firmware update(a.k.a FWU) feature is part of the TBB architecture.
BL1 is responsible for carrying out the FWU process if platform
specific code detects that it is needed.

This patch adds support for FWU feature support in BL1 which is
included by enabling `TRUSTED_BOARD_BOOT` compile time flag.

This patch adds bl1_fwu.c which contains all the core operations
of FWU, which are; SMC handler, image copy, authentication, execution
and resumption. It also adds bl1.h introducing #defines for all
BL1 SMCs.

Following platform porting functions are introduced:

int bl1_plat_mem_check(uintptr_t mem_base, unsigned int mem_size,
unsigned int flags);
	This function can be used to add platform specific memory checks
	for the provided base/size for the given security state.
	The weak definition will invoke `assert()` and return -ENOMEM.

__dead2 void bl1_plat_fwu_done(void *cookie, void *reserved);
	This function can be used to initiate platform specific procedure
	to mark completion of the FWU process.
	The weak definition waits forever calling `wfi()`.

plat_bl1_common.c contains weak definitions for above functions.

FWU process starts when platform detects it and return the image_id
other than BL2_IMAGE_ID by using `bl1_plat_get_next_image_id()` in
`bl1_main()`.

NOTE: User MUST provide platform specific real definition for
bl1_plat_mem_check() in order to use it for Firmware update.

Change-Id: Ice189a0885d9722d9e1dd03f76cac1aceb0e25ed

Add descriptor based image management support in BL1 ...

As of now BL1 loads and execute BL2 based on hard coded information
provided in BL1. But due to addition of support for upcoming Firmware
Update feature, BL1 now require more flexible approach to load and
run different images using information provided by the platform.

This patch adds new mechanism to load and execute images based on
platform provided image id's. BL1 now queries the platform to fetch
the image id of the next image to be loaded and executed. In order
to achieve this, a new struct image_desc_t was added which holds the
information about images, such as: ep_info and image_info.

This patch introduces following platform porting functions:

unsigned int bl1_plat_get_next_image_id(void);
	This is used to identify the next image to be loaded
	and executed by BL1.

struct image_desc *bl1_plat_get_image_desc(unsigned int image_id);
	This is used to retrieve the image_desc for given image_id.

void bl1_plat_set_ep_info(unsigned int image_id,
struct entry_point_info *ep_info);
	This function allows platforms to update ep_info for given
	image_id.

The plat_bl1_common.c file provides default weak implementations of
all above functions, the `bl1_plat_get_image_desc()` always return
BL2 image descriptor, the `bl1_plat_get_next_image_id()` always return
BL2 image ID and `bl1_plat_set_ep_info()` is empty and just returns.
These functions gets compiled into all BL1 platforms by default.

Platform setup in BL1, using `bl1_platform_setup()`, is now done
_after_ the initialization of authentication module. This change
provides the opportunity to use authentication while doing the
platform setup in BL1.

In order to store secure/non-secure context, BL31 uses percpu_data[]
to store context pointer for each core. In case of BL1 only the
primary CPU will be active hence percpu_data[] is not required to
store the context pointer.

This patch introduce bl1_cpu_context[] and bl1_cpu_context_ptr[] to
store the context and context pointers respectively. It also also
re-defines cm_get_context() and cm_set_context() for BL1 in
bl1/bl1_context_mgmt.c.

BL1 now follows the BL31 pattern of using SP_EL0 for the C runtime
environment, to support resuming execution from a previously saved
context.

NOTE: THE `bl1_plat_set_bl2_ep_info()` PLATFORM PORTING FUNCTION IS
      NO LONGER CALLED BY BL1 COMMON CODE. PLATFORMS THAT OVERRIDE
      THIS FUNCTION MAY NEED TO IMPLEMENT `bl1_plat_set_ep_info()`
      INSTEAD TO MAINTAIN EXISTING BEHAVIOUR.

Change-Id: Ieee4c124b951c2e9bc1c1013fa2073221195d881

Move context management code to common location ...

The upcoming Firmware Update feature needs transitioning across
Secure/Normal worlds to complete the FWU process and hence requires
context management code to perform this task.

Currently context management code is part of BL31 stage only.
This patch moves the code from (include)/bl31 to (include)/common.
Some function declarations/definitions and macros have also moved
to different files to help code sharing.

Change-Id: I3858b08aecdb76d390765ab2b099f457873f7b0c

Pass the entry point info to bl1_plat_prepare_exit() ...

This patch modifies the prototype of the bl1_plat_prepare_exit()
platform API to pass the address of the entry point info structure
received from BL2. The structure contains information that can be
useful, depending on the kind of clean up or bookkeeping operations
to perform.

The weak implementation of this function ignores this argument to
preserve platform backwards compatibility.

NOTE: THIS PATCH MAY BREAK PLATFORM PORTS THAT ARE RELYING ON THE
FORMER PROTOTYPE OF THE BL1_PLAT_PREPARE_EXIT() API.

Change-Id: I3fc18f637de06c85719c4ee84c85d6a4572a0fdb

Introduce SPIN_ON_BL1_EXIT build flag ...

This patch introduces a new build flag, SPIN_ON_BL1_EXIT, which
puts an infinite loop in BL1. It is intended to help debugging
the post-BL2 phase of the Trusted Firmware by stopping execution
in BL1 just before handing over to BL31. At this point, the
developer may take control of the target using a debugger.

This feature is disabled by default and can be enabled by
rebuilding BL1 with SPIN_ON_BL1_EXIT=1.

User Guide updated accordingly.

Change-Id: I6b6779d5949c9e5571dd371255520ef1ac39685c

Improve display_boot_progress() function ...

- Remove out-dated information about the use of printf() in the
   function comment.

 - Make the argument const, as the function doesn't need to modify it.

 - Rename the function into bl1_print_bl31_ep_info() to make its
   purpose clearer.

Change-Id: I2a9d215a37f0ec11aefce0c5c9e050473b7a6b25

Add optional bl1_plat_prepare_exit() API ...

This patch adds an optional API to the platform port:

    void bl1_plat_prepare_exit(void);

This function is called prior to exiting BL1 in response to the
RUN_IMAGE_SMC request raised by BL2. It should be used to perform
platform specific clean up or bookkeeping operations before
transferring control to the next image.

A weak empty definition of this function has been provided to
preserve platform backwards compatibility.

Change-Id: Iec09697de5c449ae84601403795cdb6aca166ba1

Break down BL1 AArch64 synchronous exception handler ...

The AArch64 synchronous exception vector code in BL1 is almost
reaching its architectural limit of 32 instructions. This means
there is very little space for this code to grow.

This patch reduces the size of the exception vector code by
moving most of its code in a function to which we branch from
SynchronousExceptionA64.

Change-Id: Ib35351767a685fb2c2398029d32e54026194f7ed

Miscellaneous documentation fixes ...

This patch gathers miscellaneous minor fixes to the documentation, and comments
in the source code.

Change-Id: I631e3dda5abafa2d90f464edaee069a1e58b751b
Co-Authored-By: Soby Mathew <soby.mathew@arm.com>
Co-Authored-By: Dan Handley <dan.handley@arm.com>

Unmask SError interrupt and clear SCR_EL3.EA bit ...

This patch disables routing of external aborts from lower exception levels to
EL3 and ensures that a SError interrupt generated as a result of execution in
EL3 is taken locally instead of a lower exception level.

The SError interrupt is enabled in the TSP code only when the operation has not
been directly initiated by the normal world. This is to prevent the possibility
of an asynchronous external abort which originated in normal world from being
taken when execution is in S-EL1.

Fixes ARM-software/tf-issues#153

Change-Id: I157b996c75996d12fd86d27e98bc73dd8bce6cd5

Populate BL31 input parameters as per new spec ...

This patch is based on spec published at
https://github.com/ARM-software/tf-issues/issues/133

It rearranges the bl31_args struct into
bl31_params and bl31_plat_params which provide the
information needed for Trusted firmware and platform
specific data via x0 and x1

On the FVP platform BL3-1 params and BL3-1 plat params
and its constituents are stored at the start of TZDRAM.

The information about memory availability and size for
BL3-1, BL3-2 and BL3-3 is moved into platform specific data.

Change-Id: I8b32057a3d0dd3968ea26c2541a0714177820da9

Rework handover interface between BL stages ...

This patch reworks the handover interface from: BL1 to BL2 and
BL2 to BL3-1. It removes the raise_el(), change_el(), drop_el()
and run_image() functions as they catered for code paths that were
never exercised.
BL1 calls bl1_run_bl2() to jump into BL2 instead of doing the same
by calling run_image(). Similarly, BL2 issues the SMC to transfer
execution to BL3-1 through BL1 directly. Only x0 and x1 are used
to pass arguments to BL31. These arguments and parameters for
running BL3-1 are passed through a reference to a
'el_change_info_t' structure. They were being passed value in
general purpose registers earlier.

Change-Id: Id4fd019a19a9595de063766d4a66295a2c9307e1

Access system registers directly in assembler ...

Instead of using the system register helper functions to read
or write system registers, assembler coded functions should
use MRS/MSR instructions. This results in faster and more
compact code.

This change replaces all usage of the helper functions with
direct register accesses.

Change-Id: I791d5f11f257010bb3e6a72c6c5ab8779f1982b3

Replace disable_mmu with assembler version ...

disable_mmu() cannot work as a C function as there is no control
over data accesses generated by the compiler between disabling and
cleaning the data cache. This results in reading stale data from
main memory.

As assembler version is provided for EL3, and a variant that also
disables the instruction cache which is now used by the BL1
exception handling function.

Fixes ARM-software/tf-issues#147

Change-Id: I0cf394d2579a125a23c2f2989c2e92ace6ddb1a6

Correct usage of data and instruction barriers ...

The current code does not always use data and instruction
barriers as required by the architecture and frequently uses
barriers excessively due to their inclusion in all of the
write_*() helper functions.

Barriers should be used explicitly in assembler or C code
when modifying processor state that requires the barriers in
order to enable review of correctness of the code.

This patch removes the barriers from the helper functions and
introduces them as necessary elsewhere in the code.

PORTING NOTE: check any port of Trusted Firmware for use of
system register helper functions for reliance on the previous
barrier behaviour and add explicit barriers as necessary.

Fixes ARM-software/tf-issues#92

Change-Id: Ie63e187404ff10e0bdcb39292dd9066cb84c53bf

Reduce deep nesting of header files ...

Reduce the number of header files included from other header
files as much as possible without splitting the files. Use forward
declarations where possible. This allows removal of some unnecessary
"#ifndef __ASSEMBLY__" statements.

Also, review the .c and .S files for which header files really need
including and reorder the #include statements alphabetically.

Fixes ARM-software/tf-issues#31

Change-Id: Iec92fb976334c77453e010b60bcf56f3be72bd3e

Place assembler functions in separate sections ...

This extends the --gc-sections behaviour to the many assembler
support functions in the firmware images by placing each function
into its own code section. This is achieved by creating a 'func'
macro used to declare each function label.

Fixes ARM-software/tf-issues#80

Change-Id: I301937b630add292d2dec6d2561a7fcfa6fec690