Refactor ARMv8.3 Pointer Authentication support code ...

This patch provides the following features and makes modifications
listed below:
- Individual APIAKey key generation for each CPU.
- New key generation on every BL31 warm boot and TSP CPU On event.
- Per-CPU storage of APIAKey added in percpu_data[]
  of cpu_data structure.
- `plat_init_apiakey()` function replaced with `plat_init_apkey()`
  which returns 128-bit value and uses Generic timer physical counter
  value to increase the randomness of the generated key.
  The new function can be used for generation of all ARMv8.3-PAuth keys
- ARMv8.3-PAuth specific code placed in `lib\extensions\pauth`.
- New `pauth_init_enable_el1()` and `pauth_init_enable_el3()` functions
  generate, program and enable APIAKey_EL1 for EL1 and EL3 respectively;
  pauth_disable_el1()` and `pauth_disable_el3()` functions disable
  PAuth for EL1 and EL3 respectively;
  `pauth_load_bl31_apiakey()` loads saved per-CPU APIAKey_EL1 from
  cpu-data structure.
- Combined `save_gp_pauth_registers()` function replaces calls to
  `save_gp_registers()` and `pauth_context_save()`;
  `restore_gp_pauth_registers()` replaces `pauth_context_restore()`
  and `restore_gp_registers()` calls.
- `restore_gp_registers_eret()` function removed with corresponding
  code placed in `el3_exit()`.
- Fixed the issue when `pauth_t pauth_ctx` structure allocated space
  for 12 uint64_t PAuth registers instead of 10 by removal of macro
  CTX_PACGAKEY_END from `include/lib/el3_runtime/aarch64/context.h`
  and assigning its value to CTX_PAUTH_REGS_END.
- Use of MODE_SP_ELX and MODE_SP_EL0 macro definitions
  in `msr	spsel`  instruction instead of hard-coded values.
- Changes in documentation related to ARMv8.3-PAuth and ARMv8.5-BTI.

Change-Id: Id18b81cc46f52a783a7e6a09b9f149b6ce803211
Signed-off-by: Alexei Fedorov <Alexei.Fedorov@arm.com>

Minor changes to documentation and comments ...

Fix some typos and clarify some sentences.

Change-Id: Id276d1ced9a991b4eddc5c47ad9a825e6b29ef74
Signed-off-by: Antonio Nino Diaz <antonio.ninodiaz@arm.com>

BL31: Enable pointer authentication support ...

The size increase after enabling options related to ARMv8.3-PAuth is:

+----------------------------+-------+-------+-------+--------+
|                            |  text |  bss  |  data | rodata |
+----------------------------+-------+-------+-------+--------+
| CTX_INCLUDE_PAUTH_REGS = 1 |  +192 | +1536 |   +0  |   +0   |
|                            |  0.3% |  3.1% |       |        |
+----------------------------+-------+-------+-------+--------+
| ENABLE_PAUTH = 1           | +1848 | +1536 |  +16  |   +0   |
|                            |  3.3% |  3.1% |  3.1% |        |
+----------------------------+-------+-------+-------+--------+

Results calculated with the following build configuration:

    make PLAT=fvp SPD=tspd DEBUG=1 \
    SDEI_SUPPORT=1                 \
    EL3_EXCEPTION_HANDLING=1       \
    TSP_NS_INTR_ASYNC_PREEMPT=1    \
    CTX_INCLUDE_PAUTH_REGS=1       \
    ENABLE_PAUTH=1

Change-Id: I43db7e509a4f39da6599ec2faa690d197573ec1b
Signed-off-by: Antonio Nino Diaz <antonio.ninodiaz@arm.com>

Correct typographical errors ...

Corrects typos in core code, documentation files, drivers, Arm
platforms and services.

None of the corrections affect code; changes are limited to comments
and other documentation.

Change-Id: I5c1027b06ef149864f315ccc0ea473e2a16bfd1d
Signed-off-by: Paul Beesley <paul.beesley@arm.com>

Sanitise includes across codebase ...

Enforce full include path for includes. Deprecate old paths.

The following folders inside include/lib have been left unchanged:

- include/lib/cpus/${ARCH}
- include/lib/el3_runtime/${ARCH}

The reason for this change is that having a global namespace for
includes isn't a good idea. It defeats one of the advantages of having
folders and it introduces problems that are sometimes subtle (because
you may not know the header you are actually including if there are two
of them).

For example, this patch had to be created because two headers were
called the same way: e0ea0928d5b7 ("Fix gpio includes of mt8173 platform
to avoid collision."). More recently, this patch has had similar
problems: 46f9b2c3a282 ("drivers: add tzc380 support").

This problem was introduced in commit 4ecca33988b9 ("Move include and
source files to logical locations"). At that time, there weren't too
many headers so it wasn't a real issue. However, time has shown that
this creates problems.

Platforms that want to preserve the way they include headers may add the
removed paths to PLAT_INCLUDES, but this is discouraged.

Change-Id: I39dc53ed98f9e297a5966e723d1936d6ccf2fc8f
Signed-off-by: Antonio Nino Diaz <antonio.ninodiaz@arm.com>

context_mgmt: Fix MISRA defects ...

The macro EL_IMPLEMENTED() has been deprecated in favour of the new
function el_implemented().

Change-Id: Ic9b1b81480b5e019b50a050e8c1a199991bf0ca9
Signed-off-by: Antonio Nino Diaz <antonio.ninodiaz@arm.com>

Mark BL31 initialization functions ...

Mark the initialization functions in BL31, such as context management,
EHF, RAS and PSCI as __init so that they can be reclaimed by the
platform when no longer needed

Change-Id: I7446aeee3dde8950b0f410cb766b7a2312c20130
Signed-off-by: Daniel Boulby <daniel.boulby@arm.com>

BL31: Fix warning about BL32 init function ...

The expected value for failure is 0, so the warning only has to be shown
in that case. This is the way the TSPD has done it since it was
introduced, and the way SPM and OP-TEE do it.

Trusty wrongly returns 0 on success.

In the case of TLK, the return value of tlkd_init() is passed from the
secure world in register X1 when calling the SMC TLK_ENTRY_DONE.

Change-Id: I39106d67631ee57f109619f8830bf4b9d96155e6
Signed-off-by: Antonio Nino Diaz <antonio.ninodiaz@arm.com>

Convert BL31 error message into warning ...

If BL32 isn't present or it fails to initialize the current code prints
an error message in both debug and release builds. This is too verbose
for release builds, so it has been converted into a warning.

Also, it was missing a newline at the end of the message.

Change-Id: I91e18d5d5864dbb19d47ecd54f174d2d8c06296c
Signed-off-by: Antonio Nino Diaz <antonio.ninodiaz@arm.com>

Fix MISRA defects in BL31 common code ...

Change-Id: I5993b425445ee794e6d2a792c244c0af53640655
Signed-off-by: Antonio Nino Diaz <antonio.ninodiaz@arm.com>

Fix MISRA rule 8.4 in common code ...

Rule 8.4: A compatible declaration shall be visible when
          an object or function with external linkage is defined.

Change-Id: I26e042cb251a6f9590afa1340fdac73e42f23979
Signed-off-by: Roberto Vargas <roberto.vargas@arm.com>

BL31: Introduce Exception Handling Framework ...

EHF is a framework that allows dispatching of EL3 interrupts to their
respective handlers in EL3.

This framework facilitates the firmware-first error handling policy in
which asynchronous exceptions may be routed to EL3. Such exceptions may
be handed over to respective exception handlers. Individual handlers
might further delegate exception handling to lower ELs.

The framework associates the delegated execution to lower ELs with a
priority value. For interrupts, this corresponds to the priorities
programmed in GIC; for other types of exceptions, viz. SErrors or
Synchronous External Aborts, individual dispatchers shall explicitly
associate delegation to a secure priority. In order to prevent lower
priority interrupts from preempting higher priority execution, the
framework provides helpers to control preemption by virtue of
programming Priority Mask register in the interrupt controller.

This commit allows for handling interrupts targeted at EL3. Exception
handlers own interrupts by assigning them a range of secure priorities,
and registering handlers for each priority range it owns.

Support for exception handling in BL31 image is enabled by setting the
build option EL3_EXCEPTION_HANDLING=1.

Documentation to follow.

NOTE: The framework assumes the priority scheme supported by platform
interrupt controller is compliant with that of ARM GIC architecture (v2
or later).

Change-Id: I7224337e4cea47c6ca7d7a4ca22a3716939f7e42
Signed-off-by: Jeenu Viswambharan <jeenu.viswambharan@arm.com>

Fix order of #includes ...

This fix modifies the order of system includes to meet the ARM TF coding
standard. There are some exceptions in order to retain header groupings,
minimise changes to imported headers, and where there are headers within
the #if and #ifndef statements.

Change-Id: I65085a142ba6a83792b26efb47df1329153f1624
Signed-off-by: Isla Mitchell <isla.mitchell@arm.com>

Merge pull request #927 from jeenu-arm/state-switch ...

Execution state switch

Use SPDX license identifiers ...

To make software license auditing simpler, use SPDX[0] license
identifiers instead of duplicating the license text in every file.

NOTE: Files that have been imported by FreeBSD have not been modified.

[0]: https://spdx.org/

Change-Id: I80a00e1f641b8cc075ca5a95b10607ed9ed8761a
Signed-off-by: dp-arm <dimitris.papastamos@arm.com>

Add macro to check whether the CPU implements an EL ...

Replace all instances of checks with the new macro.

Change-Id: I0eec39b9376475a1a9707a3115de9d36f88f8a2a
Signed-off-by: Jeenu Viswambharan <jeenu.viswambharan@arm.com>

Flush console where necessary ...

Call console_flush() before execution either terminates or leaves an
exception level.

Fixes: ARM-software/tf-issues#123

Change-Id: I64eeb92effb039f76937ce89f877b68e355588e3
Signed-off-by: Antonio Nino Diaz <antonio.ninodiaz@arm.com>

Add PMF instrumentation points in TF ...

In order to quantify the overall time spent in the PSCI software
implementation, an initial collection of PMF instrumentation points
has been added.

Instrumentation has been added to the following code paths:

- Entry to PSCI SMC handler.  The timestamp is captured as early
  as possible during the runtime exception and stored in memory
  before entering the PSCI SMC handler.

- Exit from PSCI SMC handler.  The timestamp is captured after
  normal return from the PSCI SMC handler or if a low power state
  was requested it is captured in the bl31 warm boot path before
  return to normal world.

- Entry to low power state.  The timestamp is captured before entry
  to a low power state which implies either standby or power down.
  As these power states are mutually exclusive, only one timestamp
  is defined to describe both.  It is possible to differentiate between
  the two power states using the PSCI STAT interface.

- Exit from low power state.  The timestamp is captured after a standby
  or power up operation has completed.

To calculate the number of cycles spent running code in Trusted Firmware
one can perform the following calculation:

(exit_psci - enter_psci) - (exit_low_pwr - enter_low_pwr).

The resulting number of cycles can be converted to time given the
frequency of the counter.

Change-Id: Ie3b8f3d16409b6703747093b3a2d5c7429ad0166
Signed-off-by: dp-arm <dimitris.papastamos@arm.com>

PSCI: Do psci_setup() as part of std_svc_setup() ...

This patch moves the invocation of `psci_setup()` from BL31 and SP_MIN
into `std_svc_setup()` as part of ARM Standard Service initialization.
This allows us to consolidate ARM Standard Service initializations which
will be added to in the future. A new function `get_arm_std_svc_args()`
is introduced to get arguments corresponding to each standard service.
This function must be implemented by the EL3 Runtime Firmware and both
SP_MIN and BL31 implement it.

Change-Id: I38e1b644f797fa4089b20574bd4a10f0419de184

PSCI: Introduce PSCI Library argument structure ...

This patch introduces a `psci_lib_args_t` structure which must be
passed into `psci_setup()` which is then used to initialize the PSCI
library. The `psci_lib_args_t` is a versioned structure so as to enable
compatibility checks during library initialization. Both BL31 and SP_MIN
are modified to use the new structure.

SP_MIN is also modified to add version string and build message as part
of its cold boot log just like the other BLs in Trusted Firmware.

NOTE: Please be aware that this patch modifies the prototype of
`psci_setup()`, which breaks compatibility with EL3 Runtime Firmware
(excluding BL31 and SP_MIN) integrated with the PSCI Library.

Change-Id: Ic3761db0b790760a7ad664d8a437c72ea5edbcd6

Introduce PSCI Library Interface ...

This patch introduces the PSCI Library interface. The major changes
introduced are as follows:

* Earlier BL31 was responsible for Architectural initialization during cold
boot via bl31_arch_setup() whereas PSCI was responsible for the same during
warm boot. This functionality is now consolidated by the PSCI library
and it does Architectural initialization via psci_arch_setup() during both
cold and warm boots.

* Earlier the warm boot entry point was always `psci_entrypoint()`. This was
not flexible enough as a library interface. Now PSCI expects the runtime
firmware to provide the entry point via `psci_setup()`. A new function
`bl31_warm_entrypoint` is introduced in BL31 and the previous
`psci_entrypoint()` is deprecated.

* The `smc_helpers.h` is reorganized to separate the SMC Calling Convention
defines from the Trusted Firmware SMC helpers. The former is now in a new
header file `smcc.h` and the SMC helpers are moved to Architecture specific
header.

* The CPU context is used by PSCI for context initialization and
restoration after power down (PSCI Context). It is also used by BL31 for SMC
handling and context management during Normal-Secure world switch (SMC
Context). The `psci_smc_handler()` interface is redefined to not use SMC
helper macros thus enabling to decouple the PSCI context from EL3 runtime
firmware SMC context. This enables PSCI to be integrated with other runtime
firmware using a different SMC context.

NOTE: With this patch the architectural setup done in `bl31_arch_setup()`
is done as part of `psci_setup()` and hence `bl31_platform_setup()` will be
invoked prior to architectural setup. It is highly unlikely that the platform
setup will depend on architectural setup and cause any failure. Please be
be aware of this change in sequence.

Change-Id: I7f497a08d33be234bbb822c28146250cb20dab73

Build option to include AArch32 registers in cpu context ...

The system registers that are saved and restored in CPU context include
AArch32 systems registers like SPSR_ABT, SPSR_UND, SPSR_IRQ, SPSR_FIQ,
DACR32_EL2, IFSR32_EL2 and FPEXC32_EL2. Accessing these registers on an
AArch64-only (i.e. on hardware that does not implement AArch32, or at
least not at EL1 and higher ELs) platform leads to an exception. This patch
introduces the build option `CTX_INCLUDE_AARCH32_REGS` to specify whether to
include these AArch32 systems registers in the cpu context or not. By default
this build option is set to 1 to ensure compatibility. AArch64-only platforms
must set it to 0. A runtime check is added in BL1 and BL31 cold boot path to
verify this.

Fixes ARM-software/tf-issues#386

Change-Id: I720cdbd7ed7f7d8516635a2ec80d025f478b95ee

Remove dashes from image names: 'BL3-x' --> 'BL3x' ...

This patch removes the dash character from the image name, to
follow the image terminology in the Trusted Firmware Wiki page:

    https://github.com/ARM-software/arm-trusted-firmware/wiki

Changes apply to output messages, comments and documentation.

non-ARM platform files have been left unmodified.

Change-Id: Ic2a99be4ed929d52afbeb27ac765ceffce46ed76

Ensure BL31 does not print to boot console by default ...

It is not ideal for BL31 to continue to use boot console at
runtime which could be potentially uninitialized. This patch
introduces a new optional platform porting API
`bl31_plat_runtime_setup()` which allows the platform to perform
any BL31 runtime setup just prior to BL31 exit during cold boot.
The default weak implementation of this function will invoke
`console_uninit()` which will suppress any BL31 runtime logs.

On the ARM Standard platforms, there is an anomaly that
the boot console will be reinitialized on resumption from
system suspend in `arm_system_pwr_domain_resume()`. This
will be resolved in the following patch.

NOTE: The default weak definition of `bl31_plat_runtime_setup()`
disables the BL31 console. To print the BL31 runtime
messages, platforms must override this API and initialize a
runtime console.

Fixes ARM-software/tf-issues#328

Change-Id: Ibaf8346fcceb447fe1a5674094c9f8eb4c09ac4a

Introduce print_entry_point_info() function ...

This patch introduces a new function called 'print_entry_point_info'
that prints an entry_point_t structure for debugging purposes.
As such, it can be used to display the entry point address, SPSR and
arguments passed from a firmware image to the next one.

This function is now called in the following images transitions:
 - BL1 to BL2
 - BL1 to BL31
 - BL31 to the next image (typically BL32 or BL33)

The following changes have been introduced:

 - Fix the output format of the SPSR value : SPSR is a 32-bit value,
   not a 64-bit one.

 - Print all arguments values.
   The entry_point_info_t structure allows to pass up to 8 arguments.
   In most cases, only the first 2 arguments were printed.
   print_entry_point_info() now prints all of them as 'VERBOSE'
   traces.

Change-Id: Ieb384bffaa7849e6cb95a01a47c0b7fc2308653a

Make generic code work in presence of system caches ...

On the ARMv8 architecture, cache maintenance operations by set/way on the last
level of integrated cache do not affect the system cache. This means that such a
flush or clean operation could result in the data being pushed out to the system
cache rather than main memory. Another CPU could access this data before it
enables its data cache or MMU. Such accesses could be serviced from the main
memory instead of the system cache. If the data in the sysem cache has not yet
been flushed or evicted to main memory then there could be a loss of
coherency. The only mechanism to guarantee that the main memory will be updated
is to use cache maintenance operations to the PoC by MVA(See section D3.4.11
(System level caches) of ARMv8-A Reference Manual (Issue A.g/ARM DDI0487A.G).

This patch removes the reliance of Trusted Firmware on the flush by set/way
operation to ensure visibility of data in the main memory. Cache maintenance
operations by MVA are now used instead. The following are the broad category of
changes:

1. The RW areas of BL2/BL31/BL32 are invalidated by MVA before the C runtime is
   initialised. This ensures that any stale cache lines at any level of cache
   are removed.

2. Updates to global data in runtime firmware (BL31) by the primary CPU are made
   visible to secondary CPUs using a cache clean operation by MVA.

3. Cache maintenance by set/way operations are only used prior to power down.

NOTE: NON-UPSTREAM TRUSTED FIRMWARE CODE SHOULD MAKE EQUIVALENT CHANGES IN
ORDER TO FUNCTION CORRECTLY ON PLATFORMS WITH SUPPORT FOR SYSTEM CACHES.

Fixes ARM-software/tf-issues#205

Change-Id: I64f1b398de0432813a0e0881d70f8337681f6e9a

PSCI: Migrate TF to the new platform API and CM helpers ...

This patch migrates the rest of Trusted Firmware excluding Secure Payload and
the dispatchers to the new platform and context management API. The per-cpu
data framework APIs which took MPIDRs as their arguments are deleted and only
the ones which take core index as parameter are retained.

Change-Id: I839d05ad995df34d2163a1cfed6baa768a5a595d

Rationalize console log output ...

Fix the following issues with the console log output:

* Make sure the welcome string is the first thing in the log output
(during normal boot).
* Prefix each message with the BL image name so it's clear which
BL the output is coming from.
* Ensure all output is wrapped in one of the log output macros so it can
be easily compiled out if necessary. Change some of the INFO() messages
to VERBOSE(), especially in the TSP.
* Create some extra NOTICE() and INFO() messages during cold boot.
* Remove all usage of \r in log output.

Fixes ARM-software/tf-issues#231

Change-Id: Ib24f7acb36ce64bbba549f204b9cde2dbb46c8a3

Support asynchronous method for BL3-2 initialization ...

This patch adds support for BL3-2 initialization by asynchronous
method where BL3-1 transfers control to BL3-2 using world switch.
After BL3-2 initialization, it transfers control to BL3-3 via SPD
service handler. The SPD service handler initializes the CPU context
to BL3-3 entrypoint depending on the return function indentifier from
TSP initialization.

Fixes ARM-software/TF-issues#184

Change-Id: I7b135c2ceeb356d3bb5b6a287932e96ac67c7a34

Rework incorrect use of assert() and panic() in codebase ...

Assert a valid security state using the macro sec_state_is_valid().
Replace assert() with panic() in those cases that might arise
because of runtime errors and not programming errors.
Replace panic() with assert() in those cases that might arise
because of programming errors.

Fixes ARM-software/tf-issues#96

Change-Id: I51e9ef0439fd5ff5e0edfef49050b69804bf14d5