Refactor ARMv8.3 Pointer Authentication support code ...

This patch provides the following features and makes modifications
listed below:
- Individual APIAKey key generation for each CPU.
- New key generation on every BL31 warm boot and TSP CPU On event.
- Per-CPU storage of APIAKey added in percpu_data[]
  of cpu_data structure.
- `plat_init_apiakey()` function replaced with `plat_init_apkey()`
  which returns 128-bit value and uses Generic timer physical counter
  value to increase the randomness of the generated key.
  The new function can be used for generation of all ARMv8.3-PAuth keys
- ARMv8.3-PAuth specific code placed in `lib\extensions\pauth`.
- New `pauth_init_enable_el1()` and `pauth_init_enable_el3()` functions
  generate, program and enable APIAKey_EL1 for EL1 and EL3 respectively;
  pauth_disable_el1()` and `pauth_disable_el3()` functions disable
  PAuth for EL1 and EL3 respectively;
  `pauth_load_bl31_apiakey()` loads saved per-CPU APIAKey_EL1 from
  cpu-data structure.
- Combined `save_gp_pauth_registers()` function replaces calls to
  `save_gp_registers()` and `pauth_context_save()`;
  `restore_gp_pauth_registers()` replaces `pauth_context_restore()`
  and `restore_gp_registers()` calls.
- `restore_gp_registers_eret()` function removed with corresponding
  code placed in `el3_exit()`.
- Fixed the issue when `pauth_t pauth_ctx` structure allocated space
  for 12 uint64_t PAuth registers instead of 10 by removal of macro
  CTX_PACGAKEY_END from `include/lib/el3_runtime/aarch64/context.h`
  and assigning its value to CTX_PAUTH_REGS_END.
- Use of MODE_SP_ELX and MODE_SP_EL0 macro definitions
  in `msr	spsel`  instruction instead of hard-coded values.
- Changes in documentation related to ARMv8.3-PAuth and ARMv8.5-BTI.

Change-Id: Id18b81cc46f52a783a7e6a09b9f149b6ce803211
Signed-off-by: Alexei Fedorov <Alexei.Fedorov@arm.com>

Add UBSAN support and handlers ...

This patch adds support for the Undefined Behaviour sanitizer. There are
two types of support offered - minimalistic trapping support which
essentially immediately crashes on undefined behaviour and full support
with full debug messages.

The full support relies on ubsan.c which has been adapted from code used
by OPTEE.

Change-Id: I417c810f4fc43dcb56db6a6a555bfd0b38440727
Signed-off-by: Justin Chadwell <justin.chadwell@arm.com>

AArch64: Disable Secure Cycle Counter ...

This patch fixes an issue when secure world timing information
can be leaked because Secure Cycle Counter is not disabled.
For ARMv8.5 the counter gets disabled by setting MDCR_El3.SCCD
bit on CPU cold/warm boot.
For the earlier architectures PMCR_EL0 register is saved/restored
on secure world entry/exit from/to Non-secure state, and cycle
counting gets disabled by setting PMCR_EL0.DP bit.
'include\aarch64\arch.h' header file was tided up and new
ARMv8.5-PMU related definitions were added.

Change-Id: I6f56db6bc77504634a352388990ad925a69ebbfa
Signed-off-by: Alexei Fedorov <Alexei.Fedorov@arm.com>

Rework smc_unknown return code path in smc_handler ...

The intention of this patch is to leverage the existing el3_exit() return
routine for smc_unknown return path rather than a custom set of instructions.
In order to leverage el3_exit(), the necessary counteraction (i.e., saving the
system registers apart from GP registers) must be performed. Hence a series of
instructions which save system registers( like SPSR_EL3, SCR_EL3 etc) to stack
are moved to the top of group of instructions which essentially decode the OEN
from the smc function identifier and obtain the specific service handler in
rt_svc_descs_array. This ensures that the control flow for both known and
unknown smc calls will be similar.

Change-Id: I67f94cfcba176bf8aee1a446fb58a4e383905a87
Signed-off-by: Madhukar Pappireddy <madhukar.pappireddy@arm.com>

Fix restoration of PAuth context ...

Replace call to pauth_context_save() with pauth_context_restore()
in case of unknown SMC call.

Change-Id: Ib863d979faa7831052b33e8ac73913e2f661f9a0
Signed-off-by: Alexei Fedorov <Alexei.Fedorov@arm.com>

Restore PAuth context in case of unknown SMC call ...

Change-Id: I8fb346743b7afddbb8bf5908db4f27ee5a26f99b
Signed-off-by: Alexei Fedorov <Alexei.Fedorov@arm.com>

Add support for pointer authentication ...

The previous commit added the infrastructure to load and save
ARMv8.3-PAuth registers during Non-secure <-> Secure world switches, but
didn't actually enable pointer authentication in the firmware.

This patch adds the functionality needed for platforms to provide
authentication keys for the firmware, and a new option (ENABLE_PAUTH) to
enable pointer authentication in the firmware itself. This option is
disabled by default, and it requires CTX_INCLUDE_PAUTH_REGS to be
enabled.

Change-Id: I35127ec271e1198d43209044de39fa712ef202a5
Signed-off-by: Antonio Nino Diaz <antonio.ninodiaz@arm.com>

Add ARMv8.3-PAuth registers to CPU context ...

ARMv8.3-PAuth adds functionality that supports address authentication of
the contents of a register before that register is used as the target of
an indirect branch, or as a load.

This feature is supported only in AArch64 state.

This feature is mandatory in ARMv8.3 implementations.

This feature adds several registers to EL1. A new option called
CTX_INCLUDE_PAUTH_REGS has been added to select if the TF needs to save
them during Non-secure <-> Secure world switches. This option must be
enabled if the hardware has the registers or the values will be leaked
during world switches.

To prevent leaks, this patch also disables pointer authentication in the
Secure world if CTX_INCLUDE_PAUTH_REGS is 0. Any attempt to use it will
be trapped in EL3.

Change-Id: I27beba9907b9a86c6df1d0c5bf6180c972830855
Signed-off-by: Antonio Nino Diaz <antonio.ninodiaz@arm.com>

Remove support for the SMC Calling Convention 2.0 ...

This reverts commit 2f370465241c ("Add support for the SMC Calling
Convention 2.0").

SMCCC v2.0 is no longer required for SPM, and won't be needed in the
future. Removing it makes the SMC handling code less complicated.

The SPM implementation based on SPCI and SPRT was using it, but it has
been adapted to SMCCC v1.0.

Change-Id: I36795b91857b2b9c00437cfbfed04b3c1627f578
Signed-off-by: Antonio Nino Diaz <antonio.ninodiaz@arm.com>

Sanitise includes across codebase ...

Enforce full include path for includes. Deprecate old paths.

The following folders inside include/lib have been left unchanged:

- include/lib/cpus/${ARCH}
- include/lib/el3_runtime/${ARCH}

The reason for this change is that having a global namespace for
includes isn't a good idea. It defeats one of the advantages of having
folders and it introduces problems that are sometimes subtle (because
you may not know the header you are actually including if there are two
of them).

For example, this patch had to be created because two headers were
called the same way: e0ea0928d5b7 ("Fix gpio includes of mt8173 platform
to avoid collision."). More recently, this patch has had similar
problems: 46f9b2c3a282 ("drivers: add tzc380 support").

This problem was introduced in commit 4ecca33988b9 ("Move include and
source files to logical locations"). At that time, there weren't too
many headers so it wasn't a real issue. However, time has shown that
this creates problems.

Platforms that want to preserve the way they include headers may add the
removed paths to PLAT_INCLUDES, but this is discouraged.

Change-Id: I39dc53ed98f9e297a5966e723d1936d6ccf2fc8f
Signed-off-by: Antonio Nino Diaz <antonio.ninodiaz@arm.com>

SPM: Introduce SMC handlers for SPCI and SPRT ...

Change-Id: I2ae9b3bb686c41b2e138132a7bed107925ac861e
Signed-off-by: Antonio Nino Diaz <antonio.ninodiaz@arm.com>

BL31: Use helper function to save registers in SMC handler ...

Use the helper function `save_gp_registers` to save the register
state to cpu_context on entry to EL3 in SMC handler. This has the
effect of saving x0 - x3 as well into the cpu_context which was
not done previously but it unifies the register save sequence
in BL31.

Change-Id: I5753c942263a5f9178deda3dba896e3220f3dd83
Signed-off-by: Soby Mathew <soby.mathew@arm.com>

RAS: Introduce handler for EL3 EAs ...

External Aborts while executing in EL3 is fatal in nature. This patch
allows for the platform to define a handler for External Aborts received
while executing in EL3. A default implementation is added which falls
back to platform unhandled exception.

Change-Id: I466f2c8113a33870f2c7d2d8f2bf20437d9fd354
Signed-off-by: Jeenu Viswambharan <jeenu.viswambharan@arm.com>

RAS: Move EA handling to a separate file ...

A new file ea_delegate.S is introduced, and all EA-related functions are
moved into it. This makes runtime_exceptions.S less crowded and reads
better.

No functional changes.

Change-Id: I64b653b3931984cffd420563f8e8d1ba263f329f
Signed-off-by: Jeenu Viswambharan <jeenu.viswambharan@arm.com>

Add end_vector_entry assembler macro ...

Check_vector_size checks if the size of the vector fits
in the size reserved for it. This check creates problems in
the Clang assembler. A new macro, end_vector_entry, is added
and check_vector_size is deprecated.

This new macro fills the current exception vector until the next
exception vector. If the size of the current vector is bigger
than 32 instructions then it gives an error.

Change-Id: Ie8545cf1003a1e31656a1018dd6b4c28a4eaf671
Signed-off-by: Roberto Vargas <roberto.vargas@arm.com>

AArch64: Introduce RAS handling ...

RAS extensions are mandatory for ARMv8.2 CPUs, but are also optional
extensions to base ARMv8.0 architecture.

This patch adds build system support to enable RAS features in ARM
Trusted Firmware. A boolean build option RAS_EXTENSION is introduced for
this.

With RAS_EXTENSION, an Exception Synchronization Barrier (ESB) is
inserted at all EL3 vector entry and exit. ESBs will synchronize pending
external aborts before entering EL3, and therefore will contain and
attribute errors to lower EL execution. Any errors thus synchronized are
detected via. DISR_EL1 register.

When RAS_EXTENSION is set to 1, HANDLE_EL3_EA_FIRST must also be set to 1.

Change-Id: I38a19d84014d4d8af688bd81d61ba582c039383a
Signed-off-by: Jeenu Viswambharan <jeenu.viswambharan@arm.com>

AArch64: Introduce External Abort handling ...

At present, any External Abort routed to EL3 is reported as an unhandled
exception and cause a panic. This patch enables ARM Trusted Firmware to
handle External Aborts routed to EL3.

With this patch, when an External Abort is received at EL3, its handling
is delegated to plat_ea_handler() function. Platforms can provide their
own implementation of this function. This patch adds a weak definition
of the said function that prints out a message and just panics.

In order to support handling External Aborts at EL3, the build option
HANDLE_EA_EL3_FIRST must be set to 1.

Before this patch, HANDLE_EA_EL3_FIRST wasn't passed down to
compilation; this patch fixes that too.

Change-Id: I4d07b7e65eb191ff72d63b909ae9512478cd01a1
Signed-off-by: Jeenu Viswambharan <jeenu.viswambharan@arm.com>

AArch64: Refactor GP register restore to separate function ...

At present, the function that restores general purpose registers also
does ERET. Refactor the restore code to restore general purpose
registers without ERET to complement the save function.

The macro save_x18_to_x29_sp_el0 was used only once, and is therefore
removed, and its contents expanded inline for readability.

No functional changes, but with this patch:

  - The SMC return path will incur an branch-return and an additional
    register load.

  - The unknown SMC path restores registers x0 to x3.

Change-Id: I7a1a63e17f34f9cde810685d70a0ad13ca3b7c50
Signed-off-by: Jeenu Viswambharan <jeenu.viswambharan@arm.com>

Add support for the SMC Calling Convention 2.0 ...

Due to differences in the bitfields of the SMC IDs, it is not possible
to support SMCCC 1.X and 2.0 at the same time.

The behaviour of `SMCCC_MAJOR_VERSION` has changed. Now, it is a build
option that specifies the major version of the SMCCC that the Trusted
Firmware supports. The only two allowed values are 1 and 2, and it
defaults to 1. The value of `SMCCC_MINOR_VERSION` is derived from it.

Note: Support for SMCCC v2.0 is an experimental feature to enable
prototyping of secure partition specifications. Support for this
convention is disabled by default and could be removed without notice.

Change-Id: I88abf9ccf08e9c66a13ce55c890edea54d9f16a7
Signed-off-by: Antonio Nino Diaz <antonio.ninodiaz@arm.com>

Redefine SMC_UNK as -1 instead of 0xFFFFFFFF ...

According to the SMC Calling Convention (ARM DEN0028B):

    The Unknown SMC Function Identifier is a sign-extended value of
    (-1) that is returned in R0, W0 or X0 register.

The value wasn't sign-extended because it was defined as a 32-bit
unsigned value (0xFFFFFFFF).

SMC_PREEMPT has been redefined as -2 for the same reason.

NOTE: This might be a compatibility break for some AArch64 platforms
that don't follow the previous version of the SMCCC (ARM DEN0028A)
correctly. That document specifies that only the bottom 32 bits of the
returned value must be checked. If a platform relies on the top 32 bits
of the result being 0 (so that SMC_UNK is 0x00000000FFFFFFFF), it will
have to fix its code to comply with the SMCCC.

Change-Id: I7f7b109f6b30c114fe570aa0ead3c335383cb54d
Signed-off-by: Antonio Nino Diaz <antonio.ninodiaz@arm.com>

runtime_exceptions: Save x4-x29 unconditionally ...

In preparation for SMCCC v1.1 support, save x4 to x29 unconditionally.
Previously we expected callers coming from AArch64 mode to preserve
x8-x17.  This is no longer the case with SMCCC v1.1 as AArch64 callers
only need to save x0-x3.

Change-Id: Ie62d620776533969ff4a02c635422f1b9208be9c
Signed-off-by: Dimitris Papastamos <dimitris.papastamos@arm.com>

Workaround for CVE-2017-5715 on Cortex A57 and A72 ...

Invalidate the Branch Target Buffer (BTB) on entry to EL3 by disabling
and enabling the MMU.  To achieve this without performing any branch
instruction, a per-cpu vbar is installed which executes the workaround
and then branches off to the corresponding vector entry in the main
vector table.  A side effect of this change is that the main vbar is
configured before any reset handling.  This is to allow the per-cpu
reset function to override the vbar setting.

This workaround is enabled by default on the affected CPUs.

Change-Id: I97788d38463a5840a410e3cea85ed297a1678265
Signed-off-by: Dimitris Papastamos <dimitris.papastamos@arm.com>

Fix x30 reporting for unhandled exceptions ...

Some error paths that lead to a crash dump will overwrite the value in
the x30 register by calling functions with the no_ret macro, which
resolves to a BL instruction. This is not very useful and not what the
reader would expect, since a crash dump should usually show all
registers in the state they were in when the exception happened. This
patch replaces the offending function calls with a B instruction to
preserve the value in x30.

Change-Id: I2a3636f2943f79bab0cd911f89d070012e697c2a
Signed-off-by: Julius Werner <jwerner@chromium.org>

Use SPDX license identifiers ...

To make software license auditing simpler, use SPDX[0] license
identifiers instead of duplicating the license text in every file.

NOTE: Files that have been imported by FreeBSD have not been modified.

[0]: https://spdx.org/

Change-Id: I80a00e1f641b8cc075ca5a95b10607ed9ed8761a
Signed-off-by: dp-arm <dimitris.papastamos@arm.com>

Define and use no_ret macro where no return is expected ...

There are many instances in ARM Trusted Firmware where control is
transferred to functions from which return isn't expected. Such jumps
are made using 'bl' instruction to provide the callee with the location
from which it was jumped to. Additionally, debuggers infer the caller by
examining where 'lr' register points to. If a 'bl' of the nature
described above falls at the end of an assembly function, 'lr' will be
left pointing to a location outside of the function range. This misleads
the debugger back trace.

This patch defines a 'no_ret' macro to be used when jumping to functions
from which return isn't expected. The macro ensures to use 'bl'
instruction for the jump, and also, for debug builds, places a 'nop'
instruction immediately thereafter (unless instructed otherwise) so as
to leave 'lr' pointing within the function range.

Change-Id: Ib34c69fc09197cfd57bc06e147cc8252910e01b0
Co-authored-by: Douglas Raillard <douglas.raillard@arm.com>
Signed-off-by: Jeenu Viswambharan <jeenu.viswambharan@arm.com>

Cosmetic change to exception table ...

* Move comments on unhandled exceptions at the right place.
* Reformat the existing comments to highlight the start of
  each block of 4 entries in the exception table to ease
  navigation (lines of dash reserved for head comments).
* Reflow comments to 80 columns.

Change-Id: I5ab88a93d0628af8e151852cb5b597eb34437677
Signed-off-by: Douglas Raillard <douglas.raillard@arm.com>

Add PMF instrumentation points in TF ...

In order to quantify the overall time spent in the PSCI software
implementation, an initial collection of PMF instrumentation points
has been added.

Instrumentation has been added to the following code paths:

- Entry to PSCI SMC handler.  The timestamp is captured as early
  as possible during the runtime exception and stored in memory
  before entering the PSCI SMC handler.

- Exit from PSCI SMC handler.  The timestamp is captured after
  normal return from the PSCI SMC handler or if a low power state
  was requested it is captured in the bl31 warm boot path before
  return to normal world.

- Entry to low power state.  The timestamp is captured before entry
  to a low power state which implies either standby or power down.
  As these power states are mutually exclusive, only one timestamp
  is defined to describe both.  It is possible to differentiate between
  the two power states using the PSCI STAT interface.

- Exit from low power state.  The timestamp is captured after a standby
  or power up operation has completed.

To calculate the number of cycles spent running code in Trusted Firmware
one can perform the following calculation:

(exit_psci - enter_psci) - (exit_low_pwr - enter_low_pwr).

The resulting number of cycles can be converted to time given the
frequency of the counter.

Change-Id: Ie3b8f3d16409b6703747093b3a2d5c7429ad0166
Signed-off-by: dp-arm <dimitris.papastamos@arm.com>

Introduce some helper macros for exception vectors ...

This patch introduces some assembler macros to simplify the
declaration of the exception vectors. It abstracts the section
the exception code is put into as well as the alignments
constraints mandated by the ARMv8 architecture. For all TF images,
the exception code has been updated to make use of these macros.

This patch also updates some invalid comments in the exception
vector code.

Change-Id: I35737b8f1c8c24b6da89b0a954c8152a4096fa95

Move context management code to common location ...

The upcoming Firmware Update feature needs transitioning across
Secure/Normal worlds to complete the FWU process and hence requires
context management code to perform this task.

Currently context management code is part of BL31 stage only.
This patch moves the code from (include)/bl31 to (include)/common.
Some function declarations/definitions and macros have also moved
to different files to help code sharing.

Change-Id: I3858b08aecdb76d390765ab2b099f457873f7b0c

Remove the IMF_READ_INTERRUPT_ID build option ...

The IMF_READ_INTERRUPT_ID build option enables a feature where the interrupt
ID of the highest priority pending interrupt is passed as a parameter to the
interrupt handler registered for that type of interrupt. This additional read
of highest pending interrupt id from GIC is problematic as it is possible that
the original interrupt may get deasserted and another interrupt of different
type maybe become the highest pending interrupt. Hence it is safer to prevent
such behaviour by removing the IMF_READ_INTERRUPT_ID build option.

The `id` parameter of the interrupt handler `interrupt_type_handler_t` is
now made a reserved parameter with this patch. It will always contain
INTR_ID_UNAVAILABLE.

Fixes ARM-software/tf-issues#307

Change-Id: I2173aae1dd37edad7ba6bdfb1a99868635fa34de