lib: el3_runtime: Fix SPE system registers in el2_sysregs_context ...

Include EL2 registers related to SPE in EL2 context save/restore
routines if architecture supports it and platform wants to use these
features in Secure world.

Change-Id: Ie01a2c38fa5f6c907276eddec120fdfb222561a6
Signed-off-by: Arunachalam Ganapathy <arunachalam.ganapathy@arm.com>

lib: el3_runtime: Conditionally save/restore EL2 NEVE registers ...

Include EL2 registers related to Nested Virtualization in EL2 context
save/restore routines if architecture supports it and platform wants to
use these features in Secure world.

Change-Id: If006ab83bbc2576488686f5ffdff88b91adced5c
Signed-off-by: Arunachalam Ganapathy <arunachalam.ganapathy@arm.com>

lib: el3_runtime: Fix aarch32 system registers in el2_sysregs_context ...

AArch64-only platforms do not implement AArch32 at EL1 and higher ELs.
In such cases the build option CTX_INCLUDE_AARCH32_REGS is set to 0.
So don't save/restore aarch32 system registers in el2_sysregs_context
save/restore routines if CTX_INCLUDE_AARCH32_REGS is set to 0.

Change-Id: I229cdd46136c4b4bc9623b02eb444d904e09ce5a
Signed-off-by: Arunachalam Ganapathy <arunachalam.ganapathy@arm.com>

Increase type widths to satisfy width requirements ...

Usually, C has no problem up-converting types to larger bit sizes. MISRA
rule 10.7 requires that you not do this, or be very explicit about this.
This resolves the following required rule:

    bl1/aarch64/bl1_context_mgmt.c:81:[MISRA C-2012 Rule 10.7 (required)]<None>
    The width of the composite expression "0U | ((mode & 3U) << 2U) | 1U |
    0x3c0U" (32 bits) is less that the right hand operand
    "18446744073709547519ULL" (64 bits).

This also resolves MISRA defects such as:

    bl2/aarch64/bl2arch_setup.c:18:[MISRA C-2012 Rule 12.2 (required)]
    In the expression "3U << 20", shifting more than 7 bits, the number
    of bits in the essential type of the left expression, "3U", is
    not allowed.

Further, MISRA requires that all shifts don't overflow. The definition of
PAGE_SIZE was (1U << 12), and 1U is 8 bits. This caused about 50 issues.
This fixes the violation by changing the definition to 1UL << 12. Since
this uses 32bits, it should not create any issues for aarch32.

This patch also contains a fix for a build failure in the sun50i_a64
platform. Specifically, these misra fixes removed a single and
instruction,

    92407e73        and     x19, x19, #0xffffffff

from the cm_setup_context function caused a relocation in
psci_cpus_on_start to require a linker-generated stub. This increased the
size of the .text section and caused an alignment later on to go over a
page boundary and round up to the end of RAM before placing the .data
section. This sectionn is of non-zero size and therefore causes a link
error.

The fix included in this reorders the functions during link time
without changing their ording with respect to alignment.

Change-Id: I76b4b662c3d262296728a8b9aab7a33b02087f16
Signed-off-by: Jimmy Brisson <jimmy.brisson@arm.com>

SPM: Change condition on saving/restoring EL2 registers ...

Make this more scalable by explicitly checking internal and hardware
states at run_time

Signed-off-by: Ruari Phipps <ruari.phipps@arm.com>
Change-Id: I1c6ed1c1badb3538a93bff3ac5b5189b59cccfa1

runtime_exceptions: Update AT speculative workaround ...

As per latest mailing communication [1], we decided to
update AT speculative workaround implementation in order to
disable page table walk for lower ELs(EL1 or EL0) immediately
after context switching to EL3 from lower ELs.

Previous implementation of AT speculative workaround is available
here: 45aecff00

AT speculative workaround is updated as below:
1. Avoid saving and restoring of SCTLR and TCR registers for EL1
   in context save and restore routine respectively.
2. On EL3 entry, save SCTLR and TCR registers for EL1.
3. On EL3 entry, update EL1 system registers to disable stage 1
   page table walk for lower ELs (EL1 and EL0) and enable EL1
   MMU.
4. On EL3 exit, restore SCTLR and TCR registers for EL1 which
   are saved in step 2.

[1]:
https://lists.trustedfirmware.org/pipermail/tf-a/2020-July/000586.html

Change-Id: Iee8de16f81dc970a8f492726f2ddd57e7bd9ffb5
Signed-off-by: Manish V Badarkhe <Manish.Badarkhe@arm.com>

el3_runtime: Rearrange context offset of EL1 sys registers ...

SCTLR and TCR registers of EL1 plays role in enabling/disabling of
page table walk for lower ELs (EL0 and EL1).
Hence re-arranged EL1 context offsets to have SCTLR and TCR registers
values one after another in the stack so that these registers values
can be saved and restored using stp and ldp instruction respectively.

Change-Id: Iaa28fd9eba82a60932b6b6d85ec8857a9acd5f8b
Signed-off-by: Manish V Badarkhe <Manish.Badarkhe@arm.com>

el3_runtime: Update context save and restore routines for EL1 and EL2 ...

As per latest mailing communication [1], we decided
not to update SCTLR and TCR registers in EL1 and EL2 context
restore routine when AT speculative workaround is enabled
hence reverted the changes done as part of this commit: 45aecff00.

[1]:
https://lists.trustedfirmware.org/pipermail/tf-a/2020-July/000586.html

Change-Id: I8c5f31d81fcd53770a610e302a5005d98772b71f
Signed-off-by: Manish V Badarkhe <Manish.Badarkhe@arm.com>

Prevent RAS register access from lower ELs ...

This patch adds a build config 'RAS_TRAP_LOWER_EL_ERR_ACCESS' to set
SCR_EL3.TERR during CPU boot. This bit enables trapping RAS register
accesses from EL1 or EL2 to EL3.

RAS_TRAP_LOWER_EL_ERR_ACCESS is disabled by default.

Signed-off-by: Varun Wadekar <vwadekar@nvidia.com>
Change-Id: Ifb0fb0afedea7dd2a29a0b0491a1161ecd241438

Enable ARMv8.6-ECV Self-Synch when booting to EL2 ...

Enhanced Counter Virtualization, ECV, is an architecture extension introduced
in ARMv8.6. This extension allows the hypervisor, at EL2, to setup
self-synchronizing views of the timers for it's EL1 Guests. This patch pokes the
control register to enable this extension when booting a hypervisor at EL2.

Change-Id: I4e929ecdf400cea17eff1de5cf8704aa7e40973d
Signed-off-by: Jimmy Brisson <jimmy.brisson@arm.com>

Enable ARMv8.6-FGT when booting to EL2 ...

The Fine Grained Traps (FGT) architecture extension was added to aarch64 in
ARMv8.6. This extension primarily allows hypervisors, at EL2, to trap specific
instructions in a more fine grained manner, with an enable bit for each
instruction. This patch adds support for this extension by enabling the
extension when booting an hypervisor at EL2.

Change-Id: Idb9013ed118b6a1b7b76287237096de992ca4da3
Signed-off-by: Jimmy Brisson <jimmy.brisson@arm.com>

Enable v8.6 WFE trap delays ...

This patch enables the v8.6 extension to add a delay before WFE traps
are taken. A weak hook plat_arm_set_twedel_scr_el3 has been added in
plat/common/aarch64/plat_common.c that disables this feature by default
but platform-specific code can override it when needed.

The only hook provided sets the TWED fields in SCR_EL3, there are similar
fields in HCR_EL2, SCTLR_EL2, and SCTLR_EL1 to control WFE trap delays in
lower ELs but these should be configured by code running at EL2 and/or EL1
depending on the platform configuration and is outside the scope of TF-A.

Signed-off-by: John Powell <john.powell@arm.com>
Change-Id: I0a9bb814205efeab693a3d0a0623e62144abba2d

Fix exception in save/restore of EL2 registers. ...

Removing FPEXC32_EL2 from the register save/restore routine for EL2
registers since it is already a part of save/restore routine for
fpregs.

Signed-off-by: Max Shvetsov <maksims.svecovs@arm.com>
Change-Id: I5ed45fdbf7c8efa8dcfcd96586328d4f6b256bc4

Implement workaround for AT speculative behaviour ...

During context switching from higher EL (EL2 or higher)
to lower EL can cause incorrect translation in TLB due to
speculative execution of AT instruction using out-of-context
translation regime.

Workaround is implemented as below during EL's (EL1 or EL2)
"context_restore" operation:
1. Disable page table walk using SCTLR.M and TCR.EPD0 & EPD1
   bits for EL1 or EL2 (stage1 and stage2 disabled)
2. Save all system registers except TCR and SCTLR (for EL1 and EL2)
3. Do memory barrier operation (isb) to ensure all
   system register writes are done.
4. Restore TCR and SCTLR registers (for EL1 and EL2)

Errata details are available for various CPUs as below:
Cortex-A76: 1165522
Cortex-A72: 1319367
Cortex-A57: 1319537
Cortex-A55: 1530923
Cortex-A53: 1530924

More details can be found in mail-chain:
https://lists.trustedfirmware.org/pipermail/tf-a/2020-April/000445.html

Currently, Workaround is implemented as build option which is default
disabled.

Signed-off-by: Manish V Badarkhe <Manish.Badarkhe@arm.com>
Change-Id: If8545e61f782cb0c2dda7ffbaf50681c825bd2f0

context: TPIDR_EL2 register not saved/restored ...

TPIDR_EL2 is missing from the EL2 state register save/restore
sequence. This patch adds it to the context save restore routines.

Signed-off-by: Olivier Deprez <olivier.deprez@arm.com>
Change-Id: I35fc5ee82f97b72bcedac57c791312e7b3a45251

SPMD: Adds partially supported EL2 registers. ...

This patch adds EL2 registers that are supported up to ARMv8.6.
ARM_ARCH_MINOR has to specified to enable save/restore routine.

Note: Following registers are still not covered in save/restore.
 * AMEVCNTVOFF0<n>_EL2
 * AMEVCNTVOFF1<n>_EL2
 * ICH_AP0R<n>_EL2
 * ICH_AP1R<n>_EL2
 * ICH_LR<n>_EL2

Change-Id: I4813f3243e56e21cb297b31ef549a4b38d4876e1
Signed-off-by: Max Shvetsov <maksims.svecovs@arm.com>

SPMD: save/restore EL2 system registers. ...

NOTE: Not all EL-2 system registers are saved/restored.
This subset includes registers recognized by ARMv8.0

Change-Id: I9993c7d78d8f5f8e72d1c6c8d6fd871283aa3ce0
Signed-off-by: Jose Marinho <jose.marinho@arm.com>
Signed-off-by: Olivier Deprez <olivier.deprez@arm.com>
Signed-off-by: Artsem Artsemenka <artsem.artsemenka@arm.com>
Signed-off-by: Max Shvetsov <maksims.svecovs@arm.com>

Merge "Coverity: remove unnecessary header file includes" into integration

Merge changes from topic "mp/separate_nobits" into integration ...

* changes:
  plat/arm: Add support for SEPARATE_NOBITS_REGION
  Changes necessary to support SEPARATE_NOBITS_REGION feature

Coverity: remove unnecessary header file includes ...

This patch removes unnecessary header file includes
discovered by Coverity HFA option.

Change-Id: I2827c37c1c24866c87db0e206e681900545925d4
Signed-off-by: Zelalem <zelalem.aweke@arm.com>

Use correct type when reading SCR register ...

The Secure Configuration Register is 64-bits in AArch64 and 32-bits in
AArch32. Use u_register_t instead of unsigned int to reflect this.

Change-Id: I51b69467baba36bf0cfaec2595dc8837b1566934
Signed-off-by: Louis Mayencourt <louis.mayencourt@arm.com>

Changes necessary to support SEPARATE_NOBITS_REGION feature ...

Since BL31 PROGBITS and BL31 NOBITS sections are going to be
in non-adjacent memory regions, potentially far from each other,
some fixes are needed to support it completely.

1. adr instruction only allows computing the effective address
of a location only within 1MB range of the PC. However, adrp
instruction together with an add permits position independent
address of any location with 4GB range of PC.

2. Since BL31 _RW_END_ marks the end of BL31 image, care must be
taken that it is aligned to page size since we map this memory
region in BL31 using xlat_v2 lib utils which mandate alignment of
image size to page granularity.

Change-Id: I3451cc030d03cb2032db3cc088f0c0e2c84bffda
Signed-off-by: Madhukar Pappireddy <madhukar.pappireddy@arm.com>

Prevent speculative execution past ERET ...

Even though ERET always causes a jump to another address, aarch64 CPUs
speculatively execute following instructions as if the ERET
instruction was not a jump instruction.
The speculative execution does not cross privilege-levels (to the jump
target as one would expect), but it continues on the kernel privilege
level as if the ERET instruction did not change the control flow -
thus execution anything that is accidentally linked after the ERET
instruction. Later, the results of this speculative execution are
always architecturally discarded, however they can leak data using
microarchitectural side channels. This speculative execution is very
reliable (seems to be unconditional) and it manages to complete even
relatively performance-heavy operations (e.g. multiple dependent
fetches from uncached memory).

This was fixed in Linux, FreeBSD, OpenBSD and Optee OS:
https://github.com/torvalds/linux/commit/679db70801da9fda91d26caf13bf5b5ccc74e8e8
https://github.com/freebsd/freebsd/commit/29fb48ace4186a41c409fde52bcf4216e9e50b61
https://github.com/openbsd/src/commit/3a08873ece1cb28ace89fd65e8f3c1375cc98de2
https://github.com/OP-TEE/optee_os/commit/abfd092aa19f9c0251e3d5551e2d68a9ebcfec8a

It is demonstrated in a SafeSide example:
https://github.com/google/safeside/blob/master/demos/eret_hvc_smc_wrapper.cc
https://github.com/google/safeside/blob/master/kernel_modules/kmod_eret_hvc_smc/eret_hvc_smc_module.c

Signed-off-by: Anthony Steinhauser <asteinhauser@google.com>
Change-Id: Iead39b0b9fb4b8d8b5609daaa8be81497ba63a0f

aarch64: Fix stack pointer maintenance on EA handling path ...

EA handlers for exceptions taken from lower ELs at the end invokes
el3_exit function. However there was a bug with sp maintenance which
resulted in el3_exit setting runtime stack to context. This in turn
caused memory corruption on consecutive EL3 entries.

Signed-off-by: Jan Dabros <jsd@semihalf.com>
Change-Id: I0424245c27c369c864506f4baa719968890ce659

S-EL2 Support: Check for AArch64 ...

Check that entry point information requesting S-EL2
has AArch64 as an execution state during context setup.

Signed-off-by: Artsem Artsemenka <artsem.artsemenka@arm.com>
Change-Id: I447263692fed6e55c1b076913e6eb73b1ea735b7

Add support for enabling S-EL2 ...

This patch adds support for enabling S-EL2 if this EL is specified in the entry
point information being used to initialise a secure context. It is the caller's
responsibility to check if S-EL2 is available on the system before requesting
this EL through the entry point information.

Signed-off-by: Achin Gupta <achin.gupta@arm.com>
Change-Id: I2752964f078ab528b2e80de71c7d2f35e60569e1

Merge "AArch32: Disable Secure Cycle Counter" into integration

AArch32: Disable Secure Cycle Counter ...

This patch changes implementation for disabling Secure Cycle
Counter. For ARMv8.5 the counter gets disabled by setting
SDCR.SCCD bit on CPU cold/warm boot. For the earlier
architectures PMCR register is saved/restored on secure
world entry/exit from/to Non-secure state, and cycle counting
gets disabled by setting PMCR.DP bit.
In 'include\aarch32\arch.h' header file new
ARMv8.5-PMU related definitions were added.

Change-Id: Ia8845db2ebe8de940d66dff479225a5b879316f8
Signed-off-by: Alexei Fedorov <Alexei.Fedorov@arm.com>

Fix MTE support from causing unused variable warnings ...

assert() calls are removed in release builds, and if that assert call is
the only use of a variable, an unused variable warning will be triggered
in a release build. This patch fixes this problem when
CTX_INCLUDE_MTE_REGS by not using an intermediate variable to store the
results of get_armv8_5_mte_support().

Change-Id: I529e10ec0b2c8650d2c3ab52c4f0cecc0b3a670e
Signed-off-by: Justin Chadwell <justin.chadwell@arm.com>

Merge changes from topic "db/unsigned_long" into integration ...

* changes:
  Unsigned long should not be used as per coding guidelines
  SCTLR and ACTLR are 32-bit for AArch32 and 64-bit for AArch64