PSCI: Build option to enable D-Caches early in warmboot ...

This patch introduces a build option to enable D-cache early on the CPU
after warm boot. This is applicable for platforms which do not require
interconnect programming to enable cache coherency (eg: single cluster
platforms). If this option is enabled, then warm boot path enables
D-caches immediately after enabling MMU.

Fixes ARM-Software/tf-issues#456

Change-Id: I44c8787d116d7217837ced3bcf0b1d3441c8d80e
Signed-off-by: Soby Mathew <soby.mathew@arm.com>

Add support for GCC stack protection ...

Introduce new build option ENABLE_STACK_PROTECTOR. It enables
compilation of all BL images with one of the GCC -fstack-protector-*
options.

A new platform function plat_get_stack_protector_canary() is introduced.
It returns a value that is used to initialize the canary for stack
corruption detection. Returning a random value will prevent an attacker
from predicting the value and greatly increase the effectiveness of the
protection.

A message is printed at the ERROR level when a stack corruption is
detected.

To be effective, the global data must be stored at an address
lower than the base of the stacks. Failure to do so would allow an
attacker to overwrite the canary as part of an attack which would void
the protection.

FVP implementation of plat_get_stack_protector_canary is weak as
there is no real source of entropy on the FVP. It therefore relies on a
timer's value, which could be predictable.

Change-Id: Icaaee96392733b721fa7c86a81d03660d3c1bc06
Signed-off-by: Douglas Raillard <douglas.raillard@arm.com>

Merge pull request #860 from jeenu-arm/hw-asstd-coh ...

Patches for platforms with hardware-assisted coherency

Simplify translation tables headers dependencies ...

The files affected by this patch don't really depend on `xlat_tables.h`.
By changing the included file it becomes easier to switch between the
two versions of the translation tables library.

Change-Id: Idae9171c490e0865cb55883b19eaf942457c4ccc
Signed-off-by: Antonio Nino Diaz <antonio.ninodiaz@arm.com>

Enable data caches early with hardware-assisted coherency ...

At present, warm-booted CPUs keep their caches disabled when enabling
MMU, and remains so until they enter coherency later.

On systems with hardware-assisted coherency, for which
HW_ASSISTED_COHERENCY build flag would be enabled, warm-booted CPUs can
have both caches and MMU enabled at once.

Change-Id: Icb0adb026e01aecf34beadf49c88faa9dd368327
Signed-off-by: Jeenu Viswambharan <jeenu.viswambharan@arm.com>

Replace some memset call by zeromem ...

Replace all use of memset by zeromem when zeroing moderately-sized
structure by applying the following transformation:
memset(x, 0, sizeof(x)) => zeromem(x, sizeof(x))

As the Trusted Firmware is compiled with -ffreestanding, it forbids the
compiler from using __builtin_memset and forces it to generate calls to
the slow memset implementation. Zeromem is a near drop in replacement
for this use case, with a more efficient implementation on both AArch32
and AArch64.

Change-Id: Ia7f3a90e888b96d056881be09f0b4d65b41aa79e
Signed-off-by: Douglas Raillard <douglas.raillard@arm.com>

Introduce unified API to zero memory ...

Introduce zeromem_dczva function on AArch64 that can handle unaligned
addresses and make use of DC ZVA instruction to zero a whole block at a
time. This zeroing takes place directly in the cache to speed it up
without doing external memory access.

Remove the zeromem16 function on AArch64 and replace it with an alias to
zeromem. This zeromem16 function is now deprecated.

Remove the 16-bytes alignment constraint on __BSS_START__ in
firmware-design.md as it is now not mandatory anymore (it used to comply
with zeromem16 requirements).

Change the 16-bytes alignment constraints in SP min's linker script to a
8-bytes alignment constraint as the AArch32 zeromem implementation is now
more efficient on 8-bytes aligned addresses.

Introduce zero_normalmem and zeromem helpers in platform agnostic header
that are implemented this way:
* AArch32:
	* zero_normalmem: zero using usual data access
	* zeromem: alias for zero_normalmem
* AArch64:
	* zero_normalmem: zero normal memory  using DC ZVA instruction
	                  (needs MMU enabled)
	* zeromem: zero using usual data access

Usage guidelines: in most cases, zero_normalmem should be preferred.

There are 2 scenarios where zeromem (or memset) must be used instead:
* Code that must run with MMU disabled (which means all memory is
  considered device memory for data accesses).
* Code that fills device memory with null bytes.

Optionally, the following rule can be applied if performance is
important:
* Code zeroing small areas (few bytes) that are not secrets should use
  memset to take advantage of compiler optimizations.

  Note: Code zeroing security-related critical information should use
  zero_normalmem/zeromem instead of memset to avoid removal by
  compilers' optimizations in some cases or misbehaving versions of GCC.

Fixes ARM-software/tf-issues#408

Change-Id: Iafd9663fc1070413c3e1904e54091cf60effaa82
Signed-off-by: Douglas Raillard <douglas.raillard@arm.com>

Abort preempted TSP STD SMC after PSCI CPU suspend ...

Standard SMC requests that are handled in the secure-world by the Secure
Payload can be preempted by interrupts that must be handled in the
normal world. When the TSP is preempted the secure context is stored and
control is passed to the normal world to handle the non-secure
interrupt. Once completed the preempted secure context is restored. When
restoring the preempted context, the dispatcher assumes that the TSP
preempted context is still stored as the SECURE context by the context
management library.

However, PSCI power management operations causes synchronous entry into
TSP. This overwrites the preempted SECURE context in the context
management library. When restoring back the SECURE context, the Secure
Payload crashes because this context is not the preempted context
anymore.

This patch avoids corruption of the preempted SECURE context by aborting
any preempted SMC during PSCI power management calls. The
abort_std_smc_entry hook of the TSP is called when aborting the SMC
request.

It also exposes this feature as a FAST SMC callable from normal world to
abort preempted SMC with FID TSP_FID_ABORT.

Change-Id: I7a70347e9293f47d87b5de20484b4ffefb56b770
Signed-off-by: Douglas Raillard <douglas.raillard@arm.com>

Merge pull request #775 from soby-mathew/sm/AArch32_stack_align ...

AArch32: Fix the stack alignment issue

AArch32: Fix the stack alignment issue ...

The AArch32 Procedure call Standard mandates that the stack must be aligned
to 8 byte boundary at external interfaces. This patch does the required
changes.

This problem was detected when a crash was encountered in
`psci_print_power_domain_map()` while printing 64 bit values. Aligning
the stack to 8 byte boundary resolved the problem.

Fixes ARM-Software/tf-issues#437

Change-Id: I517bd8203601bb88e9311bd36d477fb7b3efb292
Signed-off-by: Soby Mathew <soby.mathew@arm.com>

Define and use no_ret macro where no return is expected ...

There are many instances in ARM Trusted Firmware where control is
transferred to functions from which return isn't expected. Such jumps
are made using 'bl' instruction to provide the callee with the location
from which it was jumped to. Additionally, debuggers infer the caller by
examining where 'lr' register points to. If a 'bl' of the nature
described above falls at the end of an assembly function, 'lr' will be
left pointing to a location outside of the function range. This misleads
the debugger back trace.

This patch defines a 'no_ret' macro to be used when jumping to functions
from which return isn't expected. The macro ensures to use 'bl'
instruction for the jump, and also, for debug builds, places a 'nop'
instruction immediately thereafter (unless instructed otherwise) so as
to leave 'lr' pointing within the function range.

Change-Id: Ib34c69fc09197cfd57bc06e147cc8252910e01b0
Co-authored-by: Douglas Raillard <douglas.raillard@arm.com>
Signed-off-by: Jeenu Viswambharan <jeenu.viswambharan@arm.com>

PSCI: Do psci_setup() as part of std_svc_setup() ...

This patch moves the invocation of `psci_setup()` from BL31 and SP_MIN
into `std_svc_setup()` as part of ARM Standard Service initialization.
This allows us to consolidate ARM Standard Service initializations which
will be added to in the future. A new function `get_arm_std_svc_args()`
is introduced to get arguments corresponding to each standard service.
This function must be implemented by the EL3 Runtime Firmware and both
SP_MIN and BL31 implement it.

Change-Id: I38e1b644f797fa4089b20574bd4a10f0419de184

PSCI: Introduce PSCI Library argument structure ...

This patch introduces a `psci_lib_args_t` structure which must be
passed into `psci_setup()` which is then used to initialize the PSCI
library. The `psci_lib_args_t` is a versioned structure so as to enable
compatibility checks during library initialization. Both BL31 and SP_MIN
are modified to use the new structure.

SP_MIN is also modified to add version string and build message as part
of its cold boot log just like the other BLs in Trusted Firmware.

NOTE: Please be aware that this patch modifies the prototype of
`psci_setup()`, which breaks compatibility with EL3 Runtime Firmware
(excluding BL31 and SP_MIN) integrated with the PSCI Library.

Change-Id: Ic3761db0b790760a7ad664d8a437c72ea5edbcd6

AArch32: Support in SP_MIN to receive arguments from BL2 ...

This patch adds support in SP_MIN to receive generic and
platform specific arguments from BL2.

The new signature is as following:
    void sp_min_early_platform_setup(void *from_bl2,
         void *plat_params_from_bl2);

ARM platforms have been modified to use this support.

Note: Platforms may break if using old signature.
      Default value for RESET_TO_SP_MIN is changed to 0.

Change-Id: I008d4b09fd3803c7b6231587ebf02a047bdba8d0

AArch32: Refactor SP_MIN to support RESET_TO_SP_MIN ...

This patch uses the `el3_entrypoint_common` macro to initialize
CPU registers, in SP_MIN entrypoint.s file, in both cold and warm
boot path. It also adds conditional compilation, in cold and warm
boot entry path, based on RESET_TO_SP_MIN.

Change-Id: Id493ca840dc7b9e26948dc78ee928e9fdb76b9e4

AArch32: add a minimal secure payload (SP_MIN) ...

This patch adds a minimal AArch32 secure payload SP_MIN. It relies on PSCI
library to initialize the normal world context. It runs in Monitor mode
and uses the runtime service framework to handle SMCs. It is added as
a BL32 component in the Trusted Firmware source tree.

Change-Id: Icc04fa6b242025a769c1f6c7022fde19459c43e9

Move spinlock library code to AArch64 folder ...

This patch moves the assembly exclusive lock library code
`spinlock.S` into architecture specific folder `aarch64`.
A stub file which includes the file from new location is
retained at the original location for compatibility. The BL
makefiles are also modified to include the file from the new
location.

Change-Id: Ide0b601b79c439e390c3a017d93220a66be73543

TSP: Print BL32_BASE rather than __RO_START__ ...

In debug builds, the TSP prints its image base address and size.
The base address displayed corresponds to the start address of the
read-only section, as defined in the linker script.

This patch changes this to use the BL32_BASE address instead, which is
the same address as __RO_START__ at the moment but has the advantage
to be independent of the linker symbols defined in the linker script
as well as the layout and order of the sections.

Change-Id: I032d8d50df712c014cbbcaa84a9615796ec902cc

Introduce SEPARATE_CODE_AND_RODATA build flag ...

At the moment, all BL images share a similar memory layout: they start
with their code section, followed by their read-only data section.
The two sections are contiguous in memory. Therefore, the end of the
code section and the beginning of the read-only data one might share
a memory page. This forces both to be mapped with the same memory
attributes. As the code needs to be executable, this means that the
read-only data stored on the same memory page as the code are
executable as well. This could potentially be exploited as part of
a security attack.

This patch introduces a new build flag called
SEPARATE_CODE_AND_RODATA, which isolates the code and read-only data
on separate memory pages. This in turn allows independent control of
the access permissions for the code and read-only data.

This has an impact on memory footprint, as padding bytes need to be
introduced between the code and read-only data to ensure the
segragation of the two. To limit the memory cost, the memory layout
of the read-only section has been changed in this case.

 - When SEPARATE_CODE_AND_RODATA=0, the layout is unchanged, i.e.
   the read-only section still looks like this (padding omitted):

   |        ...        |
   +-------------------+
   | Exception vectors |
   +-------------------+
   |  Read-only data   |
   +-------------------+
   |       Code        |
   +-------------------+ BLx_BASE

   In this case, the linker script provides the limits of the whole
   read-only section.

 - When SEPARATE_CODE_AND_RODATA=1, the exception vectors and
   read-only data are swapped, such that the code and exception
   vectors are contiguous, followed by the read-only data. This
   gives the following new layout (padding omitted):

   |        ...        |
   +-------------------+
   |  Read-only data   |
   +-------------------+
   | Exception vectors |
   +-------------------+
   |       Code        |
   +-------------------+ BLx_BASE

   In this case, the linker script now exports 2 sets of addresses
   instead: the limits of the code and the limits of the read-only
   data. Refer to the Firmware Design guide for more details. This
   provides platform code with a finer-grained view of the image
   layout and allows it to map these 2 regions with the appropriate
   access permissions.

Note that SEPARATE_CODE_AND_RODATA applies to all BL images.

Change-Id: I936cf80164f6b66b6ad52b8edacadc532c935a49

Introduce some helper macros for exception vectors ...

This patch introduces some assembler macros to simplify the
declaration of the exception vectors. It abstracts the section
the exception code is put into as well as the alignments
constraints mandated by the ARMv8 architecture. For all TF images,
the exception code has been updated to make use of these macros.

This patch also updates some invalid comments in the exception
vector code.

Change-Id: I35737b8f1c8c24b6da89b0a954c8152a4096fa95

Make:Remove calls to shell from makefiles. ...

As an initial stage of making Trusted Firmware build environment more
portable, we remove most uses of the $(shell ) function and replace them
with more portable make function based solutions.

Note that the setting of BUILD_STRING still uses $(shell ) since it's
not possible to reimplement this as a make function. Avoiding invocation
of this on incompatible host platforms will be implemented separately.

Change-Id: I768e2f9a265c78814a4adf2edee4cc46cda0f5b8

Remove all non-configurable dead loops ...

Added a new platform porting function plat_panic_handler, to allow
platforms to handle unexpected error situations. It must be
implemented in assembly as it may be called before the C environment
is initialized. A default implementation is provided, which simply
spins.

Corrected all dead loops in generic code to call this function
instead. This includes the dead loop that occurs at the end of the
call to panic().

All unnecesary wfis from bl32/tsp/aarch64/tsp_exceptions.S have
been removed.

Change-Id: I67cb85f6112fa8e77bd62f5718efcef4173d8134

Remove dashes from image names: 'BL3-x' --> 'BL3x' ...

This patch removes the dash character from the image name, to
follow the image terminology in the Trusted Firmware Wiki page:

    https://github.com/ARM-software/arm-trusted-firmware/wiki

Changes apply to output messages, comments and documentation.

non-ARM platform files have been left unmodified.

Change-Id: Ic2a99be4ed929d52afbeb27ac765ceffce46ed76

TSP: Allow preemption of synchronous S-EL1 interrupt handling ...

Earlier the TSP only ever expected to be preempted during Standard SMC
processing. If a S-EL1 interrupt triggered while in the normal world, it
will routed to S-EL1 `synchronously` for handling. The `synchronous` S-EL1
interrupt handler `tsp_sel1_intr_entry` used to panic if this S-EL1 interrupt
was preempted by another higher priority pending interrupt which should be
handled in EL3 e.g. Group0 interrupt in GICv3.

With this patch, the `tsp_sel1_intr_entry` now expects `TSP_PREEMPTED` as the
return code from the `tsp_common_int_handler` in addition to 0 (interrupt
successfully handled) and in both cases it issues an SMC with id
`TSP_HANDLED_S_EL1_INTR`. The TSPD switches the context and returns back
to normal world. In case a higher priority EL3 interrupt was pending, the
execution will be routed to EL3 where interrupt will be handled. On return
back to normal world, the pending S-EL1 interrupt which was preempted will
get routed to S-EL1 to be handled `synchronously` via `tsp_sel1_intr_entry`.

Change-Id: I2087c7fedb37746fbd9200cdda9b6dba93e16201

Enable use of FIQs and IRQs as TSP interrupts ...

On a GICv2 system, interrupts that should be handled in the secure world are
typically signalled as FIQs. On a GICv3 system, these interrupts are signalled
as IRQs instead. The mechanism for handling both types of interrupts is the same
in both cases. This patch enables the TSP to run on a GICv3 system by:

1. adding support for handling IRQs in the exception handling code.
2. removing use of "fiq" in the names of data structures, macros and functions.

The build option TSPD_ROUTE_IRQ_TO_EL3 is deprecated and is replaced with a
new build flag TSP_NS_INTR_ASYNC_PREEMPT. For compatibility reasons, if the
former build flag is defined, it will be used to define the value for the
new build flag. The documentation is also updated accordingly.

Change-Id: I1807d371f41c3656322dd259340a57649833065e

Unify interrupt return paths from TSP into the TSPD ...

The TSP is expected to pass control back to EL3 if it gets preempted due to
an interrupt while handling a Standard SMC in the following scenarios:

1. An FIQ preempts Standard SMC execution and that FIQ is not a TSP Secure
   timer interrupt or is preempted by a higher priority interrupt by the time
   the TSP acknowledges it. In this case, the TSP issues an SMC with the ID
   as `TSP_EL3_FIQ`. Currently this case is never expected to happen as only
   the TSP Secure Timer is expected to generate FIQ.

2. An IRQ preempts Standard SMC execution and in this case the TSP issues
   an SMC with the ID as `TSP_PREEMPTED`.

In both the cases, the TSPD hands control back to the normal world and returns
returns an error code to the normal world to indicate that the standard SMC it
had issued has been preempted but not completed.

This patch unifies the handling of these two cases in the TSPD and ensures that
the TSP only uses TSP_PREEMPTED instead of separate SMC IDs. Also instead of 2
separate error codes, SMC_PREEMPTED and TSP_EL3_FIQ, only SMC_PREEMPTED is
returned as error code back to the normal world.

Background information: On a GICv3 system, when the secure world has affinity
routing enabled, in 2. an FIQ will preempt TSP execution instead of an IRQ. The
FIQ could be a result of a Group 0 or a Group 1 NS interrupt. In both case, the
TSPD passes control back to the normal world upon receipt of the TSP_PREEMPTED
SMC. A Group 0 interrupt will immediately preempt execution to EL3 where it
will be handled. This allows for unified interrupt handling in TSP for both
GICv3 and GICv2 systems.

Change-Id: I9895344db74b188021e3f6a694701ad272fb40d4

Make generic code work in presence of system caches ...

On the ARMv8 architecture, cache maintenance operations by set/way on the last
level of integrated cache do not affect the system cache. This means that such a
flush or clean operation could result in the data being pushed out to the system
cache rather than main memory. Another CPU could access this data before it
enables its data cache or MMU. Such accesses could be serviced from the main
memory instead of the system cache. If the data in the sysem cache has not yet
been flushed or evicted to main memory then there could be a loss of
coherency. The only mechanism to guarantee that the main memory will be updated
is to use cache maintenance operations to the PoC by MVA(See section D3.4.11
(System level caches) of ARMv8-A Reference Manual (Issue A.g/ARM DDI0487A.G).

This patch removes the reliance of Trusted Firmware on the flush by set/way
operation to ensure visibility of data in the main memory. Cache maintenance
operations by MVA are now used instead. The following are the broad category of
changes:

1. The RW areas of BL2/BL31/BL32 are invalidated by MVA before the C runtime is
   initialised. This ensures that any stale cache lines at any level of cache
   are removed.

2. Updates to global data in runtime firmware (BL31) by the primary CPU are made
   visible to secondary CPUs using a cache clean operation by MVA.

3. Cache maintenance by set/way operations are only used prior to power down.

NOTE: NON-UPSTREAM TRUSTED FIRMWARE CODE SHOULD MAKE EQUIVALENT CHANGES IN
ORDER TO FUNCTION CORRECTLY ON PLATFORMS WITH SUPPORT FOR SYSTEM CACHES.

Fixes ARM-software/tf-issues#205

Change-Id: I64f1b398de0432813a0e0881d70f8337681f6e9a

Pass the target suspend level to SPD suspend hooks ...

In certain Trusted OS implementations it is a requirement to pass them the
highest power level which will enter a power down state during a PSCI
CPU_SUSPEND or SYSTEM_SUSPEND API invocation. This patch passes this power level
to the SPD in the "max_off_pwrlvl" parameter of the svc_suspend() hook.

Currently, the highest power level which was requested to be placed in a low
power state (retention or power down) is passed to the SPD svc_suspend_finish()
hook. This hook is called after emerging from the low power state. It is more
useful to pass the highest power level which was powered down instead. This
patch does this by changing the semantics of the parameter passed to an SPD's
svc_suspend_finish() hook. The name of the parameter has been changed from
"suspend_level" to "max_off_pwrlvl" as well. Same changes have been made to the
parameter passed to the tsp_cpu_resume_main() function.

NOTE: THIS PATCH CHANGES THE SEMANTICS OF THE EXISTING "svc_suspend_finish()"
      API BETWEEN THE PSCI AND SPD/SP IMPLEMENTATIONS. THE LATTER MIGHT NEED
      UPDATES TO ENSURE CORRECT BEHAVIOUR.

Change-Id: If3a9d39b13119bbb6281f508a91f78a2f46a8b90

PSCI: Migrate SPDs and TSP to the new platform and framework API ...

The new PSCI frameworks mandates that the platform APIs and the various
frameworks in Trusted Firmware migrate away from MPIDR based core
identification to one based on core index. Deprecated versions of the old
APIs are still present to provide compatibility but their implementations
are not optimal. This patch migrates the various SPDs exisiting within
Trusted Firmware tree and TSP to the new APIs.

Change-Id: Ifc37e7071c5769b5ded21d0b6a071c8c4cab7836

Add linker symbol declarations to bl_common.h ...

Add extern declarations of linker symbols to bl_common.h. These are
used by platform ports to determine the memory layout of BL images.
Adding the declarations to this file facilitates removal of these
declarations from the platform porting source files in subsequent
patches.

Also remove the linker symbol declarations from common TSP source
code.

Change-Id: I8ed0426bc815317c4536b588e4e78bc15b4fe91c