Sanitise includes across codebase ...

Enforce full include path for includes. Deprecate old paths.

The following folders inside include/lib have been left unchanged:

- include/lib/cpus/${ARCH}
- include/lib/el3_runtime/${ARCH}

The reason for this change is that having a global namespace for
includes isn't a good idea. It defeats one of the advantages of having
folders and it introduces problems that are sometimes subtle (because
you may not know the header you are actually including if there are two
of them).

For example, this patch had to be created because two headers were
called the same way: e0ea0928d5b7 ("Fix gpio includes of mt8173 platform
to avoid collision."). More recently, this patch has had similar
problems: 46f9b2c3a282 ("drivers: add tzc380 support").

This problem was introduced in commit 4ecca33988b9 ("Move include and
source files to logical locations"). At that time, there weren't too
many headers so it wasn't a real issue. However, time has shown that
this creates problems.

Platforms that want to preserve the way they include headers may add the
removed paths to PLAT_INCLUDES, but this is discouraged.

Change-Id: I39dc53ed98f9e297a5966e723d1936d6ccf2fc8f
Signed-off-by: Antonio Nino Diaz <antonio.ninodiaz@arm.com>

xlat v2: Support the EL2 translation regime ...

The translation library is useful elsewhere. Even though this repository
doesn't exercise the EL2 support of the library, it is better to have it
here as well to make it easier to maintain.

enable_mmu_secure() and enable_mmu_direct() have been deprecated. The
functions are still present, but they are behind ERROR_DEPRECATED and
they call the new functions enable_mmu_svc_mon() and
enable_mmu_direct_svc_mon().

Change-Id: I13ad10cd048d9cc2d55e0fff9a5133671b67dcba
Signed-off-by: Antonio Nino Diaz <antonio.ninodiaz@arm.com>

xlat v2: Flush xlat tables after being modified ...

During cold boot, the initial translation tables are created with data
caches disabled, so all modifications go to memory directly. After the
MMU is enabled and data cache is enabled, any modification to the tables
goes to data cache, and eventually may get flushed to memory.

If CPU0 modifies the tables while CPU1 is off, CPU0 will have the
modified tables in its data cache. When CPU1 is powered on, the MMU is
enabled, then it enables coherency, and then it enables the data cache.
Until this is done, CPU1 isn't in coherency, and the translation tables
it sees can be outdated if CPU0 still has some modified entries in its
data cache.

This can be a problem in some cases. For example, the warm boot code
uses only the tables mapped during cold boot, which don't normally
change. However, if they are modified (and a RO page is made RW, or a XN
page is made executable) the CPU will see the old attributes and crash
when it tries to access it.

This doesn't happen in systems with HW_ASSISTED_COHERENCY or
WARMBOOT_ENABLE_DCACHE_EARLY. In these systems, the data cache is
enabled at the same time as the MMU. As soon as this happens, the CPU is
in coherency.

There was an attempt of a fix in psci_helpers.S, but it didn't solve the
problem. That code has been deleted. The code was introduced in commit
<264410306381> ("Invalidate TLB entries during warm boot").

Now, during a map or unmap operation, the memory associated to each
modified table is flushed. Traversing a table will also flush it's
memory, as there is no way to tell in the current implementation if the
table that has been traversed has also been modified.

Change-Id: I4b520bca27502f1018878061bc5fb82af740bb92
Signed-off-by: Antonio Nino Diaz <antonio.ninodiaz@arm.com>

xlat: Use bool instead of int ...

Change-Id: I35d5b6a7c219f6f38983b30f157c1ed3808af17f
Signed-off-by: Antonio Nino Diaz <antonio.ninodiaz@arm.com>

xlat: Fix MISRA defects ...

Fix defects of MISRA C-2012 rules 8.13, 10.1, 10.3, 10.4, 10.8, 11.6,
14.4, 15.7, 17.8, 20.10, 20.12, 21.1 and Directive 4.9.

Change-Id: I7ff61e71733908596dbafe2e99d99b4fce9765bd
Signed-off-by: Antonio Nino Diaz <antonio.ninodiaz@arm.com>

xlat: Remove references to the Trusted Firmware ...

This library can be used in other projects. All comments that talk about
the Trusted Firmware should be talking about the library itself.

Change-Id: I3b98d42f7132be72c1f8a4900acfaa78dbd2daa2
Signed-off-by: Antonio Nino Diaz <antonio.ninodiaz@arm.com>

xlat v2: Make setup_mmu_config public ...

This allows other parts of the code to reuse it. No functional changes.

Change-Id: Ib052ae235c422d9179958bd3016c3e678779ae9b
Signed-off-by: Antonio Nino Diaz <antonio.ninodiaz@arm.com>

xlat v2: Turn MMU parameters into 64-bit values ...

Most registers are 64-bit wide, even in AArch32 mode:

- MAIR_ELx is equivalent to MAIR0 and MAIR1.
- TTBR is 64 bit in both AArch64 and AArch32.

The only difference is the TCR register, which is 32 bit in AArch32 and
in EL3 in AArch64. For consistency with the rest of ELs in AArch64, it
makes sense to also have it as a 64-bit value.

Change-Id: I2274d66a28876702e7085df5f8aad0e7ec139da9
Signed-off-by: Antonio Nino Diaz <antonio.ninodiaz@arm.com>

xlat v2: Remove IMAGE_EL define ...

The Exception Level is now detected at runtime. This means that it is not
needed to hardcode the EL used by each image.

This doesn't result in a substantial increase of the image size because
the initialization functions that aren't used are garbage-collected by
the linker.

In AArch32 the current EL has been changed from EL3 to EL1 because the
the AArch32 PL1&0 translation regime behaves more like the AArch64 EL1&0
translation regime than the EL3 one.

Change-Id: I941404299ebe7666ca17619207c923b49a55cb73
Signed-off-by: Antonio Nino Diaz <antonio.ninodiaz@arm.com>

xlat v2: Remove unused tlbi helper ...

xlat_arch_tlbi_va_regime() isn't used, so it has been renamed to
xlat_arch_tlbi_va() and the previous implementation has been removed.

Change-Id: Ic118bed3fb68234748d86b2e9e95b25650289276
Signed-off-by: Antonio Nino Diaz <antonio.ninodiaz@arm.com>

xlat v2: Remove architectural headers ...

They only contained one function that is easily integrated in the private
library header and the existing architectural C files.

This also helps making the library more portable, as the Makefile of the
library now doesn't use the variable INCLUDES, which is specific to this
codebase and doesn't respect the namespace of the library.

Change-Id: I22228e6a97e9b4f346f5cd8947609263e8df71d8
Signed-off-by: Antonio Nino Diaz <antonio.ninodiaz@arm.com>

xlat v2: Split code into separate files ...

Instead of having one big file with all the code, it's better to have
a few smaller files that are more manageable:

- xlat_tables_core.c: Code related to the core functionality of the
  library (map and unmap regions, initialize xlat context).
- xlat_tables_context.c: Instantiation of the active image context
  as well as APIs to manipulate it.
- xlat_tables_utils.c: Helper code that isn't part of the core
  functionality (change attributes, debug print messages).

Change-Id: I3ea956fc1afd7473c0bb5e7c6aab3b2e5d88c711
Signed-off-by: Antonio Nino Diaz <antonio.ninodiaz@arm.com>

xlat v2: Split MMU setup and enable ...

At present, the function provided by the translation library to enable
MMU constructs appropriate values for translation library, and programs
them to the right registers. The construction of initial values,
however, is only required once as both the primary and secondaries
program the same values.

Additionally, the MMU-enabling function is written in C, which means
there's an active stack at the time of enabling MMU. On some systems,
like Arm DynamIQ, having active stack while enabling MMU during warm
boot might lead to coherency problems.

This patch addresses both the above problems by:

  - Splitting the MMU-enabling function into two: one that sets up
    values to be programmed into the registers, and another one that
    takes the pre-computed values and writes to the appropriate
    registers. With this, the primary effectively calls both functions
    to have the MMU enabled, but secondaries only need to call the
    latter.

  - Rewriting the function that enables MMU in assembly so that it
    doesn't use stack.

This patch fixes a bunch of MISRA issues on the way.

Change-Id: I0faca97263a970ffe765f0e731a1417e43fbfc45
Signed-off-by: Jeenu Viswambharan <jeenu.viswambharan@arm.com>

xlat v2: Introduce xlat granule size helpers ...

The function xlat_arch_is_granule_size_supported() can be used to check
if a specific granule size is supported. In Armv8, AArch32 only supports
4 KiB pages. AArch64 supports 4 KiB, 16 KiB or 64 KiB depending on the
implementation, which is detected at runtime.

The function xlat_arch_get_max_supported_granule_size() returns the max
granule size supported by the implementation.

Even though right now they are only used by SPM, they may be useful in
other places in the future. This patch moves the code currently in SPM
to the xlat tables lib so that it can be reused.

Change-Id: If54624a5ecf20b9b9b7f38861b56383a03bbc8a4
Signed-off-by: Antonio Nino Diaz <antonio.ninodiaz@arm.com>

Update ULL() macro and instances of ull to comply with MISRA ...

MISRA C-2012 Rule 7.3 violation: lowercase l shall not be used as literal suffixes.

This patch resolves this for the ULL() macro by using ULL suffix instead
of the ull suffix.

Change-Id: Ia8183c399e74677e676956e8653e82375d0e0a01
Signed-off-by: David Cunado <david.cunado@arm.com>

ARMv7 may not support large page addressing ...

ARCH_SUPPORTS_LARGE_PAGE_ADDRESSING allows build environment to
handle specific case when target ARMv7 core only supports 32bit MMU
descriptor mode.

If ARMv7 based platform does not set ARM_CORTEX_Ax=yes, platform
shall define ARMV7_SUPPORTS_LARGE_PAGE_ADDRESSING to enable
large page addressing support.

Signed-off-by: Etienne Carriere <etienne.carriere@linaro.org>

xlat: Introduce API to change memory attributes of a region ...

This patch introduces a new API in the translation tables library
(v2), that allows to change the memory attributes of a memory
region. It may be used to change its execution permissions and
data access permissions.

As a prerequisite, the memory must be already mapped. Moreover, it
must be mapped at the finest granularity (currently 4 KB).

Change-Id: I242a8c6f0f3ef2b0a81a61e28706540462faca3c
Co-authored-by: Sandrine Bailleux <sandrine.bailleux@arm.com>
Co-authored-by: Antonio Nino Diaz <antonio.ninodiaz@arm.com>
Signed-off-by: Antonio Nino Diaz <antonio.ninodiaz@arm.com>

xlat: Add support for EL0 and EL1 mappings ...

This patch introduces the ability of the xlat tables library to manage
EL0 and EL1 mappings from a higher exception level.

Attributes MT_USER and MT_PRIVILEGED have been added to allow the user
specify the target EL in the translation regime EL1&0.

REGISTER_XLAT_CONTEXT2 macro is introduced to allow creating a
xlat_ctx_t that targets a given translation regime (EL1&0 or EL3).

A new member is added to xlat_ctx_t to represent the translation regime
the xlat_ctx_t manages. The execute_never mask member is removed as it
is computed from existing information.

Change-Id: I95e14abc3371d7a6d6a358cc54c688aa9975c110
Co-authored-by: Douglas Raillard <douglas.raillard@arm.com>
Co-authored-by: Sandrine Bailleux <sandrine.bailleux@arm.com>
Co-authored-by: Antonio Nino Diaz <antonio.ninodiaz@arm.com>
Signed-off-by: Antonio Nino Diaz <antonio.ninodiaz@arm.com>

xlat: Introduce function xlat_arch_tlbi_va_regime() ...

Introduce a variant of the TLB invalidation helper function that
allows the targeted translation regime to be specified, rather than
defaulting to the current one.

This new function is useful in the context of EL3 software managing
translation tables for the S-EL1&0 translation regime, as then it
might need to invalidate S-EL1&0 TLB entries rather than EL3 ones.

Define a new enumeration to be able to represent translation regimes in
the xlat tables library.

Change-Id: Ibe4438dbea2d7a6e7470bfb68ff805d8bf6b07e5
Co-authored-by: Sandrine Bailleux <sandrine.bailleux@arm.com>
Co-authored-by: Douglas Raillard <douglas.raillard@arm.com>
Co-authored-by: Antonio Nino Diaz <antonio.ninodiaz@arm.com>
Signed-off-by: Antonio Nino Diaz <antonio.ninodiaz@arm.com>

Enable CnP bit for ARMv8.2 CPUs ...

This patch enables the CnP (Common not Private) bit for secure page
tables so that multiple PEs in the same Inner Shareable domain can use
the same translation table entries for a given stage of translation in
a particular translation regime. This only takes effect when ARM
Trusted Firmware is built with ARM_ARCH_MINOR >= 2.

ARM Trusted Firmware Design has been updated to include a description
of this feature usage.

Change-Id: I698305f047400119aa1900d34c65368022e410b8
Signed-off-by: Isla Mitchell <isla.mitchell@arm.com>

xlat lib: Fix some types ...

Fix the type length and signedness of some of the constants and
variables used in the translation table library.

This patch supersedes Pull Request #1018:
https://github.com/ARM-software/arm-trusted-firmware/pull/1018

Change-Id: Ibd45faf7a4fb428a0bf71c752551d35800212fb2
Signed-off-by: Sandrine Bailleux <sandrine.bailleux@arm.com>

xlat lib v2: Remove hard-coded virtual address space size ...

Previous patches have made it possible to specify the physical and
virtual address spaces sizes for each translation context. However,
there are still some places in the code where the physical (resp.
virtual) address space size is assumed to be PLAT_PHY_ADDR_SPACE_SIZE
(resp. PLAT_VIRT_ADDR_SPACE_SIZE).

This patch removes them and reads the relevant address space size
from the translation context itself instead. This information is now
passed in argument to the enable_mmu_arch() function, which needs it
to configure the TCR_ELx.T0SZ field (in AArch64) or the TTBCR.T0SZ
field (in AArch32) appropriately.

Change-Id: I20b0e68b03a143e998695d42911d9954328a06aa
Signed-off-by: Sandrine Bailleux <sandrine.bailleux@arm.com>

xlat lib v2: Refactor the functions enabling the MMU ...

This patch refactors both the AArch32 and AArch64 versions of the
function enable_mmu_arch().

In both versions, the code now computes the VMSA-related system
registers upfront then program them in one go (rather than interleaving
the 2).

In the AArch64 version, this allows to reduce the amount of code
generated by the C preprocessor and limits it to the actual differences
between EL1 and EL3.

In the AArch32 version, this patch also removes the function
enable_mmu_internal_secure() and moves its code directly inside
enable_mmu_arch(), as it was its only caller.

Change-Id: I35c09b6db4404916cbb2e2fd3fda2ad59f935954
Signed-off-by: Sandrine Bailleux <sandrine.bailleux@arm.com>

xlat lib v2: Remove init_xlat_tables_arch() function ...

In both the AArch32 and AArch64 versions, this function used to check
the sanity of the PLAT_PHY_ADDR_SPACE_SIZE in regard to the
architectural maximum value. Instead, export the
xlat_arch_get_max_supported_pa() function and move the debug
assertion in AArch-agnostic code.

The AArch64 used to also precalculate the TCR.PS field value, based
on the size of the physical address space. This is now done directly
by enable_mmu_arch(), which now receives the physical address space size
in argument.

Change-Id: Ie77ea92eb06db586f28784fdb479c6e27dd1acc1
Signed-off-by: Sandrine Bailleux <sandrine.bailleux@arm.com>

aarch32: Apply workaround for errata 813419 of Cortex-A57 ...

TLBI instructions for monitor mode won't have the desired effect under
specific circumstances in Cortex-A57 r0p0. The workaround is to
execute DSB and TLBI twice each time.

Even though this errata is only needed in r0p0, the current errata
framework is not prepared to apply run-time workarounds. The current one
is always applied if compiled in, regardless of the CPU or its revision.

The `DSB` instruction used when initializing the translation tables has
been changed to `DSB ISH` as an optimization and to be consistent with
the barriers used for the workaround.

NOTE: This workaround is present in AArch64 TF and already enabled by
default on Juno.

Change-Id: I10b0baa304ed64b13b7b26ea766e61461e759dfa
Signed-off-by: Dimitris Papastamos <dimitris.papastamos@arm.com>

Merge pull request #924 from antonio-nino-diaz-arm/an/fix-xn-bit ...

Fix execute-never permissions in xlat tables libs

Use SPDX license identifiers ...

To make software license auditing simpler, use SPDX[0] license
identifiers instead of duplicating the license text in every file.

NOTE: Files that have been imported by FreeBSD have not been modified.

[0]: https://spdx.org/

Change-Id: I80a00e1f641b8cc075ca5a95b10607ed9ed8761a
Signed-off-by: dp-arm <dimitris.papastamos@arm.com>

Fix execute-never permissions in xlat tables libs ...

Translation regimes that only support one virtual address space (such as
the ones for EL2 and EL3) can flag memory regions as execute-never by
setting to 1 the XN bit in the Upper Attributes field in the translation
tables descriptors. Translation regimes that support two different
virtual address spaces (such as the one shared by EL1 and EL0) use bits
PXN and UXN instead.

The Trusted Firmware runs at EL3 and EL1, it has to handle translation
tables of both translation regimes, but the previous code handled both
regimes the same way, as if both had only 1 VA range.

When trying to set a descriptor as execute-never it would set the XN
bit correctly in EL3, but it would set the XN bit in EL1 as well. XN is
at the same bit position as UXN, which means that EL0 was being
prevented from executing code at this region, not EL1 as the code
intended. Therefore, the PXN bit was unset to 0 all the time. The result
is that, in AArch64 mode, read-only data sections of BL2 weren't
protected from being executed.

This patch adds support of translation regimes with two virtual address
spaces to both versions of the translation tables library, fixing the
execute-never permissions for translation tables in EL1.

The library currently does not support initializing translation tables
for EL0 software, therefore it does not set/unset the UXN bit. If EL1
software needs to initialize translation tables for EL0 software, it
should use a different library instead.

Change-Id: If27588f9820ff42988851d90dc92801c8ecbe0c9
Signed-off-by: Antonio Nino Diaz <antonio.ninodiaz@arm.com>

Control inclusion of helper code used for asserts ...

Many asserts depend on code that is conditionally compiled based on the
DEBUG define. This patch modifies the conditional inclusion of such code
so that it is based on the ENABLE_ASSERTIONS build option.

Change-Id: I6406674788aa7e1ad7c23d86ce94482ad3c382bd
Signed-off-by: Antonio Nino Diaz <antonio.ninodiaz@arm.com>

Add support to change xlat_tables to non-cacheable ...

This patch adds an additional flag `XLAT_TABLE_NC` which marks the
translation tables as Non-cacheable for MMU accesses.

Change-Id: I7c28ab87f0ce67da237fadc3627beb6792860fd4
Signed-off-by: Summer Qin <summer.qin@arm.com>