cpus: higher performance non-cacheable load forwarding ...

The CPUACTLR_EL1 register on Cortex-A57 CPUs supports a bit to enable
non-cacheable streaming enhancement. Platforms can set this bit only
if their memory system meets the requirement that cache line fill
requests from the Cortex-A57 processor are atomic.

This patch adds support to enable higher performance non-cacheable load
forwarding for such platforms. Platforms must enable this support by
setting the 'A57_ENABLE_NONCACHEABLE_LOAD_FWD' flag from their
makefiles. This flag is disabled by default.

Change-Id: Ib27e55dd68d11a50962c0bbc5b89072208b4bac5
Signed-off-by: Varun Wadekar <vwadekar@nvidia.com>

Merge "Update docs with PMU security information" into integration

Update docs with PMU security information ...

This patch adds information on the PMU configuration registers
and security considerations related to the PMU.

Signed-off-by: Petre-Ionut Tudor <petre-ionut.tudor@arm.com>
Change-Id: I36b15060b9830a77d3f47f293c0a6dafa3c581fb

Adds option to read ROTPK from registers for FVP ...

Enables usage of ARM_ROTPK_LOCATION=regs for FVP board.
Removes hard-coded developer keys. Instead, setting
ARM_ROTPK_LOCATION=devel_* takes keys from default directory.
In case of ROT_KEY specified - generates a new hash and replaces the
original.

Note: Juno board was tested by original feature author and was not tested
for this patch since we don't have access to the private key. Juno
implementation was moved to board-specific file without changing
functionality. It is not known whether byte-swapping is still needed
for this platform.

Change-Id: I0fdbaca0415cdcd78f3a388551c2e478c01ed986
Signed-off-by: Max Shvetsov <maksims.svecovs@arm.com>

Neovers N1: added support to update presence of External LLC ...

CPUECTLR_EL1.EXTLLC bit indicates the presense of internal or external
last level cache(LLC) in the system, the reset value is internal LLC.

To cater for the platforms(like N1SDP) which has external LLC present
introduce a new build option 'NEOVERSE_N1_EXTERNAL_LLC' which can be
enabled by platform port.

Signed-off-by: Manish Pandey <manish.pandey2@arm.com>
Change-Id: Ibf475fcd6fd44401897a71600f4eafe989921363

Merge "bl31: Split into two separate memory regions" into integration

bl31: Split into two separate memory regions ...

Some platforms are extremely memory constrained and must split BL31
between multiple non-contiguous areas in SRAM. Allow the NOBITS
sections (.bss, stacks, page tables, and coherent memory) to be placed
in a separate region of RAM from the loaded firmware image.

Because the NOBITS region may be at a lower address than the rest of
BL31, __RW_{START,END}__ and __BL31_{START,END}__ cannot include this
region, or el3_entrypoint_common would attempt to invalidate the dcache
for the entire address space. New symbols __NOBITS_{START,END}__ are
added when SEPARATE_NOBITS_REGION is enabled, and the dcached for the
NOBITS region is invalidated separately.

Signed-off-by: Samuel Holland <samuel@sholland.org>
Change-Id: Idedfec5e4dbee77e94f2fdd356e6ae6f4dc79d37

Workaround for Hercules erratum 1688305 ...

Erratum 1688305 is a Cat B erratum present in r0p0, r0p1 versions
of Hercules core. The erratum can be avoided by setting bit 1 of the
implementation defined register CPUACTLR2_EL1 to 1 to prevent store-
release from being dispatched before it is the oldest.

Change-Id: I2ac04f5d9423868b6cdd4ceb3d0ffa46e570efed
Signed-off-by: Madhukar Pappireddy <madhukar.pappireddy@arm.com>

Merge "Changes to support updated register usage in SMCCC v1.2" into integration

doc: Split the User Guide into multiple files ...

The User Guide document has grown organically over time and
now covers a wide range of topics, making it difficult to
skim read and extract information from. Currently, it covers
these topics and maybe a couple more:

- Requirements (hardware, tools, libs)
- Checking out the repo
- Basic build instructions
- A comprehensive list of build flags
- FIP packaging
- Building specifically for Juno
- Firmware update images
- EL3 payloads
- Preloaded BL33 boot flow
- Running on FVPs
- Running on Juno

I have separated these out into a few groups that become new
documents. Broadly speaking, build instructions for the tools,
for TF-A generally, and for specific scenarios are separated.
Content relating to specific platforms (Juno and the FVPs are
Arm-specific platforms, essentially) has been moved into the
documentation that is specific to those platforms, under
docs/plat/arm.

Change-Id: Ica87c52d8cd4f577332be0b0738998ea3ba3bbec
Signed-off-by: Paul Beesley <paul.beesley@arm.com>

Changes to support updated register usage in SMCCC v1.2 ...

From AArch64 state, arguments are passed in registers W0-W7(X0-X7)
and results are returned in W0-W7(X0-X7) for SMC32(SMC64) calls.
From AArch32 state, arguments are passed in registers R0-R7 and
results are returned in registers R0-R7 for SMC32 calls.

Most of the functions and macros already existed to support using
upto 8 registers for passing/returning parameters/results. Added
few helper macros for SMC calls from AArch32 state.

Link to the specification:
https://developer.arm.com/docs/den0028/c

Change-Id: I87976b42454dc3fc45c8343e9640aa78210e9741
Signed-off-by: Madhukar Pappireddy <madhukar.pappireddy@arm.com>

doc: Misc syntax and spelling fixes ...

Tidying up a few Sphinx warnings that had built-up over time.
None of these are critical but it cleans up the Sphinx output.

At the same time, fixing some spelling errors that were detected.

Change-Id: I38209e235481eed287f8008c6de9dedd6b12ab2e
Signed-off-by: Paul Beesley <paul.beesley@arm.com>

doc: Convert internal links to RST format ...

Currently links between documents are using the format:

<path/to/><filename>.rst

This was required for services like GitHub because they render each
document in isolation - linking to another document is like linking
to any other file, just provide the full path.

However, with the new approach, the .rst files are only the raw
source for the documents. Once the documents have been rendered
the output is now in another format (HTML in our case) and so,
when linking to another document, the link must point to the
rendered version and not the .rst file.

The RST spec provides a few methods for linking between content.
The parent of this patch enabled the automatic creation of anchors
for document titles - we will use these anchors as the targets for
our links. Additional anchors can be added by hand if needed, on
section and sub-section titles, for example.

An example of this new format, for a document with the title
"Firmware Design" is :ref:`Firmware Design`.

One big advantage of this is that anchors are not dependent on
paths. We can then move documents around, even between directories,
without breaking any links between documents. Links will need to be
updated only if the title of a document changes.

Change-Id: I9e2340a61dd424cbd8fd1ecc2dc166f460d81703
Signed-off-by: Paul Beesley <paul.beesley@arm.com>

Merge "Neoverse N1 Errata Workaround 1542419" into integration

Neoverse N1 Errata Workaround 1542419 ...

Coherent I-cache is causing a prefetch violation where when the core
executes an instruction that has recently been modified, the core might
fetch a stale instruction which violates the ordering of instruction
fetches.

The workaround includes an instruction sequence to implementation
defined registers to trap all EL0 IC IVAU instructions to EL3 and a trap
handler to execute a TLB inner-shareable invalidation to an arbitrary
address followed by a DSB.

Signed-off-by: Lauren Wehrmeister <lauren.wehrmeister@arm.com>
Change-Id: Ic3b7cbb11cf2eaf9005523ef5578a372593ae4d6

Fix the CAS spinlock implementation ...

Make the spinlock implementation use ARMv8.1-LSE CAS instruction based
on a platform build option. The CAS-based implementation used to be
unconditionally selected for all ARM8.1+ platforms.

The previous CAS spinlock implementation had a bug wherein the spin_unlock()
implementation had an `sev` after `stlr` which is not sufficient. A dsb is
needed to ensure that the stlr completes prior to the sev. Having a dsb is
heavyweight and a better solution would be to use load exclusive semantics
to monitor the lock and wake up from wfe when a store happens to the lock.
The patch implements the same.

Change-Id: I5283ce4a889376e4cc01d1b9d09afa8229a2e522
Signed-off-by: Soby Mathew <soby.mathew@arm.com>
Signed-off-by: Olivier Deprez <olivier.deprez@arm.com>

Add documentation for new KEY_SIZE option ...

This patch adds documentation for the new KEY_SIZE build option that is
exposed by cert_create, and instructions on how to use it.

Change-Id: I09b9b052bfdeeaca837e0f0026e2b01144f2472c
Signed-off-by: Justin Chadwell <justin.chadwell@arm.com>

Add documentation for CTX_INCLUDE_MTE_REGS ...

A new build flag, CTX_INCLUDE_MTE_REGS, has been added; this patch adds
documentation for it in the User Guide along with instructions of what
different values mean.

Change-Id: I430a9c6ced06b1b6be317edbeff4f5530e30f63a
Signed-off-by: Justin Chadwell <justin.chadwell@arm.com>

Merge changes from topic "lw/n1_errata_fixes" into integration ...

* changes:
  Removing redundant ISB instructions
  Workaround for Neoverse N1 erratum 1275112
  Workaround for Neoverse N1 erratum 1262888
  Workaround for Neoverse N1 erratum 1262606
  Workaround for Neoverse N1 erratum 1257314
  Workaround for Neoverse N1 erratum 1220197
  Workaround for Neoverse N1 erratum 1207823
  Workaround for Neoverse N1 erratum 1165347
  Workaround for Neoverse N1 erratum 1130799
  Workaround for Neoverse N1 erratum 1073348

Workaround for Neoverse N1 erratum 1275112 ...

Neoverse N1 erratum 1275112 is a Cat B erratum [1],
present in older revisions of the Neoverse N1 processor core.
The workaround is to set a bit in the implementation defined
CPUACTLR_EL1 system register, which delays instruction fetch after
branch misprediction.

[1] http://infocenter.arm.com/help/index.jsp?topic=/com.arm.doc.pjdoc-466751330-10325/index.html

Change-Id: If7fe55fe92e656fa6aea12327ab297f2e6119833
Signed-off-by: Lauren Wehrmeister <lauren.wehrmeister@arm.com>

Workaround for Neoverse N1 erratum 1262888 ...

Neoverse N1 erratum 1262888 is a Cat B erratum [1],
present in older revisions of the Neoverse N1 processor core.
The workaround is to set a bit in the implementation defined
CPUECTLR_EL1 system register, which disables the MMU hardware prefetcher.

[1] http://infocenter.arm.com/help/index.jsp?topic=/com.arm.doc.pjdoc-466751330-10325/index.html

Change-Id: Ib733d748e32a7ea6a2783f3d5a9c5e13eee01105
Signed-off-by: Lauren Wehrmeister <lauren.wehrmeister@arm.com>

Workaround for Neoverse N1 erratum 1262606 ...

Neoverse N1 erratum 1262606 is a Cat B erratum [1],
present in older revisions of the Neoverse N1 processor core.
The workaround is to set a bit in the implementation defined
CPUACTLR_EL1 system register, which delays instruction fetch after
branch misprediction.

[1] http://infocenter.arm.com/help/index.jsp?topic=/com.arm.doc.pjdoc-466751330-10325/index.html

Change-Id: Idd980e9d5310232d38f0ce272862e1fb0f02ce9a
Signed-off-by: Lauren Wehrmeister <lauren.wehrmeister@arm.com>

Workaround for Neoverse N1 erratum 1257314 ...

Neoverse N1 erratum 1257314 is a Cat B erratum [1],
present in older revisions of the Neoverse N1 processor core.
The workaround is to set a bit in the implementation defined
CPUACTLR3_EL1 system register, which prevents parallel
execution of divide and square root instructions.

[1] http://infocenter.arm.com/help/index.jsp?topic=/com.arm.doc.pjdoc-466751330-10325/index.html

Change-Id: I54f0f40ff9043efee40d51e796b92ed85b394cbb
Signed-off-by: Lauren Wehrmeister <lauren.wehrmeister@arm.com>

Workaround for Neoverse N1 erratum 1220197 ...

Neoverse N1 erratum 1220197 is a Cat B erratum [1],
present in older revisions of the Neoverse N1 processor core.
The workaround is to set two bits in the implementation defined
CPUECTLR_EL1 system register, which disables write streaming to the L2.

[1] http://infocenter.arm.com/help/index.jsp?topic=/com.arm.doc.pjdoc-466751330-10325/index.html

Change-Id: I9c3373f1b6d67d21ee71b2b80aec5e96826818e8
Signed-off-by: Lauren Wehrmeister <lauren.wehrmeister@arm.com>

Workaround for Neoverse N1 erratum 1207823 ...

Neoverse N1 erratum 1207823 is a Cat B erratum [1],
present in older revisions of the Neoverse N1 processor core.
The workaround is to set a bit in the implementation defined
CPUACTLR2_EL1 system register.

[1] http://infocenter.arm.com/help/index.jsp?topic=/com.arm.doc.pjdoc-466751330-10325/index.html

Change-Id: Ia932337821f1ef0d644db3612480462a8d924d21
Signed-off-by: Lauren Wehrmeister <lauren.wehrmeister@arm.com>

Workaround for Neoverse N1 erratum 1165347 ...

Neoverse N1 erratum 1165347 is a Cat B erratum [1],
present in older revisions of the Neoverse N1 processor core.
The workaround is to set two bits in the implementation defined
CPUACTLR2_EL1 system register.

[1] http://infocenter.arm.com/help/index.jsp?topic=/com.arm.doc.pjdoc-466751330-10325/index.html

Change-Id: I163d0ea00578245c1323d2340314cdc3088c450d
Signed-off-by: Lauren Wehrmeister <lauren.wehrmeister@arm.com>

Workaround for Neoverse N1 erratum 1130799 ...

Neoverse N1 erratum 1130799 is a Cat B erratum [1],
present in older revisions of the Neoverse N1 processor core.
The workaround is to set a bit in the implementation defined
CPUACTLR2_EL1 system register.

[1] http://infocenter.arm.com/help/index.jsp?topic=/com.arm.doc.pjdoc-466751330-10325/index.html

Change-Id: I252bc45f9733443ba0503fefe62f50fdea61da6d
Signed-off-by: Lauren Wehrmeister <lauren.wehrmeister@arm.com>

Workaround for Neoverse N1 erratum 1073348 ...

Neoverse N1 erratum 1073348 is a Cat B erratum [1],
present in older revisions of the Neoverse N1 processor core.
The workaround is to set a bit in the implementation defined
CPUACTLR_EL1 system register, which disables static prediction.

[1] http://infocenter.arm.com/help/index.jsp?topic=/com.arm.doc.pjdoc-466751330-10325/index.html

Change-Id: I674126c0af6e068eecb379a190bcf7c75dcbca8e
Signed-off-by: Lauren Wehrmeister <lauren.wehrmeister@arm.com>

doc: Fix typo in file interrupt-framework-design.rst ...

Signed-off-by: Peng Donglin <dolinux.peng@gmail.com>
Change-Id: I459e7d056735222f6f34e275dbdaf9a389d193fc

Neoverse N1: Introduce workaround for Neoverse N1 erratum 1315703 ...

Neoverse N1 erratum 1315703 is a Cat A (rare) erratum [1], present in
older revisions of the Neoverse N1 processor core.
The workaround is to set a bit in the implementation defined CPUACTLR2_EL1
system register, which will disable the load-bypass-store feature.

[1] http://infocenter.arm.com/help/index.jsp?topic=/com.arm.doc.pjdocpjdoc-466751330-1032/index.html

Change-Id: I5c708dbe0efa4daa0bcb6bd9622c5efe19c03af9
Signed-off-by: Andre Przywara <andre.przywara@arm.com>