cpus: denver: disable cycle counter when event counting is prohibited ...

The Denver CPUs implement support for PMUv3 for ARMv8.1 and expect the
PMCR_EL0 to be saved in non-secure context.

This patch disables cycle counter when event counting is prohibited
immediately on entering the secure world to avoid leaking useful
information about the PMU counters. The context saving code later
saves the value of PMCR_EL0 to the non-secure world context.

Verified with 'PMU Leakage' test suite.

 ******************************* Summary *******************************
 > Test suite 'PMU Leakage'
                                                                 Passed
 =================================
 Tests Skipped : 2
 Tests Passed  : 2
 Tests Failed  : 0
 Tests Crashed : 0
 Total tests   : 4
 =================================

Signed-off-by: Varun Wadekar <vwadekar@nvidia.com>
Change-Id: I3675e2b99b44ed23d86e29a5af1b496e80324875

cpus: denver: fixup register used to store return address ...

The denver_enable_dco and denver_disable_dco use register X3 to store
the return address. But X3 gets over-written by other functions,
downstream.

This patch stores the return address to X18 instead, to fix this
anomaly.

Change-Id: Ic40bfc1d9abaa7b90348843b9ecd09521bb4ee7b
Signed-off-by: Kalyani Chidambaram <kalyanic@nvidia.com>

cpus: denver: Implement static workaround for CVE-2018-3639 ...

For Denver CPUs, this approach enables the mitigation during EL3
initialization, following every PE reset. No mechanism is provided to
disable the mitigation at runtime.

This approach permanently mitigates the EL3 software stack only. Other
software components are responsible to enable it for their exception
levels.

TF-A implements this approach for the Denver CPUs with DENVER_MIDR_PN3
and earlier:

*   By setting bit 11 (Disable speculative store buffering) of
    `ACTLR_EL3`

*   By setting bit 9 (Disable speculative memory disambiguation) of
    `ACTLR_EL3`

TF-A implements this approach for the Denver CPUs with DENVER_MIDR_PN4
and later:

*   By setting bit 18 (Disable speculative store buffering) of
    `ACTLR_EL3`

*   By setting bit 17 (Disable speculative memory disambiguation) of
    `ACTLR_EL3`

Change-Id: If1de96605ce3f7b0aff5fab2c828e5aecb687555
Signed-off-by: Varun Wadekar <vwadekar@nvidia.com>

cpus: denver: reset power state to 'C1' on boot ...

Denver CPUs expect the power state field to be reset to 'C1'
during boot. This patch updates the reset handler to reset the
ACTLR_.PMSTATE field to 'C1' state during CPU boot.

Change-Id: I7cb629627a4dd1a30ec5cbb3a5e90055244fe30c
Signed-off-by: Varun Wadekar <vwadekar@nvidia.com>

denver: use plat_my_core_pos() to get core position ...

The current functions to disable and enable Dynamic Code Optimizer
(DCO) assume that all denver cores are in the same cluster. They
ignore AFF1 field of the mpidr_el1 register, which leads to
incorect logical core id calculation.

This patch calls the platform handler, plat_my_core_pos(), to get
the logical core id to disable/enable DCO for the core.

Original change by: Krishna Sitaraman <ksitaraman@nvidia.com>

Change-Id: I45fbd1f1eb032cc1db677a4fdecc554548b4a830
Signed-off-by: Varun Wadekar <vwadekar@nvidia.com>

cpus: denver: report CVE_2017_5715 mitigation to higher layers ...

This patch uses the 'declare_cpu_ops_wa' macro, to set the check function,
to report that Denver cores are mitigated.

Denver cores are vulnerable to this anomaly and require the mitigation to
be enabled always.

Change-Id: I1bb6eefdec8c01fb8b645e112f8d04d4bb8811ef
Signed-off-by: Varun Wadekar <vwadekar@nvidia.com>

Add end_vector_entry assembler macro ...

Check_vector_size checks if the size of the vector fits
in the size reserved for it. This check creates problems in
the Clang assembler. A new macro, end_vector_entry, is added
and check_vector_size is deprecated.

This new macro fills the current exception vector until the next
exception vector. If the size of the current vector is bigger
than 32 instructions then it gives an error.

Change-Id: Ie8545cf1003a1e31656a1018dd6b4c28a4eaf671
Signed-off-by: Roberto Vargas <roberto.vargas@arm.com>

Workaround for CVE-2017-5715 on NVIDIA Denver CPUs ...

Flush the indirect branch predictor and RSB on entry to EL3 by issuing
a newly added instruction for Denver CPUs. Support for this operation
can be determined by comparing bits 19:16 of ID_AFR0_EL1 with 0b0001.

To achieve this without performing any branch instruction, a per-cpu
vbar is installed which executes the workaround and then branches off
to the corresponding vector entry in the main vector table. A side
effect of this change is that the main vbar is configured before any
reset handling. This is to allow the per-cpu reset function to override
the vbar setting.

Change-Id: Ief493cd85935bab3cfee0397e856db5101bc8011
Signed-off-by: Varun Wadekar <vwadekar@nvidia.com>

Use SPDX license identifiers ...

To make software license auditing simpler, use SPDX[0] license
identifiers instead of duplicating the license text in every file.

NOTE: Files that have been imported by FreeBSD have not been modified.

[0]: https://spdx.org/

Change-Id: I80a00e1f641b8cc075ca5a95b10607ed9ed8761a
Signed-off-by: dp-arm <dimitris.papastamos@arm.com>

cpus: denver: remove barrier from denver_enable_dco() ...

This patch removes unnecessary `isb` from the enable DCO sequence as
there is no need to synchronize this operation.

Change-Id: I0191e684bbc7fdba635c3afbc4e4ecd793b6f06f
Signed-off-by: Varun Wadekar <vwadekar@nvidia.com>

cpus: denver: disable DCO operations from platform code ...

This patch moves the code to disable DCO operations out from common
CPU files. This allows the platform code to call thsi API as and
when required. There are certain CPU power down states which require
the DCO to be kept ON and platforms can decide selectively now.

Change-Id: Icb946fe2545a7d8c5903c420d1ee169c4921a2d1
Signed-off-by: Varun Wadekar <vwadekar@nvidia.com>

cpus: Add support for all Denver variants ...

This patch adds support for all variants of the Denver CPUs. The
variants export their cpu_ops to allow all Denver platforms to run
the Trusted Firmware stack.

Change-Id: I1488813ddfd506ffe363d8a32cda1b575e437035
Signed-off-by: Varun Wadekar <vwadekar@nvidia.com>

Add provision to extend CPU operations at more levels ...

Various CPU drivers in ARM Trusted Firmware register functions to handle
power-down operations. At present, separate functions are registered to
power down individual cores and clusters.

This scheme operates on the basis of core and cluster, and doesn't cater
for extending the hierarchy for power-down operations. For example,
future CPUs might support multiple threads which might need powering
down individually.

This patch therefore reworks the CPU operations framework to allow for
registering power down handlers on specific level basis. Henceforth:

  - Generic code invokes CPU power down operations by the level
    required.

  - CPU drivers explicitly mention CPU_NO_RESET_FUNC when the CPU has no
    reset function.

  - CPU drivers register power down handlers as a list: a mandatory
    handler for level 0, and optional handlers for higher levels.

All existing CPU drivers are adapted to the new CPU operations framework
without needing any functional changes within.

Also update firmware design guide.

Change-Id: I1826842d37a9e60a9e85fdcee7b4b8f6bc1ad043
Signed-off-by: Jeenu Viswambharan <jeenu.viswambharan@arm.com>

Add "Project Denver" CPU support ...

Denver is NVIDIA's own custom-designed, 64-bit, dual-core CPU which is
fully ARMv8 architecture compatible.  Each of the two Denver cores
implements a 7-way superscalar microarchitecture (up to 7 concurrent
micro-ops can be executed per clock), and includes a 128KB 4-way L1
instruction cache, a 64KB 4-way L1 data cache, and a 2MB 16-way L2
cache, which services both cores.

Denver implements an innovative process called Dynamic Code Optimization,
which optimizes frequently used software routines at runtime into dense,
highly tuned microcode-equivalent routines. These are stored in a
dedicated, 128MB main-memory-based optimization cache. After being read
into the instruction cache, the optimized micro-ops are executed,
re-fetched and executed from the instruction cache as long as needed and
capacity allows.

Effectively, this reduces the need to re-optimize the software routines.
Instead of using hardware to extract the instruction-level parallelism
(ILP) inherent in the code, Denver extracts the ILP once via software
techniques, and then executes those routines repeatedly, thus amortizing
the cost of ILP extraction over the many execution instances.

Denver also features new low latency power-state transitions, in addition
to extensive power-gating and dynamic voltage and clock scaling based on
workloads.

Signed-off-by: Varun Wadekar <vwadekar@nvidia.com>