PSCI: Implement platform compatibility layer ...

The new PSCI topology framework and PSCI extended State framework introduces
a breaking change in the platform port APIs. To ease the migration of the
platform ports to the new porting interface, a compatibility layer is
introduced which essentially defines the new platform API in terms of the
old API. The old PSCI helpers to retrieve the power-state, its associated
fields and the highest coordinated physical OFF affinity level of a core
are also implemented for compatibility. This allows the existing
platform ports to work with the new PSCI framework without significant
rework. This layer will be enabled by default once the switch to the new
PSCI framework is done and is controlled by the build flag ENABLE_PLAT_COMPAT.

Change-Id: I4b17cac3a4f3375910a36dba6b03d8f1700d07e3

PSCI: Unify warm reset entry points ...

There used to be 2 warm reset entry points:

 - the "on finisher", for when the core has been turned on using a
   PSCI CPU_ON call;

 - the "suspend finisher", entered upon resumption from a previous
   PSCI CPU_SUSPEND call.

The appropriate warm reset entry point used to be programmed into the
mailboxes by the power management hooks.

However, it is not required to provide this information to the PSCI
entry point code, as it can figure it out by itself. By querying affinity
info state, a core is able to determine on which execution path it is.
If the state is ON_PENDING then it means it's been turned on else
it is resuming from suspend.

This patch unifies the 2 warm reset entry points into a single one:
psci_entrypoint(). The patch also implements the necessary logic
to distinguish between the 2 types of warm resets in the power up
finisher.

The plat_setup_psci_ops() API now takes the
secure entry point as an additional parameter to enable the platforms
to configure their mailbox. The platform hooks `pwr_domain_on`
and `pwr_domain_suspend` no longer take secure entry point as
a parameter.

Change-Id: I7d1c93787b54213aefdbc046b8cd66a555dfbfd9

PSCI: Add framework to handle composite power states ...

The state-id field in the power-state parameter of a CPU_SUSPEND call can be
used to describe composite power states specific to a platform. The current PSCI
implementation does not interpret the state-id field. It relies on the target
power level and the state type fields in the power-state parameter to perform
state coordination and power management operations. The framework introduced
in this patch allows the PSCI implementation to intepret generic global states
like RUN, RETENTION or OFF from the State-ID to make global state coordination
decisions and reduce the complexity of platform ports. It adds support to
involve the platform in state coordination which facilitates the use of
composite power states and improves the support for entering standby states
at multiple power domains.

The patch also includes support for extended state-id format for the power
state parameter as specified by PSCIv1.0.

The PSCI implementation now defines a generic representation of the power-state
parameter. It depends on the platform port to convert the power-state parameter
(possibly encoding a composite power state) passed in a CPU_SUSPEND call to this
representation via the `validate_power_state()` plat_psci_ops handler. It is an
array where each index corresponds to a power level. Each entry contains the
local power state the power domain at that power level could enter.

The meaning of the local power state values is platform defined, and may vary
between levels in a single platform. The PSCI implementation constrains the
values only so that it can classify the state as RUN, RETENTION or OFF as
required by the specification:
   * zero means RUN
   * all OFF state values at all levels must be higher than all RETENTION
     state values at all levels
   * the platform provides PLAT_MAX_RET_STATE and PLAT_MAX_OFF_STATE values
     to the framework

The platform also must define the macros PLAT_MAX_RET_STATE and
PLAT_MAX_OFF_STATE which lets the PSCI implementation find out which power
domains have been requested to enter a retention or power down state. The PSCI
implementation does not interpret the local power states defined by the
platform. The only constraint is that the PLAT_MAX_RET_STATE <
PLAT_MAX_OFF_STATE.

For a power domain tree, the generic implementation maintains an array of local
power states. These are the states requested for each power domain by all the
cores contained within the domain. During a request to place multiple power
domains in a low power state, the platform is passed an array of requested
power-states for each power domain through the plat_get_target_pwr_state()
API. It coordinates amongst these states to determine a target local power
state for the power domain. A default weak implementation of this API is
provided in the platform layer which returns the minimum of the requested
power-states back to the PSCI state coordination.

Finally, the plat_psci_ops power management handlers are passed the target
local power states for each affected power domain using the generic
representation described above. The platform executes operations specific to
these target states.

The platform power management handler for placing a power domain in a standby
state (plat_pm_ops_t.pwr_domain_standby()) is now only used as a fast path for
placing a core power domain into a standby or retention state should now be
used to only place the core power domain in a standby or retention state.

The extended state-id power state format can be enabled by setting the
build flag PSCI_EXTENDED_STATE_ID=1 and it is disabled by default.

Change-Id: I9d4123d97e179529802c1f589baaa4101759d80c

PSCI: Introduce new platform interface to describe topology ...

This patch removes the assumption in the current PSCI implementation that MPIDR
based affinity levels map directly to levels in a power domain tree. This
enables PSCI generic code to support complex power domain topologies as
envisaged by PSCIv1.0 specification. The platform interface for querying
the power domain topology has been changed such that:

1. The generic PSCI code does not generate MPIDRs and use them to query the
   platform about the number of power domains at a particular power level. The
   platform now provides a description of the power domain tree on the SoC
   through a data structure. The existing platform APIs to provide the same
   information have been removed.

2. The linear indices returned by plat_core_pos_by_mpidr() and
   plat_my_core_pos() are used to retrieve core power domain nodes from the
   power domain tree. Power domains above the core level are accessed using a
   'parent' field in the tree node descriptors.

The platform describes the power domain tree in an array of 'unsigned
char's. The first entry in the array specifies the number of power domains at
the highest power level implemented in the system. Each susbsequent entry
corresponds to a power domain and contains the number of power domains that are
its direct children. This array is exported to the generic PSCI implementation
via the new `plat_get_power_domain_tree_desc()` platform API.

The PSCI generic code uses this array to populate its internal power domain tree
using the Breadth First Search like algorithm. The tree is split into two
arrays:

1. An array that contains all the core power domain nodes

2. An array that contains all the other power domain nodes

A separate array for core nodes allows certain core specific optimisations to
be implemented e.g. remove the bakery lock, re-use per-cpu data framework for
storing some information.

Entries in the core power domain array are allocated such that the
array index of the domain is equal to the linear index returned by
plat_core_pos_by_mpidr() and plat_my_core_pos() for the MPIDR
corresponding to that domain. This relationship is key to be able to use
an MPIDR to find the corresponding core power domain node, traverse to higher
power domain nodes and index into arrays that contain core specific
information.

An introductory document has been added to briefly describe the new interface.

Change-Id: I4b444719e8e927ba391cae48a23558308447da13

PSCI: Introduce new platform and CM helper APIs ...

This patch introduces new platform APIs and context management helper APIs
to support the new topology framework based on linear core position. This
framework will be introduced in the follwoing patch and it removes the
assumption that the MPIDR based affinity levels map directly to levels
in a power domain tree. The new platforms APIs and context management
helpers based on core position are as described below:

* plat_my_core_pos() and plat_core_pos_by_mpidr()

These 2 new mandatory platform APIs are meant to replace the existing
'platform_get_core_pos()' API. The 'plat_my_core_pos()' API returns the
linear index of the calling core and 'plat_core_pos_by_mpidr()' returns
the linear index of a core specified by its MPIDR. The latter API will also
validate the MPIDR passed as an argument and will return an error code (-1)
if an invalid MPIDR is passed as the argument. This enables the caller to
safely convert an MPIDR of another core to its linear index without querying
the PSCI topology tree e.g. during a call to PSCI CPU_ON.

Since the 'plat_core_pos_by_mpidr()' API verifies an MPIDR, which is always
platform specific, it is no longer possible to maintain a default implementation
of this API. Also it might not be possible for a platform port to verify an
MPIDR before the C runtime has been setup or the topology has been initialized.
This would prevent 'plat_core_pos_by_mpidr()' from being callable prior to
topology setup. As a result, the generic Trusted Firmware code does not call
this API before the topology setup has been done.

The 'plat_my_core_pos' API should be able to run without a C runtime.
Since this API needs to return a core position which is equal to the one
returned by 'plat_core_pos_by_mpidr()' API for the corresponding MPIDR,
this too cannot have default implementation and is a mandatory API for
platform ports. These APIs will be implemented by the ARM reference platform
ports later in the patch stack.

* plat_get_my_stack() and plat_set_my_stack()

These APIs are the stack management APIs which set/return stack addresses
appropriate for the calling core. These replace the 'platform_get_stack()' and
'platform_set_stack()' APIs. A default weak MP version and a global UP version
of these APIs are provided for the platforms.

* Context management helpers based on linear core position

A set of new context management(CM) helpers viz cm_get_context_by_index(),
cm_set_context_by_index(), cm_init_my_context() and cm_init_context_by_index()
are defined which are meant to replace the old helpers which took MPIDR
as argument. The old CM helpers are implemented based on the new helpers to
allow for code consolidation and will be deprecated once the switch to the new
framework is done.

Change-Id: I89758632b370c2812973a4b2efdd9b81a41f9b69

PSCI: Remove references to affinity based power management ...

As per Section 4.2.2. in the PSCI specification, the term "affinity"
is used in the context of describing the hierarchical arrangement
of cores. This often, but not always, maps directly to the processor
power domain topology of the system. The current PSCI implementation
assumes that this is always the case i.e. MPIDR based levels of
affinity always map to levels in a power domain topology tree.

This patch is the first in a series of patches which remove this
assumption. It removes all occurences of the terms "affinity
instances and levels" when used to describe the power domain
topology. Only the terminology is changed in this patch. Subsequent
patches will implement functional changes to remove the above
mentioned assumption.

Change-Id: Iee162f051b228828310610c5a320ff9d31009b4e

PSCI: Invoke PM hooks only for the highest level ...

This patch optimizes the invocation of the platform power management hooks for
ON, OFF and SUSPEND such that they are called only for the highest affinity
level which will be powered off/on. Earlier, the hooks were being invoked for
all the intermediate levels as well.

This patch requires that the platforms migrate to the new semantics of the PM
hooks.  It also removes the `state` parameter from the pm hooks as the `afflvl`
parameter now indicates the highest affinity level for which power management
operations are required.

Change-Id: I57c87931d8a2723aeade14acc710e5b78ac41732

PSCI: Create new directory to implement new frameworks ...

This patch creates a copy of the existing PSCI files and related psci.h and
platform.h header files in a new `PSCI1.0` directory. The changes for the
new PSCI power domain topology and extended state-ID frameworks will be
added incrementally to these files. This incremental approach will
aid in review and in understanding the changes better. Once all the
changes have been introduced, these files will replace the existing PSCI
files.

Change-Id: Ibb8a52e265daa4204e34829ed050bddd7e3316ff

PSCI: Add SYSTEM_SUSPEND API support ...

This patch adds support for SYSTEM_SUSPEND API as mentioned in the PSCI 1.0
specification. This API, on being invoked on the last running core on a
supported platform, will put the system into a low power mode with memory
retention.

The psci_afflvl_suspend() internal API has been reused as most of the actions
to suspend a system are the same as invoking the PSCI CPU_SUSPEND API with the
target affinity level as 'system'. This API needs the 'power state' parameter
for the target low power state. This parameter is not passed by the caller of
the SYSTEM_SUSPEND API. Hence, the platform needs to implement the
get_sys_suspend_power_state() platform function to provide this information.
Also, the platform also needs to add support for suspending the system to the
existing 'plat_pm_ops' functions: affinst_suspend() and
affinst_suspend_finish().

Change-Id: Ib6bf10809cb4e9b92f463755608889aedd83cef5

Initialise cpu ops after enabling data cache ...

The cpu-ops pointer was initialized before enabling the data cache in the cold
and warm boot paths. This required a DCIVAC cache maintenance operation to
invalidate any stale cache lines resident in other cpus.

This patch moves this initialization to the bl31_arch_setup() function
which is always called after the data cache and MMU has been enabled.

This change removes the need:
 1. for the DCIVAC cache maintenance operation.
 2. to initialise the CPU ops upon resumption from a PSCI CPU_SUSPEND
    call since memory contents are always preserved in this case.

Change-Id: Ibb2fa2f7460d1a1f1e721242025e382734c204c6

Demonstrate model for routing IRQs to EL3 ...

This patch provides an option to specify a interrupt routing model
where non-secure interrupts (IRQs) are routed to EL3 instead of S-EL1.
When such an interrupt occurs, the TSPD arranges a return to
the normal world after saving any necessary context. The interrupt
routing model to route IRQs to EL3 is enabled only during STD SMC
processing. Thus the pre-emption of S-EL1 is disabled during Fast SMC
and Secure Interrupt processing.

A new build option TSPD_ROUTE_NS_INT_EL3 is introduced to change
the non secure interrupt target execution level to EL3.

Fixes ARM-software/tf-issues#225

Change-Id: Ia1e779fbbb6d627091e665c73fa6315637cfdd32

Increment the PSCI VERSION to 1.0 ...

This patch:

   * Bumps the PSCI VERSION to 1.0. This means that
     the PSCI_VERSION API will now return the value 0x00010000
     to indicate the version as 1.0. The firmware remains
     compatible with PSCI v0.2 clients.

   * The firmware design guide is updated to document the
     APIs supported by the Trusted Firmware generic code.

   * The FVP Device Tree Sources (dts) and Blobs(dtb) are also
     updated to add "psci-1.0" and "psci-0.2" to the list of
     compatible PSCI versions.

Change-Id: Iafc2f549c92651dcd65d7e24a8aae35790d00f8a

Implement PSCI_FEATURES API ...

This patch implements the PSCI_FEATURES function which is a mandatory
API in the PSCI 1.0 specification. A capability variable is
constructed during initialization by examining the plat_pm_ops and
spd_pm_ops exported by the platform and the Secure Payload Dispatcher.
This is used by the PSCI FEATURES function to determine which
PSCI APIs are supported by the platform.

Change-Id: I147ffc1bd5d90b469bd3cc4bbe0a20e95c247df7

Rework the PSCI migrate APIs ...

This patch reworks the PSCI MIGRATE, MIGRATE_INFO_TYPE and
MIGRATE_INFO_UP_CPU support for Trusted Firmware. The
implementation does the appropriate validation of parameters
and invokes the appropriate hook exported by the SPD.

The TSP is a MP Trusted OS. Hence the ability to actually
migrate a Trusted OS has not been implemented. The
corresponding function is not populated in the spd_pm_hooks
structure for the TSPD.

The `spd_pm_ops_t` has undergone changes with this patch.
SPD PORTS MAY NEED TO BE UPDATED.

Fixes ARM-software/tf-issues#249

Change-Id: Iabd87521bf7c530a5e4506b6d3bfd4f1bf87604f

Validate power_state and entrypoint when executing PSCI calls ...

This patch allows the platform to validate the power_state and
entrypoint information from the normal world early on in PSCI
calls so that we can return the error safely. New optional
pm_ops hooks `validate_power_state` and `validate_ns_entrypoint`
are introduced to do this.

As a result of these changes, all the other pm_ops handlers except
the PSCI_ON handler are expected to be successful. Also, the PSCI
implementation will now assert if a PSCI API is invoked without the
corresponding pm_ops handler being registered by the platform.

NOTE : PLATFORM PORTS WILL BREAK ON MERGE OF THIS COMMIT. The
pm hooks have 2 additional optional callbacks and the return type
of the other hooks have changed.

Fixes ARM-Software/tf-issues#229

Change-Id: I036bc0cff2349187c7b8b687b9ee0620aa7e24dc

Save 'power_state' early in PSCI CPU_SUSPEND call ...

This patch adds support to save the "power state" parameter before the
affinity level specific handlers are called in a CPU_SUSPEND call.
This avoids the need to pass the power_state as a parameter to the
handlers and Secure Payload Dispatcher (SPD) suspend spd_pm_ops.
The power_state arguments in the spd_pm_ops operations are now reserved
and must not be used. The SPD can query the relevant power_state fields
by using the psci_get_suspend_afflvl() & psci_get_suspend_stateid() APIs.

NOTE: THIS PATCH WILL BREAK THE SPD_PM_OPS INTERFACE. HENCE THE SECURE
PAYLOAD DISPATCHERS WILL NEED TO BE REWORKED TO USE THE NEW INTERFACE.

Change-Id: I1293d7dc8cf29cfa6a086a009eee41bcbf2f238e

Remove `ns_entrypoint` and `mpidr` from parameters in pm_ops ...

This patch removes the non-secure entry point information being passed
to the platform pm_ops which is not needed. Also, it removes the `mpidr`
parameter for  platform pm hooks which are meant to do power management
operations only on the current cpu.

NOTE: PLATFORM PORTS MUST BE UPDATED AFTER MERGING THIS COMMIT.

Change-Id: If632376a990b7f3b355f910e78771884bf6b12e7

Move bakery algorithm implementation out of coherent memory ...

This patch moves the bakery locks out of coherent memory to normal memory.
This implies that the lock information needs to be placed on a separate cache
line for each cpu. Hence the bakery_lock_info_t structure is allocated in the
per-cpu data so as to minimize memory wastage. A similar platform per-cpu
data is introduced for the platform locks.

As a result of the above changes, the bakery lock api is completely changed.
Earlier, a reference to the lock structure was passed to the lock implementation.
Now a unique-id (essentially an index into the per-cpu data array) and an offset
into the per-cpu data for bakery_info_t needs to be passed to the lock
implementation.

Change-Id: I1e76216277448713c6c98b4c2de4fb54198b39e0

Invalidate the dcache after initializing cpu-ops ...

This patch fixes a crash due to corruption of cpu_ops
data structure. During the secondary CPU boot, after the
cpu_ops has been initialized in the per cpu-data, the
dcache lines need to invalidated so that the update in
memory can be seen later on when the dcaches are turned ON.
Also, after initializing the psci per cpu data, the dcache
lines are flushed so that they are written back to memory
and dirty dcache lines are avoided.

Fixes ARM-Software/tf-issues#271

Change-Id: Ia90f55e9882690ead61226eea5a5a9146d35f313

Introduce framework for CPU specific operations ...

This patch introduces a framework which will allow CPUs to perform
implementation defined actions after a CPU reset, during a CPU or cluster power
down, and when a crash occurs. CPU specific reset handlers have been implemented
in this patch. Other handlers will be implemented in subsequent patches.

Also moved cpu_helpers.S to the new directory lib/cpus/aarch64/.

Change-Id: I1ca1bade4d101d11a898fb30fea2669f9b37b956

Miscellaneous PSCI code cleanups ...

This patch implements the following cleanups in PSCI generic code:

1. It reworks the affinity level specific handlers in the PSCI implementation
   such that.

   a. Usage of the 'rc' local variable is restricted to only where it is
      absolutely needed

   b. 'plat_state' local variable is defined only when a direct invocation of
      plat_get_phys_state() does not suffice.

   c. If a platform handler is not registered then the level specific handler
      returns early.

2. It limits the use of the mpidr_aff_map_nodes_t typedef to declaration of
   arrays of the type instead of using it in function prototypes as well.

3. It removes dangling declarations of __psci_cpu_off() and
   __psci_cpu_suspend(). The definitions of these functions were removed in
   earlier patches.

Change-Id: I51e851967c148be9c2eeda3a3c41878f7b4d6978

Add APIs to preserve highest affinity level in OFF state ...

This patch adds APIs to find, save and retrieve the highest affinity level which
will enter or exit from the physical OFF state during a PSCI power management
operation. The level is stored in per-cpu data.

It then reworks the PSCI implementation to perform cache maintenance only
when the handler for the highest affinity level to enter/exit the OFF state is
called.

For example. during a CPU_SUSPEND operation, state management is done prior to
calling the affinity level specific handlers. The highest affinity level which
will be turned off is determined using the psci_find_max_phys_off_afflvl()
API. This level is saved using the psci_set_max_phys_off_afflvl() API. In the
code that does generic handling for each level, prior to performing cache
maintenance it is first determined if the current affinity level matches the
value returned by psci_get_max_phys_off_afflvl(). Cache maintenance is done if
the values match.

This change allows the last CPU in a cluster to perform cache maintenance
independently. Earlier, cache maintenance was started in the level 0 handler and
finished in the level 1 handler. This change in approach will facilitate
implementation of tf-issues#98.

Change-Id: I57233f0a27b3ddd6ddca6deb6a88b234525b0ae6

Add PSCI service specific per-CPU data ...

This patch adds a structure defined by the PSCI service to the per-CPU data
array. The structure is used to save the 'power_state' parameter specified
during a 'cpu_suspend' call on the current CPU. This parameter was being saved
in the cpu node in the PSCI topology tree earlier.

The existing API to return the state id specified during a PSCI CPU_SUSPEND call
i.e. psci_get_suspend_stateid(mpidr) has been renamed to
psci_get_suspend_stateid_by_mpidr(mpidr). The new psci_get_suspend_stateid() API
returns the state id of the current cpu.

The psci_get_suspend_afflvl() API has been changed to return the target affinity
level of the current CPU. This was specified using the 'mpidr' parameter in the
old implementation.

The behaviour of the get_power_on_target_afflvl() has been tweaked such that
traversal of the PSCI topology tree to locate the affinity instance node for the
current CPU is done only in the debug build as it is an expensive operation.

Change-Id: Iaad49db75abda471f6a82d697ee6e0df554c4caf

Add macro to flush per-CPU data ...

This patch adds a macro which will flush the contents of the specified member of
the per-CPU data structure to the PoC. This is required to enable an update of a
per-CPU data member to be visible to all observers.

Change-Id: I20e0feb9b9f345dc5a1162e88adc7956a7ad7a64

Add support for PSCI SYSTEM_OFF and SYSTEM_RESET APIs ...

This patch adds support for SYSTEM_OFF and SYSTEM_RESET PSCI
operations. A platform should export handlers to complete the
requested operation. The FVP port exports fvp_system_off() and
fvp_system_reset() as an example.

If the SPD provides a power management hook for system off and
system reset, then the SPD is notified about the corresponding
operation so it can do some bookkeeping. The TSPD exports
tspd_system_off() and tspd_system_reset() for that purpose.

Versatile Express shutdown and reset methods have been removed
from the FDT as new PSCI sys_poweroff and sys_reset services
have been added. For those kernels that do not support yet these
PSCI services (i.e. GICv3 kernel), the original dtsi files have
been renamed to *-no_psci.dtsi.

Fixes ARM-software/tf-issues#218

Change-Id: Ic8a3bf801db979099ab7029162af041c4e8330c8

Optimize EL3 register state stored in cpu_context structure ...

This patch further optimizes the EL3 register state stored in
cpu_context. The 2 registers which are removed from cpu_context are:

  * cntfrq_el0 is the system timer register which is writable
    only in EL3 and it can be programmed during cold/warm boot. Hence
    it need not be saved to cpu_context.

  * cptr_el3 controls access to Trace, Floating-point, and Advanced
    SIMD functionality and it is programmed every time during cold
    and warm boot. The current BL3-1 implementation does not need to
    modify the access controls during normal execution and hence
    they are expected to remain static.

Fixes ARM-software/tf-issues#197

Change-Id: I599ceee3b73a7dcfd37069fd41b60e3d397a7b18

Merge pull request #172 from soby-mathew/sm/asm_assert ...

Introduce asm assert and optimize crash reporting

Rework the crash reporting in BL3-1 to use less stack ...

This patch reworks the crash reporting mechanism to further
optimise the stack and code size. The reporting makes use
of assembly console functions to avoid calling C Runtime
to report the CPU state. The crash buffer requirement is
reduced to 64 bytes with this implementation. The crash
buffer is now part of per-cpu data which makes retrieving
the crash buffer trivial.

Also now panic() will use crash reporting if
invoked from BL3-1.

Fixes ARM-software/tf-issues#199

Change-Id: I79d27a4524583d723483165dc40801f45e627da5

Remove coherent stack usage from the warm boot path ...

This patch uses stacks allocated in normal memory to enable the MMU early in the
warm boot path thus removing the dependency on stacks allocated in coherent
memory. Necessary cache and stack maintenance is performed when a cpu is being
powered down and up. This avoids any coherency issues that can arise from
reading speculatively fetched stale stack memory from another CPUs cache. These
changes affect the warm boot path in both BL3-1 and BL3-2.

The EL3 system registers responsible for preserving the MMU state are not saved
and restored any longer. Static values are used to program these system
registers when a cpu is powered on or resumed from suspend.

Change-Id: I8357e2eb5eb6c5f448492c5094b82b8927603784

Allow FP register context to be optional at build time ...

CTX_INCLUDE_FPREGS make variable allows us to include or exclude FP
registers from context structure, in case FP is not used by TSPD.

Fixes ARM-software/tf-issues#194

Change-Id: Iee41af382d691340c7ae21830ad1bbf95dad1f4b