Make generic code work in presence of system caches ...

On the ARMv8 architecture, cache maintenance operations by set/way on the last
level of integrated cache do not affect the system cache. This means that such a
flush or clean operation could result in the data being pushed out to the system
cache rather than main memory. Another CPU could access this data before it
enables its data cache or MMU. Such accesses could be serviced from the main
memory instead of the system cache. If the data in the sysem cache has not yet
been flushed or evicted to main memory then there could be a loss of
coherency. The only mechanism to guarantee that the main memory will be updated
is to use cache maintenance operations to the PoC by MVA(See section D3.4.11
(System level caches) of ARMv8-A Reference Manual (Issue A.g/ARM DDI0487A.G).

This patch removes the reliance of Trusted Firmware on the flush by set/way
operation to ensure visibility of data in the main memory. Cache maintenance
operations by MVA are now used instead. The following are the broad category of
changes:

1. The RW areas of BL2/BL31/BL32 are invalidated by MVA before the C runtime is
   initialised. This ensures that any stale cache lines at any level of cache
   are removed.

2. Updates to global data in runtime firmware (BL31) by the primary CPU are made
   visible to secondary CPUs using a cache clean operation by MVA.

3. Cache maintenance by set/way operations are only used prior to power down.

NOTE: NON-UPSTREAM TRUSTED FIRMWARE CODE SHOULD MAKE EQUIVALENT CHANGES IN
ORDER TO FUNCTION CORRECTLY ON PLATFORMS WITH SUPPORT FOR SYSTEM CACHES.

Fixes ARM-software/tf-issues#205

Change-Id: I64f1b398de0432813a0e0881d70f8337681f6e9a

Re-design bakery lock memory allocation and algorithm ...

This patch unifies the bakery lock api's across coherent and normal
memory implementation of locks by using same data type `bakery_lock_t`
and similar arguments to functions.

A separate section `bakery_lock` has been created and used to allocate
memory for bakery locks using `DEFINE_BAKERY_LOCK`. When locks are
allocated in normal memory, each lock for a core has to spread
across multiple cache lines. By using the total size allocated in a
separate cache line for a single core at compile time, the memory for
other core locks is allocated at link time by multiplying the single
core locks size with (PLATFORM_CORE_COUNT - 1). The normal memory lock
algorithm now uses lock address instead of the `id` in the per_cpu_data.
For locks allocated in coherent memory, it moves locks from
tzfw_coherent_memory to bakery_lock section.

The bakery locks are allocated as part of bss or in coherent memory
depending on usage of coherent memory. Both these regions are
initialised to zero as part of run_time_init before locks are used.
Hence, bakery_lock_init() is made an empty function as the lock memory
is already initialised to zero.

The above design lead to the removal of psci bakery locks from
non_cpu_power_pd_node to psci_locks.

NOTE: THE BAKERY LOCK API WHEN USE_COHERENT_MEM IS NOT SET HAS CHANGED.
THIS IS A BREAKING CHANGE FOR ALL PLATFORM PORTS THAT ALLOCATE BAKERY
LOCKS IN NORMAL MEMORY.

Change-Id: Ic3751c0066b8032dcbf9d88f1d4dc73d15f61d8b

PSCI: Rework generic code to conform to coding guidelines ...

This patch reworks the PSCI generic implementation to conform to ARM
Trusted Firmware coding guidelines as described here:
https://github.com/ARM-software/arm-trusted-firmware/wiki

This patch also reviews the use of signed data types within PSCI
Generic code and replaces them with their unsigned counterparts wherever
they are not appropriate. The PSCI_INVALID_DATA macro which was defined
to -1 is now replaced with PSCI_INVALID_PWR_LVL macro which is defined
to PLAT_MAX_PWR_LVL + 1.

Change-Id: Iaea422d0e46fc314e0b173c2b4c16e0d56b2515a

PSCI: Switch to the new PSCI frameworks ...

This commit does the switch to the new PSCI framework implementation replacing
the existing files in PSCI folder with the ones in PSCI1.0 folder. The
corresponding makefiles are modified as required for the new implementation.
The platform.h header file is also is switched to the new one
as required by the new frameworks. The build flag ENABLE_PLAT_COMPAT defaults
to 1 to enable compatibility layer which let the existing platform ports to
continue to build and run with minimal changes.

The default weak implementation of platform_get_core_pos() is now removed from
platform_helpers.S and is provided by the compatibility layer.

Note: The Secure Payloads and their dispatchers still use the old platform
and framework APIs and hence it is expected that the ENABLE_PLAT_COMPAT build
flag will remain enabled in subsequent patch. The compatibility for SPDs using
the older APIs on platforms migrated to the new APIs will be added in the
following patch.

Change-Id: I18c51b3a085b564aa05fdd98d11c9f3335712719

PSCI: Add SYSTEM_SUSPEND API support ...

This patch adds support for SYSTEM_SUSPEND API as mentioned in the PSCI 1.0
specification. This API, on being invoked on the last running core on a
supported platform, will put the system into a low power mode with memory
retention.

The psci_afflvl_suspend() internal API has been reused as most of the actions
to suspend a system are the same as invoking the PSCI CPU_SUSPEND API with the
target affinity level as 'system'. This API needs the 'power state' parameter
for the target low power state. This parameter is not passed by the caller of
the SYSTEM_SUSPEND API. Hence, the platform needs to implement the
get_sys_suspend_power_state() platform function to provide this information.
Also, the platform also needs to add support for suspending the system to the
existing 'plat_pm_ops' functions: affinst_suspend() and
affinst_suspend_finish().

Change-Id: Ib6bf10809cb4e9b92f463755608889aedd83cef5

Export maximum affinity using PLATFORM_MAX_AFFLVL macro ...

This patch removes the plat_get_max_afflvl() platform API
and instead replaces it with a platform macro PLATFORM_MAX_AFFLVL.
This is done because the maximum affinity level for a platform
is a static value and it is more efficient for it to be defined
as a platform macro.

NOTE: PLATFORM PORTS NEED TO BE UPDATED ON MERGE OF THIS COMMIT

Fixes ARM-Software/tf-issues#265

Change-Id: I31d89b30c2ccda30d28271154d869060d50df7bf

Implement PSCI_FEATURES API ...

This patch implements the PSCI_FEATURES function which is a mandatory
API in the PSCI 1.0 specification. A capability variable is
constructed during initialization by examining the plat_pm_ops and
spd_pm_ops exported by the platform and the Secure Payload Dispatcher.
This is used by the PSCI FEATURES function to determine which
PSCI APIs are supported by the platform.

Change-Id: I147ffc1bd5d90b469bd3cc4bbe0a20e95c247df7

Remove coherent memory from the BL memory maps ...

This patch extends the build option `USE_COHERENT_MEMORY` to
conditionally remove coherent memory from the memory maps of
all boot loader stages. The patch also adds necessary
documentation for coherent memory removal in firmware-design,
porting and user guides.

Fixes ARM-Software/tf-issues#106

Change-Id: I260e8768c6a5c2efc402f5804a80657d8ce38773

Move bakery algorithm implementation out of coherent memory ...

This patch moves the bakery locks out of coherent memory to normal memory.
This implies that the lock information needs to be placed on a separate cache
line for each cpu. Hence the bakery_lock_info_t structure is allocated in the
per-cpu data so as to minimize memory wastage. A similar platform per-cpu
data is introduced for the platform locks.

As a result of the above changes, the bakery lock api is completely changed.
Earlier, a reference to the lock structure was passed to the lock implementation.
Now a unique-id (essentially an index into the per-cpu data array) and an offset
into the per-cpu data for bakery_info_t needs to be passed to the lock
implementation.

Change-Id: I1e76216277448713c6c98b4c2de4fb54198b39e0

Invalidate the dcache after initializing cpu-ops ...

This patch fixes a crash due to corruption of cpu_ops
data structure. During the secondary CPU boot, after the
cpu_ops has been initialized in the per cpu-data, the
dcache lines need to invalidated so that the update in
memory can be seen later on when the dcaches are turned ON.
Also, after initializing the psci per cpu data, the dcache
lines are flushed so that they are written back to memory
and dirty dcache lines are avoided.

Fixes ARM-Software/tf-issues#271

Change-Id: Ia90f55e9882690ead61226eea5a5a9146d35f313

Fix CPU_SUSPEND when invoked with affinity level higher than get_max_afflvl() ...

This patch fixes the assertion failure when CPU_SUSPEND is invoked with
an affinity level higher than supported by the platform by adding suitable
checks for affinity level within `psci_cpu_suspend`. Also added suitable
bound checks within `psci_aff_map_get_idx` to prevent indexing beyond array
limits.

Fixes ARM-software/tf-issues#260

Change-Id: I04b75c49729e6c6d1983add590f60146c8fc3630

Miscellaneous PSCI code cleanups ...

This patch implements the following cleanups in PSCI generic code:

1. It reworks the affinity level specific handlers in the PSCI implementation
   such that.

   a. Usage of the 'rc' local variable is restricted to only where it is
      absolutely needed

   b. 'plat_state' local variable is defined only when a direct invocation of
      plat_get_phys_state() does not suffice.

   c. If a platform handler is not registered then the level specific handler
      returns early.

2. It limits the use of the mpidr_aff_map_nodes_t typedef to declaration of
   arrays of the type instead of using it in function prototypes as well.

3. It removes dangling declarations of __psci_cpu_off() and
   __psci_cpu_suspend(). The definitions of these functions were removed in
   earlier patches.

Change-Id: I51e851967c148be9c2eeda3a3c41878f7b4d6978

Add APIs to preserve highest affinity level in OFF state ...

This patch adds APIs to find, save and retrieve the highest affinity level which
will enter or exit from the physical OFF state during a PSCI power management
operation. The level is stored in per-cpu data.

It then reworks the PSCI implementation to perform cache maintenance only
when the handler for the highest affinity level to enter/exit the OFF state is
called.

For example. during a CPU_SUSPEND operation, state management is done prior to
calling the affinity level specific handlers. The highest affinity level which
will be turned off is determined using the psci_find_max_phys_off_afflvl()
API. This level is saved using the psci_set_max_phys_off_afflvl() API. In the
code that does generic handling for each level, prior to performing cache
maintenance it is first determined if the current affinity level matches the
value returned by psci_get_max_phys_off_afflvl(). Cache maintenance is done if
the values match.

This change allows the last CPU in a cluster to perform cache maintenance
independently. Earlier, cache maintenance was started in the level 0 handler and
finished in the level 1 handler. This change in approach will facilitate
implementation of tf-issues#98.

Change-Id: I57233f0a27b3ddd6ddca6deb6a88b234525b0ae6

Add PSCI service specific per-CPU data ...

This patch adds a structure defined by the PSCI service to the per-CPU data
array. The structure is used to save the 'power_state' parameter specified
during a 'cpu_suspend' call on the current CPU. This parameter was being saved
in the cpu node in the PSCI topology tree earlier.

The existing API to return the state id specified during a PSCI CPU_SUSPEND call
i.e. psci_get_suspend_stateid(mpidr) has been renamed to
psci_get_suspend_stateid_by_mpidr(mpidr). The new psci_get_suspend_stateid() API
returns the state id of the current cpu.

The psci_get_suspend_afflvl() API has been changed to return the target affinity
level of the current CPU. This was specified using the 'mpidr' parameter in the
old implementation.

The behaviour of the get_power_on_target_afflvl() has been tweaked such that
traversal of the PSCI topology tree to locate the affinity instance node for the
current CPU is done only in the debug build as it is an expensive operation.

Change-Id: Iaad49db75abda471f6a82d697ee6e0df554c4caf

Eliminate psci_suspend_context array ...

psci_suspend_context is an array of cache-line aligned structures
containing the single power_state integer per cpu. This array is
the only structure indexed by the aff_map_node.data integer.

This patch saves 2KB of BL3-1 memory by placing the CPU
power_state value directly in the aff_map_node structure. As a
result, this value is now never cached and the cache clean when
writing the value is no longer required.

Fixes ARM-software/tf-issues#195

Change-Id: Ib4c70c8f79eed295ea541e7827977a588a19ef9b

Initialise CPU contexts from entry_point_info ...

Consolidate all BL3-1 CPU context initialization for cold boot, PSCI
and SPDs into two functions:
*  The first uses entry_point_info to initialize the relevant
   cpu_context for first entry into a lower exception level on a CPU
*  The second populates the EL1 and EL2 system registers as needed
   from the cpu_context to ensure correct entry into the lower EL

This patch alters the way that BL3-1 determines which exception level
is used when first entering EL1 or EL2 during cold boot - this is now
fully determined by the SPSR value in the entry_point_info for BL3-3,
as set up by the platform code in BL2 (or otherwise provided to BL3-1).

In the situation that EL1 (or svc mode) is selected for a processor
that supports EL2, the context management code will now configure all
essential EL2 register state to ensure correct execution of EL1. This
allows the platform code to run non-secure EL1 payloads directly
without requiring a small EL2 stub or OS loader.

Change-Id: If9fbb2417e82d2226e47568203d5a369f39d3b0f

Provide cm_get/set_context() for current CPU ...

All callers of cm_get_context() pass the calling CPU MPIDR to the
function. Providing a specialised version for the current
CPU results in a reduction in code size and better readability.

The current function has been renamed to cm_get_context_by_mpidr()
and the existing name is now used for the current-CPU version.

The same treatment has been done to cm_set_context(), although
only both forms are used at present in the PSCI and TSPD code.

Change-Id: I91cb0c2f7bfcb950a045dbd9ff7595751c0c0ffb

Remove unused data declarations ...

Some data variables were declared but not used. These have been
removed.

Change-Id: I038632af3c32d88984cd25b886c43ff763269bf9

Rework BL3-1 unhandled exception handling and reporting ...

This patch implements the register reporting when unhandled exceptions are
taken in BL3-1. Unhandled exceptions will result in a dump of registers
to the console, before halting execution by that CPU. The Crash Stack,
previously called the Exception Stack, is used for this activity.
This stack is used to preserve the CPU context and runtime stack
contents for debugging and analysis.

This also introduces the per_cpu_ptr_cache, referenced by tpidr_el3,
to provide easy access to some of BL3-1 per-cpu data structures.
Initially, this is used to provide a pointer to the Crash stack.

panic() now prints the the error file and line number in Debug mode
and prints the PC value in release mode.

The Exception Stack is renamed to Crash Stack with this patch.
The original intention of exception stack is no longer valid
since we intend to support several valid exceptions like IRQ
and FIQ in the trusted firmware context. This stack is now
utilized for dumping and reporting the system state when a
crash happens and hence the rename.

Fixes ARM-software/tf-issues#79 Improve reporting of unhandled exception

Change-Id: I260791dc05536b78547412d147193cdccae7811a

Reduce deep nesting of header files ...

Reduce the number of header files included from other header
files as much as possible without splitting the files. Use forward
declarations where possible. This allows removal of some unnecessary
"#ifndef __ASSEMBLY__" statements.

Also, review the .c and .S files for which header files really need
including and reorder the #include statements alphabetically.

Fixes ARM-software/tf-issues#31

Change-Id: Iec92fb976334c77453e010b60bcf56f3be72bd3e

Always use named structs in header files ...

Add tag names to all unnamed structs in header files. This
allows forward declaration of structs, which is necessary to
reduce header file nesting (to be implemented in a subsequent
commit).

Also change the typedef names across the codebase to use the _t
suffix to be more conformant with the Linux coding style. The
coding style actually prefers us not to use typedefs at all but
this is considered a step too far for Trusted Firmware.

Also change the IO framework structs defintions to use typedef'd
structs to be consistent with the rest of the codebase.

Change-Id: I722b2c86fc0d92e4da3b15e5cab20373dd26786f

Make use of user/system includes more consistent ...

Make codebase consistent in its use of #include "" syntax for
user includes and #include <> syntax for system includes.

Fixes ARM-software/tf-issues#65

Change-Id: If2f7c4885173b1fd05ac2cde5f1c8a07000c7a33

Preserve PSCI cpu_suspend 'power_state' parameter. ...

This patch saves the 'power_state' parameter prior to suspending
a cpu and invalidates it upon its resumption. The 'affinity level'
and 'state id' fields of this parameter can be read using a set of
public and private apis. Validation of power state parameter is
introduced which checks for SBZ bits are zero.
This change also takes care of flushing the parameter from the cache
to main memory. This ensures that it is available after cpu reset
when the caches and mmu are turned off. The earlier support for
saving only the 'affinity level' field of the 'power_state' parameter
has also been reworked.

Fixes ARM-Software/tf-issues#26
Fixes ARM-Software/tf-issues#130

Change-Id: Ic007ccb5e39bf01e0b67390565d3b4be33f5960a

Implement ARM Standard Service ...

This patch implements ARM Standard Service as a runtime service and adds
support for call count, UID and revision information SMCs. The existing
PSCI implementation is subsumed by the Standard Service calls and all
PSCI calls are therefore dispatched by the Standard Service to the PSCI
handler.

At present, PSCI is the only specification under Standard Service. Thus
call count returns the number of PSCI calls implemented. As this is the
initial implementation, a revision number of 0.1 is returned for call
revision.

Fixes ARM-software/tf-issues#62

Change-Id: I6d4273f72ad6502636efa0f872e288b191a64bc1