ARM: MMU: fix arch_remap_range() across section boundaries ...

Fixes: e3e54c6441 ARM: mmu: Implement on-demand PTE allocation

PGD_FLAGS_WC_V7 lacks the PMD_TYPE_SECT and PMD_SECT_BUFFERABLE flags.
Without them a dma_alloc_writecombine() creates an invalid section when
it crosses a section boundary.

Signed-off-by: Sascha Hauer <s.hauer@pengutronix.de>
Reviewed-by: Lucas Stach <l.stach@pengutronix.de>

ARM: mmu: only create flat mapping when early MMU hasn't done it already ...

Signed-off-by: Sascha Hauer <s.hauer@pengutronix.de>

ARM: mmu: Introduce ARM_TTB_SIZE ...

Commit 1c33aac ("ARM: use memalign to
allocate page table"), reasonalby changed TTB allocation size from
SZ_32K to SZ_16K (TTB's real size), but it also changed alignment from
SZ_16K to SZ_64K for unclear reasons.

Reading various TTBR related ARM documentation it seems that worst
case alignment for it is 16KiB (bits [0, 13 - N] must be zero) which
also matches early TTB allocation code.

Since both early and regular MMU code has to share this paramter,
introduce ARM_TTB_SIZE and use it in both cases for both size and
alignment.

Signed-off-by: Andrey Smirnov <andrew.smirnov@gmail.com>
Signed-off-by: Sascha Hauer <s.hauer@pengutronix.de>

ARM: mmu: Implement on-demand PTE allocation ...

Allocating PTEs for every 4K page corresponding to SDRAM upfront costs
us quite a bit of memory: 1KB per 1MB or RAM. This is far from being a
deal-breaker for majority of use-cases, but for builds where amount of
free memory is in hundres of KBs* it becomes a real hurdle for being
able to use MMU (which also means no L1 cache).

Given how we really only need PTEs for a very few regions of memory
dedicated from DMA buffers (Ethernet, USB, etc), changing MMU code to
do on-demand section splitting can allow us to save significant amount
of memory without any functionality loss.

Below is a very trivial comparison of memory usages on start before
and after this patch is applied.

Before:

barebox@ZII VF610 Development Board, Rev B:/ meminfo
used: 1271584
free: 265553032

After:

barebox@ZII VF610 Development Board, Rev B:/ meminfo
used: 795276
free: 266024448

Tested on:

       - VF610 Tower Board,
       - VF610 ZII Development Board (Rev. C)
       - i.MX51 Babbage Board
       - i.MX7 SabreSD Board
       - i.MX6 ZII RDU2 Board
        - AT91SAM9X5-EK Board

* One example of such use-case is memory testing while running purely
  out of SRAM

Signed-off-by: Andrey Smirnov <andrew.smirnov@gmail.com>
Signed-off-by: Sascha Hauer <s.hauer@pengutronix.de>

ARM: mmu: Simplify the use of dma_flush_range() ...

Simplify the use of dma_flush_range() by changing its signature to
accept pointer to start of the data and data size. This change allows
us to avoid a whole bunch of repetitive arithmetic currently done by
all of the callers.

Reviewed-by: Lucas Stach <l.stach@pengutronix.de>
Signed-off-by: Andrey Smirnov <andrew.smirnov@gmail.com>
Signed-off-by: Sascha Hauer <s.hauer@pengutronix.de>

ARM: mmu: Use dma_inv_range() in dma_sync_single_for_cpu() ...

The code in the if () statement is identical to already existing
dma_inv_rand(). Use it instead.

Reviewed-by: Lucas Stach <l.stach@pengutronix.de>
Signed-off-by: Andrey Smirnov <andrew.smirnov@gmail.com>
Signed-off-by: Sascha Hauer <s.hauer@pengutronix.de>

ARM: mmu: Use find_pte() to find PTE in create_vector_table() ...

There's already a function that implement necessary arithemtic to find
offset within page table for a given address, so make use of it
instead of re-implementing it again.

Signed-off-by: Andrey Smirnov <andrew.smirnov@gmail.com>
Signed-off-by: Sascha Hauer <s.hauer@pengutronix.de>

ARM: mmu: Make sure that address is 1M aligned in arm_create_pte() ...

If address passed arm_create_pte() is not 1M (PGDIR_SIZE) aligned,
page table that is created will end up having unexpected mapping
offset, breaking "1:1 mapping" assumption and leading to bugs that are
not immediately obvious in their nature.

To prevent this and because all of the callers already do said
alignement in-place, move the alignment code to be a part of
arm_create_pte().

Signed-off-by: Andrey Smirnov <andrew.smirnov@gmail.com>
Signed-off-by: Sascha Hauer <s.hauer@pengutronix.de>

ARM: mmu: Pass PTE flags a parameter to arm_create_pte() ...

In order to make it possible to use this functions in contexts where
creating a new PTE of uncached pages in not appropriate, pass PTE
flags a parameter to arm_create_pte() and fix all of the current users
as necessary.

Signed-off-by: Andrey Smirnov <andrew.smirnov@gmail.com>
Signed-off-by: Sascha Hauer <s.hauer@pengutronix.de>

ARM: mmu: Share code between dma_alloc_*() functions ...

Code of dma_alloc_coherent() and dma_alloc_writecombine() is almost
identical with exception of the flags passed to undelying call to
__remap_range(). Move commong code into a shared subroutine and
convert both functions to use it.

Signed-off-by: Andrey Smirnov <andrew.smirnov@gmail.com>
Signed-off-by: Sascha Hauer <s.hauer@pengutronix.de>

ARM: mmu: Use xmemalign in mmu_init() ...

We don't handle OOM case in that code, so using xmemalign seems like a
better option.

Signed-off-by: Andrey Smirnov <andrew.smirnov@gmail.com>
Signed-off-by: Sascha Hauer <s.hauer@pengutronix.de>

ARM: mmu: Use xmemalign in arm_create_pte() ...

We don't handle the OOM case in that code, so using xmemalign seems
like a better option.

Signed-off-by: Andrey Smirnov <andrew.smirnov@gmail.com>
Signed-off-by: Sascha Hauer <s.hauer@pengutronix.de>

ARM: mmu: Use PAGE_SIZE instead of magic right shift by 12 ...

Signed-off-by: Andrey Smirnov <andrew.smirnov@gmail.com>
Signed-off-by: Sascha Hauer <s.hauer@pengutronix.de>

ARM: mmu: Define and use PTRS_PER_PTE ...

Signed-off-by: Andrey Smirnov <andrew.smirnov@gmail.com>
Signed-off-by: Sascha Hauer <s.hauer@pengutronix.de>

ARM: mmu: Use PAGE_SIZE when specifying size of one page ...

Signed-off-by: Andrey Smirnov <andrew.smirnov@gmail.com>
Signed-off-by: Sascha Hauer <s.hauer@pengutronix.de>

ARM: mmu: Replace various SZ_1M with PGDIR_SIZE ...

Signed-off-by: Andrey Smirnov <andrew.smirnov@gmail.com>
Signed-off-by: Sascha Hauer <s.hauer@pengutronix.de>

ARM: mmu: Trivial simplification in arm_mmu_remap_sdram() ...

Signed-off-by: Andrey Smirnov <andrew.smirnov@gmail.com>
Signed-off-by: Sascha Hauer <s.hauer@pengutronix.de>

ARM: mmu: Replace hardcoded shifts with pgd_index() from Linux ...

Signed-off-by: Andrey Smirnov <andrew.smirnov@gmail.com>
Signed-off-by: Sascha Hauer <s.hauer@pengutronix.de>

ARM: mmu: Drop needless shifting in map_io_sections() ...

Instead of shifting phys right by 20 and then again left by the same
amount, just convert the code to expect it to be in unit of bytes all
the time.

Signed-off-by: Andrey Smirnov <andrew.smirnov@gmail.com>
Signed-off-by: Sascha Hauer <s.hauer@pengutronix.de>

ARM: mmu: Share code for initial flat mapping creation ...

Code creating inital 4GiB flat mapping is identical between mmu.c and
mmu-early.c, so move it to mmu.h and share.

Signed-off-by: Andrey Smirnov <andrew.smirnov@gmail.com>
Signed-off-by: Sascha Hauer <s.hauer@pengutronix.de>

ARM: mmu: Specify size in bytes in create_sections() ...

Seeing

        create_sections(ttb, 0, PAGE_SIZE, ...);

as the code the creates initial flat 4 GiB mapping is a bit less
intuitive then

        create_sections(ttb, 0, SZ_4G - 1, ...);

so, for the sake of clarification, convert create_sections() to accept
address of the last byte in the region instead of size in MiB. Note
the alternative of converting size to units of bytes was not chosen to
avoid converting 3-rd function argument into a 64-bit number.

Signed-off-by: Andrey Smirnov <andrew.smirnov@gmail.com>
Signed-off-by: Sascha Hauer <s.hauer@pengutronix.de>

ARM: mmu: Share code for create_sections() ...

Regular MMU code never creates anything but 1:1 mapping, and barring
that plus the call to __mmu_cache_flush(), early MMU code version of
the function is pretty much identical. To avoid code duplication, move
it to mmu.h and convert both regular and early MMU code to use it.

Signed-off-by: Andrey Smirnov <andrew.smirnov@gmail.com>
Signed-off-by: Sascha Hauer <s.hauer@pengutronix.de>

ARM: mmu: Introduce set_domain() ...

Port set_domain() form Linux kernel and share it between regular and
early MMU code to avoid duplication.

Signed-off-by: Andrey Smirnov <andrew.smirnov@gmail.com>
Signed-off-by: Sascha Hauer <s.hauer@pengutronix.de>

ARM: mmu: Introduce set_ttbr() ...

Signed-off-by: Andrey Smirnov <andrew.smirnov@gmail.com>
Signed-off-by: Sascha Hauer <s.hauer@pengutronix.de>

ARM: mmu: Make use of IS_ALIGNED in arm_mmu_remap_sdram() ...

Signed-off-by: Andrey Smirnov <andrew.smirnov@gmail.com>
Signed-off-by: Sascha Hauer <s.hauer@pengutronix.de>

ARM: mmu: Remove unused ARM_VECTORS_SIZE ...

Signed-off-by: Andrey Smirnov <andrew.smirnov@gmail.com>
Signed-off-by: Sascha Hauer <s.hauer@pengutronix.de>

ARM: implement dma mapping functions ...

Implement basic dma mapping functions. For now just assume every address
is valid for dma mapping.

Signed-off-by: Sascha Hauer <s.hauer@pengutronix.de>

dma: Use dma_addr_t as type for DMA addresses ...

DMA addresses are not necessarily the same as unsigned long. Fix
the type for the dma_sync_single_* operations.

Signed-off-by: Sascha Hauer <s.hauer@pengutronix.de>

ARM: move linker variable declarations to sections.h ...

We collected most linker variable declarations in asm/sections.h, so
move __exceptions_start/__exceptions_stop there aswell.

Signed-off-by: Sascha Hauer <s.hauer@pengutronix.de>

ARM: Move mmu_disable to mmu.c ...

mmu_64 has mmu_disable() in mmu_64.c. Move to mmu.c for the 32bit
case aswell to be consistent.

Signed-off-by: Sascha Hauer <s.hauer@pengutronix.de>