kbuild: switch over to thin archive ...

Starting v4.13, Linux always uses thin archive instead of relocatable
ELF to combine builtin objects.

The thin archive is basically a text file that only contains paths to
object files.

As Linux commit 98ced886dd79 mentioned, this has a lot of benefits:

 - save disk space for builds

 - speed-up building a little

 - fix some issues when liking a giant executable (for example,
   allyesconfig on ARM Linux)

 - work better with dead code elimination

The last one is important for barebox because pbl highly relies on the
dead code elimination (-f{function,data}-sections and --gc-sections)
dropping symbols that are unreachable from the linker's entry point.
The dead code elimination does not work if the same symbol names are
used in the pevious incremental link mechanism because the same name
symbols all go into the same section.

Commit a83c97f2a406 ("ARM: socfpga: generate smaller images when
multiple boards are selected") worked around it by giving a dedicate
section to each of the same name symbols. This workaround can go away.

built-in.o was renamed to built-in.a since it is now an archive.
built-in-pbl.o was renamed to built-in.pbl.a for consistency.

Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Signed-off-by: Sascha Hauer <s.hauer@pengutronix.de>

kbuild: unify barebox and pbl build commands ...

The current Linux uses the unified build command for built-in objects
and module objects. This is possible because part-of-module is set to
y when the target is being built for a module. If so, quiet_modtag is
set to [M], and modkern_cflags is set to KBUILD_CFLAGS_MODULE instead
of KBUILD_CFLAGS_KERNEL.

Currently, barebox uses a separate build command for pbl, but the
difference between cmd_cc_o_c and cmd_pbl_cc_o_c is just
"-D__PBL__ $(PBL_CPPFLAGS)".

By using the same approach as Linux, they can be unified.

Set part-of-pbl flag to y when the target is being built for pbl.
Merge cmd_as_o_S and cmd_link_o_target as well.

This also changes the shortlog style as follows:

  PBLCC  ->  CC [P]
  PBLAS  ->  AS [P]
  PBLLD  ->  LD [P]

This is more consistent with the policy in the Linux build system.

In Linux, a capital letter enclosed square blackets indicates which
target the object is being built for.

  CC      object built for built-in
  CC [M]  object built for modules
  CC [U]  object built for userspace (proposed for Linux 5.8-rc1)

The idea behind this is we use the same compiler $(CC) for them.
Only the difference is the compiler flags. On the other hand, host
programs are compiled by a different tool, $(HOSTCC), hence the
shortlog is 'HOSTCC'.

Barebox supports module builds (it seems), but the [P] notation does
not conflict with [M] because pbl has a limited memory footprint.
We never expect pbl can support the module feature.

Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Signed-off-by: Sascha Hauer <s.hauer@pengutronix.de>

kbuild: resync modkern_{c, a}flags and quiet_modtag with Linux 5.7-rc6 ...

The latest Linux calculates them more simply; part-of-module is set
when the target is being built for a module.

Also, rename real-objs-{y,m} to real-obj-{y,m}.

Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Signed-off-by: Sascha Hauer <s.hauer@pengutronix.de>

kbuild: append $(bbenv-y) to obj-y earlier ...

You can save one line.

Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Signed-off-by: Sascha Hauer <s.hauer@pengutronix.de>

kbuild: resync mkdir code with Linux 5.7-rc6 ...

Kbuild automatically creates the output directories for O= builds.
Previously it called mkdir too much. Linux optimized this a lot.
Let's import the outcome so it works faster.

Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Signed-off-by: Sascha Hauer <s.hauer@pengutronix.de>

kbuild: remove obj-dtb-y, pbl-dtb-y, lwl-dtb-y syntax ...

The embedded DTBs are compiled by the chain of pattern rules as follows:

  %.dts -> %.dtb -> %.dtb.S -> %.dtb.o       for barebox proper
  %.dts -> %.dtb -> %.dtb.S -> %.dtb.pbl.o   for pbl

Barebox introduced {obj,pbl,lwl}-dtb-y syntax to put the intermediate
files into extra-y. The purposes of doing so were:

 [1] prevent GNU Make from deleting the intermediate files
 [2] include .*.cmd files

In contrast, Linux does not use a special syntax for embedding DTBs
into the kernel.

For example, as you see in arch/sh/boot/dts/Makefile of Linux 5.6,

   obj-y += <basename>.dtb.o

... just works.

This is because scripts/Kbuild.include specifies .SECONDARY to cater
to [1], and scripts/Makefile.build adds the intermediates to 'targets'
to deal with [2].

Barebox had already imported the same code from Linux, so you can use
obj-y instead of obj-dtb-y, like Linux.

pbl-dtb-y and lwl-dtb-y are barebox-specific cases, so I added the
%.dtb.pbl.o pattern to intermediate_targets.

Going forward, please use obj-y, pbl-y, or lwl-y.

Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Signed-off-by: Sascha Hauer <s.hauer@pengutronix.de>

kbuild: rename pbl object pbl-*.o to *.pbl.o ...

Currently, pbl objects are output to:

  <directory-path>/pbl-<basename>.o

This commit changes as follows:

  <directory-path>/<basename>.pbl.o

The motivation is not only to get rid of the ugly code introduced by
commit 257abdaa36c8 ("Do not rm the path from pbl-y target"), but also
to make it easier to remove obj-dtb-y, pbl-dtb-y, lwl-dtb-y syntax in
the next commit.

Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Signed-off-by: Sascha Hauer <s.hauer@pengutronix.de>

kbuild: resync top Makefile, mkmakefile, tags.sh with Linux 5.7-rc4 ...

The prologue of the build system is not specific to Linux or Barebox.
Paritially resync with Linux 5.7-rc4

- In Linux, $(objtree) is always '.', and $(srctree) might be relative.
  I am keeping both absolute for now to avoid unexpected breakage.

- I did not resync single target for now. We need to touch
  Makefile.build a lot if we want to resync it.

- 'export CDPATH=' is Barebox-only code, which was added by
  commit 6cc8d0544658 ("Makefile: disable CDPATH"). Keep it.

- KBUILD_SRC no longer exists in the upstream Linux. I replaced it
  with building_out_of_srctree.

- scripts/mkmakefile does not contain anything specific to Linux or
  Barebox. Resync it with Linux 5.7-rc4

- scripts/tags was imported from Linux 3.6 with some cherry-picks on
  top of that. Resync it with Linux 5.7-rc4

Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Signed-off-by: Sascha Hauer <s.hauer@pengutronix.de>

kbuild: prefix compiler flag variables with KBUILD_ ...

In old days, Linux also used to use well-known variables such CFLAGS,
CPPFLAGS, etc.

They were prefixed with KBUILD_ presumably for preventing users from
overriding them accidentally.

Rename as follows:
  CFLAGS          ->  KBUILD_CFLAGS
  AFLAGS          ->  KBUILD_AFLAGS
  CPPFLAGS        ->  KBUILD_CPPFLAGS
  LDFLAGS         ->  KBUILD_LDFLAGS
  HOSTCFLAGS      ->  KBUILD_HOSTCFLAGS
  HOSTCXXFLAGS    ->  KBUILD_HOSTCXXFLAGS
  HOSTLDFLAGS     ->  KBUILD_HOSTLDFLAGS
  HOST_LOADLIBES  ->  KBUILD_HOSTLDLIBS

HOSTCFLAGS, HOSTCXXFLAGS, HOSTLDFLAGS, HOSTLDLIBS are re-used to allow
users to pass-in additional flags to the host compiler.

Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Signed-off-by: Sascha Hauer <s.hauer@pengutronix.de>

kbuild: support ccflags-y, asflags-y, cppflags-y, and ldflags-y ...

Some Makefiles use ccflags-y, but it is not actually supported.
Support ccflags-y, asflags-y, cppflags-y, and ldflags-y like Linux.

Remove the workaround in drivers/pci/Makefile.

Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Signed-off-by: Sascha Hauer <s.hauer@pengutronix.de>

kbuild: update flex and bison rules to Linux 5.7-rc1 ...

Import the following commits from Linux, and re-sync.

- 6ba7dc6616ce
  "kbuild: make bison create C file and header in a single pattern rule"

- cf8dfd15e5fb
  "kbuild: move flex and bison rules to Makefile.host"

Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Signed-off-by: Sascha Hauer <s.hauer@pengutronix.de>

kbuild: rename hostprogs-y/always to hostprogs/always-y ...

[ Linux commit 5f2fb52 ]

In old days, the "host-progs" syntax was used for specifying host
programs. It was renamed to the current "hostprogs-y" in 2004.

It is typically useful in scripts/Makefile because it allows Kbuild to
selectively compile host programs based on the kernel configuration.

This commit renames like follows:

  always       ->  always-y
  hostprogs-y  ->  hostprogs

So, scripts/Makefile will look like this:

  always-$(CONFIG_BUILD_BIN2C) += ...
  always-$(CONFIG_KALLSYMS)    += ...
      ...
  hostprogs := $(always-y) $(always-m)

I think this makes more sense because a host program is always a host
program, irrespective of the kernel configuration. We want to specify
which ones to compile by CONFIG options, so always-y will be handier.

The "always", "hostprogs-y", "hostprogs-m" will be kept for backward
compatibility for a while.

Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Signed-off-by: Sascha Hauer <s.hauer@pengutronix.de>

kbuild: change *FLAGS_<basetarget>.o to take the path relative to $(obj) ...

[ Linux commit 54b8ae6 ]

Kbuild provides per-file compiler flag addition/removal:

  CFLAGS_<basetarget>.o
  CFLAGS_REMOVE_<basetarget>.o
  AFLAGS_<basetarget>.o
  AFLAGS_REMOVE_<basetarget>.o
  CPPFLAGS_<basetarget>.lds
  HOSTCFLAGS_<basetarget>.o
  HOSTCXXFLAGS_<basetarget>.o

The <basetarget> is the filename of the target with its directory and
suffix stripped.

This syntax comes into a trouble when two files with the same basename
appear in one Makefile, for example:

  obj-y += foo.o
  obj-y += dir/foo.o
  CFLAGS_foo.o := <some-flags>

Here, the <some-flags> applies to both foo.o and dir/foo.o

The real world problem is:

  scripts/kconfig/util.c
  scripts/kconfig/lxdialog/util.c

Both files are compiled into scripts/kconfig/mconf, but only the
latter should be given with the ncurses flags.

It is more sensible to use the relative path to the Makefile, like this:

  obj-y += foo.o
  CFLAGS_foo.o := <some-flags>
  obj-y += dir/foo.o
  CFLAGS_dir/foo.o := <other-flags>

At first, I attempted to replace $(basetarget) with $*. The $* variable
is replaced with the stem ('%') part in a pattern rule. This works with
most of cases, but does not for explicit rules.

For example, arch/ia64/lib/Makefile reuses rule_as_o_S in its own
explicit rules, so $* will be empty, resulting in ignoring the per-file
AFLAGS.

I introduced a new variable, target-stem, which can be used also from
explicit rules.

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Signed-off-by: Sascha Hauer <s.hauer@pengutronix.de>

Merge branch 'for-next/rsa'

Merge branch 'for-next/of-overlay'

rsa: Allow to directly compile in rsa public keys ...

So far we relied on the U-Boot mkimage tool to generate us device tree
snippets containing rsa public keys which we then compiled into barebox.
Make this easier and allow to directly specify a filename or PKCS#11 URI
in Kconfig. With this we no longer need the U-Boot mkimage tool here and
no more external steps to prepare device tree snippets.

With this rsa public keys can be directly compiled as C structs into
barebox which is much more direct than putting it into the device tree.

Signed-off-by: Sascha Hauer <s.hauer@pengutronix.de>

ARM: Compile dtbs for lowlevel code ...

When no PBL is used then the DTB is used in the real barebox binary. We
already have lwl-y for code which needs to be in PBL when existing and
in the real barebox binary otherwise. Do the same for the DTBs and
compile them with lwl-dtb-y.

Signed-off-by: Sascha Hauer <s.hauer@pengutronix.de>

dtc: optionally add add __symbols__ to build-in devicetree ...

The devicetree overlay driver requires the __symbols__ node to resolve
phandles to the base devicetree. If Barebox has to apply the overlay to
the live devicetree, the build-in devicetree must be built with the
__symbols__ node.

It is configurable, because adding __symbols__ significantly increases
the size of the devicetree binary. When configuring Barebox, a developer
should be able to enable or disable devicetree overlay support and as
this only affects the build-in devicetree, which is linked to the image
anyway, use a Kconfig item for configuring the __symbols__.

Signed-off-by: Michael Tretter <m.tretter@pengutronix.de>
Signed-off-by: Sascha Hauer <s.hauer@pengutronix.de>

common: add generic CONFIG_UBSAN plumbing ...

-fsanitize=undefined allows compile-time instrumentation of code to detect
some classes of runtime undefined behavior.

In preparation for allowing arches to provide infrastructure in support
of this feature, add some generic UBSAN options and associated plumbing.

These are only shown in the debug menu when the arch selects the
appropriate symbol. The option is named equally to their Linux counterparts.

Signed-off-by: Ahmad Fatoum <ahmad@a3f.at>
Signed-off-by: Sascha Hauer <s.hauer@pengutronix.de>

images: add support for generating STM32MP images ...

Both STM32MP BootROM and TF-A first stage expect subsequent bootloader
stages to feature a specific STM32 file header. Add a stm32image
utility to address this.

Signed-off-by: Marco Felsch <marco.felsch@gmail.com>
Signed-off-by: Ahmad Fatoum <a.fatoum@pengutronix.de>
Signed-off-by: Sascha Hauer <s.hauer@pengutronix.de>

images/mvebu: Convert Armada machines to mvebuimg ...

The results are bytewise identical apart from armada-xp-db where the
(AFAIK unused) parameters for the binary header are different (i.e.
0000005b 00000000 instead of 0000005b 00000068 for all other boards).

The advantage is that the build system now knows about the binary
headers and changes to them result in a rebuild of the images.

Signed-off-by: Uwe Kleine-König <u.kleine-koenig@pengutronix.de>
Signed-off-by: Sascha Hauer <s.hauer@pengutronix.de>

kbuild: Use ls(1) instead of stat(1) to obtain file size ...

stat(1) is not standardized and different implementations have their own
(conflicting) flags for querying the size of a file.

ls(1) provides the same information (value of st.st_size) in the 5th
column, except when the file is a character or block device. This output
is standardized[0]. The -n option turns on -l, which writes lines
formatted like

  "%s %u %s %s %u %s %s\n", <file mode>, <number of links>,
      <owner name>, <group name>, <size>, <date and time>,
      <pathname>

but instead of writing the <owner name> and <group name>, it writes the
numeric owner and group IDs (this avoids /etc/passwd and /etc/group
lookups as well as potential field splitting issues).

The <size> field is specified as "the value that would be returned for
the file in the st_size field of struct stat".

To avoid duplicating logic in several locations in the tree, create
scripts/file-size.sh and update callers to use that instead of stat(1).

[0] http://pubs.opengroup.org/onlinepubs/9699919799/utilities/ls.html#tag_20_73_10

Signed-off-by: Michael Forney <forney@google.com>
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
[afa: imported script and adjusted barebox stat(1) callsites]
Signed-off-by: Ahmad Fatoum <a.fatoum@pengutronix.de>
Signed-off-by: Sascha Hauer <s.hauer@pengutronix.de>

ARM: i.MX: Add missing quotes to HAB specific defines ...

The Freescale Code Signing Tool (cst) needs quotes around the filenames
passed in the csf file. Add these quotes when the CONFIG_HAB* variables
are passed from the command line.

Signed-off-by: Sascha Hauer <s.hauer@pengutronix.de>

Merge branch 'for-next/layerscape'

dtc: disable noisy warnings ...

Disable the following warnings from dtc:

-Wno-alias_paths
-Wno-graph_child_address
-Wno-simple_bus_reg
-Wno-unique_unit_address

This syncs the warning flags with the Kernel.

Signed-off-by: Sascha Hauer <s.hauer@pengutronix.de>

pbl multiimage: Allow to check image sizes ...

PBL images are often constrained in size by limitations exposed by
the SoCs SRAM size or partition sizes on the boot device. So far
we tried to configure these limits in Kconfig, but with PBL multi
images and thus different limitations for the different supported
images this no longer works. This patch has another approach for
it:

During build time make variables containing the relevant sizes for
each image are created. These are:

PBL_CODE_SIZE_$(symbol)
PBL_MEMORY_SIZE_$(symbol)
PBL_IMAGE_SIZE_$(symbol)

PBL_CODE_SIZE_$(symbol) contains the pure code size of the PBL, it
should be smaller than the available SRAM during boot. Normally the
PBL's bss segment also needs to be in the initial SRAM, for this
case PBL_MEMORY_SIZE_$(symbol) is the relevant variable.
PBL_IMAGE_SIZE_$(symbol) contains the full size of the PBL image
including the compressed payload (but without any image headers
created later by SoC specific image tools).

$(symbol) is a placeholder for the start symbol used for this PBL image,
thus for the i.MX53 QSB with entry start_imx53_loco

PBL_CODE_SIZE_start_imx53_loco

will be created. The images/Makefile.* can use these variables directly
to check sizes or specify the same variables with a "MAX_" prefix. So
when images/Makefile.imx specifies

MAX_PBL_CODE_SIZE_start_imx53_loco = 0x10000

then the build system will make sure that the PBL code for the QSB will
not get bigger than 64KiB.

Also included in this patch are the size restrictions for the i.MX8MQ
images as an example how to use this.

Signed-off-by: Sascha Hauer <s.hauer@pengutronix.de>

defaultenv: Fix dependencies ...

The defaultenv should be rebuilt once a file in it has changed.
the genenv script always generates the environment file to a temporary
file. Only if it has changed to the last target file the temporary file
is moved over the target file. This means we always have to call genenv,
thus replace "if_changed" with "cmd".
With this dependencies are correctly tracked. New or changed files
result in new image builds whereas unchanged environments do not
unnecessarily result in new images.

Signed-off-by: Sascha Hauer <s.hauer@pengutronix.de>

kbuild: cherry-pick changes from Linux v5.0-rc3 ...

Not synced completely.

Just cherry-picking low-hanging fruits.

  0b35786d77ba  kbuild: call make once for all targets when O=.. is used
  5e54d5e5fbc1  kbuild: kill EXTRA_ARFLAGS
  836caba77c29  kbuild: kill backward compatibility checks
  3156fd0529b5  kbuild: fix some minor typoes
  b8b0618cf6fa  kbuild: remove extra ifdef/endif of top Makefile
  16f890988114  kbuild: Remove reference to uninitialised variable
  9319f4539c18  kbuild: support simultaneous "make %config" and "make all"
  9d5db8949f1e  scripts/Makefile.clean: clean also $(extra-m) and $(extra-)
  a4954fd7724c  kbuild: remove obj-n and lib-n handling
  371fdc77af44  kbuild: collect shorthands into scripts/Kbuild.include
  a29b82326ed4  kbuild: Remove duplicate $(cmd) definition in Makefile.clean
  a1e7b7bb1ab5  Makefile: sort list of defconfig targets in make help output
  34948e0bbf98  kbuild: Drop support for clean-rule
  4218affdf57f  kbuild: remove warning about "make depend"
  9c8fa9bc08f6  kbuild: fix if_change and friends to consider argument order
  39a33ff80a25  kbuild: remove cc-option-align
  a7f924190924  kbuild: add %.dtb.S and %.dtb to 'targets' automatically
  54a702f70589  kbuild: mark $(targets) as .SECONDARY and remove .PRECIOUS markers
  9564a8cf422d  Kbuild: fix # escaping in .cmd files for future Make
  bd412d81b7ea  kbuild: .PHONY is not a variable, but PHONY is
  6916162c7308  kbuild: remove duplicated comments about PHONY
  d6c6ab93e17f  kbuild: remove deprecated host-progs variable
  7d0ea2524202  kbuild: use 'else ifeq' for checksrc to improve readability
  3f80babd9ca4  kbuild: remove unused cc-fullversion variable
  bd352a739fde  kbuild: remove unused baseprereq
  b421b8a6cb87  kbuild: remove unused archmrproper

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Signed-off-by: Sascha Hauer <s.hauer@pengutronix.de>

kbuild: remove unnecessary LEX_PREFIX and YACC_PREFIX ...

Because the latest Kconfig uses the default 'yy' prefix, this is
no longer unneeded.

Based on Linux commit eea199b445f6 ("kbuild: remove unnecessary
LEX_PREFIX and YACC_PREFIX").

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Signed-off-by: Sascha Hauer <s.hauer@pengutronix.de>

kbuild: generate lexer and parser during build instead of shipping ...

Recent kernel versions run flex and bison to generate lexers and
parsers from real source files such as *.l and *.y . This provides
better maintainability than version-controlling pre-generated C files
with a "_shipped" suffix.

This commit imports flex and bison rules from Linux, and deletes
pre-generated parsers and lexers.

Refer to the following commits in Linux:

 - 033dba2ec06c ("kbuild: prepare to remove C files pre-generated by
   flex and bison")

 - 29c833061c1d ("kconfig: generate lexer and parser during build
   instead of shipping")

 - e039139be8c2 ("scripts/dtc: generate lexer and parser during
   build instead of shipping")

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Signed-off-by: Sascha Hauer <s.hauer@pengutronix.de>