Boot failure on s390 with llvm-objcopy

[nathanchance]

I have been testing the s390 llvm-objcopy pull request, which results in no new build warnings or errors on next-20240215:

$ make -skj"$(nproc)" ARCH=s390 LLVM=1 OBJDUMP=s390x-linux-gnu-objdump mrproper defconfig all

However, the resulting kernel image does not boot:

$ boot-qemu.py -a s390 -k .
...
Linux version 6.8.0-rc4-next-20240215 (nathan@dev-fedora.aadp) #1 SMP Thu Feb 15 19:50:28 MST 2024
Out of memory allocating 6d7800 bytes 8 aligned in range 0:20000000
Reserved memory ranges:
0000000000000000 a394c3c30d90cdaf DECOMPRESSOR
Usable online memory ranges (info source: sclp read info [3]):
0000000000000000 0000000020000000
Usable online memory total: 20000000 Reserved: a394c3c30d90cdaf Free: 0
Call Trace:
(sp:0000000000033e90 [<0000000000012fbc>] physmem_alloc_top_down+0x5c/0x104)
 sp:0000000000033f00 [<0000000000011d56>] startup_kernel+0x3a6/0x77c
 sp:0000000000033f60 [<00000000000100f4>] startup_normal+0xd4/0xd4


 -- System halted

With OBJCOPY=s390x-linux-gnu-objcopy, there are no issues when booting:

$ boot-qemu.py -a s390 -k .
...

KASLR disabled: CPU has no PRNG
KASLR disabled: CPU has no PRNG
[    0.851887] Linux version 6.8.0-rc4-next-20240215 (nathan@dev-fedora.aadp) (ClangBuiltLinux clang version 19.0.0git (https://github.com/llvm/llvm-project db4ea21dfde307f4fc873a6fe56791cd7ae3f0a1), ClangBuiltLinux LLD 19.0.0) #1 SMP Thu Feb 15 19:53:34 MST 2024
[    0.854515] setup: Linux is running under KVM in 64-bit mode
[    0.878762] setup: The maximum memory size is 512MB
[    0.879514] setup: Relocating AMODE31 section of size 0x00003000
...

Looking at the files that are generated by objcopy during the build, I notice that arch/s390/boot/info.bin is different between the two builds...

$ file {gnu,llvm}-objcopy/arch/s390/boot/info.bin
gnu-objcopy/arch/s390/boot/info.bin:  data
llvm-objcopy/arch/s390/boot/info.bin: empty

If I take a directory built with OBJCOPY=llvm-obcopy and move the arch/s390/boot/info.bin from a directory built with OBJCOPY=s390x-linux-gnu-objcopy and run make again, the resulting image boots.

Looking at the build logs, info.bin is generated from vmlinux with

llvm-objcopy -O binary --only-section=.vmlinux.info --set-section-flags .vmlinux.info=load vmlinux arch/s390/boot/info.bin

$ llvm-readelf --section-headers vmlinux | rg -F .vmlinux.info
  [42] .vmlinux.info     PROGBITS        0000000000000000 1d45000 000078 00   W  0   0  1

$ llvm-objcopy -O binary --only-section=.vmlinux.info --set-section-flags .vmlinux.info=load vmlinux llvm-info.bin

$ s390x-linux-gnu-objcopy -O binary --only-section=.vmlinux.info --set-section-flags .vmlinux.info=load vmlinux gnu-info.bin

$ file {gnu,llvm}-info.bin
gnu-info.bin:  data
llvm-info.bin: empty

This was done initially as part of https://git.kernel.org/linus/369f91c374514f9491d52fec12f7ee9ef6d44b23.

cc @MaskRay and @uweigand (even though this doesn't seem like anything specific to s390 but just in case)

[uweigand]

I had a quick look, and it seems llvm-objcopy with -O binary only outputs sections marked as ALLOC. Due the --sec-section-flags argument, the flags of .vmlinux.info are set to LOAD only, which means the ALLOC flag is removed.

Now, I'm not quite sure what LOAD without ALLOC even means exactly. (ALLOC without LOAD is used for .bss, ALLOC and LOAD together are used for sections mapped at run-time, and neither ALLOC nor LOAD is used for debug and similar sections.)

Should the kernel Makefile use --set-section-flags .vmlinux.info=alloc,load instead here?

[nathanchance]

Should the kernel Makefile use --set-section-flags .vmlinux.info=alloc,load instead here?

Yeah, seems entirely reasonable to me, especially since I see no visible difference with GNU objcopy.

Patch submitted: https://lore.kernel.org/20240216-s390-fix-boot-with-llvm-objcopy-v1-1-0ac623daf42b@kernel.org/

[nathanchance]

Patch accepted: https://git.kernel.org/s390/c/9ea30fd166e9b869546059f714beab78cb150b11