Ideas to unify ARM boot sequences and kernel namings

ARM General ARM Discussion
11

And does systemd-boot probe for available kernels, or does it have to be defined somewhere?

Needs manual defining, but should be pretty simple, at least to me, much simpler than grub probing:

https://wiki.archlinux.org/title/systemd-boot

One of the advantage here is, we can add systemd-boot entry for each kernel, thus no need to probe at all, and allowing user to choose which kernel to boot.
Just put something like /boot/loader/entries/linux-rpi4-mainline.conf for linux-rpi4-mainline and /boot/loader/entries/linux.conf for upstream linux package.

Let me re-compile the linux-rpi4-mainline with my uboot-raspberry dependency and test systemd-boot to choose kernels.
As I just want to change the eeprom for my CM4, but upstream doesn’t support the rpi user space tool.

And IIRC for UEFI boot, the Arch install ISO uses systemd-boot by default.

12

Sure, but as I said already. We can’t expect regular users to make these kinds of changes, when they want to try other kernels. So it needs to be generated automatically.

But yeah. If the kernel packages themselves add a loader entry for itself, it should be doable.

Do you know which config options are needed from the kernel to do this and how to get uboot to start systemd-boot?

13

Do you know which config options are needed from the kernel to do this

It’s already in upstream kernel.
And quite some other vendor kernels also have them compiled in already.

The configs are

CONFIG_EFI=y
CONFIG_EFI_STUB=y

and how to get uboot to start systemd-boot?

We can go uboot script, but I want to make it the fallback method if we can’t find anything better.

I see the uboot has some command compiled in to load EFI from /boot/efi/boot/bootaa64.efi, which is exactly the same file installed by systemd-boot.

But I haven’t yet found a solid doc on the boot sequence, I’ll need to do more test to see what’s needed to make it to boot EFI executable by default.
(I hope removing extlinux directory would do the trick, but needs more testing)

14

Great. :slight_smile:

Looking forward to your solution.

15

BTW, currently the most time consuming part is recompiling both linux and linux-rpi4-mainline to get an environment with two kernels to select.

If you guys can use my diff to build test packages, it would save me quite some time.

16

You would only have to compile them both once, to get a test environment.
We also compile “on device”, so it takes us a bit of time as well. Plus, we can’t add them to the repo, since the package names will be the same as existing ones.

17

You would only have to compile them both once, to get a test environment.

That would only be true if the modified linux-rpi4-mainline kernel has enough change to avoid any file conflicts…
Or even with distcc, I still need to wait for one hour to build it.

At least the upstream version linux has been recompiled and booted.

Furthermore, it looks like at least the rpi uboot has the following boot sequence:

- scan_dev_for_extlinux
  The current one for "extlinux.confg".

- scan_dev_for_scripts
  The old uboot script, at least now we don't use that

- scan_dev_for_efi
  What I want.

Thus it just means, if I removed extlinux.conf, it would be enough to load EFI.

18

Just found this documentation on uboot.

It states that:

Since most of our devices are big.little endian, EFI can’t be built for most of our devices. :frowning:
Unless I misinterpret that statement.

And I don’t see

CONFIG_CMD_BOOTEFI=y
CONFIG_EFI_LOADER=y

in any of the uboot defconfigs for our supported boards. Not even the raspberry pi.

19

Since most of our devices are big.little endian

You got it completely wrong. Little endian is the default endian for x86, and most distros go little endian even for arches which support both big and little endian.

Thus it’s not a problem at all, because no one really use big endian at all.

I guess you get confused with BIG.little cores layout, which is completely unrelated to memory endian.

in any of the uboot defconfigs for our supported boards. Not even the raspberry pi.

Because those are Uboot default for almost all boards already.

Just compile one Uboot, and check the .config:

$ pwd
/home/adam/uboot-raspberrypi/src/u-boot-2021.04
$ grep EFI .config
...
CONFIG_EFI_PARTITION=y
CONFIG_EFI_PARTITION_ENTRIES_NUMBERS=128
CONFIG_EFI_PARTITION_ENTRIES_OFF=0
CONFIG_EFI_LOADER=y
CONFIG_CMD_BOOTEFI_BOOTMGR=y
CONFIG_EFI_VARIABLE_FILE_STORE=y
...

Trust me, EFI is more popular than you thought.

20

So what you are saying is, that EFI support should already be in the uboots we build ourselves?

21

Yes.

Just grab one board, boot into Uboot cmdline, check if you have bootefi command.

22

Cool, so what we just need to do, is create a uboot script that launches the EFI file you described here?:

23

Cool, so what we just need to do, is create a uboot script that launches the EFI file you described here?:

In fact, we don’t even need to do that.
Just remove /boot/extlinux/ directly, then uboot with EFI support will try to load the EFI file in the described location.

Although this only works for systemd-boot.

All we really need to do is to update various uboot, linux and vendor kernel packages, to allow them to coexist with each other, without any naming/file conflicts.
And add extra systemd-boot config file for each kernel.

Then, make sure all boards already have their fs using GPT table, and have an ESP partition (normally also used as /boot), which is a requirement for systemd-boot.

Finally, on those boards, run bootctl install --esp-path=/boot --graceful.

Then enjoy the wonder.

There are other small modifications, like the vendor kernels also needs extra hooks for libalpm to update their initramfs.

But for my CM4, I just changed the following packages to allow them co-exists:

  • uboot-raspberry
    Of course, still need all my existing changes to support CM4

  • linux
    Remove the conflicts/provides

  • linux-rpi4-mainline
    Remove the conflicts/provides
    Add rename the kernel/initramfs, add extra kernel string to avoid name conflicts
    Add new mkinitramfs.d preset

For now, I just created the systemd-boot config manually: (Note I’m booting from my NVME driver, and also uses LVM, thus not using the MANJARO partition naming)

$ cat /boot/loader/entries/linux.conf 
title   Manjaro ARM
linux   /Image
initrd  /initramfs-linux.img
options console=ttyAMA0,115200 root=/dev/arm_nvme/root rw rootwait earlycon=pl011,0xfe201000 loglevel=6 audit=0

$ cat /boot/loader/entries/linux-rpi-mainline.conf 
title   Manjaro ARM - rpi kernel
linux   /Image-rpi4-mainline
initrd  /initramfs-rpi-mainline.img
options console=ttyAMA0,115200 root=/dev/arm_nvme/root rw rootwait earlycon=pl011,0xfe201000 loglevel=6 audit=0

Finally I removed /boot/extlinux directly completely.

Then I reboot, uboot starts up and find the EFI file automatically, and boots into systemd-boot, which loads the default one of upstream kernel.

So yes, I tested it with my CM4, and it works as expected.

I’ll upload the PKGBUILDs tomorrow, maybe add those systemd-boot config entries too.
(Why I can’t even fork those projects inside gitlab?)

Full boot sequence can be found here:

24

And switching to linux-rpi4-mainline kernel just needs me to press ‘up’ during systemd-boot screen.

Awesome!

Here is the photo for the selection screen:
(Sorry for the crappy camera and the extra mosaic)

IMG_20210613_213106
IMG_20210613_2131062000×1125 395 KB

1 Like
25

Nice. Looks great.

I do see one issue though.

Since we are thinking of making the kernel packages themselves supply the loader entry conf files, the below example (from the main linux package) would only be usable on the Raspberry Pi, because of the “options” line. Different devices have different console names and baud-rates.

Also, some devices (looking at amlogic) use a different initramfs file name (ending in .uimg). How would we do the conf to only look for that if it exists? :slight_smile:

26

the below example (from the main linux package) would only be usable on the Raspberry Pi, because of the “options” line. Different devices have different console names and baud-rates.

Regular users won’t need those earlycon nor console lines, those are for my debugging purpose.

For end users, just remove the console, earlycon, loglevel options, and call it a day.

(And of course different root option, unless Manjaro ARM moves to LVM by default)

Also, some devices (looking at amlogic) use a different initramfs file name (ending in .uimg).

The maintainer for the package should move to regular filename.
.uimg looks like uboot image, which is completely unnecessary.

From ancient age, U-boot can already load plain kernel/initramfs without any problem.

(Unless it’s the amlogic Uboot too old, then it’s another problem and we need to compile a newer Uboot for such board).

BTW, I also find that, Uboot itself will try its best to load FDT automatically.
Thus under most case, as long as the device tree name is correct, it can load the device tree even without any config file.

27

My Odroid does not boot using the regaular .img file, so it has to be the .uimg file, created with mkimage.

They do use old uboots, since I just use the uboots they provide, since I can’t compile/package mainline for them on my device (because of the x86_64 binary thing).

Most uboots have a default FTD defined at built time, but it differs slightly from where our kernel puts it. That can ofcourse be changed in the kernel.

28

My Odroid does not boot using the regaular .img file, so it has to be the .uimg file, created with mkimage.

Mind to provide the Uboot version, and env print output?

If it’s amlogic based, I’m afraid if it has eMMC, we can’t even try to chainload upstream Uboot…

29

BTW, even for the damn prebuilt Uboot for certain boards, I guess they still load uboot script.
And I believe that’s the reason why you need uboot image, as it’s hardcoded into the uboot script.
And of course you can change the uboot script.

That’s why I prefer to manually use uboot command to verify what’s the real capability of the vendor uboot.

As long as it’s not too ancient, we can still use uboot script to teach it to load systemd-boot (as long as they have EFI built in).

So that we can continue our chain load sequence, while keeps the dirty hacks inside the uboot.

30