Restarting after update...and UEFI

[deemon]

all of you who claim, that you have zero problems here… maybe for you the fallback option still works perfectly and you have no idea, that the actual main option doesn’t even exist half of the time? … just saying.

[deemon]

@soundofthunder

The files on the $ESP are not affected by a BIOS update; only the reference to them in BIOS can sometimes be lost – considering you just replaced the entire BIOS firmware it’s understandable that it might not automatically exist – typically, the BIOS/mainboard manufacturer is responsible for how well the process works, or doesn’t.

Exactly that! And if there is no reference the UEFI still has some mechanisms to detect bootable media. How well or sophiscated those mechanisms work is anyones guess, but one would assume, if the boot media has STANDARD solution (EFI/boot/bootx64.efi for example) is picks it up 99.9% of the time. If it has some semi-weird approach, which requires the boot installer generate boot files into the UEFI variables (I assume it’s in the /sys/firmware/efi/efivars/ folder) for anything to find it … it’s robustness is less reliable. Some manufacturers custom UEFI’s might just scan the $ESP for any *.efi files (Exactly like manjaro ISO “detect EFI bootloaders” does it), but it’s not a standard and doesn’t always work this well.

@soundofthunder

There are ways to generate the entries again - I believe some were already mentioned in this thread. There’s also plenty to be found on the topic with a forum Search.

and the problem is, that Manjaro default installation relies on those entries there heavily, instead of going the standard single boot option.

At least that is how it looks to me.

Just looking my Ubuntu laptop efi folder. they also have BOOT + ubuntu actually… but their ubuntu folder has extra files in them, one of which is BOOTX64.CSV with content that points to shimx64.efi … and also some grub.cfg which helps to locate the correct disk?
AND in ubuntu, they have automatically copied the /efi/ubuntu/shimx64.efi to /BOOT/BOOTX64.EFI on the grub-install … they have identical date and time and md5sum. I think this is the main thing why Ubuntu doesn’t fail like Manjaro does.

[Aragorn]

@deemon

one would assume, if the boot media has STANDARD solution (EFI/boot/bootx64.efi for example) is picks it up 99.9% of the time. If it has some semi-weird approach, which requires the boot installer generate boot files into the UEFI variables (I assume it’s in the /sys/firmware/efi/efivars/ folder) for anything to find it …

No, EFI variables are stored in the firmware’s non-volatile memory, also referred to as the CMOS memory. They are not stored on disk.

/sys/firmware/efi/efivars/ is a virtual filesystem — its contents represent information about the hardware that the kernel exports to userspace in the form of directories and files.

/boot/efi/EFI/boot/bootx64.efi is not an EFI variable. It’s a 64-bit boot loader — usually GRUB, but it could be something else altogether — in the form of an EFI extension, so that the EFI firmware can load and execute it. This is the opposite of a legacy GRUB, which is just i8086 16-bit real-mode code.

[deemon]

@Aragorn

No, EFI variables are stored in the firmware’s non-volatile memory, also referred to as the CMOS memory. They are not stored on disk.

/sys/firmware/efi/efivars/ is a virtual filesystem — its contents represent information about the hardware that the kernel exports to userspace in the form of directories and files.

I think you are mistaken in the second paragraph. Correct would be:
“/sys/firmware/efi/efivars/ is a virtual filesystem — that links to the motherboards NVRAM efi configuration folder or something”

@Aragorn

/boot/efi/EFI/bootx64.efi is not an EFI variable. It’s a 64-bit boot loader — usually GRUB, but it could be something else altogether — in the form of an EFI extension, so that the EFI firmware can load and execute it. This is the opposite of a legacy GRUB, which is just i8086 16-bit real-mode code.

nobody said, that it’s efi variable. I am saying, it’s standard file with standard location and name.
https://uefi.org/specs/UEFI/2.10/03_Boot_Manager.html?highlight=bootx64%20efi
Whereas Manjaro/grubx64.efi definately is not standard (and therefore needs some pointer to it or uefi to have some kind of extra search functionality built in).

Sadly just about now found this good explanation in second post there, which pretty much tells almost the same story I had assumed until now:
https://superuser.com/questions/1662123/how-does-uefi-decide-whoch-efi-file-to-boot-into

Let me quote the important part from there:

And if you’re booting from removable media (which the NVRAM of course won’t know about) – or if you’re booting from an internal disk, but none of the BootOrder options worked – the firmware will then look for one specific path in each EFI System Partition – always \EFI\Boot\BootX64.efi on x86_64 systems. (Section “3.5.1.1 Removable Media Boot Behavior” lists names for each CPU architecture.)

Most OS installers place a copy of their bootloader at \EFI\Boot\BootX64.efi to make sure the system still boots into something even if the NVRAM data is lost.

And the second paragraph from this quote is my problem with Manjaro, which obviously doesn’t do that, because it’s “undesirable” for couple of people… … I mean, it initially did on first install? But there is no mechanism or anything that will refresh or update this bootx64.efi when it gets updated for Manjaro/grub itself. I can only assume once again, that this bootx64.efi actually (if it was originally some grubx64.efi copy) that it has to actually match the executables and configurations and stuff in /boot/grub folder and when it doesn’t (I mean when the contents in grub folder gets updated to newer version on grub upgrade to newer version) and the bootx64.efi isn’t, then they don’t match and kinda breaks the system?

Is there some way or option somewhere to restore this grub-install functionality, that it will make this copy into BOOT/BOOTX64.EFI every time I update grub itself??

[deemon]

@deemon

Is there some way or option somewhere to restore this grub-install functionality, that it will make this copy into BOOT/BOOTX64.EFI every time I update grub itself??

hah! Found myself semi ok solution…

sudo grub-install --removable

Doesn’t make a copy, just installs the grub to EFI/BOOT/BOOTX64.EFI instead of EFI/Manjaro/grubx64.efi. So I can just easily update it manually to the right location. (Or fix or upgrade also the “fallback option”)

[Aragorn]

@deemon

@Aragorn

/sys/firmware/efi/efivars/ is a virtual filesystem — its contents represent information about the hardware that the kernel exports to userspace in the form of directories and files.

I think you are mistaken in the second paragraph. Correct would be:
“/sys/firmware/efi/efivars/ is a virtual filesystem — that links to the motherboards NVRAM efi configuration folder or something”

Well, no. All information under /sys comes straight from the kernel, and the kernel in turn gets its information about the hardware and firmware from the UEFI, or — on legacy-booting systems — from the 16-bit real-mode code in the initramfs that accesses the firmware through legacy BIOS calls. But in the latter case, there are no efivars, of course.

@deemon

@Aragorn

/boot/efi/EFI/bootx64.efi is not an EFI variable. It’s a 64-bit boot loader — usually GRUB, but it could be something else altogether — in the form of an EFI extension, so that the EFI firmware can load and execute it. This is the opposite of a legacy GRUB, which is just i8086 16-bit real-mode code.

nobody said, that it’s efi variable. I am saying, it’s standard file with standard location and name.
https://uefi.org/specs/UEFI/2.10/03_Boot_Manager.html?highlight=bootx64%20efi
Whereas Manjaro/grubx64.efi definately is not standard (and therefore needs some pointer to it or uefi to have some kind of extra search functionality built in).

Okay, let’s pick this apart.

First of all, you have /boot/efi/EFI/boot/bootx64.efi, and then there is /boot/efi/EFI/manjaro/grubx64.efi.

That latter one is the EFI executable that will be invoked if you directly choose “Manjaro” from the UEFI boot manager menu. The former one is whatever the default UEFI executable will be for that particular drive, and thus, the one that will be invoked if you select “UEFI OS” from the UEFI boot manager menu.

In other words, /boot/efi/EFI/boot/x64.efi could just as easily be the Microsoft Windows boot loader — and if you dual-boot with MS-Windows, then that’s what Microsoft will always make sure that it is.

@deemon

And if you’re booting from removable media (which the NVRAM of course won’t know about) – or if you’re booting from an internal disk, but none of the BootOrder options worked – the firmware will then look for one specific path in each EFI System Partition – always \EFI\Boot\BootX64.efi on x86_64 systems. (Section “3.5.1.1 Removable Media Boot Behavior” lists names for each CPU architecture.)

Exactly, yes.

@deemon

And the second paragraph from this quote is my problem with Manjaro, which obviously doesn’t do that, because it’s “undesirable” for couple of people… … I mean, it initially did on first install?

Yes, it did that on my system as well — I do have a /boot/efi/EFI/boot/bootx64.efi on my system.

And if you run install-grub — it’s a separate package from the repo, and not to be confused with grub-install — then Manjaro will make sure that this file exists upon every major update.

@deemon

Is there some way or option somewhere to restore this grub-install functionality, that it will make this copy into BOOT/BOOTX64.EFI every time I update grub itself??

sudo pacman -Syu install-grub

---

@deemon

hah! Found myself semi ok solution…

sudo grub-install --removable

Yes, that’s the manual way of doing the same thing.

[deemon]

@Aragorn

sudo pacman -Syu install-grub

Except it doesn’t… it still just upgrades EFI/Manjaro/grubx64.efi and not EFI/BOOT/BOOTX64.EFI. At least by default. And I can’t seem to find any documentation for this.

    /boot/efi/EFI  sudo install-grub                                                                                              2 ✘
Grub will be installed on: EFI
Installing for x86_64-efi platform.
Installation finished. No error reported.
Generating grub configuration file ...
Found theme: /usr/share/grub/themes/manjaro/theme.txt
Found linux image: /boot/vmlinuz-6.12-x86_64
Found initrd image: /boot/amd-ucode.img /boot/initramfs-6.12-x86_64.img
Found initrd fallback image: /boot/initramfs-6.12-x86_64-fallback.img
Found linux image: /boot/vmlinuz-6.6-x86_64
Found initrd image: /boot/amd-ucode.img /boot/initramfs-6.6-x86_64.img
Found initrd fallback image: /boot/initramfs-6.6-x86_64-fallback.img
Found linux image: /boot/vmlinuz-6.1-x86_64
Found initrd image: /boot/amd-ucode.img /boot/initramfs-6.1-x86_64.img
Found initrd fallback image: /boot/initramfs-6.1-x86_64-fallback.img
Warning: os-prober will not be executed to detect other bootable partitions.
Systems on them will not be added to the GRUB boot configuration.
Check GRUB_DISABLE_OS_PROBER documentation entry.
Adding boot menu entry for UEFI Firmware Settings ...
Found memtest86+ EFI image: /boot/memtest86+/memtest.efi
/usr/bin/grub-probe: warning: unknown device type nvme0n1.
done    /boot/efi/EFI  l Manjaro                                                                                                        ✔
drwxr-xr-x      - root  5 juuni  2019  .
drwxr-xr-x      - root  7 jaan  17:23  ..
.rwxr-xr-x 139264 root  7 jaan  18:25  grubx64.efi              <= reinstalled.
    /boot/efi/EFI  l BOOT                                                                                                           ✔
drwxr-xr-x      - root  7 jaan  17:22  .
drwxr-xr-x      - root  7 jaan  17:23  ..
.rwxr-xr-x 139264 root  7 jaan  18:10  BOOTX64.EFI             <= not-reinstalled. :-(
    /boot/efi/EFI  man install-grub                                                                                                 ✔
No manual entry for install-grub

[Aragorn]

@deemon

@Aragorn

sudo pacman -Syu install-grub

Except it doesn’t… it still just upgrades EFI/Manjaro/grubx64.efi and not EFI/BOOT/BOOTX64.EFI. At least by default. And I can’t seem to find any documentation for this.

Perhaps you could put in a feature request? It’s maintained by @philm, so let’s see whether he can offer some comments.

[deemon]

there is in the script variable fallback_is_same=false which should copy it, but even if I edit it to true, it still doesn’t work … so the script is broken?

[Mirdarthos]

@Aragorn

Perhaps you could put in a feature request? It’s maintained by @philm, so let’s see whether he can offer some comments.

I’d say

[deemon]

And where exactly am I supposed to post this request to @philm ?
Who’s even the author of this script? Is it like GNU thing or @philm creation?

[anon33601770]

@deemon

there is in the script variable fallback_is_same=false which should copy it

no - that is not how that script works

@deemon

but even if I edit it to true, it still doesn’t work … so the script is broken?

no, it’s not - look at how that variable is used throughout the script, (try to) understand how it works

@deemon

fallback_is_same=false

that is just initializing the variable which is then later used …

You are drawing the wrong conclusions based upon the variable name.

---

---

@deemon

meanwhile while @philm fixes it or not, will use my pacman hook:

what would you say: what is there to fix - and how ?

[deemon]

meanwhile while @philm fixes it or not, will use my pacman hook:

$cat /etc/pacman.d/hooks/99-upgrade-grub.hook
[Trigger]
Operation = Install
Operation = Upgrade
Type = Package
Target = grub

[Action]
Description = Updating Grub-Fallback option
When = PostTransaction
Exec = /usr/bin/grub-install --removable

to just fix the fallback every time there is grub upgrade.

[deemon]

@anon33601770

no, it’s not - look at how that variable is used throughout the script, (try to) understand how it works

not bash guru, but it seems it fails to find any fallback file. because it looks for lower case filename whereas the filename is in upper case normally (when installed with sudo grub-install --removable).

anyway fixed it myself.
Now where do I bugreport it?

$diff /usr/bin/install-grub install-grub-fixed
127c127
<                 efi_target_file="$(find $path -name $efi_boot_file)"
---
>                 efi_target_file="$(find $path -iname $efi_boot_file)"

Now it does work and replaces the file in EFI/BOOT also, may it be bootx64.efi or BOOTX64.EFI or BOOTx64.EFI or BoOtX.64.eFi

(in the grand scheme of things the filename case should not matter anyhow, because efi partition is in vfat, which usually is treated and mounted as case-insensitive as far as I know.)

[deemon]

@anon33601770

what would you say: what is there to fix - and how ?

check previous post

[anon33601770]

…
find $path -iname
may indeed be better than
find $path -name

although in a pure Arch/Manjaro system, like in my VM, the directory name as well as the file name are both lower case

In my Mint system (real hardware, no dual boot),
both are (appear as) upper case.

As I seem to recall, case should not matter in a vfat file system, which the ESP always is.

Perhaps a change to the mount options of that vfat file system could achieve the same?
Don’t know.
Am not a bash guru myself.

You made it work.
Good!

[Aragorn]

@deemon

And where exactly am I supposed to post this request to @philm ?

We’ve already pinged him several times now on this thread, so he’ll likely take a look at it. And if not, there’s always the Feature Request category.

@deemon

Who’s even the author of this script? Is it like GNU thing or @philm creation?

The latter.

[soundofthunder]

@deemon

Kernel 6.6.65 has worked perfectly those 2 days now. No random weird reboots, no weird hardware error messages … so far so good.

Well, this topic has meandered along it’s merry way since the original Support request. I note that updating the BIOS as suggested has resolved the issues specified by the OP and that both kernel 6.6 (LTS) and kernel 6.12 appear to allow booting without issue:

@deemon

Ou… and didn’t get those journalctl Hardware errors on boot with kernel 6.12 this time …

The appropriate post has been marked as the Solution; and this topic is now closed. Feel free to continue the conversation in the Members area if you wish.

Regards.