Install-grub: a new way to keep your EFI/MBR in-sync with grub package

[Teo]

Just tested, works perfectly fine. Updated the efi/manjaro and didn’t do anything to the fallback, because there is actually no fallback on this ESP (i made another just for manjaro).

[teo@teo-lenovo-v15 ~]$ sudo ls /boot/efi -Rl
[sudo] password for teo: 
/boot/efi:
total 1
drwx------ 3 root root 512 24. Jun 2023   EFI
drwx------ 2 root root 512 25. Jun 2023  'System Volume Information'

/boot/efi/EFI:
total 1
drwx------ 2 root root 512 24. Jun 2023  manjaro

/boot/efi/EFI/manjaro:
total 140
-rwx------ 1 root root 143360 23. Dez 16:38 grubx64.efi

'/boot/efi/System Volume Information':
total 0
[teo@teo-lenovo-v15 ~]$ sudo pacman -S grub
[sudo] password for teo: 
warning: grub-2.12-3 is up to date -- reinstalling
resolving dependencies...
looking for conflicting packages...

Packages (1) grub-2.12-3

Total Download Size:   15,08 MiB
Total Installed Size:  47,86 MiB
Net Upgrade Size:       0,00 MiB

:: Proceed with installation? [Y/n] y
:: Retrieving packages...
 grub-2.12-3-x86_64                        15,1 MiB  15,3 MiB/s 00:01 [--------------------------------------] 100%
(1/1) checking keys in keyring                                        [--------------------------------------] 100%
(1/1) checking package integrity                                      [--------------------------------------] 100%
(1/1) loading package files                                           [--------------------------------------] 100%
(1/1) checking for file conflicts                                     [--------------------------------------] 100%
(1/1) checking available disk space                                   [--------------------------------------] 100%
:: Running pre-transaction hooks...
(1/1) Creating Timeshift snapshot before upgrade...
==> skipping timeshift-autosnap due skipRsyncAutosnap in /etc/timeshift-autosnap.conf set to TRUE.
:: Processing package changes...
(1/1) reinstalling grub                                               [--------------------------------------] 100%
:: Running post-transaction hooks...
(1/4) Arming ConditionNeedsUpdate...
(2/4) Installing Grub to MBR/EFI
Grub will be installed on: EFI
Installing for x86_64-efi platform.
Installation finished. No error reported.
Generating grub configuration file ...
Found theme: /usr/share/grub/themes/manjaro/theme.txt
Found linux image: /boot/vmlinuz-6.6-x86_64
Found initrd image: /boot/intel-ucode.img /boot/initramfs-6.6-x86_64.img
Found initrd fallback image: /boot/initramfs-6.6-x86_64-fallback.img
Found linux image: /boot/vmlinuz-6.1-x86_64
Found initrd image: /boot/intel-ucode.img /boot/initramfs-6.1-x86_64.img
Found initrd fallback image: /boot/initramfs-6.1-x86_64-fallback.img
Warning: os-prober will be executed to detect other bootable partitions.
Its output will be used to detect bootable binaries on them and create new boot entries.
Found Windows Boot Manager on /dev/nvme0n1p1@/efi/Microsoft/Boot/bootmgfw.efi
Adding boot menu entry for UEFI Firmware Settings ...
Root filesystem isn't btrfs
If you think an error has occurred, please file a bug report at "https://github.com/Antynea/grub-btrfs"
Found memtest86+ image: /boot/memtest86+/memtest.bin
/usr/bin/grub-probe: warning: unknown device type nvme0n1.
Found memtest86+ EFI image: /boot/memtest86+/memtest.efi
/usr/bin/grub-probe: warning: unknown device type nvme0n1.
done
(3/4) Checking which packages need to be rebuilt
(4/4) Updating the info directory file...
[teo@teo-lenovo-v15 ~]$ sudo ls /boot/efi -Rl
/boot/efi:
total 1
drwx------ 3 root root 512 24. Jun 2023   EFI
drwx------ 2 root root 512 25. Jun 2023  'System Volume Information'

/boot/efi/EFI:
total 1
drwx------ 2 root root 512 24. Jun 2023  manjaro

/boot/efi/EFI/manjaro:
total 140
-rwx------ 1 root root 143360 25. Dez 16:39 grubx64.efi

'/boot/efi/System Volume Information':
total 0
[teo@teo-lenovo-v15 ~]$ sudo ls /boot/efi -Rl
/boot/efi:
total 1
drwx------ 3 root root 512 24. Jun 2023   EFI
drwx------ 2 root root 512 25. Jun 2023  'System Volume Information'

/boot/efi/EFI:
total 1
drwx------ 2 root root 512 24. Jun 2023  manjaro

/boot/efi/EFI/manjaro:
total 140
-rwx------ 1 root root 143360 25. Dez 16:39 grubx64.efi

'/boot/efi/System Volume Information':
total 0

[soundofthunder]

@philm

No, as $efi_directory/EFI/$bootloader_id/$efi_grub_file is generated via grub-install and needs to been copied over to the UEFI Fallback file …

@Teo

as far as i know the fallback should always be called bootx. It is hardcoded in some more stupid uefis.

Yes, a typo. However, that’s all the more reason to protect any boot file that previously exists; for example, in the case of rEFInd if it has previously been moved to the fallback location.

The option to move rEFInd to the fallback location is a valid option with an established procedure outlined in the rEFInd ArchWiki. This is most useful when multibooting and wishing to retain rEFInd as the default loader (while keeping Grub/Other loaders in the usual defined locations).

Again, this is a scenario where the optional check previously mentioned is valid.

In this scenario the grub boot file could arguably still be copied to the fallback location, but with a version appendum bootx64-grub-2.16 for example.
(However, that can open the door for further complication. What I usually do is keep an additional zipped copy of the rEFInd boot file in case Grub eventually obliterates it.)

@Teo
Stupid UEFIs: Yes, I’m aware of some very flaky implementations; the early Gigabyte DualBIOS UEFI boards, in particular, come to mind.

[philm]

Well, seems Garuda also has some similar in their hooks to update and install grub. There are some more safety checks we may want to adopt: garuda-hooks/garuda-hooks-runner · main · Garuda Linux 🦅 / PKGBUILDs · GitLab

[cscs]

Most recent run seems to finish successfully, but spits warnings first.

$ sudo install-grub
find: ‘/EFI/Manjaro’: No such file or directory
cmp: missing operand after '-s'
cmp: Try 'cmp --help' for more information.
cmp: invalid --ignore-initial value '/efi/EFI/Manjaro/grubx64.efi'
cmp: Try 'cmp --help' for more information.
Grub will be installed on: EFI
Installing for x86_64-efi platform.
Installation finished. No error reported.
Generating grub configuration file ...
Found theme: /usr/share/grub/themes/manjaro/theme.txt
Found linux image: /boot/vmlinuz-linux-amd-znver3
Found initrd image: /boot/amd-ucode.img /boot/initramfs-linux-amd-znver3.img
Found initrd fallback image: /boot/initramfs-linux-amd-znver3-fallback.img
Found linux image: /boot/vmlinuz-6.1-x86_64
Found initrd image: /boot/amd-ucode.img /boot/initramfs-6.1-x86_64.img
Found initrd fallback image: /boot/initramfs-6.1-x86_64-fallback.img
Warning: os-prober will be executed to detect other bootable partitions.
Its output will be used to detect bootable binaries on them and create new boot entries.
Found Windows Boot Manager on /dev/nvme0n1p1@/EFI/Microsoft/Boot/bootmgfw.efi
Adding boot menu entry for UEFI Firmware Settings ...
Found memtest86+ EFI image: /boot/memtest86+/memtest.efi
/usr/bin/grub-probe: warning: unknown device type nvme0n1.
Adding reboot option.
Adding poweroff option.
done

[vfbsilva]

Running the last update breaks my machine with:
.1.6408601 Failed to execute /init (error -2) 3.6423901 Kernel panic - not syncing: No working snit found. Try passing init= option to kernel. See Linux Documentation/admin-guide/init.rst for [ 3.643821] CPU: 5 Pit: 1 Comm: swapper/0 Not tainted 6.1.69-1-MRNJRRO 01 fb174fce6e87ae85fd1a8166263822f41d203eb6 [ 3.643852] Hardware name: 115115 System Product Name/TUF GRMING X670E-PLUS WIFI, BIOS 1813 10/13/2023 [ 3.643876] Call Trace: 3.6438832 <TRSK> [ 3.643890] dump_stack_1v1+0x48/0x60 3.6439031 ? rest_init+Ox90/0xd0 3.6439121 panic+Oxl1b/Ox2ba 3.6439221 ? rest_init.OxdB/OxdO 3.643931] kernel_init+Ox130/0x130 3.6439401 ret_from_fork+Ox22/0x30 3.6439521 </TRSK> 3.6441081 Kernel Offset: 0xe600000 from 8xffffffff81000000 (relocation range: Oxffffffff80000000-Oxffffffffbfffffff) 3.6441461 ---1 end Kernel panic - not syncing: No working init found. Try passing init= option to kernel. See Linux Documentation/admin-guide/init.rst for guidance. 1--

I have not tried the script so far should I test it?

[cscs]

…is your problem related to failing to install grub?

@vfbsilva

No working init found

Maybe … I might think its more mkinitcpio.

The script is really just an attempt at automating the system-dependent string of, ex:

sudo grub-install --target=x86_64-efi --efi-directory=/boot/efi --bootloader-id=Manjaro --recheck

Looks like there is more of a guide here:

[root tip] [How To] Primer on handling a grub package update

How To update the system boot after updating grub itself With the grub update 2023-12-22T23:00:00Z (UTC) to v2.12, it was thus advised to ensure all parts of grub on the system are populated with the new binary and updated configuration. This has revealed that a lot of users have doubts on how to correctly identify if their system needs special handling e.g. identifying whether to use EFI or BIOS commands. This topic is aimed at being a primer to properly assess how to update grub on a syste…

[vfbsilva]

@cscs I really don’t know, updating to the latest kernel results in the error I posted. Running the grub-install line you provide gives me no errors. Grub is installed properly but it fails to generate a valid config to the new kernel it does not find init.

[cscs]

sudo pacman -Syu 
sudo mkinitcpio -P
sudo update-grub

[philm]

@cscs

Most recent run seems to finish successfully, but spits warnings first.

the issue is that you have it installed to /efi/EFI/Manjaro instead of /boot/efi/EFI/Manjaro. I’m still implementing that. Check the latest script: install-grub ($976) · Snippets · GitLab

[cscs]

Well, yeah, I know, that hasnt changed from my initial report.
I fixed it by adding ~5 characters to the script… you told me to try this one, so I did, that was the result.

Trying the newest iteration from the snippet results in total failure.

$ sudo /usr/bin/install-grub2
find: ‘/EFI/Manjaro’: No such file or directory
cmp: missing operand after '-s'
cmp: Try 'cmp --help' for more information.
cmp: invalid --ignore-initial value '/efi/EFI/Manjaro/grubx64.efi'
cmp: Try 'cmp --help' for more information.
WARNING: Grub is not properly installed, system may not be bootable or other bootloader is used

[philm]

@cscs

Trying the newest iteration from the snippet results in total failure.

Are you sure? does your script have:

    for path in /boot /efi; do
        if [[ -e $path ]]; then
            efi_directory="$(find $path -name $efi_grub_file | awk -F '/EFI' '{print $1}')"
            [[ -z $efi_directory ]] && break
            efi_source_file="$(find $efi_directory/EFI/$bootloader_id -name $efi_grub_file)"
            efi_target_file="$(find $path -name $efi_boot_file)"
            cmp -s $efi_source_file $efi_target_file && fallback_is_same=true
            [[ ! -z $efi_directory ]] && break
        fi
    done

cos on your end /boot is empty and it should break and do it with /efi.

[vfbsilva]

sudo mkinitcpio -P
[sudo] senha para vfbsilva: 
==> Building image from preset: /etc/mkinitcpio.d/linux510.preset: 'default'
==> Using configuration file: '/etc/mkinitcpio.conf'
  -> -k /boot/vmlinuz-5.10-x86_64 -c /etc/mkinitcpio.conf -g /boot/initramfs-5.10-x86_64.img
==> Starting build: '5.10.203-1-MANJARO'
  -> Running build hook: [base]
  -> Running build hook: [udev]
  -> Running build hook: [autodetect]
  -> Running build hook: [modconf]
  -> Running build hook: [block]
  -> Running build hook: [keyboard]
  -> Running build hook: [keymap]
  -> Running build hook: [consolefont]
==> WARNING: consolefont: no font found in configuration
  -> Running build hook: [filesystems]
  -> Running build hook: [fsck]
==> Generating module dependencies
==> Creating gzip-compressed initcpio image: '/boot/initramfs-5.10-x86_64.img'
==> Image generation successful
==> Building image from preset: /etc/mkinitcpio.d/linux510.preset: 'fallback'
==> Using configuration file: '/etc/mkinitcpio.conf'
  -> -k /boot/vmlinuz-5.10-x86_64 -c /etc/mkinitcpio.conf -g /boot/initramfs-5.10-x86_64-fallback.img -S autodetect
==> Starting build: '5.10.203-1-MANJARO'
  -> Running build hook: [base]
  -> Running build hook: [udev]
  -> Running build hook: [modconf]
  -> Running build hook: [block]
==> WARNING: Possibly missing firmware for module: 'qed'
==> WARNING: Possibly missing firmware for module: 'wd719x'
==> WARNING: Possibly missing firmware for module: 'qla2xxx'
==> WARNING: Possibly missing firmware for module: 'aic94xx'
==> WARNING: Possibly missing firmware for module: 'bfa'
==> WARNING: Possibly missing firmware for module: 'qla1280'
  -> Running build hook: [keyboard]
  -> Running build hook: [keymap]
  -> Running build hook: [consolefont]
==> WARNING: consolefont: no font found in configuration
  -> Running build hook: [filesystems]
  -> Running build hook: [fsck]
==> Generating module dependencies
==> Creating gzip-compressed initcpio image: '/boot/initramfs-5.10-x86_64-fallback.img'
==> Image generation successful
==> Building image from preset: /etc/mkinitcpio.d/linux61.preset: 'default'
==> Using default configuration file: '/etc/mkinitcpio.conf'
  -> -k /boot/vmlinuz-6.1-x86_64 -g /boot/initramfs-6.1-x86_64.img --microcode /boot/amd-ucode.img
==> Starting build: '6.1.67-2-MANJARO'
  -> Running build hook: [base]
  -> Running build hook: [udev]
  -> Running build hook: [autodetect]
  -> Running build hook: [modconf]
  -> Running build hook: [block]
==> WARNING: Possibly missing firmware for module: 'xhci_pci'
  -> Running build hook: [keyboard]
  -> Running build hook: [keymap]
  -> Running build hook: [consolefont]
==> WARNING: consolefont: no font found in configuration
  -> Running build hook: [filesystems]
  -> Running build hook: [fsck]
==> Generating module dependencies
==> Creating gzip-compressed initcpio image: '/boot/initramfs-6.1-x86_64.img'
==> Image generation successful
==> Building image from preset: /etc/mkinitcpio.d/linux61.preset: 'fallback'
==> Using default configuration file: '/etc/mkinitcpio.conf'
  -> -k /boot/vmlinuz-6.1-x86_64 -g /boot/initramfs-6.1-x86_64-fallback.img -S autodetect --microcode /boot/amd-ucode.img
==> Starting build: '6.1.67-2-MANJARO'
  -> Running build hook: [base]
  -> Running build hook: [udev]
  -> Running build hook: [modconf]
  -> Running build hook: [block]
==> WARNING: Possibly missing firmware for module: 'qed'
==> WARNING: Possibly missing firmware for module: 'qla2xxx'
==> WARNING: Possibly missing firmware for module: 'aic94xx'
==> WARNING: Possibly missing firmware for module: 'bfa'
==> WARNING: Possibly missing firmware for module: 'qla1280'
==> WARNING: Possibly missing firmware for module: 'wd719x'
==> WARNING: Possibly missing firmware for module: 'xhci_pci'
  -> Running build hook: [keyboard]
  -> Running build hook: [keymap]
  -> Running build hook: [consolefont]
==> WARNING: consolefont: no font found in configuration
  -> Running build hook: [filesystems]
  -> Running build hook: [fsck]
==> Generating module dependencies
==> Creating gzip-compressed initcpio image: '/boot/initramfs-6.1-x86_64-fallback.img'
==> Image generation successful
==> Building image from preset: /etc/mkinitcpio.d/linux66.preset: 'default'
==> Using default configuration file: '/etc/mkinitcpio.conf'
  -> -k /boot/vmlinuz-6.6-x86_64 -g /boot/initramfs-6.6-x86_64.img --microcode /boot/amd-ucode.img
==> Starting build: '6.6.8-2-MANJARO'
  -> Running build hook: [base]
  -> Running build hook: [udev]
  -> Running build hook: [autodetect]
  -> Running build hook: [modconf]
  -> Running build hook: [block]
==> WARNING: Possibly missing firmware for module: 'xhci_pci'
  -> Running build hook: [keyboard]
  -> Running build hook: [keymap]
  -> Running build hook: [consolefont]
==> WARNING: consolefont: no font found in configuration
  -> Running build hook: [filesystems]
  -> Running build hook: [fsck]
==> Generating module dependencies
==> Creating gzip-compressed initcpio image: '/boot/initramfs-6.6-x86_64.img'
==> Image generation successful
==> Building image from preset: /etc/mkinitcpio.d/linux66.preset: 'fallback'
==> Using default configuration file: '/etc/mkinitcpio.conf'
  -> -k /boot/vmlinuz-6.6-x86_64 -g /boot/initramfs-6.6-x86_64-fallback.img -S autodetect --microcode /boot/amd-ucode.img
==> Starting build: '6.6.8-2-MANJARO'
  -> Running build hook: [base]
  -> Running build hook: [udev]
  -> Running build hook: [modconf]
  -> Running build hook: [block]
==> WARNING: Possibly missing firmware for module: 'qed'
==> WARNING: Possibly missing firmware for module: 'qla2xxx'
==> WARNING: Possibly missing firmware for module: 'aic94xx'
==> WARNING: Possibly missing firmware for module: 'bfa'
==> WARNING: Possibly missing firmware for module: 'qla1280'
==> WARNING: Possibly missing firmware for module: 'wd719x'
==> WARNING: Possibly missing firmware for module: 'xhci_pci'
  -> Running build hook: [keyboard]
  -> Running build hook: [keymap]
  -> Running build hook: [consolefont]
==> WARNING: consolefont: no font found in configuration
  -> Running build hook: [filesystems]
  -> Running build hook: [fsck]
==> Generating module dependencies
==> Creating gzip-compressed initcpio image: '/boot/initramfs-6.6-x86_64-fallback.img'
==> Image generation successful
(base) vfbsilva@isengard ~ $ sudo update-grub
Gerando o arquivo de configuração do grub...
Tema encontrado: /usr/share/grub/themes/manjaro/theme.txt
Imagem Linux encontrada: /boot/vmlinuz-6.6-x86_64
Imagem initrd encontrada: /boot/amd-ucode.img /boot/initramfs-6.6-x86_64.img
Found initrd fallback image: /boot/initramfs-6.6-x86_64-fallback.img
Imagem Linux encontrada: /boot/vmlinuz-6.1-x86_64
Imagem initrd encontrada: /boot/amd-ucode.img /boot/initramfs-6.1-x86_64.img
Found initrd fallback image: /boot/initramfs-6.1-x86_64-fallback.img
Imagem Linux encontrada: /boot/vmlinuz-5.10-x86_64
Imagem initrd encontrada: /boot/amd-ucode.img /boot/initramfs-5.10-x86_64.img
Found initrd fallback image: /boot/initramfs-5.10-x86_64-fallback.img
Aviso: os-prober será executado para detectar outras partições de arranque.
A sua saída será usada para detectar binários de arranque nessas partições e criar novas entradas.
Encontrado Manjaro Linux (23.0.0) em /dev/sda1
Encontrado Windows Boot Manager em /dev/sdd2@/EFI/Microsoft/Boot/bootmgfw.efi
A adicionar entrada de menu para UEFI Firmware Settings ...
Root filesystem isn't btrfs
If you think an error has occurred, please file a bug report at "https://github.com/Antynea/grub-btrfs"
Found memtest86+ image: /boot/memtest86+/memtest.bin
/usr/bin/grub-probe: aviso: tipo de dispositivo nvme0n1 desconhecido.
concluído

Broke initrd.

[cscs]

@philm

Are you sure? does your script have

Yes, very sure.
The only difference between your posted file and the version on my filesystem is a newline at the end.

[philm]

OK, then try with:

    for path in /efi /boot; do
        if [[ -e $path ]]; then
            echo "Checking: $path"
            efi_directory="$(find $path -name $efi_grub_file | awk -F '/EFI' '{print $1}')"
            if [[ ! -z $efi_directory ]]; then
                efi_source_file="$(find $efi_directory/EFI/$bootloader_id -name $efi_grub_file)"
                efi_target_file="$(find $path -name $efi_boot_file)"
                cmp -s $efi_source_file $efi_target_file && fallback_is_same=true
                break
            fi
        fi
    done

[cscs]

Some difference, but still fail.

$ sudo /usr/bin/install-grub3
Checking: /efi
cmp: invalid --ignore-initial value '/efi/EFI/Manjaro/grubx64.efi'
cmp: Try 'cmp --help' for more information.
WARNING: Grub is not properly installed, system may not be bootable or other bootloader is used

[philm]

Do you have a fallback image?

    for path in /efi /boot; do
        if [[ -e $path ]]; then
            efi_directory="$(find $path -name $efi_grub_file | awk -F '/EFI' '{print $1}')"
            if [[ ! -z $efi_directory ]]; then
                efi_source_file="$(find $efi_directory/EFI/$bootloader_id -name $efi_grub_file)"
                efi_target_file="$(find $path -name $efi_boot_file)"
                if [[ ! -z $efi_source_file ]] && [[ ! -z $efi_target_file ]]; then 
                    cmp -s $efi_source_file $efi_target_file && fallback_is_same=true
                fi
                break
            fi
        fi
    done

[philm]

@cscs: can you give install-grub 2.12-3.5 a whirl? I also tested it with an empty /efi and non-existing folder. Should work now. Also it checks now if grub got installed and skips if not.

[cscs]

Packages (1) install-grub-2.12-3.5

Total Download Size:   0.02 MiB
Total Installed Size:  0.00 MiB
Net Upgrade Size:      0.00 MiB

:: Proceed with installation? [Y/n] 
:: Retrieving packages...
 install-grub-2.12-3.5-x86_64                                                       21.9 KiB  74.7 KiB/s 00:00 [##################################################################] 100%
(1/1) checking keys in keyring                                                                                 [##################################################################] 100%
(1/1) checking package integrity                                                                               [##################################################################] 100%
(1/1) loading package files                                                                                    [##################################################################] 100%
(1/1) checking for file conflicts                                                                              [##################################################################] 100%
(1/1) checking available disk space                                                                            [##################################################################] 100%
:: Processing package changes...
(1/1) upgrading install-grub                                                                                   [##################################################################] 100%
:: Running post-transaction hooks...
(1/1) Arming ConditionNeedsUpdate...

$ sudo install-grub
cmp: invalid --ignore-initial value '/efi/EFI/Manjaro/grubx64.efi'
cmp: Try 'cmp --help' for more information.
WARNING: Grub is not properly installed, system may not be bootable or other bootloader is used

[philm]

@cscs: does this work: install-grub ($976) · Snippets · GitLab?

More or less we check:

cmp -s /efi/EFI/Manjaro/grubx64.efi /efi/EFI/boot/bootx64.efi && fallback_is_same=true

In my case:

sudo tree /boot/efi
/boot/efi
└── EFI
    ├── boot
    │   └── bootx64.efi
    └── Manjaro
        └── grubx64.efi

4 directories, 2 files

[cscs]

It has successfully whittled down the output

$ sudo install-grub3
WARNING: Grub is not properly installed, system may not be bootable or other bootloader is used