Install-grub: a new way to keep your EFI/MBR in-sync with grub package

soundofthunder · 26 December 2023 09:55

Is memtest86-efi from the AUR the one you would use?

cscs · 26 December 2023 09:58

No, I meant writing my own file instead of 60_memtest86+ provided by the memtest86+-efi package (while still using that package).

bedna · 26 December 2023 11:47

What if the user accidently formated the boot partition (or accidently removed the files somehow, it’s known to happen) and want to reinstall.

philm:

    for path in /boot /efi; do
        if [[ -e $path ]]; then
            efi_directory="$(find $path -name $efi_grub_file | awk -F '/EFI' '{print $1}')"
            if [[ ! -z "$efi_directory" ]]; then
                efi_source_file="$efi_directory/EFI/$bootloader_id/$efi_grub_file"
                efi_target_file="$(find $path -name $efi_boot_file)"
                if [[ -e "$efi_source_file" ]] && [[ -e "$efi_target_file" ]]; then
                    cmp -s "$efi_source_file" "$efi_target_file" && fallback_is_same=true
                fi
                break

Would that work then?

What if you also check the fstab where the mount target is supposed to be?

Teo · 26 December 2023 12:33

I do not think the purpose of this project should be to cover all scenarios, including a very broken system without ESP, kernel, and so on. It should be a hook at grub update, then assuming a working system detecting where the bootloader is and updating it or throwing an error as is now.

bedna · 26 December 2023 12:46

I thought that was exactly what it was supposed to be, because experienced users knows where the stuff is anyway and most likely wont use this.
If not, you are correct.

Since it involves other distros in the checking, I would assume it could be a great feature if it also checks fstab.

How will it handle a live environment?
If you chroot, and the boot is for some reason not getting mounted correctly/not there, what would happen?
A fstab check would fix that, at least the path. Assuming the fstab is not the REASON for the fail, but then find might figure it out instead.
IDK, I feel like it could be done if done cleverly.

cat /etc/fstab | grep 'boot' | awk '{print $2}'

Edit
Wait, that won’t work with cscs setup, hmmm.
But then again the find would fix it in that case…
I feel the idea is there, execution, not so much.

Mirdarthos · 26 December 2023 14:55

I feel this is a better option:

Wollie · 26 December 2023 15:05

What I miss is a clear separation between

sytem analysis and reporting
suggestion of what exactly to do
prompting for approval to go

bedna · 26 December 2023 15:19

Witch links BACK to this thread. xD

I was under the assumption this tool was for less knowledgeable people to fix grub. Maybe I’m mistaken and back off then.

Mirdarthos · 26 December 2023 15:37

This one?

https://i.imgur.com/MKTxNlX.png

https://i.imgur.com/U6sb53m.jpg

https://i.imgur.com/bdeDeua.png

I don’t know, but I meant the instructions on finding the mountpoint of the EFI partition…

bedna · 26 December 2023 15:47

[root tip] [How To] Primer on handling a grub package update

Also see

Install-grub: a new way to keep your EFI/MBR in-sync with grub package
Some might not know that when updating the grub package you’re actually only update GRUB partly. This also creates some issues with your system if an older grub got installed but doesn’t match configurations and scripts. This script fixes most of our installation cases. If you have a fancy partition layout or some extra stuff ongoing, it may fail as explained here: Re: [Regression] efi: Don’t display a uefi-firmware entry if it’s not su You can now install the package install-grub from our unst…

The last tip of that tutorial is TO THIS THREAD. xD

Not here to argue, wanted to provide input and ideas.

Mirdarthos · 26 December 2023 15:48

Yeah, I know. And I still think it’s a good way of determining the mountpoint.

philm · 26 December 2023 16:58

Then he has to fix it manually.

UEFI systems tend to have a separate partition for the bootloader. On MBR most of the files can be also on the root partition and no extra mountpoint may exist. That is why we search for the path and not a mountpoint. I added /efi support beside the regular /boot/efi folder Manjaro usually ships, cos @cscs mentioned that it is also kind of a standard to use just /efi.

You are already in the manual fixing mode. The script won’t find it and error out.

This script is not for that. It should automate the update without any user interaction. It is used for users who don’t know how grub was installed and also don’t care about it. It is also designed for those who use Calamares auto-partitioning and standard setup. If you want more, you can dig into grub-multi-install ($974) · Snippets · GitLab and postinst.in ($975) · Snippets · GitLab, which is used at Ubuntu. I tried to only get a working essence for now. I’m open for new features covering other cases as I and the community came up with so far …

It is designed to upgrade a working grub install to keep the binaries of the package grub and the install of grub at MBR/EFI in sync. The reason is pointed out here: Re: [Regression] efi: Don't display a uefi-firmware entry if it's not su

philm · 26 December 2023 17:34

If we talk about what this script should cover, we mostly talk about the bootloader module of Calamares 3.2.x. There we have two cmds for grub-install:

EFI systems

    if is_zfs:
        check_target_env_call(["sh", "-c", "ZPOOL_VDEV_NAME_PATH=1 " + libcalamares.job.configuration["grubInstall"]
                               + " --target=" + efi_target + " --efi-directory=" + efi_directory
                               + " --bootloader-id=" + efi_bootloader_id + " --force"])
    else:
        check_target_env_call([libcalamares.job.configuration["grubInstall"],
                               "--target=" + efi_target,
                               "--efi-directory=" + efi_directory,
                               "--bootloader-id=" + efi_bootloader_id,
                               "--force"])

BIOS systems

    if is_zfs:
        check_target_env_call(["sh", "-c", "ZPOOL_VDEV_NAME_PATH=1 "
                               + libcalamares.job.configuration["grubInstall"]
                               + " --target=i386-pc --recheck --force "
                               + boot_loader["installPath"]])
    else:
        check_target_env_call([libcalamares.job.configuration["grubInstall"],
                               "--target=i386-pc",
                               "--recheck",
                               "--force",
                               boot_loader["installPath"]])

Most cases of EFI installs should be covered by now. On BIOS side only the variant via MBR so far. GPT I’ve to check again. On the EFI side of things --no-vram was added. The path for the efi executable stays the same, updating the bootloader entry in the nvram just messes with the priorities and invites issues from non compliant UEFI implementation. It’s easier and better to just not deal with that.

Since we don’t support ZFS with our install medias, that option is not yet covered but most likely easy to add.

andreas85 · 26 December 2023 17:49

I’m not 100% sure, but it seems to me that some UEFI variants change the order of boot entries when grub is reinstalled.

I don’t know what triggers this, but maybe it can be detected.

If the order

has changed (while running install-grub),

it would be good to spit out a warning that the user may have to manually (once) select the correct entry when booting.

extended 2 times

soundofthunder · 26 December 2023 17:56

Reinstallation of Grub on Debian, and SUSE, in particular, seems to do just that every time. I don’t think that can be easily detected in advance; predicted maybe.

philm · 26 December 2023 17:59

Most likely it will change the priority. Garuda for example uses this to detect that:

local current_grub_entry=“$(efibootmgr | awk ‘match($0,/^BootCurrent: ([0-9a-fA-F]{4})$/,out) { current=out[1] } match($0,/^Boot([0-9a-fA-F]{4})*? ([^:]+)\t/,out) { if (out[1]==current) print out[2] }’)”

Then they check with:

	if [ "${current_grub_entry,,}" == "garuda" ] && [ "$did_update" == "true" ]; then
		should_act_safe=false
	fi

And spit out the notice depending of the result like:

	if [ "$did_update" != "true" ]; then
		garudalib_add_update_notice "The GRUB bootloader could not be updated automatically safely. Steps to prevent any major issues caused by this have been taken automatically, but if possible, you should update your bootloader using 'grub-install'."
	elif [ "$should_act_safe" == "true" ]; then
		garudalib_add_update_notice "The GRUB bootloader at /EFI/Garuda was updated, but it seems like you are not using the Garuda Linux bootloader by default. If you are using 'rEFInd' bootloader, you can safely ignore this. Otherwise, please change your EFI boot priorities to prioritize the \"Garuda\" bootloader. Running 'grub-install' will do this automatically."
	else
		garudalib_add_update_notice "The GRUB bootloader at /EFI/Garuda has been updated/reinstalled using 'grub-install'. If this looks correct, no further action has to be taken."
	fi

So if the Bootloader-ID differs some other Bootloader entry was used.

soundofthunder · 26 December 2023 18:17

This scenario seems to be repeating itself lately.

anon6547204 · 26 December 2023 18:23

It probably got lost in the discussion, so I’ll repeat my question again: What about Btrfs? Is there a difference and if so, what should be done?

philm · 26 December 2023 18:23

I just tested BIOS/GPT install. This is what I got after using the manual mode to create the needed 8MiB partition:

12:44:59 [6]: void Calamares::JobThread::finalize()
    There are 36 jobs, total weight 33 
    .. Job 1 "Manage auto-mount settings" +wt 0.1 tot.wt 0.1 
    .. Job 2 "Clear all temporary mounts." +wt 0.1 tot.wt 0.2 
    .. Job 3 "Clear mounts for partitioning operations on /dev/sda" +wt 0.1 tot.wt 0.3 
    .. Job 4 "Create new gpt partition table on /dev/sda." +wt 0.1 tot.wt 0.4 
    .. Job 5 "Create new 8MiB partition on /dev/sda (VBOX HARDDISK)." +wt 0.1 tot.wt 0.5 
    .. Job 6 "Set flags on 8MiB unformatted partition." +wt 0.1 tot.wt 0.6 
    .. Job 7 "Create new 12620MiB partition on /dev/sda (VBOX HARDDISK)." +wt 0.1 tot.wt 0.7 
    .. Job 8 "Set flags on 12620MiB ext4 partition." +wt 0.1 tot.wt 0.8 
    .. Job 9 "Set partition information" +wt 0.1 tot.wt 0.9 
    .. Job 10 "Manage auto-mount settings" +wt 0.1 tot.wt 1 
    .. Job 11 "mount" +wt 1 tot.wt 2 
    .. Job 12 "unpackfs" +wt 12 tot.wt 14 
    .. Job 13 "networkcfg" +wt 1 tot.wt 15 
    .. Job 14 "Generate machine-id." +wt 1 tot.wt 16 
    .. Job 15 "fstab" +wt 1 tot.wt 17 
    .. Job 16 "Set timezone to America/New_York" +wt 1 tot.wt 18 
    .. Job 17 "Set keyboard model to pc105, layout to us-" +wt 1 tot.wt 19 
    .. Job 18 "localecfg" +wt 1 tot.wt 20 
    .. Job 19 "Configuring encrypted swap." +wt 1 tot.wt 21 
    .. Job 20 "Configuring LUKS key file." +wt 1 tot.wt 22 
    .. Job 21 "initcpiocfg" +wt 1 tot.wt 23 
    .. Job 22 "Creating initramfs with mkinitcpio." +wt 1 tot.wt 24 
    .. Job 23 "Configure <pre>sudo</pre> users." +wt 0.166667 tot.wt 24.1667 
    .. Job 24 "Preparing groups." +wt 0.166667 tot.wt 24.3333 
    .. Job 25 "Create user j" +wt 0.166667 tot.wt 24.5 
    .. Job 26 "Set password for user j" +wt 0.166667 tot.wt 24.6667 
    .. Job 27 "Set password for user root" +wt 0.166667 tot.wt 24.8333 
    .. Job 28 "Set hostname j-virtualbox" +wt 0.166667 tot.wt 25 
    .. Job 29 "displaymanager" +wt 1 tot.wt 26 
    .. Job 30 "mhwdcfg" +wt 1 tot.wt 27 
    .. Job 31 "hwclock" +wt 1 tot.wt 28 
    .. Job 32 "services" +wt 1 tot.wt 29 
    .. Job 33 "grubcfg" +wt 1 tot.wt 30 
    .. Job 34 "bootloader" +wt 1 tot.wt 31 
    .. Job 35 "postcfg" +wt 1 tot.wt 32 
    .. Job 36 "Unmount file systems." +wt 1 tot.wt 33

Based on @linux-aarhus tutorial we assume grub to be installed on a partition, rather MBR. Well calamares installed on MBR:

[PYTHON JOB]: "Bootloader: grub (bios)" 
    .. Running ("grub-install", "--target=i386-pc", "--recheck", "--force", "/dev/sda") 
    .. Finished. Exit code: 0 output:
 Installing for i386-pc platform.
Installation finished. No error reported.
    .. Running ("grub-mkconfig", "-o", "/boot/grub/grub.cfg") 
    .. Finished. Exit code: 0 output:
 Generating grub configuration file ...
Found theme: /usr/share/grub/themes/manjaro/theme.txt
Found linux image: /boot/vmlinuz-6.6-x86_64
Found initrd image: /boot/intel-ucode.img /boot/amd-ucode.img /boot/initramfs-6.6-x86_64.img
Found initrd fallback image: /boot/initramfs-6.6-x86_64-fallback.img
Warning: os-prober will be executed to detect other bootable partitions.
Its output will be used to detect bootable binaries on them and create new boot entries.
Adding boot menu entry for UEFI Firmware Settings ...
Root filesystem isn't btrfs
If you think an error has occurred, please file a bug report at "https://github.com/Antynea/grub-btrfs"
Found memtest86+ image: /boot/memtest86+/memtest.bin
Found memtest86+ EFI image: /boot/memtest86+/memtest.efi
done

Calamares offers MBR or installing the bootloader to /. The later I may test another day. However it worked just fine:

 ~ sudo install-grub
Grub will be installed on: MBR
Installing for i386-pc platform.
Installation finished. No error reported.
Generating grub configuration file ...
Found theme: /usr/share/grub/themes/manjaro/theme.txt
Found linux image: /boot/vmlinuz-6.6-x86_64
Found initrd image: /boot/intel-ucode.img /boot/initramfs-6.6-x86_64.img
Found initrd fallback image: /boot/initramfs-6.6-x86_64-fallback.img
Warning: os-prober will be executed to detect other bootable partitions.
Its output will be used to detect bootable binaries on them and create new boot entries.
Root filesystem isn't btrfs
If you think an error has occurred, please file a bug report at "https://github.com/Antynea/grub-btrfs"
Found memtest86+ image: /boot/memtest86+/memtest.bin
Found memtest86+ EFI image: /boot/memtest86+/memtest.efi
done

philm · 26 December 2023 18:26

I mostly tested Calamares auto-partition with ext4 so far on BIOS and UEFI. So you tell me if some is needed regarding BTRFS. Be brave and have the live-ISO at hand if you break things