Can't boot LUKS encrypted install after 2023-03-31 stable update

@philm Thank Phil for taking the time to post a thorough response - it certainly puts things in perspective for me & I know now what to look out for. More power to Manjaro & everyone contributing to the project…

1 Like

I have a hard time processing… :dizzy_face:


If i summarize:

Did i get that right?
Also, can the fix be applied before rebooting in order to prevent the issue?

2 Likes

Thank you!!!

I made myself an account just to thank you! I had a nice start to Fool’s Day, but your kindness in sharing saved my day!

It is not clear if there are any changes needed to get your system booting. Most say: Hey I've this cryptodisk issue message asking me to press any key and it boots normal. Others claim their system is not bootable. For example I’ve a similar issue when I update grub on my Macbook M1 Air using our ARM version of Manjaro natively. There the issue is with grubenv not being valid. Since we know how the partitioning is done, grub gets actually installed to UEFI when it gets updated due to a special version of update-grub. So I’ve to fix that manually until the next update-grub call. There I have our update-grub named update-grub-menu as that is actually what our regular script is doing.

Installing grub to MBR/UEFI is needed to keep the distributed binary files in-sync with the version you have installed to boot your system. If you don’t want to bother with the bootloader simply put it into the ignore list and skip it from updating all together. As if security issues is your concern, you need to use grub-install plus the flags and info you used when you installed grub the first time - well most of us let Calamares do that. So they might not even know how grub got installed…

So what to do:

  1. check if some has changed in /etc/default/grub and fix that
  2. try to update your grub menu via update-grub
  3. always consider installing grub to MBR/UEFI as your last resort

Having a Linux Live-USB is always handy, even if you forgot or never had a Windows password when you have physical access to a PC or Laptop :smile_cat:

1 Like

Thanks for the additional comments. Is there a risk to setting “IgnorePkg = grub” in the Pacman config file? I don’t want to replace one risk with another? Thanks, R

Had same issue and did changes to /etc/default/grub
Also did:

$ sudo mkinitcpio -P
$ sudo update-grub

But issue persisted.

Ran:

$ lsblk

Got:

NAME                                          MAJ:MIN RM   SIZE RO TYPE  MOUNTPOINTS
nvme0n1                                       259:0    0 931.5G  0 disk  
└─nvme0n1p1                                   259:1    0 931.5G  0 part  
  └─luks-9d1a0ad4-ef26-41c2-bafe-4d3dcf7a750c 254:0    0 931.5G  0 crypt /

Then ran:

$ sudo grub-install /dev/nvme0n1

Problem fixed.
Hope it helps other newbies like me…


Moderator edit: In the future, please use proper formatting: [HowTo] Post command output and file content as formatted text

4 Likes

Thanks @kentwillumsen - that’s a useful suggestion to add into the mix.

Thank you kentwillumsen!!!
This suggestion fixed the problem for me…

Even though I kept fairly detailed notes of my initial installation procedure, there was nothing about grub-install. So, I followed the arch link that Phil posted and discovered some strange things, namely grub-install command has the path rather than the device.

# grub-install --target=x86_64-efi --efi-directory=esp --bootloader-id=GRUB

Then, they have this note.

You might note the absence of a device_path option (e.g.: /dev/sda) in the grub-install command. In fact any device_path provided will be ignored by the GRUB UEFI install script. Indeed, UEFI boot loaders do not use a MBR bootcode or partition boot sector at all.

Because of that note, I decided to follow arch wiki. (Side note: My EFI partition is not encrypted, i.e. /boot/efi is on /dev/nvme0n1p1 and the encrypted partition is on /dev/nvme0n1p2.) The problem is that there are two directories in /boot/efi/EFI: boot and Manjaro. Which id to use then? Interestingly, the former contains bootx64.efi and the latter has grubx64.efi but these two files are identical. I went with Manjaro.

# sudo grub-install --target=x86_64-efi --efi-directory=/boot/efi --bootloader-id=Manjaro

After reboot, I entered the passphrase and ended up in grub rescue mode after it showed something like “cannot find symbol grub-debug-malloc”.

I had to create a bootable stick, chroot into my system and copy /boot/efi/EFI/Manjaro/grubx64.efi into /boot/efi/EFI/boot/bootx64.efi. After that I was able to boot without any problems.

The whole thing is rather confusing. I have no idea why those two files need to be identical and why grub needs them both to function. On my older system without FDE, those files are not identical.

Idk, I feel like @philm is hiding something about what Calamares does to grub :rofl:

While it is true that it’s unsupported it’s 100% false to say it’s impossible. I’ve used Timeshift to restore Manjaro FDE multiple times when an update left me ■■■■■■■ Up until recently Timeshifts check/error for /boot being encrypted was non-deterministic and could be bypassed easily(which always led to successful restores for me). Unfortunately that doesn’t seem to be the case anymore IN GUI; through terminal I was still able to restore and thankfully didn’t need to wipe(the disk randomly decided it was fine and I was able to mount it and restore through the terminal).

efi/boot/bootx64.efi is the “fallback” path that is used when UEFI configuration in bios is wrong. So this file works same as MBR boot, in that when you say to bios “boot from this disk” it boots from that path. Normal UEFI boot procedure is to boot the path stored in bios (here should be /EFI/Manjaro/grubx64.efi) .

Finally! This one solved the issue for me. Thank you! Don’t really know why the fallback is being used though… My bios configuration hasn’t changed since the stable update. I guess this will reappear next time I’ll run grub-install so need to remember that. Don’t really like the ‘solution’ though. Feels like it should work without it. Does anyone have an idea how this can be fixed permanently? What I’ve noticed was that the first unlock prompt was referencing UUID without dashes, the other unlock promt (after the cryptodisk error message) was referencing UUID with dashes. I guess that’s why the decryption failed and showed the cryptodisk error message. When I cp /boot/efi/EFI/Manjaro/grubx64.efi /boot/efi/EFI/boot/bootx64.efi the first password promt had the UUID displayed with dashes and then the futher decryption worked.

UPDATE:
Just run efibootmgr -v and it seems that the bootx64.efi is set as first in order

Timeout: 1 seconds
BootOrder: 0001,0000
Boot0000* manjaro       HD(1,GPT,426c0f14-e61f-9249-97a9-4496a9a94d7f,0x1000,0x96000)/File(\EFI\manjaro\grubx64.efi)
      dp: 04 01 2a 00 01 00 00 00 00 10 00 00 00 00 00 00 00 60 09 00 00 00 00 00 14 0f 6c 42 1f e6 49 92 97 a9 44 96 a9 a9 4d 7f 02 02 / 04 04 36 00 5c 00 45 00 46 00 49 00 5c 00 6d 00 61 00 6e 00 6a 00 61 00 72 00 6f 00 5c 00 67 00 72 00 75 00 62 00 78 00 36 00 34 00 2e 00 65 00 66 00 69 00 00 00 / 7f ff 04 00
Boot0001* UEFI OS       HD(1,GPT,426c0f14-e61f-9249-97a9-4496a9a94d7f,0x1000,0x96000)/File(\EFI\BOOT\BOOTX64.EFI)0000424f
      dp: 04 01 2a 00 01 00 00 00 00 10 00 00 00 00 00 00 00 60 09 00 00 00 00 00 14 0f 6c 42 1f e6 49 92 97 a9 44 96 a9 a9 4d 7f 02 02 / 04 04 30 00 5c 00 45 00 46 00 49 00 5c 00 42 00 4f 00 4f 00 54 00 5c 00 42 00 4f 00 4f 00 54 00 58 00 36 00 34 00 2e 00 45 00 46 00 49 00 00 00 / 7f ff 04 00
    data: 00 00 42 4f

Will change to Manjaro and see if it works ok. Will update this post with results.

UPDATE 2:
After changing the boot order everything is working fine. :+1:

@varikonniemi thanks for the explanation, @Ancestr thanks for the pointers. I still don’t quite understand it. The order changed and the fallback was working albeit with the ‘no such cryptodisk found’ error message. However, after installing the new grub into Manjaro, the fallback stopped working. Shouldn’t they be independent? In any case, I also changed the order and it works.

I have got 2 encrypted drives.

❯ lsblk
NAME                  MAJ:MIN RM   SIZE RO TYPE  MOUNTPOINTS
loop0                   7:0    0 116,8M  1 loop  /var/lib/snapd/snap/core/14784
loop1                   7:1    0 116,8M  1 loop  /var/lib/snapd/snap/core/14946
loop2                   7:2    0  55,6M  1 loop  /var/lib/snapd/snap/core18/2714
loop3                   7:3    0  55,6M  1 loop  /var/lib/snapd/snap/core18/2721
sda                     8:0    0   1,8T  0 disk  
└─sda1                  8:1    0   1,8T  0 part  
  └─luks-654e7435-e8ca-4b13-8abb-18733a64eff7
                      254:1    0   1,8T  0 crypt /home
nvme0n1               259:0    0 238,5G  0 disk  
├─nvme0n1p1           259:1    0     1G  0 part  /boot/efi
└─nvme0n1p2           259:2    0 237,5G  0 part  
  └─luks-3352fb98-8855-4902-9528-db429256c07d
                      254:0    0 237,5G  0 crypt /

How do I use grub-install the right way for this setting?

so you are using EFI?

sudo grub-install --target=x86_64-efi --efi-directory=/boot/efi --bootloader-id=manjaro --recheck

and then update grub config

sudo grub-mkconfig -o /boot/grub/grub.cfg

Found some nice info in the stable branch announcement:

2 Likes

Are you sure about this?
I’ve used full disk encryption + TimeShift since original install 2 years ago aprox and
Even tho i havent restored anything last 6 months i restored snapshots many times earlier
when i was fiddling with Wayland and HW video decoding in firefox.
Never got that to work but each time i failed i restored a older snapshot
We’re talking probably about 15+ restores.

I dont think i did anything special to make it work. It’s just a basic Manjaro install
because i’m fairly new to linux and Manjaro.

List boot order
efibootmgr

…find the ‘manjaro’ number and if it isn’t the first (probably) then re-order things to put manjaro first…
sudo efibootmgr -o XXXX,YYYY... (smallcaps ‘o’)

For more details and formatting
man efibootmgr

efibootmgr is very important in this context (grub-install) and should go together, once the grub is installed we should recheck the boot order. Why the boot order gets messed up I’m not sure (could be due to some install or BIOS menu, or failure to boot). If you can’t boot into the system do the same from the Live USB.

1 Like

This solved my issue. :face_exhaling: Thanks. As a relative n00b, is this a process I should complete whenever I see GRUB mentioned in the update? Before rebooting, I imagine?

Watch for grub-related info in announcements posts, check the date of files under /boot/grub/…, do not update if you see grub / systemd / mkinitcpio / cryptsetup / lvm / %younamewhatelseyoufearbreakingof% and you know you have no time to fix it. Always works for me.

And yes: if you’re ready to update these packages, you’d better make a backup right before that.

1 Like