Cannot boot after last update

Support
1

Hello,

my hard drive is encrypted with LUKS and after my last update (two days ago), I cannot boot my system anymore.

I get past GRUB, but then I get dropped into an emergency shell because device /dev/mapper/luks-.../ cannot be found.

I already chrooted into a live system (my hard drive is also fine) and rerun GRUB install, but that did not help. maxi claimed that initrd is missing, but I am not sure if this is related to the problem.

If there is anything else on how I can explain my problem clearer, I am happy to provide it (inxi or maxi output for instance).

Thank you very much for your help.

2

so from chroot rerun update; if everything is ok and you are up to date, provide output from:
mhwd-kernel -li

3

Update went fine.

This is my output

# mhwd-kernel -li
Currently running: 5.15.12-1-MANJARO (linux515)
The following kernels are installed in your system:
   * linux515
   * linux519
   * linux60

For the record, I also tried with kernel 6.0 with the same result.

4

well thats what i wanted you to suggest, to try with different kernel… i have seen people having the same issue as you after update, and switching kernels helped…
post output from:
cat /etc/fstab/
blkid

5

Thank you for the hint.

Here is the output of /etc/fstab/

# /etc/fstab: static file system information.
#
# Use 'blkid' to print the universally unique identifier for a device; this may
# be used with UUID= as a more robust way to name devices that works even if
# disks are added and removed. See fstab(5).
#
# <file system>             <mount point>  <type>  <options>  <dump>  <pass>
UUID=BE12-A115                            /boot/efi      vfat    umask=0077 0 2
/dev/mapper/luks-45618ecb-6140-46bd-8e1d-f44cea06d15a /              ext4    defaults,noatime 0 1

and blklid

/dev/loop1: TYPE="squashfs"
/dev/nvme0n1p5: UUID="45618ecb-6140-46bd-8e1d-f44cea06d15a" TYPE="crypto_LUKS" PARTLABEL="root" PARTUUID="10144802-7e6c-4141-a4a2-3f2b9ff500be"
/dev/nvme0n1p3: LABEL="Local Disk" BLOCK_SIZE="512" UUID="EEF0142AF013F793" TYPE="ntfs" PARTLABEL="Basic data partition" PARTUUID="ccd3804b-9212-4a25-a301-7bea367a3f46"
/dev/nvme0n1p1: LABEL_FATBOOT="SYSTEM" LABEL="SYSTEM" UUID="BE12-A115" BLOCK_SIZE="512" TYPE="vfat" PARTLABEL="EFI system partition" PARTUUID="8383cd4d-12cb-4e35-8bdc-7e27ae872754"
/dev/nvme0n1p4: LABEL="Windows RE tools" BLOCK_SIZE="512" UUID="10FE165AFE163904" TYPE="ntfs" PARTLABEL="Basic data partition" PARTUUID="2c9ccd27-3000-4691-9068-070bfecd822e"
/dev/nvme0n1p2: PARTLABEL="Microsoft reserved partition" PARTUUID="4292b742-d9f2-488f-a182-626065488ca7"
/dev/loop2: TYPE="squashfs"
/dev/loop0: TYPE="squashfs"
/dev/mapper/crypto_LUKS: UUID="aa7aed65-9642-4782-9e45-16054d80609e" BLOCK_SIZE="4096" TYPE="ext4"
/dev/sda2: SEC_TYPE="msdos" LABEL_FATBOOT="MISO_EFI" LABEL="MISO_EFI" UUID="8078-6038" BLOCK_SIZE="512" TYPE="vfat"
/dev/sda1: BLOCK_SIZE="2048" UUID="2022-01-03-16-02-54-00" LABEL="MANJARO_KDE_2121" TYPE="iso9660"
/dev/loop3: TYPE="squashfs"
6

looks good, uuids match…
post logs, maybe there will be something:
journalctl -b-1 -p5 --no-pager

7

I created a pastebin link as it is quite long: Nov 01 19:14:21 X1 kernel: Linux version 6.0.2-2-MANJARO (builduser@fv-az628-669 - Pastebin.com

8

thats a lot of logs for a failed boot…
first remove the rambox, it has a lot of dumped cores:
pacman -R rambox
reinstall all your kernels:
pacman -S linux515 linux519 linux60
exit chroot and reboot and try booting with all of them…
if it doesnt work, provide logs from this:
journalctl -b-1 -p4 --no-pager

9

I will try, thank you. Just one question before that.

This is the output for reinstalling the kernels:

#  pacman -S linux515 linux519 linux60
warning: linux515-5.15.76-1 is up to date -- reinstalling
warning: linux519-5.19.17-1 is up to date -- reinstalling
warning: linux60-6.0.6-1 is up to date -- reinstalling
resolving dependencies...
looking for conflicting packages...

Packages (3) linux515-5.15.76-1  linux519-5.19.17-1  linux60-6.0.6-1

Total Installed Size:  454.15 MiB
Net Upgrade Size:        0.00 MiB

:: Proceed with installation? [Y/n] Y
(3/3) checking keys in keyring                                                                                                                [#######################################################################################] 100%
(3/3) checking package integrity                                                                                                              [#######################################################################################] 100%
(3/3) loading package files                                                                                                                   [#######################################################################################] 100%
(3/3) checking for file conflicts                                                                                                             [#######################################################################################] 100%
(3/3) checking available disk space                                                                                                           [#######################################################################################] 100%
:: Processing package changes...
(1/3) reinstalling linux515                                                                                                                   [#######################################################################################] 100%
(2/3) reinstalling linux519                                                                                                                   [#######################################################################################] 100%
>>> NOTE, 5.19.17 is the last maintenance release of the 519 kernel series.
    This kernel is now marked 'End Of Life' (EOL).
    
    It is recommend to switch to the newer linux60 series:
    'sudo mhwd-kernel -i linux60'
    
(3/3) reinstalling linux60                                                                                                                    [#######################################################################################] 100%
:: Running post-transaction hooks...
(1/4) Arming ConditionNeedsUpdate...
(2/4) Updating module dependencies...
(3/4) Updating linux initcpios...
==> Building image from preset: /etc/mkinitcpio.d/linux515.preset: 'default'
  -> -k /boot/vmlinuz-5.15-x86_64 -c /etc/mkinitcpio.conf -g /boot/initramfs-5.15-x86_64.img
==> Starting build: 5.15.76-1-MANJARO
  -> Running build hook: [base]
  -> Running build hook: [udev]
  -> Running build hook: [autodetect]
  -> Running build hook: [modconf]
  -> Running build hook: [block]
  -> Running build hook: [filesystems]
  -> Running build hook: [keyboard]
  -> Running build hook: [fsck]
==> Generating module dependencies
==> Creating gzip-compressed initcpio image: /boot/initramfs-5.15-x86_64.img
==> Image generation successful
==> Building image from preset: /etc/mkinitcpio.d/linux515.preset: 'fallback'
  -> -k /boot/vmlinuz-5.15-x86_64 -c /etc/mkinitcpio.conf -g /boot/initramfs-5.15-x86_64-fallback.img -S autodetect
==> Starting build: 5.15.76-1-MANJARO
  -> Running build hook: [base]
  -> Running build hook: [udev]
  -> Running build hook: [modconf]
  -> Running build hook: [block]
  -> Running build hook: [filesystems]
  -> Running build hook: [keyboard]
  -> Running build hook: [fsck]
==> Generating module dependencies
==> Creating gzip-compressed initcpio image: /boot/initramfs-5.15-x86_64-fallback.img
==> Image generation successful
==> Building image from preset: /etc/mkinitcpio.d/linux519.preset: 'default'
  -> -k /boot/vmlinuz-5.19-x86_64 -c /etc/mkinitcpio.conf -g /boot/initramfs-5.19-x86_64.img
==> Starting build: 5.19.17-1-MANJARO
  -> Running build hook: [base]
  -> Running build hook: [udev]
  -> Running build hook: [autodetect]
  -> Running build hook: [modconf]
  -> Running build hook: [block]
==> WARNING: Possibly missing firmware for module: xhci_pci
  -> Running build hook: [filesystems]
  -> Running build hook: [keyboard]
  -> Running build hook: [fsck]
==> Generating module dependencies
==> Creating gzip-compressed initcpio image: /boot/initramfs-5.19-x86_64.img
==> Image generation successful
==> Building image from preset: /etc/mkinitcpio.d/linux519.preset: 'fallback'
  -> -k /boot/vmlinuz-5.19-x86_64 -c /etc/mkinitcpio.conf -g /boot/initramfs-5.19-x86_64-fallback.img -S autodetect
==> Starting build: 5.19.17-1-MANJARO
  -> Running build hook: [base]
  -> Running build hook: [udev]
  -> Running build hook: [modconf]
  -> Running build hook: [block]
==> WARNING: Possibly missing firmware for module: xhci_pci
  -> Running build hook: [filesystems]
  -> Running build hook: [keyboard]
  -> Running build hook: [fsck]
==> Generating module dependencies
==> Creating gzip-compressed initcpio image: /boot/initramfs-5.19-x86_64-fallback.img
==> Image generation successful
==> Building image from preset: /etc/mkinitcpio.d/linux60.preset: 'default'
  -> -k /boot/vmlinuz-6.0-x86_64 -c /etc/mkinitcpio.conf -g /boot/initramfs-6.0-x86_64.img
==> Starting build: 6.0.6-1-MANJARO
  -> Running build hook: [base]
  -> Running build hook: [udev]
  -> Running build hook: [autodetect]
  -> Running build hook: [modconf]
  -> Running build hook: [block]
==> WARNING: Possibly missing firmware for module: xhci_pci
  -> Running build hook: [filesystems]
  -> Running build hook: [keyboard]
  -> Running build hook: [fsck]
==> Generating module dependencies
==> Creating gzip-compressed initcpio image: /boot/initramfs-6.0-x86_64.img
==> Image generation successful
==> Building image from preset: /etc/mkinitcpio.d/linux60.preset: 'fallback'
  -> -k /boot/vmlinuz-6.0-x86_64 -c /etc/mkinitcpio.conf -g /boot/initramfs-6.0-x86_64-fallback.img -S autodetect
==> Starting build: 6.0.6-1-MANJARO
  -> Running build hook: [base]
  -> Running build hook: [udev]
  -> Running build hook: [modconf]
  -> Running build hook: [block]
==> WARNING: Possibly missing firmware for module: xhci_pci
  -> Running build hook: [filesystems]
  -> Running build hook: [keyboard]
  -> Running build hook: [fsck]
==> Generating module dependencies
==> Creating gzip-compressed initcpio image: /boot/initramfs-6.0-x86_64-fallback.img
==> Image generation successful
(4/4) Updating Grub-Bootmenu
Generating grub configuration file ...
Found linux image: /boot/vmlinuz-6.0-x86_64
Found initrd image: /boot/intel-ucode.img /boot/initramfs-6.0-x86_64.img
Found initrd fallback image: /boot/initramfs-6.0-x86_64-fallback.img
Found linux image: /boot/vmlinuz-5.19-x86_64
Found initrd image: /boot/intel-ucode.img /boot/initramfs-5.19-x86_64.img
Found initrd fallback image: /boot/initramfs-5.19-x86_64-fallback.img
Found linux image: /boot/vmlinuz-5.15-x86_64
Found initrd image: /boot/intel-ucode.img /boot/initramfs-5.15-x86_64.img
Found initrd fallback image: /boot/initramfs-5.15-x86_64-fallback.img
Warning: os-prober will be executed to detect other bootable partitions.
Its output will be used to detect bootable binaries on them and create new boot entries.
ERROR: mkdir /var/lock/dmraid
grub-probe: error: cannot find a GRUB drive for /dev/sda1.  Check your device.map.
grub-probe: error: cannot find a GRUB drive for /dev/sda1.  Check your device.map.
Adding boot menu entry for UEFI Firmware Settings ...
Found memtest86+ image: /boot/memtest86+/memtest.bin
done

Should I be concerned about the grub-probe errors? Or is that due to the live system?

10

everything looks ok…
the grub probe errors are as you say from the live usb, and nothing to worry about, ignore them

11

The output for the journal is still quite long: Nov 01 19:14:21 X1 kernel: MMIO Stale Data CPU bug present and SMT on, data leak - Pastebin.com

EDIT: But I just noticed that the rambox messages are still part of this log.

12

yes, did you remove it?
and you have these:

Nov 02 03:36:12 X1 systemd-cryptsetup[18572]: Device luks-45618ecb-6140-46bd-8e1d-f44cea06d15a is still in use.

Nov 02 03:36:12 X1 systemd-cryptsetup[18572]: Failed to deactivate: Device or resource busy

Nov 02 03:36:12 X1 systemd[1]: systemd-cryptsetup@luks\x2d45618ecb\x2d6140\x2d46bd\x2d8e1d\x2df44cea06d15a.service: Failed with result 'exit-code'.

the same message that had the other users that i mentioned above…

try installing the 5.10 kernel:
pacman -S linux510
and try ro boot with it…

13

Thank you for the hint. Unfortunately, it still ends with the same result.

14

post output from:
lsblk -f

15 
# lsblk -f
NAME            FSTYPE FSVER LABEL UUID FSAVAIL FSUSE% MOUNTPOINTS
loop0                                                  
loop1                                                  
loop2                                                  
loop3                                                  
sda                                                    
|-sda1                                                 
`-sda2                                                 
nvme0n1                                                
|-nvme0n1p1                              211.9M    17% /boot/efi
|-nvme0n1p2                                            
|-nvme0n1p3                                            
|-nvme0n1p4                                            
`-nvme0n1p5                                            
  `-crypto_LUKS                            9.8G    90% /
16

run these commands but not from chroot, if you are chrooted exit:
sudo umount -l /dev/nvme0n1p5

17

Same situation for all kernels, I still end up in the emergency shell. Although, I actually had to unmount /dev/mapper/crypto_LUKS as /dev/nvme0n1p5 returned “not mounted”.

EDIT: and the journal still says LUKS device still in use.

18

so you unmounted everyhting and it still didnt worked?
are you on stable branch?

19

Yes, I am on the stable branch (I do not even know how to change to a testing branch).

I unmounted all volumes (and checked that they are not mounted anymore), and this time I also closed cryptsetup, still with the same result, I am ending in the emergency shell.

20

thats what i wanted to suggest, to try the testing branch, im on it and no issues what so ever, so you shouldnt be worried, and it cant get worse than this… but first, are you using virtual machine? there were issues with it since this last stable update, maybe related to your issue…