LUKS Resume Fails at Boot (/etc/default/grub) Issue?

I’m in the process of restoring my old encrypted partition to a new install from the Manjaro installer.

So far I’ve:
-Migrated the Data Over from a Live USB
-Edited the UUID’s in /etc/fstab & /etc/crypttab
-Installed and Updated Grub & EFI

I reach the password screen from a reboot and enter the password and it says 'slot 0 open" which is great. The OS starts to boot, but then hits a brick wall with “device dev/disk-by-uuid not found” which I discovered is due to /etc/default/grub being old.

Here is my fstab:

UUID=4B3B-683F                            /boot/efi      vfat    umask=0077 0 2
/dev/mapper/luks-disk /              btrfs   subvol=@,defaults,noatime,space_cache,compress=zstd:9 0 1
/dev/mapper/luks-disk /home          btrfs   subvol=@home,defaults,noatime,space_cache,compress=zstd:9 0 2
/dev/mapper/luks-swap swap           swap    defaults,noatime 0 2

#MSS EXtended Automount & Properties
UUID=20339e0b-8343-433e-9209-ac67e31d9d73 /run/media/sarah/MSS_EXtended xfs defaults,noatime 0 0

lsblk -f :

nvme0n1                                                                                                       
├─nvme0n1p1 vfat        FAT32            EFI              8922-E04D                                           
├─nvme0n1p2 crypto_LUKS 1                                 58b7410b-402a-42e6-9514-cce9560b0a76                
└─nvme0n1p3 crypto_LUKS 1                                 059df4b4-5be4-44d6-a23a-de81135eb5b4                
  └─DISK    btrfs                        MSS              f11386cf-342d-47ac-84e6-484b7b2f377d  324.2G    65% /mnt/D
nvme1n1                                                                                                       
├─nvme1n1p1 xfs                          MSS EXtended     20339e0b-8343-433e-9209-ac67e31d9d73                
├─nvme1n1p2                                                                                                   
└─nvme1n1p3 ntfs                                          E658FD3258FD025B```

Crypttab:



## cryptsetup open $swap_device $crypt_swap_name
## get uuid using e.g. lsblk -f
swap_device=/dev/disk/by-uuid/58b7410b-402a-42e6-9514-cce9560b0a76
crypt_swap_name=luks-swap

## one can optionally provide a keyfile device and path on this device
## to the keyfile
keyfile_device=/dev/mapper/luks-disk
keyfile_filename=crypto_keyfile.bin

## additional arguments are given to mount for keyfile_device
## has to start with --options (if so desired)
keyfile_device_mount_options="--options=subvol=@"

## additional arguments are given to cryptsetup
## --allow-discards options is desired in case swap is on SSD partition
cryptsetup_options="--type luks"
luks-disk UUID=059df4b4-5be4-44d6-a23a-de81135eb5b4     /crypto_keyfile.bin luks
luks-swap UUID=58b7410b-402a-42e6-9514-cce9560b0a76     /crypto_keyfile.bin luks

/etc/openswap.conf:

## cryptsetup open $swap_device $crypt_swap_name
## get uuid using e.g. lsblk -f
swap_device=/dev/disk/by-uuid/58b7410b-402a-42e6-9514-cce9560b0a76
crypt_swap_name=luks-swap

## one can optionally provide a keyfile device and path on this device
## to the keyfile
keyfile_device=/dev/mapper/luks-disk
keyfile_filename=crypto_keyfile.bin

## additional arguments are given to mount for keyfile_device
## has to start with --options (if so desired)
keyfile_device_mount_options="--options=subvol=@"

## additional arguments are given to cryptsetup
## --allow-discards options is desired in case swap is on SSD partition
cryptsetup_options="--type luks"

/etc/default/grub:

GRUB_DEFAULT=saved
GRUB_TIMEOUT=10
GRUB_TIMEOUT_STYLE=hidden
GRUB_DISTRIBUTOR="Manjaro"
GRUB_CMDLINE_LINUX_DEFAULT="quiet cryptdevice=UUID=059df4b4-5be4-44d6-a23a-de81135eb5b4:luks-disk root=/dev/mapper/luks-disk apparmor=1 security=apparmor resume=/dev/mapper/luks-81b2dc57-06f5-4471-b484-77c3a516f307 udev.log_priority=3"
GRUB_CMDLINE_LINUX=""

# If you want to enable the save default function, uncomment the following
# line, and set GRUB_DEFAULT to saved.
#GRUB_SAVEDEFAULT="true"

# Preload both GPT and MBR modules so that they are not missed
GRUB_PRELOAD_MODULES="part_gpt part_msdos"

# Uncomment to enable booting from LUKS encrypted devices
#GRUB_ENABLE_CRYPTODISK=y

# Uncomment to use basic console
GRUB_TERMINAL_INPUT=console

# Uncomment to disable graphical terminal
#GRUB_TERMINAL_OUTPUT=console

# The resolution used on graphical terminal
# note that you can use only modes which your graphic card supports via VBE
# you can see them in real GRUB with the command 'videoinfo'
GRUB_GFXMODE=auto

# Uncomment to allow the kernel use the same resolution used by grub
GRUB_GFXPAYLOAD_LINUX=keep

# Uncomment if you want GRUB to pass to the Linux kernel the old parameter
# format "root=/dev/xxx" instead of "root=/dev/disk/by-uuid/xxx"
#GRUB_DISABLE_LINUX_UUID=true

# Uncomment to disable generation of recovery mode menu entries
GRUB_DISABLE_RECOVERY=true

# Uncomment and set to the desired menu colors.  Used by normal and wallpaper
# modes only.  Entries specified as foreground/background.
GRUB_COLOR_NORMAL="light-gray/black"
GRUB_COLOR_HIGHLIGHT="green/black"

# Uncomment one of them for the gfx desired, a image background or a gfxtheme
#GRUB_BACKGROUND="/usr/share/grub/background.png"
GRUB_THEME="/usr/share/grub/themes/manjaro/theme.txt"

# Uncomment to get a beep at GRUB start
#GRUB_INIT_TUNE="480 440 1"
GRUB_ENABLE_CRYPTODISK=y

# Uncomment this option to enable os-prober execution in the grub-mkconfig command
GRUB_DISABLE_OS_PROBER=false

#Custom Options for Dual Graphics Cards
GRUB_CMDLINE_LINUX="radeon.modeset=1 nvdia-drm.modeset=1"

I’m guessing that I’m supposed to modify the GRUB_CMDLINE_LINUX_DEFAULT variable, but I’m not sure what to modify it to. There is obviously a format to the cryptdevice=UUID=059df4b4-5be4-44d6-a23a-de81135eb5b4:luks-disk root=/dev/mapper/luks-disk apparmor=1 security=apparmor resume=/dev/mapper/luks-81b2dc57-06f5-4471-b484-77c3a516f307, but I have no idea what it is.

Is it cryptdevice=UUID=ACTUAL_DISK:luks-disk root=/dev/mapper/luks-disk apparmor=1 security=apparmor resume=/dev/mapper/luks-81b2dc57-06f5-4471-b484-77c3a516f307

or something else?

Can someone please help me solve this? :slight_smile:

EDIT 0:

Main hypothesis, format is:
GRUB_CMDLINE_LINUX_DEFAULT="quiet cryptdevice=UUID=059df4b4-5be4-44d6-a23a-de81135eb5b4:luks-disk root=/dev/mapper/luks-disk apparmor=1 security=apparmor resume=/dev/mapper/luks-swap udev.log_priority=3"

Essentially of the form:
GRUB_CMDLINE_LINUX_DEFAULT="quiet cryptdevice=UUID=UUID_OF_ROOT:luks-disk root=/dev/mapper/luks-disk apparmor=1 security=apparmor resume=/dev/mapper/luks-swap udev.log_priority=3"

With the last tag being the swap ID.

EDIT 1:

After re-making the grub config file again (with all the grub-install options after mounting the EFI partition) and running update-grub, the following error occurs:

(Updated the tables with the latest values since editing)
Upon getting past the password screen, I get:

[FAILED] Failed to Start Remount Root and Kernel File Systems

Using journalctl -p 2..3 -b -1 I got these errors:


Sep 10 10:43:23 MidnightStarSign kernel: 
Sep 10 10:43:24 MidnightStarSign kernel: BTRFS error (device dm-0: state M): cannot disable free space tree
Sep 10 10:43:23 MidnightStarSign systemd-remount-fs[659]: /usr/bin/mount for / exited with exit status 32.
Sep 10 10:43:23 MidnightStarSign systemd-udevd[704]: /usr/lib/udev/rules.d/40-brscan5.rules:17 Invalid key 'SYSFS'
Sep 10 10:43:25 MidnightStarSign systemd-remount-fs[1815]: /usr/bin/mount for / exited with exit status 32.
Sep 10 10:43:25 MidnightStarSign systemd[1]: Failed to start Remount Root and Kernel File Systems.
Sep 10 10:43:25 MidnightStarSign kernel: BTRFS error (device dm-0: state M): cannot disable free space tree
Sep 10 10:43:25 MidnightStarSign systemd-remount-fs[1893]: /usr/bin/mount for / exited with exit status 32.
Sep 10 10:43:25 MidnightStarSign systemd[1]: Failed to start Remount Root and Kernel File Systems.
Sep 10 10:43:25 MidnightStarSign kernel: BTRFS error (device dm-0: state M): cannot disable free space tree
Sep 10 10:43:25 MidnightStarSign systemd-remount-fs[1968]: /usr/bin/mount for / exited with exit status 32.
Sep 10 10:43:25 MidnightStarSign systemd[1]: Failed to start Remount Root and Kernel File Systems.
Sep 10 10:43:25 MidnightStarSign kernel: BTRFS error (device dm-0: state M): cannot disable free space tree
Sep 10 10:43:25 MidnightStarSign kernel: usb 5-3: 3:1: cannot get freq at ep 0x84
Sep 10 10:43:26 MidnightStarSign systemd-remount-fs[2115]: /usr/bin/mount for / exited with exit status 32.
Sep 10 10:43:26 MidnightStarSign systemd[1]: Failed to start Remount Root and Kernel File Systems.
Sep 10 10:43:26 MidnightStarSign kernel: BTRFS error (device dm-0: state M): cannot disable free space tree
Sep 10 10:43:26 MidnightStarSign systemd[1]: Failed to start Remount Root and Kernel File Systems.
Sep 10 10:43:26 MidnightStarSign systemd[1]: Failed to start Remount Root and Kernel File Systems.
Sep 10 10:43:26 MidnightStarSign systemd[1]: Failed to start Remount Root and Kernel File Systems.
Sep 10 10:43:43 MidnightStarSign systemd-cryptsetup[2318]: Device luks-disk is still in use.
Sep 10 10:43:43 MidnightStarSign systemd-cryptsetup[2318]: Failed to deactivate: Device or resource busy
Sep 10 10:43:43 MidnightStarSign systemd-cryptsetup[2341]: device-mapper: remove ioctl on luks-swap  failed: Device or resource busy
Sep 10 10:43:44 MidnightStarSign kernel: watchdog: watchdog0: watchdog did not stop!

Any ideas for what I should edit to get it to boot properly?

I think general principle from this topic can be applied for yours as well

1 Like

A reason is that this LUKS ID does not exist in Fstab.

That being said, this LUKS ID is old in Grub config after restoring backup, but it does not match the ID in Fstab.

Try to change it to luks-58b7410b-402a-42e6-9514-cce9560b0a76 in Grub config, then run update-grub

1 Like

Ok, I switched the /etc/openswap.conf to:

## cryptsetup open $swap_device $crypt_swap_name
## get uuid using e.g. lsblk -f
swap_device=/dev/disk/by-uuid/58b7410b-402a-42e6-9514-cce9560b0a76
crypt_swap_name=luks-swap

## one can optionally provide a keyfile device and path on this device
## to the keyfile
keyfile_device=/dev/mapper/luks-disk
keyfile_filename=crypto_keyfile.bin

## additional arguments are given to mount for keyfile_device
## has to start with --options (if so desired)
keyfile_device_mount_options="--options=subvol=@"

## additional arguments are given to cryptsetup
## --allow-discards options is desired in case swap is on SSD partition
cryptsetup_options="--type luks"

And executed both:

mkinitcpio -P
update-grub

But still no luck. The error message about /dev/disk/by-uuid disappeared, but was replaced with
[Failed] Failed to start Remount Root and Kernel File Systems

Any ideas?

This was from the journalctl -b -1 (It seems that it can’t mount / for some reason):

Sep 10 10:43:24 MidnightStarSign audit[673]: SYSCALL arch=c000003e syscall=1 success=yes exit=30921 a0=6 a1=5622b725ba80 a2=78c9 a3=0 items=0 ppid=671 pid=673 auid=4294967295 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=(none) ses=4294967295 comm="apparmor_parser" exe="/usr/bin/apparmor_parser" subj==unconfined key=(null)
Sep 10 10:43:24 MidnightStarSign audit: PROCTITLE proctitle=2F7362696E2F61707061726D6F725F706172736572002D2D7265706C616365002D2D002F6574632F61707061726D6F722E64
Sep 10 10:43:23 MidnightStarSign systemd-remount-fs[659]: /usr/bin/mount for / exited with exit status 32.
Sep 10 10:43:23 MidnightStarSign systemd-modules-load[657]: Inserted module 'crypto_user'
Sep 10 10:43:23 MidnightStarSign systemd-modules-load[657]: Inserted module 'sg'
Sep 10 10:43:23 MidnightStarSign systemd-udevd[704]: /usr/lib/udev/rules.d/40-brscan5.rules:17 Invalid key 'SYSFS'
Sep 10 10:43:24 MidnightStarSign audit[1]: SERVICE_START pid=1 uid=0 auid=4294967295 ses=4294967295 subj==unconfined msg='unit=systemd-journald comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'
Sep 10 10:43:23 MidnightStarSign systemd-modules-load[657]: Inserted module 'vboxdrv'
Sep 10 10:43:24 MidnightStarSign audit[1]: SERVICE_START pid=1 uid=0 auid=4294967295 ses=4294967295 subj==unconfined msg='unit=systemd-sysctl comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'
Sep 10 10:43:24 MidnightStarSign apparmor.systemd[649]: Restarting AppArmor
Sep 10 10:43:24 MidnightStarSign apparmor.systemd[649]: Reloading AppArmor profiles
Sep 10 10:43:23 MidnightStarSign systemd-modules-load[657]: Inserted module 'vboxnetadp'
Sep 10 10:43:23 MidnightStarSign systemd-modules-load[657]: Inserted module 'vboxnetflt'
Sep 10 10:43:23 MidnightStarSign systemd-modules-load[657]: Inserted module 'i2c_dev'
Sep 10 10:43:23 MidnightStarSign systemd-modules-load[657]: Inserted module 'uinput'
Sep 10 10:43:24 MidnightStarSign systemd[1]: Starting Flush Journal to Persistent Storage...
Sep 10 10:43:24 MidnightStarSign systemd[1]: Finished Apply Kernel Variables.
Sep 10 10:43:24 MidnightStarSign systemd-remount-fs[662]: mount: /: mount point not mounted or bad option.
Sep 10 10:43:24 MidnightStarSign systemd-remount-fs[662]:        dmesg(1) may have more information after failed mount system call.
Sep 10 10:43:24 MidnightStarSign systemd-journald[656]: Time spent on flushing to /var/log/journal/646316c5de21472bbbb570c8c7d19086 is 18.209ms for 1636 entries.
Sep 10 10:43:24 MidnightStarSign systemd-journald[656]: System Journal (/var/log/journal/646316c5de21472bbbb570c8c7d19086) is 2.5G, max 4.0G, 1.4G free.
Sep 10 10:43:24 MidnightStarSign systemd-journald[656]: Received client request to flush runtime journal.
Sep 10 10:43:24 MidnightStarSign kernel: loop: module loaded
Sep 10 10:43:24 MidnightStarSign kernel: loop3: detected capacity change from 0 to 126888

I mean that you should change LUKS ID for swap in /etc/default/grub

Change

to

 resume=/dev/mapper/luks-58b7410b-402a-42e6-9514-cce9560b0a76

I did not mean /etc/openswap.conf

1 Like

I believe I did that too, sorry:


GRUB_CMDLINE_LINUX_DEFAULT="quiet cryptdevice=UUID=059df4b4-5be4-44d6-a23a-de81135eb5b4:luks-disk root=/dev/mapper/luks-disk apparmor=1 security=apparmor resume=/dev/mapper/luks-swap udev.log_priority=3"

I updated the original post with the latest values, and adjusted some of the formatting of the tables.

I updated the original post with the journalctl entry from last boot.

@megavolt If you have any input, it would be greatly appreciated :smiley:

I guess the issue is related to the old version of space_cache in fstab.

Try to change space_cache to space_cache=v2 in fstab.
OR
Remove space_cache

1 Like

Yep, I did that and it booted.

That’s great, but I can’t log in to the main account or the root account on TTY.

It says that the password is incorrect for both, but it is not (as verified from the KDE login screen “see your password” feature.

I didn’t think this was a big deal, so I live-usb’d and chrooted into the @ directory and reset both via passwd. Then I tried the log in again, and nothing :frowning:

Still says password is incorrect.

This was the journalctl messages:

Sep 10 12:10:16 MidnightStarSign kernel: 
Sep 10 12:10:16 MidnightStarSign kernel: [drm:radeon_dp_link_train [radeon]] *ERROR* clock recovery reached max voltage
Sep 10 12:10:16 MidnightStarSign kernel: [drm:radeon_dp_link_train [radeon]] *ERROR* clock recovery failed
Sep 10 12:10:16 MidnightStarSign systemd-udevd[711]: /usr/lib/udev/rules.d/40-brscan5.rules:17 Invalid key 'SYSFS'
Sep 10 12:10:18 MidnightStarSign systemd[1]: Failed to start Snap Daemon.
Sep 10 12:10:18 MidnightStarSign systemd[1]: Failed to start Snap Daemon.
Sep 10 12:10:18 MidnightStarSign kernel: usb 5-3: 3:1: cannot get freq at ep 0x84
Sep 10 12:10:18 MidnightStarSign systemd[1]: Failed to start Snap Daemon.
Sep 10 12:10:19 MidnightStarSign systemd[1]: Failed to start Snap Daemon.
Sep 10 12:10:19 MidnightStarSign systemd[1]: Failed to start Snap Daemon.
Sep 10 12:10:20 MidnightStarSign systemd[1]: Failed to start Snap Daemon.
Sep 10 12:10:47 MidnightStarSign sddm-helper[2789]: pam_shells(sddm:auth): /etc/shells is either world writable or not a normal file
Sep 10 12:11:15 MidnightStarSign sddm-helper[2797]: pam_shells(sddm:auth): /etc/shells is either world writable or not a normal file
Sep 10 12:11:31 MidnightStarSign login[2799]: pam_securetty(login:auth): /etc/securetty is either world writable or not a normal file
Sep 10 12:11:31 MidnightStarSign login[2799]: pam_shells(login:auth): /etc/shells is either world writable or not a normal file
Sep 10 12:11:40 MidnightStarSign login[2799]: pam_securetty(login:auth): /etc/securetty is either world writable or not a normal file
Sep 10 12:11:40 MidnightStarSign login[2799]: pam_shells(login:auth): /etc/shells is either world writable or not a normal file
Sep 10 12:12:01 MidnightStarSign login[2799]: pam_shells(login:auth): /etc/shells is either world writable or not a normal file
Sep 10 12:12:31 MidnightStarSign sddm-helper[2807]: pam_shells(sddm:auth): /etc/shells is either world writable or not a normal file
Sep 10 12:12:36 MidnightStarSign systemd-cryptsetup[3065]: Device luks-disk is still in use.
Sep 10 12:12:36 MidnightStarSign systemd-cryptsetup[3065]: Failed to deactivate: Device or resource busy
Sep 10 12:12:36 MidnightStarSign systemd-cryptsetup[3067]: device-mapper: remove ioctl on luks-swap  failed: Device or resource busy
Sep 10 12:12:36 MidnightStarSign kernel: watchdog: watchdog0: watchdog did not stop!

It looks like the permission is wrong after restore.

Run manjaro-chroot to access your system.
Then check what is the output:

ls -al /etc/shells
ls -al /etc/securetty
1 Like

Looks ok to me

sh-5.1# ls -al /etc/shells
-rwxrwxrwx 1 root root 124 Jun  6  2020 /etc/shells
sh-5.1# ls -al /etc/securetty
-rwxrwxrwx 1 root root 139 Jun  9 03:53 /etc/securetty

All permissions are unfortunately incorrect.

That should be:

ls -al /etc/shells /etc/securetty 
-rw-r--r-- 1 root root 139  9. Jun 09:53 /etc/securetty
-rw-r--r-- 1 root root 126  5. Aug 20:31 /etc/shells

Try to change them:

# chmod 644 /etc/securetty
# chmod 644 /etc/shells

How did you restore this backup?
I think that you did not restore the backup but used cp which causes a lot of incorrect permissions.

1 Like

Yeah, was using cp -av, I would have expected it to copy the permissions :frowning:

Delete the wrong system, then you can restore the backup again.

Use rsync -a that preserves the same permissions, ownership and timestamp of files and folders.

1 Like

If you use both same btrfs on two different disks (backup and your system), then use the native CLI btrfs send | btrfs receive

How to copy any btrfs volume to your disk:

1 Like

Yep, doing that now, it’s actually very fast :slight_smile:

This topic was automatically closed 2 days after the last reply. New replies are no longer allowed.