Manjaro not booting after updating

Support
1

Today I try to boot up manjaro like normal, after entering the password to decrypt the partition like usual my computer decide not to work. After using the liveUSB to see the error logs this is what I find :

Feb 15 23:58:35 simple-x555qg kernel: ACPI BIOS Error (bug): Could not resolve symbol [^^^PB2.VGA.AFN7], AE_NOT_FOUND (20210730/psargs-330)
Feb 15 23:58:35 simple-x555qg kernel: ACPI Error: Aborting method \_SB.PCI0.VGA.LCDD._BCM due to previous error (AE_NOT_FOUND) (20210730/psparse-529)
Feb 15 23:58:35 simple-x555qg kernel: tpm_crb MSFT0101:00: [Firmware Bug]: ACPI region does not cover the entire command/response buffer. [mem 0xcd958000-0xcd958fff flags 0x200] vs cd958000 4000
Feb 15 23:58:35 simple-x555qg kernel: tpm_crb MSFT0101:00: can't request region for resource [mem 0xcd958000-0xcd958fff]
Feb 15 23:58:35 simple-x555qg kernel: sp5100-tco sp5100-tco: Watchdog hardware is disabled
Feb 15 23:58:45 simple-x555qg kernel: debugfs: File 'radeon_ring_gfx' in directory '0' already present!
Feb 15 23:58:45 simple-x555qg kernel: debugfs: File 'radeon_ring_cp1' in directory '0' already present!
Feb 15 23:58:45 simple-x555qg kernel: debugfs: File 'radeon_ring_cp2' in directory '0' already present!
Feb 15 23:58:45 simple-x555qg kernel: debugfs: File 'radeon_ring_dma1' in directory '0' already present!
Feb 15 23:58:45 simple-x555qg kernel: debugfs: File 'radeon_ring_dma2' in directory '0' already present!
Feb 15 23:58:59 simple-x555qg kernel: debugfs: File 'radeon_ring_gfx' in directory '0' already present!
Feb 15 23:58:59 simple-x555qg kernel: debugfs: File 'radeon_ring_cp1' in directory '0' already present!
Feb 15 23:58:59 simple-x555qg kernel: debugfs: File 'radeon_ring_cp2' in directory '0' already present!
Feb 15 23:58:59 simple-x555qg kernel: debugfs: File 'radeon_ring_dma1' in directory '0' already present!
Feb 15 23:58:59 simple-x555qg kernel: debugfs: File 'radeon_ring_dma2' in directory '0' already present!
Feb 15 23:59:18 simple-x555qg lightdm[977]: gkr-pam: unable to locate daemon control file
Feb 16 00:13:14 simple-x555qg systemd-coredump[89349]: Failed to connect to coredump service: Connection refused
Feb 16 00:13:15 simple-x555qg kernel: debugfs: File 'radeon_ring_gfx' in directory '0' already present!
Feb 16 00:13:15 simple-x555qg kernel: debugfs: File 'radeon_ring_cp1' in directory '0' already present!
Feb 16 00:13:15 simple-x555qg kernel: debugfs: File 'radeon_ring_cp2' in directory '0' already present!
Feb 16 00:13:15 simple-x555qg kernel: debugfs: File 'radeon_ring_dma1' in directory '0' already present!
Feb 16 00:13:15 simple-x555qg kernel: debugfs: File 'radeon_ring_dma2' in directory '0' already present!
Feb 16 00:13:16 simple-x555qg systemd-cryptsetup[89485]: Device luks-51fc926e-223c-49f7-a3e9-8f79d4243fd9 is still in use.
Feb 16 00:13:16 simple-x555qg systemd-cryptsetup[89485]: Failed to deactivate: Device or resource busy
Feb 16 00:13:16 simple-x555qg systemd-cryptsetup[89492]: device-mapper: remove ioctl on luks-695ef6c2-cd3a-44f1-8a0b-96b6743efa2a  failed: Device or resource busy

And this is the inxi data:

12System:
  12Kernel 5.15.16-1-MANJARO 12arch x86_64 12bits 64 12Desktop Xfce 12v 4.16.0 12Distro Manjaro Linux

12Machine:
  12Type Laptop 12System ASUSTeK 12product X555QG 12v 1.0 12serial <filter>
  12Mobo ASUSTeK 12model X555QG 12v 1.0 12serial <filter> 12UEFI American Megatrends 12v X555QG.329
    12date 05/10/2019

12Battery:
  12ID-1 BAT0 12charge 2.0 Wh (100.0%) 12condition 2.0/37.3 Wh (5.3%) 12volts 7.6 12min 7.6

12CPU:
  12Info quad core 12model AMD A12-9720P RADEON R7 12 COMPUTE CORES 4C+8G 12bits 64 12type MT MCP 12cache
    12L2 2 MiB
  12Speed (MHz) 12avg 1400 12min/max 1400/2700 12cores 121 1400 122 1400 123 1400 124 1400

12Graphics:
  12Device-1 AMD Wani [Radeon R5/R6/R7 Graphics] 12driver amdgpu 12v kernel
  12Device-2 AMD Sun XT [Radeon HD 8670A/8670M/8690M / R5 M330 M430 Radeon 520 Mobile]
    12driver radeon 12v kernel
  12Device-3 Chicony USB2.0 VGA UVC WebCam 12type USB 12driver uvcvideo
  12Display 12server X.Org 12v 1.21.1.3 12driver 12X 12loaded radeon 12unloaded amdgpu,modesetting 12gpu amdgpu
    12resolution 1366x768~60Hz
  12OpenGL 12renderer AMD Radeon R7 Graphics (CARRIZO DRM 3.42.0 5.15.16-1-MANJARO LLVM 13.0.1)
    12v 4.6 Mesa 21.3.7

12Audio:
  12Device-1 AMD Kabini HDMI/DP Audio 12driver snd_hda_intel
  12Device-2 AMD Family 15h Audio 12driver snd_hda_intel
  12Sound Server-1 ALSA 12v k5.15.16-1-MANJARO 12running yes
  12Sound Server-2 PulseAudio 12v 15.0 12running yes

12Network:
  12Device-1 Qualcomm Atheros QCA9565 / AR9565 Wireless Network Adapter 12driver ath9k
  12IF wlp1s0 12state down 12mac <filter>
  12Device-2 Realtek RTL8111/8168/8411 PCI Express Gigabit Ethernet 12driver r8169
  12IF enp2s0 12state up 12speed 1000 Mbps 12duplex full 12mac <filter>

12Bluetooth:
  12Device-1 IMC Networks 12type USB 12driver btusb
  12Report rfkill 12ID hci0 12rfk-id 1 12state down 12bt-service N/A 12rfk-block 12hardware no 12software yes
    12address see --recommends

12Drives:
  12Local Storage 12total 939.01 GiB 12used 353.49 GiB (37.6%)
  12ID-1 /dev/sda 12vendor HGST (Hitachi) 12model HTS541010A9E680 12size 931.51 GiB
  12ID-2 /dev/sdb 12type USB 12model General USB Flah Disk 12size 7.5 GiB

12Partition:
  12ID-1 / 12size 903.91 GiB 12used 353.49 GiB (39.1%) 12fs ext4 12dev /dev/dm-0

12Swap:
  12Alert No swap data was found.

12Sensors:
  12System Temperatures 12cpu 44.4 C 12mobo N/A
  12Fan Speeds (RPM) 12cpu 2500
  12GPU 12device amdgpu 12temp 44.0 C 12device radeon 12temp 36.0 C

12Info:
  12Processes 213 12Uptime 1h 34m 12Memory 10.72 GiB 12used 2.78 GiB (26.0%)
  12Client Unknown Client: systemd 12inxi 3.3.14

Thanks for the help.

2

I assume you’re able to manually unlock and use the partition, since you were able to do so in a live ISO session (and access the filesystem within.)

While in a manjaro-chroot, can you rebuild the initramfs for all kernels, and then re-update grub?

mkinitcpio -P

update-grub

Are you able to use a different kernel to boot into the system? Perhaps with LTS kernel 5.15?

Why are two different LUKS devices showing in the log?

What do the contents of the following hold:

/etc/crypttab

/etc/fstab

/etc/default/grub

/etc/mkinitcpio.conf
3

Hi, after doing the mkinitcpio -P and update-grub this were the results:
mkinitpcpio -P

sh-5.1# mkinitcpio -P
==> Building image from preset: /etc/mkinitcpio.d/linux515.preset: 'default'
  -> -k /boot/vmlinuz-5.15-x86_64 -c /etc/mkinitcpio.conf -g /boot/initramfs-5.15-x86_64.img
==> Starting build: 5.15.28-1-MANJARO
  -> Running build hook: [base]
  -> Running build hook: [udev]
  -> Running build hook: [autodetect]
  -> Running build hook: [modconf]
  -> Running build hook: [block]
  -> Running build hook: [keyboard]
  -> Running build hook: [keymap]
  -> Running build hook: [consolefont]
==> WARNING: consolefont: no font found in configuration
  -> Running build hook: [encrypt]
==> WARNING: Possibly missing firmware for module: qat_4xxx
  -> Running build hook: [openswap]
  -> Running build hook: [resume]
  -> Running build hook: [filesystems]
  -> Running build hook: [fsck]
==> Generating module dependencies
==> Creating gzip-compressed initcpio image: /boot/initramfs-5.15-x86_64.img
==> Image generation successful
==> Building image from preset: /etc/mkinitcpio.d/linux515.preset: 'fallback'
  -> -k /boot/vmlinuz-5.15-x86_64 -c /etc/mkinitcpio.conf -g /boot/initramfs-5.15-x86_64-fallback.img -S autodetect
==> Starting build: 5.15.28-1-MANJARO
  -> Running build hook: [base]
  -> Running build hook: [udev]
  -> Running build hook: [modconf]
  -> Running build hook: [block]
==> WARNING: Possibly missing firmware for module: bfa
==> WARNING: Possibly missing firmware for module: qla2xxx
==> WARNING: Possibly missing firmware for module: qed
==> WARNING: Possibly missing firmware for module: qla1280
  -> Running build hook: [keyboard]
  -> Running build hook: [keymap]
  -> Running build hook: [consolefont]
==> WARNING: consolefont: no font found in configuration
  -> Running build hook: [encrypt]
==> WARNING: Possibly missing firmware for module: qat_4xxx
  -> Running build hook: [openswap]
  -> Running build hook: [resume]
  -> Running build hook: [filesystems]
  -> Running build hook: [fsck]
==> Generating module dependencies
==> Creating gzip-compressed initcpio image: /boot/initramfs-5.15-x86_64-fallback.img
==> Image generation successful
==> Building image from preset: /etc/mkinitcpio.d/linux516.preset: 'default'
  -> -k /boot/vmlinuz-5.16-x86_64 -c /etc/mkinitcpio.conf -g /boot/initramfs-5.16-x86_64.img
==> Starting build: 5.16.14-1-MANJARO
  -> Running build hook: [base]
  -> Running build hook: [udev]
  -> Running build hook: [autodetect]
  -> Running build hook: [modconf]
  -> Running build hook: [block]
==> WARNING: Possibly missing firmware for module: xhci_pci
  -> Running build hook: [keyboard]
  -> Running build hook: [keymap]
  -> Running build hook: [consolefont]
==> WARNING: consolefont: no font found in configuration
  -> Running build hook: [encrypt]
==> WARNING: Possibly missing firmware for module: qat_4xxx
  -> Running build hook: [openswap]
  -> Running build hook: [resume]
  -> Running build hook: [filesystems]
  -> Running build hook: [fsck]
==> Generating module dependencies
==> Creating gzip-compressed initcpio image: /boot/initramfs-5.16-x86_64.img
==> Image generation successful
==> Building image from preset: /etc/mkinitcpio.d/linux516.preset: 'fallback'
  -> -k /boot/vmlinuz-5.16-x86_64 -c /etc/mkinitcpio.conf -g /boot/initramfs-5.16-x86_64-fallback.img -S autodetect
==> Starting build: 5.16.14-1-MANJARO
  -> Running build hook: [base]
  -> Running build hook: [udev]
  -> Running build hook: [modconf]
  -> Running build hook: [block]
==> WARNING: Possibly missing firmware for module: bfa
==> WARNING: Possibly missing firmware for module: qla2xxx
==> WARNING: Possibly missing firmware for module: qed
==> WARNING: Possibly missing firmware for module: qla1280
==> WARNING: Possibly missing firmware for module: xhci_pci
  -> Running build hook: [keyboard]
  -> Running build hook: [keymap]
  -> Running build hook: [consolefont]
==> WARNING: consolefont: no font found in configuration
  -> Running build hook: [encrypt]
==> WARNING: Possibly missing firmware for module: qat_4xxx
  -> Running build hook: [openswap]
  -> Running build hook: [resume]
  -> Running build hook: [filesystems]
  -> Running build hook: [fsck]
==> Generating module dependencies
==> Creating gzip-compressed initcpio image: /boot/initramfs-5.16-x86_64-fallback.img
==> Image generation successful

update-grub

sh-5.1# update-grub
Generating grub configuration file ...
Found theme: /usr/share/grub/themes/manjaro/theme.txt
Found linux image: /boot/vmlinuz-5.16-x86_64
Found initrd image: /boot/initramfs-5.16-x86_64.img
Found initrd fallback image: /boot/initramfs-5.16-x86_64-fallback.img
Found linux image: /boot/vmlinuz-5.15-x86_64
Found initrd image: /boot/initramfs-5.15-x86_64.img
Found initrd fallback image: /boot/initramfs-5.15-x86_64-fallback.img
Warning: os-prober will be executed to detect other bootable partitions.
Its output will be used to detect bootable binaries on them and create new boot entries.
grub-probe: error: cannot find a GRUB drive for /dev/sdb1.  Check your device.map.
grub-probe: error: cannot find a GRUB drive for /dev/sdb1.  Check your device.map.
Adding boot menu entry for UEFI Firmware Settings ...
done

In regard to the other files
crypttab

# /etc/crypttab: mappings for encrypted partitions.
#
# Each mapped device will be created in /dev/mapper, so your /etc/fstab
# should use the /dev/mapper/<name> paths for encrypted devices.
#
# See crypttab(5) for the supported syntax.
#
# NOTE: You need not list your root (/) partition here, but it must be set up
#       beforehand by the initramfs (/etc/mkinitcpio.conf). The same applies
#       to encrypted swap, which should be set up with mkinitcpio-openswap
#       for resume support.
#
# <name>               <device>                         <password> <options>
luks-51fc926e-223c-49f7-a3e9-8f79d4243fd9 UUID=51fc926e-223c-49f7-a3e9-8f79d4243fd9     /crypto_keyfile.bin luks
luks-695ef6c2-cd3a-44f1-8a0b-96b6743efa2a UUID=695ef6c2-cd3a-44f1-8a0b-96b6743efa2a     /crypto_keyfile.bin luks

fstab

#
# Use 'blkid' to print the universally unique identifier for a device; this may
# be used with UUID= as a more robust way to name devices that works even if
# disks are added and removed. See fstab(5).
#
# <file system>             <mount point>  <type>  <options>  <dump>  <pass>
UUID=B3E1-96FC                            /boot/efi      vfat    umask=0077 0 2
/dev/mapper/luks-51fc926e-223c-49f7-a3e9-8f79d4243fd9 /              ext4    defaults,noatime 0 1
/dev/mapper/luks-695ef6c2-cd3a-44f1-8a0b-96b6743efa2a swap           swap    defaults,noatime 0 0

grub

GRUB_DEFAULT=saved
GRUB_TIMEOUT=5
GRUB_TIMEOUT_STYLE=hidden
GRUB_DISTRIBUTOR="Manjaro"
GRUB_CMDLINE_LINUX_DEFAULT="quiet cryptdevice=UUID=51fc926e-223c-49f7-a3e9-8f79d4243fd9:luks-51fc926e-223c-49f7-a3e9-8f79d4243fd9 root=/dev/mapper/luks-51fc926e-223c-49f7-a3e9-8f79d4243fd9 apparmor=1 security=apparmor resume=/dev/mapper/luks-695ef6c2-cd3a-44f1-8a0b-96b6743efa2a udev.log_priority=3"
GRUB_CMDLINE_LINUX=""

# If you want to enable the save default function, uncomment the following
# line, and set GRUB_DEFAULT to saved.
GRUB_SAVEDEFAULT=true

# Uncomment to disable submenus in boot menu
#GRUB_DISABLE_SUBMENU=y

# Preload both GPT and MBR modules so that they are not missed
GRUB_PRELOAD_MODULES="part_gpt part_msdos"

# Uncomment to enable booting from LUKS encrypted devices
#GRUB_ENABLE_CRYPTODISK=y

# Uncomment to use basic console
GRUB_TERMINAL_INPUT=console

# Uncomment to disable graphical terminal
#GRUB_TERMINAL_OUTPUT=console

# The resolution used on graphical terminal
# note that you can use only modes which your graphic card supports via VBE
# you can see them in real GRUB with the command 'videoinfo'
GRUB_GFXMODE=auto

# Uncomment to allow the kernel use the same resolution used by grub
GRUB_GFXPAYLOAD_LINUX=keep

# Uncomment if you want GRUB to pass to the Linux kernel the old parameter
# format "root=/dev/xxx" instead of "root=/dev/disk/by-uuid/xxx"
#GRUB_DISABLE_LINUX_UUID=true

# Uncomment to disable generation of recovery mode menu entries
GRUB_DISABLE_RECOVERY=true

# Uncomment this option to enable os-prober execution in the grub-mkconfig command
GRUB_DISABLE_OS_PROBER=false

# Uncomment and set to the desired menu colors.  Used by normal and wallpaper
# modes only.  Entries specified as foreground/background.
GRUB_COLOR_NORMAL="light-gray/black"
GRUB_COLOR_HIGHLIGHT="green/black"

# Uncomment one of them for the gfx desired, a image background or a gfxtheme
#GRUB_BACKGROUND="/usr/share/grub/background.png"
GRUB_THEME="/usr/share/grub/themes/manjaro/theme.txt"

# Uncomment to get a beep at GRUB start
#GRUB_INIT_TUNE="480 440 1"

# Uncomment to ensure that the root filesystem is mounted read-only so that
# systemd-fsck can run the check automatically. We use 'fsck' by default, which
# needs 'rw' as boot parameter, to avoid delay in boot-time. 'fsck' needs to be
# removed from 'mkinitcpio.conf' to make 'systemd-fsck' work.
# See also Arch-Wiki: https://wiki.archlinux.org/index.php/Fsck#Boot_time_checking
#GRUB_ROOT_FS_RO=true
GRUB_ENABLE_CRYPTODISK=y

mkinitcpio.conf

# vim:set ft=sh
# MODULES
# The following modules are loaded before any boot hooks are
# run.  Advanced users may wish to specify all system modules
# in this array.  For instance:
#     MODULES=(piix ide_disk reiserfs)
MODULES=""

# BINARIES
# This setting includes any additional binaries a given user may
# wish into the CPIO image.  This is run last, so it may be used to
# override the actual binaries included by a given hook
# BINARIES are dependency parsed, so you may safely ignore libraries
BINARIES=()

# FILES
# This setting is similar to BINARIES above, however, files are added
# as-is and are not parsed in any way.  This is useful for config files.
FILES="/crypto_keyfile.bin"

# HOOKS
# This is the most important setting in this file.  The HOOKS control the
# modules and scripts added to the image, and what happens at boot time.
# Order is important, and it is recommended that you do not change the
# order in which HOOKS are added.  Run 'mkinitcpio -H <hook name>' for
# help on a given hook.
# 'base' is _required_ unless you know precisely what you are doing.
# 'udev' is _required_ in order to automatically load modules
# 'filesystems' is _required_ unless you specify your fs modules in MODULES
# Examples:
##   This setup specifies all modules in the MODULES setting above.
##   No raid, lvm2, or encrypted root is needed.
#    HOOKS=(base)
#
##   This setup will autodetect all modules for your system and should
##   work as a sane default
#    HOOKS=(base udev autodetect block filesystems)
#
##   This setup will generate a 'full' image which supports most systems.
##   No autodetection is done.
#    HOOKS=(base udev block filesystems)
#
##   This setup assembles a pata mdadm array with an encrypted root FS.
##   Note: See 'mkinitcpio -H mdadm' for more information on raid devices.
#    HOOKS=(base udev block mdadm encrypt filesystems)
#
##   This setup loads an lvm2 volume group on a usb device.
#    HOOKS=(base udev block lvm2 filesystems)
#
##   NOTE: If you have /usr on a separate partition, you MUST include the
#    usr, fsck and shutdown hooks.
HOOKS="base udev autodetect modconf block keyboard keymap consolefont encrypt openswap resume filesystems fsck"

# COMPRESSION
# Use this to compress the initramfs image. By default, gzip compression
# is used. Use 'cat' to create an uncompressed image.
#COMPRESSION="gzip"
#COMPRESSION="bzip2"
#COMPRESSION="lzma"
#COMPRESSION="xz"
#COMPRESSION="lzop"
#COMPRESSION="lz4"
#COMPRESSION="zstd"

# COMPRESSION_OPTIONS
# Additional options for the compressor
#COMPRESSION_OPTIONS=()
4

Something odd is happening. For some reason, it appears your filesystem is not properly unmounting before the LUKS container attempts to re-lock.

Everything looks fine, but you can try changing a few things:

  1. In your HOOKS for mkinitcpio.conf, place block before autodetect. So that it looks like:
base udev block autodetect ...

Then rebuild the initramfs again.

  1. Comment out the entry for the root LUKS device in your crypttab, since it is the initramfs that handles it (as well as already defined in Grub).

  2. In a clean environment (such as a live ISO), unlock the LUKS device, but do not mount anything. Run a full fsck.ext4 on the filesystem.

sudo cryptsetup luksOpen /dev/disk/by-uuid/51fc926e-223c-49f7-a3e9-8f79d4243fd9 crypto_check_me

sudo fsck.ext4 -f /dev/mapper/crypto_check_me

# Then after it completes...

sudo cryptsetup luksClose crypto_check_me
  1. Remove the noatime option from the entry for your swap under fstab

Everything has been working fine up until just recently?

Are you doing anything outside the default setup?

I’m not familiar with openswap.

EDIT: It appears that mkinitcpio-openswap is a HOOK found in the AUR that hasn’t been updated since 2017.

Why are you using it? Without it, your swap should still be available after booting up into a working system.

5

What I found interesting is this note from /etc/crypttab - and then … the / partition as well as the swap partition listed nonetheless …

It may be an old installation and how it was done then?
But there should have been alerts on updates that things are going to change and .pacnew files …

6

After what you suggested the problem still exists. This is the new error log:

Mar 28 21:59:33 simple-x555qg kernel: kobject_add_internal failed for ashmem with -EEXIST, don't try to register things with the same name in the same directory.
Mar 28 21:59:33 simple-x555qg kernel: ashmem: failed to register misc device!
Mar 28 21:59:33 simple-x555qg kernel: debugfs: Directory 'binder' with parent '/' already present!
Mar 28 21:59:33 simple-x555qg systemd[1]: Failed to start Load Kernel Modules.
Mar 28 21:59:30 simple-x555qg systemd-modules-load[295]: Failed to insert module 'binder_linux': Device or resource busy

As to why I’m using mkinitcpio-openswap, I don’t have any Idea. I haven’t touch the swap partition since the installation of manjaro of the beginning of this year. My best guess could be that some package from the AUR modify the swap.

7

Remove it and rebuild the initramfs.

From a manjaro-chroot:

Remove the entry for openswap from the HOOKS line in your /etc/mkinitcpio.conf file.

Then rebuild the initramfs.

mkinitpcio -P

Then remove the package.

pamac remove mkinitcpio-openswap
8

Hello, after removing the openswap, this are now the errors:

Feb 16 00:13:14 simple-x555qg systemd-coredump[89349]: Failed to connect to coredump service: Connection refused
Feb 16 00:13:15 simple-x555qg kernel: debugfs: File 'radeon_ring_gfx' in directory '0' already present!
Feb 16 00:13:15 simple-x555qg kernel: debugfs: File 'radeon_ring_cp1' in directory '0' already present!
Feb 16 00:13:15 simple-x555qg kernel: debugfs: File 'radeon_ring_cp2' in directory '0' already present!
Feb 16 00:13:15 simple-x555qg kernel: debugfs: File 'radeon_ring_dma1' in directory '0' already present!
Feb 16 00:13:15 simple-x555qg kernel: debugfs: File 'radeon_ring_dma2' in directory '0' already present!
Feb 16 00:13:16 simple-x555qg systemd-cryptsetup[89485]: Device luks-51fc926e-223c-49f7-a3e9-8f79d4243fd9 is still in use.
9

Some questions.

  1. Did the fsck.ext4 for the root filesystem return any errors?

  2. Are you using systemd-boot or Grub?

  3. Can you re-post the updated contents of the relevant files now?

10

For the first question, this is the result of fsck.ext4 in the root:

[manjaro manjaro]# fsck.ext4 -f /dev/mapper/crypto_1
e2fsck 1.46.5 (30-Dec-2021)
/dev/mapper/crypto_1: recovering journal
Pass 1: Checking inodes, blocks, and sizes
Pass 2: Checking directory structure
Pass 3: Checking directory connectivity
Pass 4: Checking reference counts
Pass 5: Checking group summary information
/dev/mapper/crypto_1: 1493335/60260352 files (0.7% non-contiguous), 96854786/241020257 blocks

Petty sure that I am using grub.
This is the crypttab:

# /etc/crypttab: mappings for encrypted partitions.
#
# Each mapped device will be created in /dev/mapper, so your /etc/fstab
# should use the /dev/mapper/<name> paths for encrypted devices.
#
# See crypttab(5) for the supported syntax.
#
# NOTE: You need not list your root (/) partition here, but it must be set up
#       beforehand by the initramfs (/etc/mkinitcpio.conf). The same applies
#       to encrypted swap, which should be set up with mkinitcpio-openswap
#       for resume support.
#
# <name>               <device>                         <password> <options>
#luks-51fc926e-223c-49f7-a3e9-8f79d4243fd9 UUID=51fc926e-223c-49f7-a3e9-8f79d4243fd9     /crypto_keyfile.bin luks
luks-695ef6c2-cd3a-44f1-8a0b-96b6743efa2a UUID=695ef6c2-cd3a-44f1-8a0b-96b6743efa2a     /crypto_keyfile.bin luks

fstab:

# /etc/fstab: static file system information.
#
# Use 'blkid' to print the universally unique identifier for a device; this may
# be used with UUID= as a more robust way to name devices that works even if
# disks are added and removed. See fstab(5).
#
# <file system>             <mount point>  <type>  <options>  <dump>  <pass>
UUID=B3E1-96FC                            /boot/efi      vfat    umask=0077 0 2
/dev/mapper/luks-51fc926e-223c-49f7-a3e9-8f79d4243fd9 /              ext4    defaults,noatime 0 1
/dev/mapper/luks-695ef6c2-cd3a-44f1-8a0b-96b6743efa2a swap           swap    defaults 0 0

mkinitcpio.conf:

# vim:set ft=sh
# MODULES
# The following modules are loaded before any boot hooks are
# run.  Advanced users may wish to specify all system modules
# in this array.  For instance:
#     MODULES=(piix ide_disk reiserfs)
MODULES=""

# BINARIES
# This setting includes any additional binaries a given user may
# wish into the CPIO image.  This is run last, so it may be used to
# override the actual binaries included by a given hook
# BINARIES are dependency parsed, so you may safely ignore libraries
BINARIES=()

# FILES
# This setting is similar to BINARIES above, however, files are added
# as-is and are not parsed in any way.  This is useful for config files.
FILES="/crypto_keyfile.bin"

# HOOKS
# This is the most important setting in this file.  The HOOKS control the
# modules and scripts added to the image, and what happens at boot time.
# Order is important, and it is recommended that you do not change the
# order in which HOOKS are added.  Run 'mkinitcpio -H <hook name>' for
# help on a given hook.
# 'base' is _required_ unless you know precisely what you are doing.
# 'udev' is _required_ in order to automatically load modules
# 'filesystems' is _required_ unless you specify your fs modules in MODULES
# Examples:
##   This setup specifies all modules in the MODULES setting above.
##   No raid, lvm2, or encrypted root is needed.
#    HOOKS=(base)
#
##   This setup will autodetect all modules for your system and should
##   work as a sane default
#    HOOKS=(base udev autodetect block filesystems)
#
##   This setup will generate a 'full' image which supports most systems.
##   No autodetection is done.
#    HOOKS=(base udev block filesystems)
#
##   This setup assembles a pata mdadm array with an encrypted root FS.
##   Note: See 'mkinitcpio -H mdadm' for more information on raid devices.
#    HOOKS=(base udev block mdadm encrypt filesystems)
#
##   This setup loads an lvm2 volume group on a usb device.
#    HOOKS=(base udev block lvm2 filesystems)
#
##   NOTE: If you have /usr on a separate partition, you MUST include the
#    usr, fsck and shutdown hooks.
HOOKS="base udev block autodetect modconf block keyboard keymap consolefont encrypt resume filesystems fsck"

# COMPRESSION
# Use this to compress the initramfs image. By default, gzip compression
# is used. Use 'cat' to create an uncompressed image.
#COMPRESSION="gzip"
#COMPRESSION="bzip2"
#COMPRESSION="lzma"
#COMPRESSION="xz"
#COMPRESSION="lzop"
#COMPRESSION="lz4"
#COMPRESSION="zstd"

# COMPRESSION_OPTIONS
# Additional options for the compressor
#COMPRESSION_OPTIONS=()

and the grub:

RUB_DEFAULT=saved
GRUB_TIMEOUT=5
GRUB_TIMEOUT_STYLE=hidden
GRUB_DISTRIBUTOR="Manjaro"
GRUB_CMDLINE_LINUX_DEFAULT="quiet cryptdevice=UUID=51fc926e-223c-49f7-a3e9-8f79d4243fd9:luks-51fc926e-223c-49f7-a3e9-8f79d4243fd9 root=/dev/mapper/luks-51fc926e-223c-49f7-a3e9-8f79d4243fd9 apparmor=1 security=apparmor resume=UUID=da453bc0-afbd-472e-90bc-7d37758b8218  udev.log_priority=3"
GRUB_CMDLINE_LINUX=""

# If you want to enable the save default function, uncomment the following
# line, and set GRUB_DEFAULT to saved.
GRUB_SAVEDEFAULT=true

# Uncomment to disable submenus in boot menu
#GRUB_DISABLE_SUBMENU=y

# Preload both GPT and MBR modules so that they are not missed
GRUB_PRELOAD_MODULES="part_gpt part_msdos"

# Uncomment to enable booting from LUKS encrypted devices
#GRUB_ENABLE_CRYPTODISK=y

# Uncomment to use basic console
GRUB_TERMINAL_INPUT=console

# Uncomment to disable graphical terminal
#GRUB_TERMINAL_OUTPUT=console

# The resolution used on graphical terminal
# note that you can use only modes which your graphic card supports via VBE
# you can see them in real GRUB with the command 'videoinfo'
GRUB_GFXMODE=auto

# Uncomment to allow the kernel use the same resolution used by grub
GRUB_GFXPAYLOAD_LINUX=keep

# Uncomment if you want GRUB to pass to the Linux kernel the old parameter
# format "root=/dev/xxx" instead of "root=/dev/disk/by-uuid/xxx"
#GRUB_DISABLE_LINUX_UUID=true

# Uncomment to disable generation of recovery mode menu entries
GRUB_DISABLE_RECOVERY=true

# Uncomment this option to enable os-prober execution in the grub-mkconfig command
#GRUB_DISABLE_OS_PROBER=false

# Uncomment and set to the desired menu colors.  Used by normal and wallpaper
# modes only.  Entries specified as foreground/background.
GRUB_COLOR_NORMAL="light-gray/black"
GRUB_COLOR_HIGHLIGHT="green/black"

# Uncomment one of them for the gfx desired, a image background or a gfxtheme
#GRUB_BACKGROUND="/usr/share/grub/background.png"
GRUB_THEME="/usr/share/grub/themes/manjaro/theme.txt"

# Uncomment to get a beep at GRUB start
#GRUB_INIT_TUNE="480 440 1"

# Uncomment to ensure that the root filesystem is mounted read-only so that
# systemd-fsck can run the check automatically. We use 'fsck' by default, which
# needs 'rw' as boot parameter, to avoid delay in boot-time. 'fsck' needs to be
# removed from 'mkinitcpio.conf' to make 'systemd-fsck' work.
# See also Arch-Wiki: https://wiki.archlinux.org/index.php/Fsck#Boot_time_checking
#GRUB_ROOT_FS_RO=true
GRUB_ENABLE_CRYPTODISK=y
11

You have block listed twice. Remove the second instance of it.

Recovering journal? If an ext4 filesystem is unmounted cleanly and/or the system shutdown properly, there’s no reason it needs to resort to to using its journal.

Something’s going on.

Anything out of the ordinary when you reboot?

12

Ok, I remove the second block and the system still doesn’t boot.
And this is the error that appears shortly after:

IMG_20220329_222702284
IMG_20220329_2227022841920×1440 171 KB

13

Just confirming you also rebuilt the initramfs after that?

Let’s get rid of hibernation as a test. Remove, for now, the resume entry in your Grub default:

resume=UUID=da453bc0-afbd-472e-90bc-7d37758b8218

And also uncomment the line about crypto disks.

# Uncomment to enable booting from LUKS encrypted devices
GRUB_ENABLE_CRYPTODISK=y

Then update-grub again.

Your ext4 filesystem still needs to recover from the journal after every reboot? :neutral_face:

Does it just pause on that message and stay like that indefinitely?

14

After rebuilding initramfs, in case that I forgot to do it, the system still not boots. So I got rid of the hibernation as you suggested and this was the result:

IMG_20220330_184754932
IMG_20220330_1847549321920×1440 227 KB

The system didn’t boot and after a few seconds this appear after:

IMG_20220330_184829483
IMG_20220330_1848294831920×1440 349 KB

Once again I run the fsck.ext4 to scan the filesystem and still recovers the boot after every log.

[manjaro manjaro]# fsck.ext4 -f /dev/mapper/crypto_1
e2fsck 1.46.5 (30-Dec-2021)
/dev/mapper/crypto_1: recovering journal
Pass 1: Checking inodes, blocks, and sizes
Pass 2: Checking directory structure
Pass 3: Checking directory connectivity
Pass 4: Checking reference counts
Pass 5: Checking group summary information
/dev/mapper/crypto_1: 1493335/60260352 files (0.7% non-contiguous), 96854786/241020257 blocks
15

Running out of ideas. I’m not sure why your ext4 filesystem needs to recover from its journal after every reboot. (You’re shutting down properly, yes?)

Last idea I can think of is that an update coincided during a drive I/O error.

First, under a live ISO (without unlocking LUKS nor mounting anything), open a terminal and check the SMART output for the Hitachi drive.

sudo smartctl -a /dev/sda

Does it show any errors or concerning values?

Try running a short SMART selftest on the Hitachi drive. Should only take 2 minutes.

sudo smartctl -t short /dev/sda

If it passes, run a long test. This can take many hours.

sudo smartctl -t long /dev/sda

Replace sda with whatever name it uses for your Hitachi drive.

16

Ok, if I don’t mount the root and tried to do the ext4 it goes to recover journals, but if I mount it first and then unmount the ext4 doesn’t recover from the journal. So I think that the system didn’t recognize the disk, but still is capable of mount it, so maybe in the boot we can use another version of autodetect, if it is a package, or mount it manually in the boot.
P.S the SMART’s didn’t show any error, if necessary I will upload the output later.

17

Sorry that my answer is very late, but do you have anbox and anbox-modules-dkms? Errors seems like mine after I install anbox-modules-dkms. This information should help a lot to us both