So far after every big update my manjaro gets this error

lsblk -f

NAME                                          FSTYPE      FSVER LABEL    UUID                                 FSAVAIL FSUSE% MOUNTPOINTS
sda                                                                                                                          
├─sda1                                        vfat        FAT32 NO_LABEL 81C5-005C                             218,6M    27% /boot/efi
├─sda2                                        crypto_LUKS 1              5ef2b8bd-072b-42e3-8132-b7b8b9df0e1a                
│ └─luks-5ef2b8bd-072b-42e3-8132-b7b8b9df0e1a ext4        1.0            face782a-ee8f-467e-ab35-9d60c7ea22f0   64,9G    81% /
└─sda3                                        crypto_LUKS 1              4b590b97-0814-4af5-9995-e5e238b0da2d                
  └─luks-4b590b97-0814-4af5-9995-e5e238b0da2d swap        1     swap     eaa871d0-314e-4fbc-8846-267094fa8e0d                [SWAP]

cat /etc/fstab

# /etc/fstab: static file system information.
#
# Use 'blkid' to print the universally unique identifier for a device; this may
# be used with UUID= as a more robust way to name devices that works even if
# disks are added and removed. See fstab(5).
#
# <file system>             <mount point>  <type>  <options>  <dump>  <pass>
UUID=81C5-005C                            /boot/efi      vfat    umask=0077 0 2
/dev/mapper/luks-5ef2b8bd-072b-42e3-8132-b7b8b9df0e1a /              ext4    defaults,noatime 0 1
/dev/mapper/luks-4b590b97-0814-4af5-9995-e5e238b0da2d swap           swap    defaults,noatime 0 0
tmpfs                                     /tmp           tmpfs   defaults,noatime,mode=1777 0 0

I get this error after changing the kernel(sudo mhwd-kernel -i linuxXX rmc) and on manjaro updates too

setfont: KDFONTOP: Function not implemented
fsck: /sbin/fsck.ext4: execute failed: Exec format error
ERROR: fsck failed on ‘/dev/mapper/luks-5ef2b8bd-072b-4203-0132-b7b6b9df8eia’ fixed
[FAILED] Failed to mount /boot/efi. [DEPEND] Dependency failed for Local File Systems.
You are in emergency mode. After logging in, type “journalctl -xb” to view system logs. “systemctl reboot” to reboot. “systemctl default” or “exit” to boot into default mode.
To continue enter root password(or press Control-D):

how do i temporary solve it:

  1. Booting into manjaro LiveCD
  2. sudo cryptsetup luksOpen /dev/sda2 lvm and sudo cryptsetup luksOpen /dev/sda3 swap
  3. GRUB/Restore the GRUB Bootloader - Manjaro
    but along with running pacman -Syu grub as wrote in this guide i run pacman -S linuxXX

how can i solve this repeating problem?

remove the fsck hook from mkinitpio.conf and rebuilt init

this fails because of the previous error

If it is a repeating problem you should investigate why it is repeating.

I cannot guess why - could be anything … impossible to answer

perhaps something with one of the devices in the volume … :man_shrugging:

The kernel seems to be unable to handle that binary? Are you sure It’s not corrupted?
Check e2fsprogs package as it’s providing/containing /bin/fsck.ext4:

$ pacman -Qkk e2fsprogs
$ paccheck --sha256sum e2fsprogs
3 Likes

pacman -Qkk e2fsprogs
Warning: e2fsprogs: /usr/bin/e2fsck (Wrong size)
Warning: e2fsprogs: /usr/bin/e2fsck (MD5 checksum mismatch)
warning: e2fsprogs: /usr/bin/e2fsck (SHA256 checksum mismatch)
warning: e2fsprogs: /usr/bin/fsck.ext2 (size mismatch)
warning: e2fsprogs: /usr/bin/fsck.ext2 (MD5 checksum mismatch)
warning: e2fsprogs: /usr/bin/fsck.ext2 (SHA256 checksum mismatch)
warning: e2fsprogs: /usr/bin/fsck.ext3 (size mismatch)
warning: e2fsprogs: /usr/bin/fsck.ext3 (MD5 checksum mismatch)
warning: e2fsprogs: /usr/bin/fsck.ext3 (SHA256 checksum mismatch)
warning: e2fsprogs: /usr/bin/fsck.ext4 (size mismatch)
warning: e2fsprogs: /usr/bin/fsck.ext4 (MD5 checksum mismatch)
warning: e2fsprogs: /usr/bin/fsck.ext4 (SHA256 checksum mismatch)
e2fsprogs: total 202 files, 4 modified files

idk but i think fsck error isn’t the reason, because it repeats on every succesful boot too

For whatever reason the binaries aren’t matching the checksums.

Reinstall that package (to fix the wrong binaries) and regenerate all initramfs:

$ sudo pacman -S e2fsprogs
$ sudo mkinitcpio -P

You can optionally check if the binaries are ok before re-generating the initramfs.

Perhaps you’ll want to check other packages/binaries as well:

$ pacman -Qq | pacman -Qkk -
1 Like

k so fsck problem has been fixed, but i dont know if system will break on next updatte again. maybe i need to share logs?

journalctl -p err --since=yesterday | nc termbin.com 9999
https://termbin.com/07nly

As you undoubtedly gatherd yourself: the filesystem is corrupt.
Boot from live usb and check/repair it.

fixed filesystem. after today’s update error repeated and i fixed it with same steps. journalctl -p err —since=today doesnt contain anything about it.

journalctl -xb | nc termbin.com 9999
https://termbin.com/lejdt

Command line: BOOT_IMAGE=/vmlinuz-6.0-x86_64 root=UUID=face782a-ee8f-467e-ab35-9d60c7ea22f0 rw quiet cryptdevice=UUID=5ef2b8bd-072b-42e3-8132-b7b8b9df0e1a:luks-5ef2b8bd-072b-42e3-8132-b7b8b9df0e1a root=/dev/mapper/luks-5ef2b8bd-072b-42e3-8132-b7b8b9df0e1a apparmor=1 security=apparmor resume=/dev/mapper/luks-4b590b97-0814-4af5-9995-e5e238b0da2d udev.log_priority=3

This looks odd to me. Is it normal to have root= called twice?

yeah, looks strange

i still need help

booting the live system (likely from USB stick)
and then providing the output of
lsblk -f
could help us to gather some info on the actual partition layout of your system

It’s definitely odd that your kernel command line shows
root=xxx
two times
… there can only be one -
if there are two, only the last occurence will be taken (that’s a guess, do not know for sure)

But there is two where only should be one.

But in the end, it still seems to work, somehow, eventually?

It’s strange, incoherent, to me.

And: it needs to be fixed - it just cant stay that way.

Trying everything else under the sun,
but not addressing this
is just a waste of time and effort.


Start from the basics:
check /etc/default/grub
check /etc/fstab
check /etc/crypttab

You just called it “lvm” -
but
from the command syntax POV, it is just a name
(you could have called it “unicorns” instead of “lvm”
and it would work just as well).

and you try to open an encrypted swap partition
not sure about that part at all

looks strange

Is there an encrypted root and swap - or is it all inside an lvm?

I could not infer the setup from the information here.

lsblk -f
cat /etc/fstab

of the actual system

to re-start this investigation

lsblk -f

NAME                                          FSTYPE      FSVER LABEL    UUID                                 FSAVAIL FSUSE% MOUNTPOINTS
sda                                                                                                                          
├─sda1                                        vfat        FAT32 NO_LABEL 81C5-005C                             218,6M    27% /boot/efi
├─sda2                                        crypto_LUKS 1              5ef2b8bd-072b-42e3-8132-b7b8b9df0e1a                
│ └─luks-5ef2b8bd-072b-42e3-8132-b7b8b9df0e1a ext4        1.0            face782a-ee8f-467e-ab35-9d60c7ea22f0   64,9G    81% /
└─sda3                                        crypto_LUKS 1              4b590b97-0814-4af5-9995-e5e238b0da2d                
  └─luks-4b590b97-0814-4af5-9995-e5e238b0da2d swap        1     swap     eaa871d0-314e-4fbc-8846-267094fa8e0d                [SWAP]

cat /etc/fstab

# /etc/fstab: static file system information.
#
# Use 'blkid' to print the universally unique identifier for a device; this may
# be used with UUID= as a more robust way to name devices that works even if
# disks are added and removed. See fstab(5).
#
# <file system>             <mount point>  <type>  <options>  <dump>  <pass>
UUID=81C5-005C                            /boot/efi      vfat    umask=0077 0 2
/dev/mapper/luks-5ef2b8bd-072b-42e3-8132-b7b8b9df0e1a /              ext4    defaults,noatime 0 1
/dev/mapper/luks-4b590b97-0814-4af5-9995-e5e238b0da2d swap           swap    defaults,noatime 0 0
tmpfs                                     /tmp           tmpfs   defaults,noatime,mode=1777 0 0

cat /etc/default/grub

GRUB_DEFAULT=saved
GRUB_TIMEOUT=5
GRUB_TIMEOUT_STYLE=hidden
GRUB_DISTRIBUTOR="Manjaro"
GRUB_CMDLINE_LINUX_DEFAULT="quiet cryptdevice=UUID=5ef2b8bd-072b-42e3-8132-b7b8b9df0e1a:luks-5ef2b8bd-072b-42e3-8132-b7b8b9df0e1a root=/dev/mapper/luks-5ef2b8bd-072b-42e3-8132-b7b8b9df0e1a apparmor=1 security=apparmor resume=/dev/mapper/luks-4b590b97-0814-4af5-9995-e5e238b0da2d udev.log_priority=3"
GRUB_CMDLINE_LINUX=""

# If you want to enable the save default function, uncomment the following
# line, and set GRUB_DEFAULT to saved.
GRUB_SAVEDEFAULT=true

# Preload both GPT and MBR modules so that they are not missed
GRUB_PRELOAD_MODULES="part_gpt part_msdos"

# Uncomment to enable booting from LUKS encrypted devices
#GRUB_ENABLE_CRYPTODISK=y

# Uncomment to use basic console
GRUB_TERMINAL_INPUT=console

# Uncomment to disable graphical terminal
#GRUB_TERMINAL_OUTPUT=console

# The resolution used on graphical terminal
# note that you can use only modes which your graphic card supports via VBE
# you can see them in real GRUB with the command 'videoinfo'
GRUB_GFXMODE=auto

# Uncomment to allow the kernel use the same resolution used by grub
GRUB_GFXPAYLOAD_LINUX=keep

# Uncomment if you want GRUB to pass to the Linux kernel the old parameter
# format "root=/dev/xxx" instead of "root=/dev/disk/by-uuid/xxx"
#GRUB_DISABLE_LINUX_UUID=true

# Uncomment to disable generation of recovery mode menu entries
GRUB_DISABLE_RECOVERY=true

# Uncomment this option to enable os-prober execution in the grub-mkconfig command
GRUB_DISABLE_OS_PROBER=false

# Uncomment and set to the desired menu colors.  Used by normal and wallpaper
# modes only.  Entries specified as foreground/background.
GRUB_COLOR_NORMAL="light-gray/black"
GRUB_COLOR_HIGHLIGHT="green/black"

# Uncomment one of them for the gfx desired, a image background or a gfxtheme
#GRUB_BACKGROUND="/usr/share/grub/background.png"
GRUB_THEME="/usr/share/grub/themes/manjaro/theme.txt"

# Uncomment to get a beep at GRUB start
#GRUB_INIT_TUNE="480 440 1"

# Uncomment to ensure that the root filesystem is mounted read-only so that
# systemd-fsck can run the check automatically. We use 'fsck' by default, which
# needs 'rw' as boot parameter, to avoid delay in boot-time. 'fsck' needs to be
# removed from 'mkinitcpio.conf' to make 'systemd-fsck' work.
# See also Arch-Wiki: https://wiki.archlinux.org/index.php/Fsck#Boot_time_checking
#GRUB_ROOT_FS_RO=true
GRUB_ENABLE_CRYPTODISK=y

sudo cat /etc/crypttab

# /etc/crypttab: mappings for encrypted partitions.
#
# Each mapped device will be created in /dev/mapper, so your /etc/fstab
# should use the /dev/mapper/<name> paths for encrypted devices.
#
# See crypttab(5) for the supported syntax.
#
# NOTE: Do not list your root (/) partition here, it must be set up
#       beforehand by the initramfs (/etc/mkinitcpio.conf). The same applies
#       to encrypted swap, which should be set up with mkinitcpio-openswap
#       for resume support.
#
# <name>               <device>                         <password> <options>
luks-5ef2b8bd-072b-42e3-8132-b7b8b9df0e1a UUID=5ef2b8bd-072b-42e3-8132-b7b8b9df0e1a     /crypto_keyfile.bin luks
luks-4b590b97-0814-4af5-9995-e5e238b0da2d UUID=4b590b97-0814-4af5-9995-e5e238b0da2d     /crypto_keyfile.bin luks

we may have found where the problem starts, look at the NOTE

Maybe.

You could just comment out the entries in /etc/crypttab
to see the effect.

It’s easily reverted in case it doesn’t work.

I have an encrypted system set up similar to yours - and my /etc/crypttab is empty.

One more thing is this line in /etc/default/grub

I’m not sure whether this part:

root=/dev/mapper/luks-5ef2b8bd-072b-42e3-8132-b7b8b9df0e1a

needs to be there at all
and if it is needed, it should contain the UUID of the DEcrypted root partition - not, as it is now, the UUID of the still ENcrypted partition.


For reference, my setup looks like this:

lsblk -f
NAME     FSTYPE      FSVER LABEL  UUID                                 FSAVAIL FSUSE% MOUNTPOINTS
sda                                                                                   
├─sda1   vfat        FAT32 EFISYS ACDE-A3C7                             510,7M     0% /boot/efi
├─sda2   ext4        1.0   boot   ef07ba8b-8853-4d17-bce2-e86737b35aaa  291,6M    33% /boot
└─sda4   crypto_LUKS 2            7b4bb6e5-f2e3-435d-834e-8c3ee34f3865                
  └─encr ext4        1.0          305afbd2-b8a3-4e18-8828-34c6d1ead4d1  796,8G    14% /
sr0                                                                                   
zram0                                                                                 [SWAP]
cat /etc/fstab 
# /etc/fstab: static file system information.
#
# Use 'blkid' to print the universally unique identifier for a device; this may
# be used with UUID= as a more robust way to name devices that works even if
# disks are added and removed. See fstab(5).
#
# <file system>             <mount point>  <type>  <options>  <dump>  <pass>
UUID=ACDE-A3C7                            /boot/efi      vfat    defaults,noatime 0 2
UUID=ef07ba8b-8853-4d17-bce2-e86737b35aaa /boot          ext4    defaults,noatime 0 2
UUID=7b4bb6e5-f2e3-435d-834e-8c3ee34f3865 /              ext4    defaults,noatime 0 1

/swapfile none swap defaults 0 0

the line in my /etc/default/grub:

GRUB_CMDLINE_LINUX_DEFAULT="loglevel=3 nowatchdog mitigations=off audit=0 cryptdevice=UUID=7b4bb6e5-f2e3-435d-834e-8c3ee34f3865:encr resume=/dev/disk/by-uuid/305afbd2-b8a3-4e18-8828-34c6d1ead4d1 resume_offset=18393088"

only contains the UUID of the still encrypted partition
and no root=xxx statement at all

I’m not at all sure how to deal with your encrypted swap partition.
I chose to use a swap file inside the encrypted / partition
so I don’t have to deal with that problem.

HTH

Sorry for a long response. I made all that you said, the problem is still here.
journalctl -xb | nc termbin.com 1 9999
https://termbin.com/zg53

After 3 months you post a log that does not indicate a problem to me - and with a lot of cyrillic script that I can read, but do not understand

Perhaps start again with what your problem is and what the relevant files look like.