Running Filesystem Check on LUKS encrypted root partition (fsck)

[vommie]

Hello,

after resuming from Hibernation, my system crashed once. Since then it seems there are some issues on the file system. Like when I try to clean the Trash in the shell, I receive the following error on two files:

rm: cannot remove '/home/vommie/.local/share/Trash/files/examplefile': Structure needs cleaning

When I try to access the Trash with Dolphin, the file manager freezes.

So I thought I should try to run fsck. I am quite new to Linux so I googled a lot and I did not found a solution to do so without having to fear I destroy my installation as my system partition is encrypted with LUKS. All settings doing so were set by the Manjaro installer.

I found out Manjaro seems to run fsck by default on boot if in the fstab the 6th parameter is set to “1”, which is it by default for my root partition:

/dev/mapper/luks-ID / ext4 defaults,noatime 0 1
(replaced the id with “ID”)

So I tried to find a log for fsck and found this command:

journalctl -b --no-pager | grep fsck

After running, I see there are a lot of errors and a recommendation to run e2fsck:

[lappie vommie]# journalctl -b --no-pager | grep fsck Jan 25 11:32:04 lappie kernel: EXT4-fs (dm-0): warning: mounting fs with errors, running e2fsck is recommended Jan 25 11:32:04 lappie kernel: EXT4-fs (dm-0): warning: mounting fs with errors, running e2fsck is recommended Jan 25 11:32:04 lappie systemd[1]: Created slice system-systemd\x2dfsck.slice. Jan 25 11:32:04 lappie systemd-fsck[560]: fsck.fat 4.1 (2017-01-24) Jan 25 11:32:04 lappie systemd-fsck[560]: /dev/nvme0n1p1: 6 files, 120/76646 clusters Jan 25 11:32:04 lappie audit[1]: SERVICE_START pid=1 uid=0 auid=4294967295 ses=4294967295 msg='unit=systemd-fsck@dev-disk-by\x2duuid-AA2B\x2dAE1D comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success' Jan 25 11:37:08 lappie kernel: EXT4-fs (dm-0): error count since last fsck: 18470

It seems fsck does not to try to repair the fs? And I did not found out how exactly to run fsck with repair option, like where do I have to place the kernel parameters (in between which parameters or in between which hooks etc.?). I found no clear examples out there.

My GRUB kernel parameters line looks like this: GRUB_CMDLINE_LINUX_DEFAULT="cryptdevice=UUID=ID:luks-ID root=/dev/mapper/luks-ID resume=/dev/mapper/luks-ID udev.log_priority=3 acpi_backlight=video"
(replaced ids with “ID”)

If I want to add the parameters “fsck.mode=force fsck.repair=yes” in there, where should I place it? Befor or after the “root” parameter? I’ve read the fs should not be mounted and I am not sure if the “root” parameter mounts the fs, looks so for me, but I am not sure.

But I am not sure if this is the right way, or if I should do something with “e2fsck”. I do not know how I can run this on an unmounted encrypted root partition and googled for hours but I am too unsure. I hope you can help me with this. I’ve backuped my drive but I want to avoid a reinstallation.

[Nachlese]

Faced this problem recently - the check that I thought would periodically happen - and automatically whenever the filesystem needed it - didn’t run.
I used the Arch wiki to resolve - will add a link later.
Basically there are two ways - I’ll describe what I did:

added “ro” to the grub boot parameters:
GRUB_CMDLINE_LINUX="cryptdevice=UUID=73213ab9-5c8c-4c2a-a397-aa321ebfd4ec:cryptroot nowatchdog ro"

… update grub afterwards

and I ran:
sudo tune2fs /dev/mapper/cryptroot -c 20 -m 1
to set the maximal mount count without check to 20
(default was either unset or zero - which I took to mean: never
as well as setting the reserved block percentage to 1% - it defaults to 5%)

at the next reboot the filesystem was checked …

With this approach, (if I remember correctly), the fsck hook in /etc/mkinitcpio.conf was explicitely not recommended - I don’t have it in my configuration and it wasn’t there to begin with.
this is mine:

HOOKS=(base udev autodetect modconf block encrypt filesystems shutdown keyboard resume)

HTH

edit:
https://wiki.archlinux.org/index.php/Fsck

@vommie

If I want to add the parameters “fsck.mode=force fsck.repair=yes” in there, where should I place it?

append it - you could even hit ESC at the Grub prompt
then “e” for edit and type it there, as a one shot option

there was also a thread here on this topic:

How to enforce boot time file system check?

My root file system doesn’t seem to be checked periodically. dumpe2fs shows this: Mount count: 252 Maximum mount count: 20 Last checked: Sat Feb 22 15:44:13 2020 Check interval: 604800 (1 week) Next check after: Sat Feb 29 15:44:13 2020 Last checked is several months ago. I’ve used tune2fs recently to enforce checks at every 20 mounts and once in every week. Even after a couple of reboots the file system doesn’t seem to be checked. The last colum…

[Nachlese]

To check/repair it without having to set up all this correctly:
boot from live media
open the LUKS container
run fsck on /dev/mapper/luks-ID

sudo cryptsetup open /dev/nvmexxx luks-ID
sudo e2fsck -v /dev/mapper/luks-ID

or along these lines

[vommie]

Thank you, Nachlese, for your fast help. I’ve tried the solution by opening the LUKS partition with a live image and it worked well. e2fsck fixed several erros, and now I can use the Trash again and it seems all is fine for now.

Next I will have to check why the sys has trouble with resuming hibernation properly, but that’s another topic