Boot fails after converting root filesystem from ext4 to btrfs

My laptop has two drives, a large hdd and a smaller ssd. I have previously converted my hdd to btrfs, and mounted it as /home in fstab. That setup has worked until now.
Since it seemed to work well, i decided to take the plunge and convert my root filesystem to btrfs as well. After following this guide (with some adaptations to manjaro) I am currently experiencing boot issues, and I’d like some help figuring them out.

inxi

inxi -Fazy
System:
Kernel: 5.4.148-1-MANJARO x86_64 bits: 64 compiler: gcc v: 11.1.0
parameters: BOOT_IMAGE=/boot/vmlinuz-x86_64 lang=en_US keytable=no tz=UTC
misobasedir=manjaro misolabel=MANJARO_XFCEM_2114 quiet systemd.show_status=1
apparmor=1 security=apparmor driver=free nouveau.modeset=1 i915.modeset=1
radeon.modeset=1
Desktop: Xfce info: xfce4-panel wm: xfwm vt: 7 dm: N/A Distro: Manjaro Linux
base: Arch Linux
Machine:
Type: Laptop System: Notebook product: P95xER v: N/A serial: N/A
Chassis: No Enclosure type: 10 serial: N/A
Mobo: Notebook model: P95xER serial: N/A UEFI: American Megatrends
v: 1.07.15 date: 11/16/2018
Battery:
ID-1: BAT0 charge: 47.2 Wh (89.4%) condition: 52.8/53.2 Wh (99.3%)
volts: 16.5 min: 15.2 model: Notebook BAT type: Li-ion serial:
status: Unknown
CPU:
Info: 6-Core model: Intel Core i7-8750H socket: BGA1440 (U3E1) note: check
bits: 64 type: MT MCP arch: Kaby Lake note: check family: 6
model-id: 9E (158) stepping: A (10) microcode: EA cache: L1: 384 KiB
L2: 9 MiB L3: 9 MiB
flags: avx avx2 lm nx pae sse sse2 sse3 sse4_1 sse4_2 ssse3 vmx
bogomips: 52815
Speed: 800 MHz min/max: 800/4100 MHz base/boost: 2079/8300 volts: 0.8 V
ext-clock: 100 MHz Core speeds (MHz): 1: 800 2: 800 3: 802 4: 800 5: 800
6: 801 7: 800 8: 800 9: 800 10: 800 11: 800 12: 801
Vulnerabilities: Type: itlb_multihit status: KVM: Split huge pages
Type: l1tf
mitigation: PTE Inversion; VMX: conditional cache flushes, SMT vulnerable
Type: mds mitigation: Clear CPU buffers; SMT vulnerable
Type: meltdown mitigation: PTI
Type: spec_store_bypass
mitigation: Speculative Store Bypass disabled via prctl and seccomp
Type: spectre_v1
mitigation: usercopy/swapgs barriers and __user pointer sanitization
Type: spectre_v2 mitigation: Full generic retpoline, IBPB: conditional,
IBRS_FW, STIBP: conditional, RSB filling
Type: srbds mitigation: Microcode
Type: tsx_async_abort status: Not affected
Graphics:
Device-1: Intel CoffeeLake-H GT2 [UHD Graphics 630] vendor: CLEVO/KAPOK
driver: i915 v: kernel bus-ID: 00:02.0 chip-ID: 8086:3e9b class-ID: 0300
Device-2: NVIDIA GP104M [GeForce GTX 1070 Mobile] vendor: CLEVO/KAPOK
driver: nouveau v: kernel bus-ID: 01:00.0 chip-ID: 10de:1ba1 class-ID: 0300
Device-3: Chicony Chicony USB 2.0 Camera type: USB driver: uvcvideo
bus-ID: 1-8:3 chip-ID: 04f2:b5a7 class-ID: 0e02
Display: x11 server: X.Org 1.20.13 compositor: xfwm4 driver:
loaded: modesetting display-ID: :0.0 screens: 1
Screen-1: 0 s-res: 2048x1152 s-dpi: 96 s-size: 542x305mm (21.3x12.0")
s-diag: 622mm (24.5")
Monitor-1: eDP-1 res: 2048x1152 hz: 60 dpi: 151 size: 345x194mm (13.6x7.6")
diag: 396mm (15.6")
OpenGL: renderer: Mesa Intel UHD Graphics 630 (CFL GT2) v: 4.6 Mesa 21.2.3
direct render: Yes
Audio:
Device-1: Intel Cannon Lake PCH cAVS vendor: CLEVO/KAPOK
driver: snd_hda_intel v: kernel bus-ID: 00:1f.3 chip-ID: 8086:a348
class-ID: 0403
Device-2: NVIDIA GP104 High Definition Audio driver: snd_hda_intel v: kernel
bus-ID: 01:00.1 chip-ID: 10de:10f0 class-ID: 0403
Sound Server-1: ALSA v: k5.4.148-1-MANJARO running: yes
Sound Server-2: sndio v: N/A running: no
Sound Server-3: JACK v: 1.9.19 running: no
Sound Server-4: PulseAudio v: 15.0 running: yes
Sound Server-5: PipeWire v: 0.3.38 running: no
Network:
Device-1: Intel Wireless-AC 9260 vendor: Rivet Networks driver: iwlwifi
v: kernel port: 4000 bus-ID: 03:00.0 chip-ID: 8086:2526 class-ID: 0280
IF: wlp3s0 state: up mac:
Device-2: Realtek RTL8111/8168/8411 PCI Express Gigabit Ethernet
vendor: CLEVO/KAPOK driver: r8169 v: kernel port: 3000 bus-ID: 04:00.0
chip-ID: 10ec:8168 class-ID: 0200
IF: enp4s0 state: down mac:
Bluetooth:
Device-1: Intel Wireless-AC 9260 Bluetooth Adapter type: USB driver: btusb
v: 0.8 bus-ID: 1-14:5 chip-ID: 8087:0025 class-ID: e001
Report: rfkill ID: hci0 rfk-id: 1 state: down bt-service: N/A rfk-block:
hardware: no software: yes address: see --recommends
Drives:
Local Storage: total: 2.07 TiB used: 94.7 GiB (4.5%)
ID-1: /dev/nvme0n1 maj-min: 259:0 vendor: Samsung model: SSD 960 EVO 250GB
size: 232.89 GiB block-size: physical: 512 B logical: 512 B speed: 31.6 Gb/s
lanes: 4 type: SSD serial: rev: 3B7QCXE7 temp: 37 Celsius C
SMART: yes health: PASSED on: 169d 16h cycles: 2,090
read-units: 22,887,199 [11.7 TB] written-units: 32,248,582 [16.5 TB]
ID-2: /dev/sda maj-min: 8:0 vendor: Seagate model: ST2000LX001-1RG174
family: FireCuda 2.5 size: 1.82 TiB block-size: physical: 4096 B
logical: 512 B sata: 3.1 speed: 6.0 Gb/s type: HDD rpm: 5400
serial: rev: SDM1 temp: 36 C
SMART: yes state: enabled health: PASSED on: 274d 1h cycles: 2089
read: 27.27 TiB written: 7.5 TiB Pre-Fail: reallocated sector: 100
threshold: 36 attribute: Spin_Retry_Count value: 100 worst: 100
threshold: 97
ID-3: /dev/sdb maj-min: 8:16 type: USB vendor: SanDisk model: USB 3.2Gen1
size: 28.65 GiB block-size: physical: 512 B logical: 512 B type: N/A
serial: rev: 1.00
SMART Message: Unknown USB bridge. Flash drive/Unsupported enclosure?
Partition:
ID-1: / raw-size: 232.59 GiB size: 232.59 GiB (100.00%)
used: 94.64 GiB (40.7%) fs: btrfs block-size: 4096 B dev: /dev/nvme0n1p2
maj-min: 259:2
ID-2: /boot raw-size: 300 MiB size: 299.4 MiB (99.80%)
used: 63.2 MiB (21.1%) fs: vfat block-size: 512 B dev: /dev/nvme0n1p1
maj-min: 259:1
Swap:
Alert: No swap data was found.
Sensors:
System Temperatures: cpu: 59.0 C mobo: N/A gpu: nouveau temp: 41.0 C
Fan Speeds (RPM): N/A
Info:
Processes: 341 Uptime: 1h 14m wakeups: 1 Memory: 31.22 GiB
used: 2.65 GiB (8.5%) Init: systemd v: 249 tool: systemctl Compilers:
gcc: 11.1.0 alt: 10 clang: 12.0.1 Packages: 1865 pacman: 1855 lib: 454
flatpak: 10 Shell: Bash v: 5.1.8 running-in: xfce4-terminal inxi: 3.3.06

fstab entry
# <file system>             		<mount point>  <type>  <options>  					<dump>  <pass>
UUID=F7A1-998C                                          /boot/efi                    vfat    umask=0077                                   0 2 
#UUID=791c4300-c5e7-41b8-bf2b-fb4d6099e2e2               /                            ext4    noatime                                      0 1 
UUID=ea10f93b-0ef1-4889-80e9-b0ca11f6999f			/			btrfs	rw,exec,subvolid=257,ssd,noatime,compress=zstd 0 0
#UUID=18c44b6f-44da-5e46-802f-6ae4a7bd4981			/			btrf	rw,exec,noatime,ssd,subvolid=257,compress=zstd 0 0
UUID=bed1ec47-85a7-428f-a273-fe9c33e1de49               /home                        btrfs   rw,exec,subvolid=257,compress=zstd,noatime   0 0 
lsblk

NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINTS
loop0 7:0 0 127.9M 1 loop
loop1 7:1 0 646.8M 1 loop
loop2 7:2 0 752.6M 1 loop
loop3 7:3 0 671M 1 loop
sda 8:0 0 1.8T 0 disk
sdb 8:16 1 28.7G 0 disk
├─sdb1 8:17 1 2.2G 0 part
└─sdb2 8:18 1 4M 0 part
nvme0n1 259:0 0 232.9G 0 disk
├─nvme0n1p1 259:1 0 300M 0 part /boot/efi
│ /boot
└─nvme0n1p2 259:2 0 232.6G 0 part /

blkid

/dev/loop1: TYPE=“squashfs”
/dev/nvme0n1p1: LABEL_FATBOOT=“NO_LABEL” LABEL=“NO_LABEL” UUID=“F7A1-998C” BLOCK_SIZE=“512” TYPE=“vfat” PARTUUID=“dd2eae74-f0b5-a041-a448-763d2bcf8fa8”
/dev/nvme0n1p2: UUID=“ea10f93b-0ef1-4889-80e9-b0ca11f6999f” UUID_SUB=“f3602d07-bd40-421e-ab87-ce0492703a7c” BLOCK_SIZE=“4096” TYPE=“btrfs” PARTUUID=“18c44b6f-44da-5e46-802f-6ae4a7bd4981”
/dev/sdb2: SEC_TYPE=“msdos” LABEL_FATBOOT=“MISO_EFI” LABEL=“MISO_EFI” UUID=“E22E-047E” BLOCK_SIZE=“512” TYPE=“vfat”
/dev/sdb1: BLOCK_SIZE=“2048” UUID=“2021-09-27-19-55-52-00” LABEL=“MANJARO_XFCEM_2114” TYPE=“iso9660”
/dev/loop2: TYPE=“squashfs”
/dev/loop0: TYPE=“squashfs”
/dev/sda: LABEL=“storage” UUID=“bed1ec47-85a7-428f-a273-fe9c33e1de49” UUID_SUB=“2fa2bbe0-3716-451a-9b2e-a2fe5a1e2ed3” BLOCK_SIZE=“4096” TYPE=“btrfs”
/dev/loop3: TYPE=“squashfs”

journal -b

dmesg

I have only little experience with btrfs - and the ones I have were not happy - but I think the UUID has changed - so verify you are using the correct UUID’s in the relevant places.

should the fstab entry be for UUID, PARTUUID or UUID_SUB?
currently, fstab lists UUID of /dev/nvme0n1p2 as mounted to /

fstab entry for /home checks out. It puzzles me a bit that subvolid=257 for both / and /home, but this might be because the two disks were converted to btrfs separately, making them separate filesystems.

An empty /home directory is listed in the / directory of nvme0n1p2. Could this ‘new’ home directory and the ‘old’ home directory on sdb be coming into conflict?

You can use UUID or PARTUUID, but then you must use the correct identifier in /etc/fstab. The advantage of the PARTUUID is that it doesn’t change if you reformat the partition, because the PARTUUID is stored in the partition table, not in the filesystem header.

Did you rebuild your initramfs after doing the conversion?

Negative. The empty directory is the mountpoint for the partition containing your user files.

I have rebuilt initramfs using mkinitcpio -P. It failed a few times because the presets had disappeared, but after reinstalling kernels with pacman, not mhwd-kernel, it succeeded. (mhwd-kernel -i linux510 failed with error message “no targets specified”).

Are any specific hooks necessary to boot with btrfs root partition, or is autodetect sufficient?

Output of mkinitcpio -P

==> Building image from preset: /etc/mkinitcpio.d/linux510.preset: ‘default’
→ -k /boot/vmlinuz-5.10-x86_64 -c /etc/mkinitcpio.conf -g /boot/initramfs-5.10-x86_64.img
==> Starting build: 5.10.70-1-MANJARO
→ Running build hook: [base]
→ Running build hook: [udev]
→ Running build hook: [autodetect]
→ Running build hook: [modconf]
→ Running build hook: [block]
→ Running build hook: [keyboard]
→ Running build hook: [keymap]
→ Running build hook: [filesystems]
→ Running build hook: [fsck]
==> Generating module dependencies
==> Creating gzip-compressed initcpio image: /boot/initramfs-5.10-x86_64.img
==> Image generation successful
==> Building image from preset: /etc/mkinitcpio.d/linux510.preset: ‘fallback’
→ -k /boot/vmlinuz-5.10-x86_64 -c /etc/mkinitcpio.conf -g /boot/initramfs-5.10-x86_64-fallback.img -S autodetect
==> Starting build: 5.10.70-1-MANJARO
→ Running build hook: [base]
→ Running build hook: [udev]
→ Running build hook: [modconf]
→ Running build hook: [block]
→ Running build hook: [keyboard]
→ Running build hook: [keymap]
→ Running build hook: [filesystems]
→ Running build hook: [fsck]
==> Generating module dependencies
==> Creating gzip-compressed initcpio image: /boot/initramfs-5.10-x86_64-fallback.img
==> Image generation successful

To be honest, I don’t know, because I have a separate /boot partition in ext4, given that GRUB used to throw a hissy fit if it encountered anything else. I also use different HOOKS ─ instead of udev and keymap, I’m using systemd and sd-vconsole. :arrow_down:

HOOKS=(base systemd autodetect modconf block keyboard sd-vconsole filesystems shutdown)

Just what exactly are your “boot issues”? It would be helpful if we had some error message to go by. :thinking:

The system would halt mid-boot and become unresponsive. Wouldn’t even react to REISUO.
Turns out the issue was caused by two dkms kernel modules: anbox-ashmem and anbox-binder.
I had installed them a few days ago to try running waydroid, but hadn’t rebooted my system until now, when I converted / to btrfs. The system booted normally after i chrooted into it and removed the two anbox kernel modules.

In other words, my boot issues were not btrfs related. Thanks for all your help.

Reminder to self; Eliminating variables is a good first step of any troubleshooting process.

ps. edited title to reflect solution

FYI, those modules are built into the kernel now, no need to install them separately anymore.

1 Like

oh wow didn’t know that, thanks for the info! I tried following the Archwiki guide for anbox/waydroid, but It’s nice that it’s simpler now

Reason: XY problem - asking about one issue - then presenting another issue and mark that as solution is rude. Changing the topic title to reflect the new issue is even more rude as every attempt to help you solve the first issue will be ignorant comments.