Home Directory is ReadOnly (NotWritable)

Harmtan13 · 4 January 2023 22:32

Manjaro - 22.00 | Sikaris
KDE - 5.26.4
CPU - Ryzen 5900x
GPU - RX 6700 XT
File system - btrfs

My system randomly becoming Read-Only, both Home & Root folders. I get error messages that read

Configuration file "/home/harmtan13/.config/dolphinrc" not writable.
Please contact your system administrator.

I’ve checked the permissions and my user “harmtan13” still owns the home Directory with all the read write & execute permissions.

In the terminal I can SU and write a test text file anywhere in the root directory but in the Home directory, I get a error message that reads

touch: cannot touch 'test.txt': Read-only file system

When I restart I the computer I get an error message that tells me I’m in an emergency mode half the time and other times it’ll boot in, but I’m stuck in this state of not being able to do anything whatsoever.

This is the 2nd time within a week this has happened. The 1st time this happened, I just backed up what I needed and nuked the drive and started fresh. It had been a while, so I figured what the hell. This reinstall has been going less than a week and it’s happened again.

Anyone know what is going on? I’m unable to find anything on the matter or maybe I’m dumb & missed it. Part of me is confused on what I should be looking for.

Edit: Below is the output for "sudo dmesg --level err -L

[ 4.862394] kvm: support for 'kvm_amd' disabled by bios
[ 5.720340] kvm: support for 'kvm_amd' disabled by bios
[ 5.943127] kvm: support for 'kvm_amd' disabled by bios
[ 6.140412] kvm: support for 'kvm_amd' disabled by bios
[ 6.344152] kvm: support for 'kvm_amd' disabled by bios
[ 6.506878] kvm: support for 'kvm_amd' disabled by bios
[ 6.654450] kvm: support for 'kvm_amd' disabled by bios
[ 6.836678] kvm: support for 'kvm_amd' disabled by bios
[ 7.013950] kvm: support for 'kvm_amd' disabled by bios
[ 7.123449] kvm: support for 'kvm_amd' disabled by bios
[ 9327.958002] _swap_info_get: Bad swap file entry 3ffffffffffff
[ 9327.958042] _swap_info_get: Bad swap file entry 3ffffffffffff
[ 9327.958076] _swap_info_get: Bad swap file entry 3ffffffffffff
[ 9364.334241] BTRFS error (device nvme0n1p3): block=59469742080 write time tree block corruption detected
[12763.456781] BTRFS error (device nvme0n1p3: state EMA): Remounting read-write after error is not allowed

UPDATE:

Bad RAM was indeed the culprit. I tried that stick in multiple slots and other sticks in its original slot. That stick indeed always failed the memtest. It also aligns with other random issues I’d run I to every once in a while.

I had 64gb of RAM, so I had plenty to go around, which is probably why it didn’t happen super frequently.
I use a custom DVR (Channels DVR) solution for my TV which allows me to keep recordings locally. For shows that I stockpile episodes for, I’ll take the video files from over the network, load them into a RAM disk, rip out the commercials, compress them, and output them back to the RAM disk before transferring them back to the original network drive the came from.

Every now & again a file would become corrupted or go missing on the RAM disk, but it was so random that I just chalked it up to kdenlive being screwy or I forgot to actually do what I was supposed to do. But literally everything has been fine since removing the stick.

Jim.B · 5 January 2023 02:07

The last entry on the error log seems to show whats going on.

[ 9364.334241] BTRFS error (device nvme0n1p3): block=59469742080 write time tree block corruption detected
[12763.456781] BTRFS error (device nvme0n1p3: state EMA): Remounting read-write after error is not allowed

If you have your home folder on nvme0n1p3 its being mounted as read only because “tree block corruption detected”. The last line says that because of the error mounting read-write isnt allowed. I would backup anything you want to keep before it fails.

andreas85 · 5 January 2023 06:51

This is a clear warning that the device has massive problems!
The last time I ignored such warning signs, it cost me more than 100€ for data recovery. It was still a spinning disk though. Since then I’ve been healed!

Since you are the admin:

Don’t repair, replace !

Harmtan13:

[ 9364.334241] BTRFS error (device nvme0n1p3): block=59469742080 write time tree block corruption detected
[12763.456781] BTRFS error (device nvme0n1p3: state EMA): Remounting read-write after error is not allowed

Most other file systems would only have reported this when data had already been lost! Because BTRFS writes (and checks) checksums, it notices the error IMMEDIATELY.

BTRFS will not allow you to write more data to a defective drive !

All data written before the message is normally intact (eventually do a btrfs scrub from within a live manjaro). But if the drive finally fails, there is no rescue other than the professionals (which costs a lot of money)

P.S.:

You will be able to mount the filesystem readonly for recovery (with rsync)
Or you could clone it with btrfs send/receive
Do recover your data using a live manjaro (iso/usb). Not with your broken system !

Zesko · 5 January 2023 07:41

Check what is the output: (Read only)

$ journalctl --no-pager -p 4 | grep 'BTRFS'

and

$ sudo btrfs device stats /

and

$ sudo btrfs scrub status /

Harmtan13:

[ 9327.958002] _swap_info_get: Bad swap file entry 3ffffffffffff
[ 9327.958042] _swap_info_get: Bad swap file entry 3ffffffffffff
[ 9327.958076] _swap_info_get: Bad swap file entry 3ffffffffffff
[ 9364.334241] BTRFS error (device nvme0n1p3): block=59469742080 write time tree block corruption detected

You got the corrupted swapfile. I think RAM is probably damaged.

After full backup, take out all RAM from your mainboard and test each individual RAM with mprime or memtest to see which one is faulty.

If all RAM are fine, your disk or mainboard (bus system) is probably faulty.

If you need a automatic Btrfs notification, you can install btrfs-desktop-notification-git from AUR ( it was created by me). It notifies you immediately to easily notice when any error/warning Btrfs message appears.

andreas85 · 5 January 2023 16:49

may be related (or not)

Harmtan13 · 5 January 2023 18:42

Haha yes, I am the admin. Unfortunately, this Admin is under-qualified for this issue by himself.

Very glad to have chosen BTRFS then.

My data is all backed up on a Separate drive. I’ve lost massive amounts of data before, so I keep multiple backups now. I’ll have to do some testing on this drive. I got it from Best Buy less than a year ago.

Same error Messages for sure.

Harmtan13 · 5 January 2023 18:55

So I’m actually not in Read-Only Mode anymore. I got frustrated and booted into windows (Installed on a completly seperate SATA drive) to play a game to blow off some steam and when I went back, everything is operating normally again. But here is the output of that below.

Dec 30 09:30:33 ManjaroDesktop kernel: BTRFS warning (device nvme2n1p1): csum failed root 5 ino 708781 off 904318976 csum 0x323b3759 expected csum 0x152449fa mirror 1
Dec 30 09:30:33 ManjaroDesktop kernel: BTRFS error (device nvme2n1p1): bdev /dev/nvme2n1p1 errs: wr 0, rd 0, flush 0, corrupt 1, gen 0
Dec 30 09:30:33 ManjaroDesktop kernel: BTRFS warning (device nvme2n1p1): csum failed root 5 ino 708781 off 904318976 csum 0x323b3759 expected csum 0x152449fa mirror 1
Dec 30 09:30:33 ManjaroDesktop kernel: BTRFS error (device nvme2n1p1): bdev /dev/nvme2n1p1 errs: wr 0, rd 0, flush 0, corrupt 2, gen 0
Dec 30 09:30:33 ManjaroDesktop kernel: BTRFS warning (device nvme2n1p1): csum failed root 5 ino 708781 off 904318976 csum 0x323b3759 expected csum 0x152449fa mirror 1
Dec 30 09:30:33 ManjaroDesktop kernel: BTRFS error (device nvme2n1p1): bdev /dev/nvme2n1p1 errs: wr 0, rd 0, flush 0, corrupt 3, gen 0
Dec 30 09:30:33 ManjaroDesktop kernel: BTRFS warning (device nvme2n1p1): csum failed root 5 ino 708781 off 904318976 csum 0x323b3759 expected csum 0x152449fa mirror 1
Dec 30 09:30:33 ManjaroDesktop kernel: BTRFS error (device nvme2n1p1): bdev /dev/nvme2n1p1 errs: wr 0, rd 0, flush 0, corrupt 4, gen 0
Dec 30 09:30:42 ManjaroDesktop kernel: BTRFS warning (device nvme2n1p1): csum failed root 5 ino 2650 off 3244359680 csum 0xdf6aa349 expected csum 0x62d99924 mirror 1
Dec 30 09:30:42 ManjaroDesktop kernel: BTRFS error (device nvme2n1p1): bdev /dev/nvme2n1p1 errs: wr 0, rd 0, flush 0, corrupt 5, gen 0
Dec 30 09:30:42 ManjaroDesktop kernel: BTRFS warning (device nvme2n1p1): csum failed root 5 ino 2650 off 3244363776 csum 0x9cedde44 expected csum 0xf52eecf0 mirror 1
Dec 30 09:30:42 ManjaroDesktop kernel: BTRFS error (device nvme2n1p1): bdev /dev/nvme2n1p1 errs: wr 0, rd 0, flush 0, corrupt 6, gen 0
Dec 30 09:30:42 ManjaroDesktop kernel: BTRFS warning (device nvme2n1p1): csum failed root 5 ino 2650 off 3244371968 csum 0x35c47b8a expected csum 0x887c2eec mirror 1
Dec 30 09:30:42 ManjaroDesktop kernel: BTRFS error (device nvme2n1p1): bdev /dev/nvme2n1p1 errs: wr 0, rd 0, flush 0, corrupt 7, gen 0
Dec 30 09:30:42 ManjaroDesktop kernel: BTRFS warning (device nvme2n1p1): csum failed root 5 ino 2650 off 3244376064 csum 0x1f9153fa expected csum 0x25d6b622 mirror 1
Dec 30 09:30:42 ManjaroDesktop kernel: BTRFS error (device nvme2n1p1): bdev /dev/nvme2n1p1 errs: wr 0, rd 0, flush 0, corrupt 8, gen 0
Dec 30 09:30:42 ManjaroDesktop kernel: BTRFS warning (device nvme2n1p1): csum failed root 5 ino 2650 off 3244380160 csum 0x9a3f91cc expected csum 0xf9ebc29b mirror 1
Dec 30 09:30:42 ManjaroDesktop kernel: BTRFS error (device nvme2n1p1): bdev /dev/nvme2n1p1 errs: wr 0, rd 0, flush 0, corrupt 9, gen 0
Dec 30 09:30:42 ManjaroDesktop kernel: BTRFS warning (device nvme2n1p1): csum failed root 5 ino 2650 off 3244384256 csum 0x3cbe691b expected csum 0x85d4c5d8 mirror 1
Dec 30 09:30:42 ManjaroDesktop kernel: BTRFS error (device nvme2n1p1): bdev /dev/nvme2n1p1 errs: wr 0, rd 0, flush 0, corrupt 10, gen 0
Dec 30 09:30:42 ManjaroDesktop kernel: BTRFS warning (device nvme2n1p1): csum failed root 5 ino 2650 off 3244388352 csum 0xd74603d9 expected csum 0x967ca4f7 mirror 1
Dec 30 09:30:42 ManjaroDesktop kernel: BTRFS error (device nvme2n1p1): bdev /dev/nvme2n1p1 errs: wr 0, rd 0, flush 0, corrupt 11, gen 0
Dec 30 09:30:42 ManjaroDesktop kernel: BTRFS warning (device nvme2n1p1): csum failed root 5 ino 2650 off 3244359680 csum 0xc3888929 expected csum 0x62d99924 mirror 1
Dec 30 09:30:42 ManjaroDesktop kernel: BTRFS error (device nvme2n1p1): bdev /dev/nvme2n1p1 errs: wr 0, rd 0, flush 0, corrupt 12, gen 0
Dec 30 09:30:42 ManjaroDesktop kernel: BTRFS warning (device nvme2n1p1): csum failed root 5 ino 2650 off 3244359680 csum 0xe6629de1 expected csum 0x62d99924 mirror 1
Dec 30 09:30:42 ManjaroDesktop kernel: BTRFS error (device nvme2n1p1): bdev /dev/nvme2n1p1 errs: wr 0, rd 0, flush 0, corrupt 13, gen 0
Dec 30 09:30:42 ManjaroDesktop kernel: BTRFS warning (device nvme2n1p1): csum failed root 5 ino 2650 off 3244359680 csum 0xc2ab98ef expected csum 0x62d99924 mirror 1
Dec 30 09:30:42 ManjaroDesktop kernel: BTRFS error (device nvme2n1p1): bdev /dev/nvme2n1p1 errs: wr 0, rd 0, flush 0, corrupt 14, gen 0
Dec 30 09:31:19 ManjaroDesktop kernel: BTRFS warning (device nvme2n1p1): csum failed root 5 ino 2650 off 3244359680 csum 0x6d278346 expected csum 0x62d99924 mirror 1
Dec 30 09:31:19 ManjaroDesktop kernel: BTRFS error (device nvme2n1p1): bdev /dev/nvme2n1p1 errs: wr 0, rd 0, flush 0, corrupt 15, gen 0
Dec 30 09:31:19 ManjaroDesktop kernel: BTRFS warning (device nvme2n1p1): csum failed root 5 ino 2650 off 3244359680 csum 0x6d278346 expected csum 0x62d99924 mirror 1
Dec 30 09:31:19 ManjaroDesktop kernel: BTRFS error (device nvme2n1p1): bdev /dev/nvme2n1p1 errs: wr 0, rd 0, flush 0, corrupt 16, gen 0
Dec 30 09:31:19 ManjaroDesktop kernel: BTRFS warning (device nvme2n1p1): csum failed root 5 ino 2650 off 3244359680 csum 0x2befa86b expected csum 0x62d99924 mirror 1
Dec 30 09:31:19 ManjaroDesktop kernel: BTRFS error (device nvme2n1p1): bdev /dev/nvme2n1p1 errs: wr 0, rd 0, flush 0, corrupt 17, gen 0
Dec 30 09:34:56 ManjaroDesktop kernel: BTRFS warning (device nvme2n1p1): csum failed root 5 ino 647807 off 334151680 csum 0x0e34908a expected csum 0x97825590 mirror 1
Dec 30 09:34:56 ManjaroDesktop kernel: BTRFS error (device nvme2n1p1): bdev /dev/nvme2n1p1 errs: wr 0, rd 0, flush 0, corrupt 18, gen 0
Dec 30 09:34:56 ManjaroDesktop kernel: BTRFS warning (device nvme2n1p1): csum failed root 5 ino 647807 off 334151680 csum 0x6a3274da expected csum 0x97825590 mirror 1
Dec 30 09:34:56 ManjaroDesktop kernel: BTRFS error (device nvme2n1p1): bdev /dev/nvme2n1p1 errs: wr 0, rd 0, flush 0, corrupt 19, gen 0
Dec 30 09:34:56 ManjaroDesktop kernel: BTRFS warning (device nvme2n1p1): csum failed root 5 ino 647807 off 334151680 csum 0x6a3274da expected csum 0x97825590 mirror 1
Dec 30 09:34:56 ManjaroDesktop kernel: BTRFS error (device nvme2n1p1): bdev /dev/nvme2n1p1 errs: wr 0, rd 0, flush 0, corrupt 20, gen 0
Dec 30 09:34:56 ManjaroDesktop kernel: BTRFS warning (device nvme2n1p1): csum failed root 5 ino 647807 off 334151680 csum 0x0e34908a expected csum 0x97825590 mirror 1
Dec 30 09:34:56 ManjaroDesktop kernel: BTRFS error (device nvme2n1p1): bdev /dev/nvme2n1p1 errs: wr 0, rd 0, flush 0, corrupt 21, gen 0
Dec 30 10:05:41 ManjaroDesktop kernel: BTRFS warning (device nvme1n1p1): csum failed root 5 ino 738 off 2393718784 csum 0xeb1f1745 expected csum 0x94df02c0 mirror 1
Dec 30 10:05:41 ManjaroDesktop kernel: BTRFS error (device nvme1n1p1): bdev /dev/nvme1n1p1 errs: wr 0, rd 0, flush 0, corrupt 2, gen 0
Dec 30 10:05:41 ManjaroDesktop kernel: BTRFS warning (device nvme1n1p1): csum failed root 5 ino 738 off 2393722880 csum 0xc7a6c9b9 expected csum 0xae65fb0d mirror 1
Dec 30 10:05:41 ManjaroDesktop kernel: BTRFS error (device nvme1n1p1): bdev /dev/nvme1n1p1 errs: wr 0, rd 0, flush 0, corrupt 3, gen 0
Dec 30 10:05:41 ManjaroDesktop kernel: BTRFS warning (device nvme1n1p1): csum failed root 5 ino 738 off 2393726976 csum 0x9712b62b expected csum 0x2ad686a0 mirror 1
Dec 30 10:05:41 ManjaroDesktop kernel: BTRFS error (device nvme1n1p1): bdev /dev/nvme1n1p1 errs: wr 0, rd 0, flush 0, corrupt 4, gen 0
Dec 30 10:05:41 ManjaroDesktop kernel: BTRFS warning (device nvme1n1p1): csum failed root 5 ino 738 off 2393731072 csum 0x2537d265 expected csum 0x7ada9ed3 mirror 1
Dec 30 10:05:41 ManjaroDesktop kernel: BTRFS error (device nvme1n1p1): bdev /dev/nvme1n1p1 errs: wr 0, rd 0, flush 0, corrupt 5, gen 0
Dec 30 10:05:41 ManjaroDesktop kernel: BTRFS warning (device nvme1n1p1): csum failed root 5 ino 738 off 2393735168 csum 0xa0c34ce7 expected csum 0xcf84f2e1 mirror 1
Dec 30 10:05:41 ManjaroDesktop kernel: BTRFS error (device nvme1n1p1): bdev /dev/nvme1n1p1 errs: wr 0, rd 0, flush 0, corrupt 6, gen 0
Dec 30 10:05:41 ManjaroDesktop kernel: BTRFS warning (device nvme1n1p1): csum failed root 5 ino 738 off 2393747456 csum 0x007a7bf7 expected csum 0x776d015d mirror 1
Dec 30 10:05:41 ManjaroDesktop kernel: BTRFS error (device nvme1n1p1): bdev /dev/nvme1n1p1 errs: wr 0, rd 0, flush 0, corrupt 7, gen 0
Dec 30 10:05:41 ManjaroDesktop kernel: BTRFS warning (device nvme1n1p1): csum failed root 5 ino 738 off 2393718784 csum 0xeb1f1745 expected csum 0x94df02c0 mirror 1
Dec 30 10:05:41 ManjaroDesktop kernel: BTRFS error (device nvme1n1p1): bdev /dev/nvme1n1p1 errs: wr 0, rd 0, flush 0, corrupt 8, gen 0
Dec 30 10:05:41 ManjaroDesktop kernel: BTRFS warning (device nvme1n1p1): csum failed root 5 ino 738 off 2393718784 csum 0xeb1f1745 expected csum 0x94df02c0 mirror 1
Dec 30 10:05:41 ManjaroDesktop kernel: BTRFS error (device nvme1n1p1): bdev /dev/nvme1n1p1 errs: wr 0, rd 0, flush 0, corrupt 9, gen 0
Dec 30 10:05:41 ManjaroDesktop kernel: BTRFS warning (device nvme1n1p1): csum failed root 5 ino 738 off 2393718784 csum 0xeb1f1745 expected csum 0x94df02c0 mirror 1
Dec 30 10:05:41 ManjaroDesktop kernel: BTRFS error (device nvme1n1p1): bdev /dev/nvme1n1p1 errs: wr 0, rd 0, flush 0, corrupt 10, gen 0
Dec 30 10:05:41 ManjaroDesktop kernel: BTRFS warning (device nvme1n1p1): csum failed root 5 ino 738 off 2393718784 csum 0xeb1f1745 expected csum 0x94df02c0 mirror 1
Dec 30 10:05:41 ManjaroDesktop kernel: BTRFS error (device nvme1n1p1): bdev /dev/nvme1n1p1 errs: wr 0, rd 0, flush 0, corrupt 11, gen 0
Jan 03 16:06:21 ManjaroDesktop kernel: BTRFS warning (device nvme1n1p1): csum failed root 5 ino 745 off 643743744 csum 0x1d6ed48e expected csum 0x12981381 mirror 1
Jan 03 16:06:21 ManjaroDesktop kernel: BTRFS error (device nvme1n1p1): bdev /dev/nvme1n1p1 errs: wr 0, rd 0, flush 0, corrupt 14, gen 0
Jan 03 16:06:21 ManjaroDesktop kernel: BTRFS warning (device nvme1n1p1): csum failed root 5 ino 745 off 643756032 csum 0x16313acb expected csum 0xf960b6e0 mirror 1
Jan 03 16:06:21 ManjaroDesktop kernel: BTRFS error (device nvme1n1p1): bdev /dev/nvme1n1p1 errs: wr 0, rd 0, flush 0, corrupt 15, gen 0
Jan 03 16:06:21 ManjaroDesktop kernel: BTRFS warning (device nvme1n1p1): csum failed root 5 ino 745 off 643772416 csum 0x6a9e18c1 expected csum 0xc0faba49 mirror 1
Jan 03 16:06:21 ManjaroDesktop kernel: BTRFS error (device nvme1n1p1): bdev /dev/nvme1n1p1 errs: wr 0, rd 0, flush 0, corrupt 16, gen 0
Jan 03 16:06:21 ManjaroDesktop kernel: BTRFS warning (device nvme1n1p1): csum failed root 5 ino 745 off 643743744 csum 0x1d6ed48e expected csum 0x12981381 mirror 1
Jan 03 16:06:21 ManjaroDesktop kernel: BTRFS error (device nvme1n1p1): bdev /dev/nvme1n1p1 errs: wr 0, rd 0, flush 0, corrupt 17, gen 0
Jan 03 16:06:21 ManjaroDesktop kernel: BTRFS warning (device nvme1n1p1): csum failed root 5 ino 745 off 643743744 csum 0x1d6ed48e expected csum 0x12981381 mirror 1
Jan 03 16:06:21 ManjaroDesktop kernel: BTRFS error (device nvme1n1p1): bdev /dev/nvme1n1p1 errs: wr 0, rd 0, flush 0, corrupt 18, gen 0
Jan 03 16:06:21 ManjaroDesktop kernel: BTRFS warning (device nvme1n1p1): csum failed root 5 ino 745 off 643743744 csum 0x1d6ed48e expected csum 0x12981381 mirror 1
Jan 03 16:06:21 ManjaroDesktop kernel: BTRFS error (device nvme1n1p1): bdev /dev/nvme1n1p1 errs: wr 0, rd 0, flush 0, corrupt 19, gen 0
Jan 03 16:06:21 ManjaroDesktop kernel: BTRFS warning (device nvme1n1p1): csum failed root 5 ino 745 off 643743744 csum 0x1d6ed48e expected csum 0x12981381 mirror 1
Jan 03 16:06:21 ManjaroDesktop kernel: BTRFS error (device nvme1n1p1): bdev /dev/nvme1n1p1 errs: wr 0, rd 0, flush 0, corrupt 20, gen 0
Jan 03 16:06:21 ManjaroDesktop kernel: BTRFS warning (device nvme1n1p1): csum failed root 5 ino 745 off 643743744 csum 0x1d6ed48e expected csum 0x12981381 mirror 1
Jan 03 16:06:21 ManjaroDesktop kernel: BTRFS error (device nvme1n1p1): bdev /dev/nvme1n1p1 errs: wr 0, rd 0, flush 0, corrupt 21, gen 0
Jan 03 16:06:21 ManjaroDesktop kernel: BTRFS warning (device nvme1n1p1): csum failed root 5 ino 745 off 643743744 csum 0x1d6ed48e expected csum 0x12981381 mirror 1
Jan 03 16:06:21 ManjaroDesktop kernel: BTRFS error (device nvme1n1p1): bdev /dev/nvme1n1p1 errs: wr 0, rd 0, flush 0, corrupt 22, gen 0
Jan 03 16:06:21 ManjaroDesktop kernel: BTRFS warning (device nvme1n1p1): csum failed root 5 ino 745 off 643756032 csum 0xf9de2251 expected csum 0xf960b6e0 mirror 1
Jan 03 16:06:21 ManjaroDesktop kernel: BTRFS error (device nvme1n1p1): bdev /dev/nvme1n1p1 errs: wr 0, rd 0, flush 0, corrupt 23, gen 0
Jan 03 16:52:30 ManjaroDesktop kernel: BTRFS warning (device nvme1n1p1): csum failed root 5 ino 481 off 338477056 csum 0x0750d42a expected csum 0x08a61325 mirror 1
Jan 03 16:52:30 ManjaroDesktop kernel: BTRFS error (device nvme1n1p1): bdev /dev/nvme1n1p1 errs: wr 0, rd 0, flush 0, corrupt 27, gen 0
Jan 03 16:52:30 ManjaroDesktop kernel: BTRFS warning (device nvme1n1p1): csum failed root 5 ino 481 off 338489344 csum 0x9d700e76 expected csum 0x9dce9ac7 mirror 1
Jan 03 16:52:30 ManjaroDesktop kernel: BTRFS error (device nvme1n1p1): bdev /dev/nvme1n1p1 errs: wr 0, rd 0, flush 0, corrupt 28, gen 0
Jan 03 16:52:30 ManjaroDesktop kernel: BTRFS warning (device nvme1n1p1): csum failed root 5 ino 481 off 338477056 csum 0x0750d42a expected csum 0x08a61325 mirror 1
Jan 03 16:52:30 ManjaroDesktop kernel: BTRFS error (device nvme1n1p1): bdev /dev/nvme1n1p1 errs: wr 0, rd 0, flush 0, corrupt 29, gen 0
Jan 03 16:52:30 ManjaroDesktop kernel: BTRFS warning (device nvme1n1p1): csum failed root 5 ino 481 off 338477056 csum 0x0750d42a expected csum 0x08a61325 mirror 1
Jan 03 16:52:30 ManjaroDesktop kernel: BTRFS error (device nvme1n1p1): bdev /dev/nvme1n1p1 errs: wr 0, rd 0, flush 0, corrupt 30, gen 0
Jan 03 16:52:30 ManjaroDesktop kernel: BTRFS warning (device nvme1n1p1): csum failed root 5 ino 481 off 338477056 csum 0x0750d42a expected csum 0x08a61325 mirror 1
Jan 03 16:52:30 ManjaroDesktop kernel: BTRFS error (device nvme1n1p1): bdev /dev/nvme1n1p1 errs: wr 0, rd 0, flush 0, corrupt 31, gen 0
Jan 03 19:20:55 ManjaroDesktop kernel: BTRFS warning (device nvme1n1p1): csum failed root 5 ino 625 off 11177984 csum 0x7659d339 expected csum 0x79af1436 mirror 1
Jan 03 19:20:55 ManjaroDesktop kernel: BTRFS error (device nvme1n1p1): bdev /dev/nvme1n1p1 errs: wr 0, rd 0, flush 0, corrupt 32, gen 0
Jan 03 19:55:19 ManjaroDesktop kernel: BTRFS warning (device nvme0n1p2): csum failed root 256 ino 250173 off 569344 csum 0xad769177 expected csum 0xa2805678 mirror 1
Jan 03 19:55:19 ManjaroDesktop kernel: BTRFS error (device nvme0n1p2): bdev /dev/nvme0n1p2 errs: wr 0, rd 0, flush 0, corrupt 1, gen 0
Jan 03 19:55:19 ManjaroDesktop kernel: BTRFS warning (device nvme0n1p2): csum failed root 256 ino 250173 off 569344 csum 0xad769177 expected csum 0xa2805678 mirror 1
Jan 03 19:55:19 ManjaroDesktop kernel: BTRFS error (device nvme0n1p2): bdev /dev/nvme0n1p2 errs: wr 0, rd 0, flush 0, corrupt 2, gen 0
Jan 03 19:55:19 ManjaroDesktop kernel: BTRFS warning (device nvme0n1p2): csum failed root 256 ino 250173 off 569344 csum 0xad769177 expected csum 0xa2805678 mirror 1
Jan 03 19:55:19 ManjaroDesktop kernel: BTRFS error (device nvme0n1p2): bdev /dev/nvme0n1p2 errs: wr 0, rd 0, flush 0, corrupt 3, gen 0
Jan 03 19:55:19 ManjaroDesktop kernel: BTRFS warning (device nvme0n1p2): csum failed root 256 ino 250173 off 569344 csum 0xad769177 expected csum 0xa2805678 mirror 1
Jan 03 19:55:19 ManjaroDesktop kernel: BTRFS error (device nvme0n1p2): bdev /dev/nvme0n1p2 errs: wr 0, rd 0, flush 0, corrupt 4, gen 0
Jan 03 19:55:19 ManjaroDesktop kernel: BTRFS warning (device nvme0n1p2): csum failed root 256 ino 250173 off 569344 csum 0xad769177 expected csum 0xa2805678 mirror 1
Jan 03 19:55:19 ManjaroDesktop kernel: BTRFS error (device nvme0n1p2): bdev /dev/nvme0n1p2 errs: wr 0, rd 0, flush 0, corrupt 5, gen 0
Jan 03 19:55:19 ManjaroDesktop kernel: BTRFS warning (device nvme0n1p2): csum failed root 256 ino 250173 off 569344 csum 0xad769177 expected csum 0xa2805678 mirror 1
Jan 03 19:55:19 ManjaroDesktop kernel: BTRFS error (device nvme0n1p2): bdev /dev/nvme0n1p2 errs: wr 0, rd 0, flush 0, corrupt 6, gen 0
Jan 03 19:55:19 ManjaroDesktop kernel: BTRFS warning (device nvme0n1p2): csum failed root 256 ino 250173 off 569344 csum 0xad769177 expected csum 0xa2805678 mirror 1
Jan 03 19:55:19 ManjaroDesktop kernel: BTRFS error (device nvme0n1p2): bdev /dev/nvme0n1p2 errs: wr 0, rd 0, flush 0, corrupt 7, gen 0
Jan 03 19:55:19 ManjaroDesktop kernel: BTRFS warning (device nvme0n1p2): csum failed root 256 ino 250173 off 569344 csum 0xad769177 expected csum 0xa2805678 mirror 1
Jan 03 19:55:19 ManjaroDesktop kernel: BTRFS error (device nvme0n1p2): bdev /dev/nvme0n1p2 errs: wr 0, rd 0, flush 0, corrupt 8, gen 0
Jan 03 19:55:19 ManjaroDesktop kernel: BTRFS warning (device nvme0n1p2): csum failed root 256 ino 250173 off 569344 csum 0xad769177 expected csum 0xa2805678 mirror 1
Jan 03 19:55:19 ManjaroDesktop kernel: BTRFS error (device nvme0n1p2): bdev /dev/nvme0n1p2 errs: wr 0, rd 0, flush 0, corrupt 9, gen 0
Jan 03 19:55:19 ManjaroDesktop kernel: BTRFS warning (device nvme0n1p2): csum failed root 256 ino 250173 off 569344 csum 0xad769177 expected csum 0xa2805678 mirror 1
Jan 03 19:55:19 ManjaroDesktop kernel: BTRFS error (device nvme0n1p2): bdev /dev/nvme0n1p2 errs: wr 0, rd 0, flush 0, corrupt 10, gen 0
Jan 04 10:46:22 ManjaroDesktop kernel: BTRFS critical (device nvme0n1p3): corrupt leaf: root=10 block=59469742080 slot=3, unexpected item end, have 268450971 expect 15515
Jan 04 10:46:22 ManjaroDesktop kernel: BTRFS error (device nvme0n1p3): block=59469742080 write time tree block corruption detected
Jan 04 10:46:22 ManjaroDesktop kernel: BTRFS: error (device nvme0n1p3) in btrfs_commit_transaction:2447: errno=-5 IO failure (Error while writing out transaction)
Jan 04 10:46:22 ManjaroDesktop kernel: BTRFS warning (device nvme0n1p3: state E): Skipping commit of aborted transaction.
Jan 04 10:46:22 ManjaroDesktop kernel: BTRFS: error (device nvme0n1p3: state EA) in cleanup_transaction:1958: errno=-5 IO failure
Jan 04 15:00:10 ManjaroDesktop kernel: BTRFS error (device nvme0n1p3: state EMA): Remounting read-write after error is not allowed

Then this one too.

[/dev/nvme0n1p2].write_io_errs    0
[/dev/nvme0n1p2].read_io_errs     0
[/dev/nvme0n1p2].flush_io_errs    0
[/dev/nvme0n1p2].corruption_errs  2345
[/dev/nvme0n1p2].generation_errs  0

And this one…


[quote="Zesko, post:4, topic:130880"]
`sudo btrfs scrub status /`
[/quote]

UUID:             83c860f4-f59b-4098-99fd-52b0ccc90010
        no stats available
Total to scrub:   19.55GiB
Rate:             0.00B/s
Error summary:    no errors found

I might venture to say that the Motherboard might be an issue. I’ve had the same RAM, CPU, GPU, and drives all on a different board with zero issues. Maybe my drive isn’t slotted in properly? I do remember having to make due with someth when I swapped cuz my brother didn’t send me all the stuff for the board. I have that now. I’ll do some investigating there.

Thanks for your assistance.

Harmtan13 · 5 January 2023 19:04

Checked the Drive Health using

sudo smartctl -a /dev/nvme0n1

and this is the output…

=== START OF INFORMATION SECTION ===
Model Number:                       WDBRPG0010BNC-WRSN
Serial Number:                      21441H802567
Firmware Version:                   111130WD
PCI Vendor/Subsystem ID:            0x15b7
IEEE OUI Identifier:                0x001b44
Total NVM Capacity:                 1,000,204,886,016 [1.00 TB]
Unallocated NVM Capacity:           0
Controller ID:                      8215
NVMe Version:                       1.3
Number of Namespaces:               1
Namespace 1 Size/Capacity:          1,000,204,886,016 [1.00 TB]
Namespace 1 Formatted LBA Size:     512
Namespace 1 IEEE EUI-64:            001b44 8b413f2542
Local Time is:                      Thu Jan  5 12:02:50 2023 MST
Firmware Updates (0x14):            2 Slots, no Reset required
Optional Admin Commands (0x0017):   Security Format Frmw_DL Self_Test
Optional NVM Commands (0x005f):     Comp Wr_Unc DS_Mngmt Wr_Zero Sav/Sel_Feat Timestmp
Log Page Attributes (0x0e):         Cmd_Eff_Lg Ext_Get_Lg Telmtry_Lg
Maximum Data Transfer Size:         128 Pages
Warning  Comp. Temp. Threshold:     84 Celsius
Critical Comp. Temp. Threshold:     88 Celsius
Namespace 1 Features (0x02):        NA_Fields

Supported Power States
St Op     Max   Active     Idle   RL RT WL WT  Ent_Lat  Ex_Lat
 0 +     6.00W       -        -    0  0  0  0        0       0
 1 +     3.50W       -        -    1  1  1  1        0       0
 2 +     3.00W       -        -    2  2  2  2        0       0
 3 -   0.1000W       -        -    3  3  3  3     4000   10000
 4 -   0.0035W       -        -    4  4  4  4     4000   40000

Supported LBA Sizes (NSID 0x1)
Id Fmt  Data  Metadt  Rel_Perf
 0 +     512       0         2
 1 -    4096       0         1

=== START OF SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED

SMART/Health Information (NVMe Log 0x02)
Critical Warning:                   0x00
Temperature:                        47 Celsius
Available Spare:                    100%
Available Spare Threshold:          10%
Percentage Used:                    0%
Data Units Read:                    32,932,272 [16.8 TB]
Data Units Written:                 10,914,989 [5.58 TB]
Host Read Commands:                 156,211,271
Host Write Commands:                93,698,639
Controller Busy Time:               185
Power Cycles:                       707
Power On Hours:                     1,210
Unsafe Shutdowns:                   36
Media and Data Integrity Errors:    0
Error Information Log Entries:      1
Warning  Comp. Temperature Time:    0
Critical Comp. Temperature Time:    0

Error Information (NVMe Log 0x01, 16 of 256 entries)
No Errors Logged

So if I’m reading this correctly, the drive appears to be ok. Guess the RAM is my next step.

andreas85 · 5 January 2023 19:40

possible

Maybe switch from a swap file to a swap partition (just to be on the safe side)

You’ve proven you’re qualified by asking the right questions at the right time

Zesko · 5 January 2023 20:35

there are many inode numbers in the log. You can find which files are corrupted:

Harmtan13:

Jan 03 19:55:19 ManjaroDesktop kernel: BTRFS warning (device nvme0n1p2): csum failed root 256 ino 250173 off 569344 csum 0xad769177 expected csum 0xa2805678 mirror 1
Jan 03 19:55:19 ManjaroDesktop kernel: BTRFS error (device nvme0n1p2): bdev /dev/nvme0n1p2 errs: wr 0, rd 0, flush 0, corrupt 9, gen 0

For example: ino is “250173”

$ sudo btrfs inspect inode-resolve 250173 /
$ sudo find / -inum 250173

Harmtan13 · 5 January 2023 21:43

Thank you. I’ll work through that log.

Appreciate it and appreciate your time in helping.

This was it, one of my most recent RAM modules failed immediately after starting a memtest. Still under warranty, so I’ll get it replaced and just live with less RAM until it’s replaced. Thanks for the help!!!

Zesko · 6 January 2023 06:36

Glad you found the bad RAM, but beware I doubt the RAM slot in the motherboard is broken.

Be careful, the test might have misinformation that this RAM is faulty.
Have you thoroughly tested other RAM in the same slot on the motherboard?

If other RAMs in the same slot of the motherboard fail after testing, that means the motherboard is faulty.

andreas85 · 6 January 2023 06:47

So the blocks of btrfs have been altered (in RAM) before they where written to the filesystem.

You could try swapping slots

system · 8 January 2023 20:48

This topic was automatically closed 2 days after the last reply. New replies are no longer allowed.