Yes, I am using snapshots.
Can you share the log with us?
$ journalctl --no-pager -p 6 --dmesg | grep BTRFS > /tmp/btrfs.log
Then copy this file /tmp/btrfs.log to the USB.
1 Like
Here you go. The log begins with these errors:
Jan 02 08:07:09 Orangekarat kernel: BTRFS info (device dm-0): first mount of filesystem 4e77bd49-b817-4ee0-a88d-ecc2af32ac71
Jan 02 08:07:09 Orangekarat kernel: BTRFS info (device dm-0): using crc32c (crc32c-intel) checksum algorithm
Jan 02 08:07:09 Orangekarat kernel: BTRFS info (device dm-0): using free space tree
Jan 02 08:07:09 Orangekarat kernel: BTRFS info (device dm-0): bdev /dev/mapper/luks-e117a149-a0c5-401b-8032-d03580a55c9b errs: wr 0, rd 0, flush 0, corrupt 789627, gen 0
Jan 02 08:07:09 Orangekarat kernel: BTRFS info (device dm-0): enabling ssd optimizations
Jan 02 08:07:09 Orangekarat kernel: BTRFS info (device dm-0: state M): turning on async discard
Jan 02 08:07:09 Orangekarat kernel: BTRFS info: devid 1 device path /dev/mapper/luks-e117a149-a0c5-401b-8032-d03580a55c9b changed to /dev/dm-0 scanned by (udev-worker) (572)
Jan 02 08:07:09 Orangekarat kernel: BTRFS info: devid 1 device path /dev/dm-0 changed to /dev/mapper/luks-e117a149-a0c5-401b-8032-d03580a55c9b scanned by (udev-worker) (572)
Jan 02 08:07:42 Orangekarat kernel: BTRFS warning (device dm-0): csum failed root 257 ino 3037134 off 253952 csum 0xc341dc86 expected csum 0xc341fc86 mirror 1
Jan 02 08:07:42 Orangekarat kernel: BTRFS error (device dm-0): bdev /dev/mapper/luks-e117a149-a0c5-401b-8032-d03580a55c9b errs: wr 0, rd 0, flush 0, corrupt 789628, gen 0
Jan 02 08:07:42 Orangekarat kernel: BTRFS warning (device dm-0): csum failed root 257 ino 3037134 off 253952 csum 0xc341dc86 expected csum 0xc341fc86 mirror 1
Jan 02 08:07:42 Orangekarat kernel: BTRFS error (device dm-0): bdev /dev/mapper/luks-e117a149-a0c5-401b-8032-d03580a55c9b errs: wr 0, rd 0, flush 0, corrupt 789629, gen 0
Jan 02 08:07:42 Orangekarat kernel: BTRFS warning (device dm-0): csum failed root 257 ino 3037134 off 253952 csum 0xc341dc86 expected csum 0xc341fc86 mirror 1
Jan 02 08:07:42 Orangekarat kernel: BTRFS error (device dm-0): bdev /dev/mapper/luks-e117a149-a0c5-401b-8032-d03580a55c9b errs: wr 0, rd 0, flush 0, corrupt 789630, gen 0
Jan 02 08:07:42 Orangekarat kernel: BTRFS warning (device dm-0): csum failed root 257 ino 3037134 off 253952 csum 0xc341dc86 expected csum 0xc341fc86 mirror 1
Jan 02 08:07:42 Orangekarat kernel: BTRFS error (device dm-0): bdev /dev/mapper/luks-e117a149-a0c5-401b-8032-d03580a55c9b errs: wr 0, rd 0, flush 0, corrupt 789631, gen 0
Jan 02 08:07:42 Orangekarat kernel: BTRFS warning (device dm-0): csum failed root 257 ino 3037134 off 253952 csum 0xc341dc86 expected csum 0xc341fc86 mirror 1
Jan 02 08:07:42 Orangekarat kernel: BTRFS error (device dm-0): bdev /dev/mapper/luks-e117a149-a0c5-401b-8032-d03580a55c9b errs: wr 0, rd 0, flush 0, corrupt 789632, gen 0
Jan 02 08:07:42 Orangekarat kernel: BTRFS warning (device dm-0): csum failed root 257 ino 3037134 off 253952 csum 0xc341dc86 expected csum 0xc341fc86 mirror 1
Jan 02 08:07:42 Orangekarat kernel: BTRFS error (device dm-0): bdev /dev/mapper/luks-e117a149-a0c5-401b-8032-d03580a55c9b errs: wr 0, rd 0, flush 0, corrupt 789633, gen 0
Jan 02 08:07:42 Orangekarat kernel: BTRFS warning (device dm-0): csum failed root 257 ino 3037134 off 253952 csum 0xc341dc86 expected csum 0xc341fc86 mirror 1
Jan 02 08:07:42 Orangekarat kernel: BTRFS error (device dm-0): bdev /dev/mapper/luks-e117a149-a0c5-401b-8032-d03580a55c9b errs: wr 0, rd 0, flush 0, corrupt 789634, gen 0
Jan 02 08:07:42 Orangekarat kernel: BTRFS warning (device dm-0): csum failed root 257 ino 3037134 off 253952 csum 0xc341dc86 expected csum 0xc341fc86 mirror 1
Jan 02 08:07:42 Orangekarat kernel: BTRFS error (device dm-0): bdev /dev/mapper/luks-e117a149-a0c5-401b-8032-d03580a55c9b errs: wr 0, rd 0, flush 0, corrupt 789635, gen 0
Jan 02 08:07:42 Orangekarat kernel: BTRFS warning (device dm-0): csum failed root 257 ino 3037134 off 253952 csum 0xc341dc86 expected csum 0xc341fc86 mirror 1
Jan 02 08:07:42 Orangekarat kernel: BTRFS error (device dm-0): bdev /dev/mapper/luks-e117a149-a0c5-401b-8032-d03580a55c9b errs: wr 0, rd 0, flush 0, corrupt 789636, gen 0
Jan 02 08:07:42 Orangekarat kernel: BTRFS warning (device dm-0): csum failed root 257 ino 3037134 off 253952 csum 0xc341dc86 expected csum 0xc341fc86 mirror 1
Jan 02 08:07:42 Orangekarat kernel: BTRFS error (device dm-0): bdev /dev/mapper/luks-e117a149-a0c5-401b-8032-d03580a55c9b errs: wr 0, rd 0, flush 0, corrupt 789637, gen 0
Jan 02 08:07:50 Orangekarat kernel: BTRFS warning (device dm-0): csum failed root 257 ino 3037134 off 253952 csum 0xc341dc86 expected csum 0xc341fc86 mirror 1
Jan 02 08:07:50 Orangekarat kernel: BTRFS error (device dm-0): bdev /dev/mapper/luks-e117a149-a0c5-401b-8032-d03580a55c9b errs: wr 0, rd 0, flush 0, corrupt 789751, gen 0
Jan 02 08:07:50 Orangekarat kernel: BTRFS warning (device dm-0): csum failed root 257 ino 3037134 off 253952 csum 0xc341dc86 expected csum 0xc341fc86 mirror 1
Jan 02 08:07:50 Orangekarat kernel: BTRFS error (device dm-0): bdev /dev/mapper/luks-e117a149-a0c5-401b-8032-d03580a55c9b errs: wr 0, rd 0, flush 0, corrupt 789752, gen 0
Jan 02 08:07:50 Orangekarat kernel: BTRFS warning (device dm-0): csum failed root 257 ino 3037134 off 253952 csum 0xc341dc86 expected csum 0xc341fc86 mirror 1
Jan 02 08:07:50 Orangekarat kernel: BTRFS error (device dm-0): bdev /dev/mapper/luks-e117a149-a0c5-401b-8032-d03580a55c9b errs: wr 0, rd 0, flush 0, corrupt 789753, gen 0
Jan 02 08:07:50 Orangekarat kernel: BTRFS warning (device dm-0): csum failed root 257 ino 3037134 off 253952 csum 0xc341dc86 expected csum 0xc341fc86 mirror 1
[I truncated this because it’s too big for the forum.]
And then it ends with this:
Jan 02 08:52:42 Orangekarat kernel: BTRFS warning (device dm-0): checksum verify failed on logical 180469760 mirror 1 wanted 0x31c6926e found 0xe6623c0e level 0
Jan 02 08:52:42 Orangekarat kernel: BTRFS warning (device dm-0): checksum verify failed on logical 180469760 mirror 1 wanted 0x31c6926e found 0xe6623c0e level 0
Jan 02 08:52:42 Orangekarat kernel: BTRFS warning (device dm-0): checksum verify failed on logical 180469760 mirror 2 wanted 0x31c6926e found 0x1bbec240 level 0
Jan 02 08:52:42 Orangekarat kernel: BTRFS error (device dm-0: state A): Transaction aborted (error -5)
Jan 02 08:52:42 Orangekarat kernel: BTRFS: error (device dm-0: state A) in btrfs_drop_snapshot:5732: errno=-5 IO failure```
1 Like
It looks like your hardware is the problem.
How to identify which file is corrupted:
For example:
Copy logical id “180469760”
Try to run
$ sudo btrfs inspect logical-resolve 180469760 /
Edit://
Copy inode id “3037134”
Run the command line to find out which file is corrupt
$ sudo btrfs inspect inode-resolve 3037134 /
The first command threw the error
ERROR: logical ino ioctl: No such file or directory
The second command threw the error
Error: ino paths ioctl: no such file or directory
Try to run
$ sudo btrfs inspect-internal logical-resolve 253952 /
$ sudo btrfs inspect-internal logical-resolve 3037134 /
$ sudo btrfs inspect-internal logical-resolve 180469760 /
ERROR: logical ino ioctl: No such file or directory
in each case.
I guess that btrfs-progs tool can’t find or read some file if this file is already gone but its checksum is present. That is why it would be marked as the corrupted file in the Btrfs log.
Edit://
This is my stupid thought, sorry.
Look at the mismatched checksum:
0xc341dc86 =/= 0xc341fc86 → That looks like d does not match f.
The damaged file must be there.
I think btrfs inspect logical-resolve <logical> <path> has some bug. I will report the bug on btrfs-prog github
I see what you are referring to. As for my immediate situation, is there any way to resolve this? Or do I need to replace the NVME drive?
I can have a look at it too. What is your status? I mean, have you backed up your data i.e. copied over what you need to an external drive? Did you make a bootable Manjaro live stick (or a different distro)?
Testing your hardware is not easy:
-
Check a cable(NVME PCIe does not use the cable) -
Test your RAMs (Remove all RAMs and test every single RAM with test tool for example
memtest -
If all RAM tests pass, then replace your disk with another disk
-
If the problem persists after replacing the drive, the mainboard or the CPU cache is probably the problem.
Let’s try to fix what’s there Zesko. Why would you assume RAM’s fault?
Did you use Timeshift to Btrfs-snaphot on regular basis? If so, about how often, hourly, daily, weekly, on boot?
A appreciate your willingness to help!
The good news is that almost all of my urgent data is in the cloud and was up to date before this disaster struck. The few items that weren’t in the cloud that have been backed up on a separate USB device. I have a live stick ready. FYI - I use an encrypted LUKS device. The most I might lose is some ancient archived emails in Thunderbird.
My chief worry at this point is if the NVME drive hardware itself is flawed, or if BTRFS was somehow the source of my problem.
I did take snaps regularly, but I can’t remember the preset frequency (it was probably weekly). The latest snap was taken at the beginning of the upgrade process where I ran into this error.
If it is new as you say, it should not be a hardware mistake. Let’s try to look at it with btrfs built-in commands.
Therefore please boot the computer into the live environment and tell me when you have, here.
I will do that, but circumstances are such that I have to be away from the machine for a few hours. May I reply to this thread when I’m available again? Again, I GREATLY appreciate your willingness to help me, a total stranger, and I want to be sure to not waste any of your time.
That is not true.
In my real experience, I had bought a new RAM, it had already been damaged (Only 1 random bit is damaged), thanks Btrfs stopped copying my new backup to other hard drive because the backup was corrupted.
Do not trust new hardware after buying it!
And don’t trust old hardware. Every hardware has a lifespan limit.
1 Like
Huh I was in the same position you are in right now so I appreciate to help (where I can). As said, I will check again into this thread until we have a solution or we are sure we can’t find one.
Plan is to find error messages and then act accordingly. I will be back tomorrow night or Friday too so don’t worry. 
Yes, RAM can be the root of the evil in some cases, true. However there are many errors btrfs can spawn and yours does not seem to liken Adram's.
There are to my knowledge several error scenarios and one should look at which method to fix the filesystem with. I will not recommend anything that is not recommended at Welcome to BTRFS documentation! — BTRFS documentation
Edit: Oops sorry Zesko your errors are in fact similar…
How do I get the strike-through option for text?
1 Like
Hey, I just wanted to let you know that I’m not going to need your help. Upon further research, the drive has failed and will need to be replaced.
I came to this conclusion after reading this article. My smartctl analysis yielded the same error, and thus it’s time to move on. Thank you again for your willingness to help!
2 Likes
This topic was automatically closed 3 hours after the last reply. New replies are no longer allowed.
