Cannot clone btrfs boot drive with clonezilla

Will try this as a last resort. Have limited knowledge.

Have some more info.

[manjaro@manjaro ~]$ sudo btrfs check /dev/sda2 
Opening filesystem to check...
Checking filesystem on /dev/sda2
UUID: 17243299-2eab-4c05-9d27-76b0968b95e5
[1/7] checking root items
[2/7] checking extents
[3/7] checking free space cache
block group 237318963200 has wrong amount of free space, free space cache has 4096 block group has 1073737728
failed to load free space cache for block group 237318963200
[4/7] checking fs roots
[5/7] checking only csums items (without verifying data)
[6/7] checking root refs
[7/7] checking quota groups
check/qgroup-verify.c:546: account_all_refs: BUG_ON `ref->num_bytes != num_bytes` triggered, value 1
btrfs(+0x1a65d)[0x55b57c0f165d]
btrfs(qgroup_verify_all+0x5ad)[0x55b57c0f27ad]
btrfs(+0x49d9a)[0x55b57c120d9a]
btrfs(main+0x8e)[0x55b57c0ed0be]
/usr/lib/libc.so.6(__libc_start_main+0xd5)[0x7fdc46329b25]
btrfs(_start+0x2e)[0x55b57c0ed3ee]
Aborted
[manjaro@manjaro ~]$ sudo btrfs check --repair /dev/sda2
enabling repair mode
WARNING:

	Do not use --repair unless you are advised to do so by a developer
	or an experienced user, and then only after having accepted that no
	fsck can successfully repair all types of filesystem corruption. Eg.
	some software or hardware bugs can fatally damage a volume.
	The operation will start in 10 seconds.
	Use Ctrl-C to stop it.
10 9 8 7 6 5 4 3 2 1^C

I did not coninue with the repair because of its warnings. Maybe try something else first??

The drive HAVE TO BE UNMOUNTED like with fsck for a check.

I was running a live iso to do that :slight_smile:

Do you use qgroups? Seems there is a problem… Maybe disable it:

btrfs quota disable /mnt/btrfs
btrfs filesystem sync /mnt/btrfs

You did a scrub above and have no errors. So the data has been checked against checksums. That should be ok.

1 Like

I have no idea if I use qgroups. Never heard of it. Run these commands from a live enviroment?:

btrfs quota disable /mnt/btrfs
btrfs filesystem sync /mnt/btrfs

How would these commands look like in my perticular situation? Replace /mnt/btrfs with /dev/sda2?
Will try tomorrow. Thank you.

The drive /dev/sda2 must be mounted to /mnt/btrfs. Doesnt matter where…

Small update. Have used fsarchiver to clone my boot drive which took forever, hence the delay.

Today I will check qgroups and rerun btrfs check to see what comes up.
Btw I have read on multiple forums that clonezilla does support btrfs and some people succesfully backup their system while others experience more or less the same problem as me. Will give a ‘‘testing’’ version of clonezilla a shot too.

ok update.

    ~  sudo btrfs quota disable /mnt/17243299-2eab-4c05-9d27-76b0968b95e5
    ~  sudo btrfs filesystem sync /mnt/17243299-2eab-4c05-9d27-76b0968b95e5
    ~  sudo btrfs check /dev/sda2                                     1 ✘ 
Opening filesystem to check...
Checking filesystem on /dev/sda2
UUID: 17243299-2eab-4c05-9d27-76b0968b95e5
[1/7] checking root items
[2/7] checking extents
[3/7] checking free space cache
block group 237318963200 has wrong amount of free space, free space cache has 4096 block group has 1073737728
failed to load free space cache for block group 237318963200
[4/7] checking fs roots
[5/7] checking only csums items (without verifying data)
[6/7] checking root refs
[7/7] checking quota groups skipped (not enabled on this FS)
found 202846408704 bytes used, no error found
total csum bytes: 187061704
total tree bytes: 3123593216
total fs tree bytes: 2674573312
total extent tree bytes: 213942272
btree space waste bytes: 508425595
file data blocks allocated: 728536821760
 referenced 308825628672

Did a reboot and tried clonezilla again with same error as result. Then from live iso:

    ~  sudo btrfs quota enable /mnt/17243299-2eab-4c05-9d27-76b0968b95e5     1 ✘ 
    ~  sudo btrfs check /dev/sda2                                      1 ✘ 

Opening filesystem to check...
Checking filesystem on /dev/sda2
UUID: 17243299-2eab-4c05-9d27-76b0968b95e5
[1/7] checking root items
[2/7] checking extents
[3/7] checking free space cache
block group 237318963200 has wrong amount of free space, free space cache has 4096 block group has 1073737728
failed to load free space cache for block group 237318963200
[4/7] checking fs roots
[5/7] checking only csums items (without verifying data)
[6/7] checking root refs
[7/7] checking quota groups
check/qgroup-verify.c:546: account_all_refs: BUG_ON `ref->num_bytes != num_bytes` triggered, value 1
btrfs(+0x1a65d)[0x5562f180265d]
btrfs(qgroup_verify_all+0x5ad)[0x5562f18037ad]
btrfs(+0x49ffa)[0x5562f1831ffa]
btrfs(main+0x8e)[0x5562f17fe0be]
/usr/lib/libc.so.6(__libc_start_main+0xd5)[0x7f90c386bb25]
btrfs(_start+0x2e)[0x5562f17fe3ee]
zsh: abort      sudo btrfs check /dev/sda2

Should I do:

sudo btrfs check --repair /dev/sdYN 

I am a bit reluctant as you might understand. I have no idea if the fsarchiver backup works…

Thank you.

Another update.

Did the following at night because it takes a long time to complete:

WARNING:

	Full balance without filters requested. This operation is very
	intense and takes potentially very long. It is recommended to
	use the balance filters to narrow down the scope of balance.
	Use 'btrfs balance start --full-balance' option to skip this
	warning. The operation will start in 10 seconds.
	Use Ctrl-C to stop it.
10 9 8 7 6 5 4 3 2 1
Starting balance without any filters.
ERROR: error during balancing '/mnt/17243299-2eab-4c05-9d27-76b0968b95e5': Read-only file system
There may be more info in syslog - try dmesg | tail

After a reboot I got multiple errors regarding not being able to write certain files and configs.Another reboot and my system was gone. It goes straight to the cli with emergency mode or whatever. Then I tried:

   ~  sudo btrfs check --repair /dev/sda2                                       32 ✘ 
enabling repair mode
WARNING:

	Do not use --repair unless you are advised to do so by a developer
	or an experienced user, and then only after having accepted that no
	fsck can successfully repair all types of filesystem corruption. Eg.
	some software or hardware bugs can fatally damage a volume.
	The operation will start in 10 seconds.
	Use Ctrl-C to stop it.
10 9 8 7 6 5 4 3 2 1
Starting repair.
Opening filesystem to check...
Checking filesystem on /dev/sda2
UUID: 17243299-2eab-4c05-9d27-76b0968b95e5
[1/7] checking root items
Fixed 0 roots.
[2/7] checking extents
No device size related problem found
[3/7] checking free space cache
cache and super generation don't match, space cache will be invalidated
[4/7] checking fs roots
[5/7] checking only csums items (without verifying data)
[6/7] checking root refs
[7/7] checking quota groups
Counts for qgroup id: 0/256 are different
our:		referenced 9621086208 referenced compressed 9621086208
disk:		referenced 9621086208 referenced compressed 9621086208
our:		exclusive 7467597824 exclusive compressed 7467597824
disk:		exclusive 7519490048 exclusive compressed 7519490048
diff:		exclusive -51892224 exclusive compressed -51892224
Counts for qgroup id: 0/257 are different
our:		referenced 90903420928 referenced compressed 90903420928
disk:		referenced 90903420928 referenced compressed 90903420928
our:		exclusive 90657878016 exclusive compressed 90657878016
disk:		exclusive 90903420928 exclusive compressed 90903420928
diff:		exclusive -245542912 exclusive compressed -245542912
Counts for qgroup id: 0/1308 are different
our:		referenced 55400407040 referenced compressed 55400407040
disk:		referenced 55400407040 referenced compressed 55400407040
our:		exclusive 158294016 exclusive compressed 158294016
disk:		exclusive 158343168 exclusive compressed 158343168
diff:		exclusive -49152 exclusive compressed -49152
Counts for qgroup id: 0/1327 are different
our:		referenced 55405174784 referenced compressed 55405174784
disk:		referenced 55405174784 referenced compressed 55405174784
our:		exclusive 162963456 exclusive compressed 162963456
disk:		exclusive 163110912 exclusive compressed 163110912
diff:		exclusive -147456 exclusive compressed -147456
Counts for qgroup id: 0/1360 are different
our:		referenced 55326171136 referenced compressed 55326171136
disk:		referenced 55326171136 referenced compressed 55326171136
our:		exclusive 659464192 exclusive compressed 659464192
disk:		exclusive 659742720 exclusive compressed 659742720
diff:		exclusive -278528 exclusive compressed -278528
Counts for qgroup id: 0/1379 are different
our:		referenced 55596335104 referenced compressed 55596335104
disk:		referenced 55596335104 referenced compressed 55596335104
our:		exclusive 635240448 exclusive compressed 635240448
disk:		exclusive 635666432 exclusive compressed 635666432
diff:		exclusive -425984 exclusive compressed -425984
Counts for qgroup id: 0/1453 are different
our:		referenced 55885750272 referenced compressed 55885750272
disk:		referenced 55885750272 referenced compressed 55885750272
our:		exclusive 5589934080 exclusive compressed 5589934080
disk:		exclusive 5591076864 exclusive compressed 5591076864
diff:		exclusive -1142784 exclusive compressed -1142784
Counts for qgroup id: 0/1807 are different
our:		referenced 58500378624 referenced compressed 58500378624
disk:		referenced 58500378624 referenced compressed 58500378624
our:		exclusive 21319680 exclusive compressed 21319680
disk:		exclusive 21368832 exclusive compressed 21368832
diff:		exclusive -49152 exclusive compressed -49152
Counts for qgroup id: 0/1844 are different
our:		referenced 59422535680 referenced compressed 59422535680
disk:		referenced 59422535680 referenced compressed 59422535680
our:		exclusive 32641024 exclusive compressed 32641024
disk:		exclusive 32657408 exclusive compressed 32657408
diff:		exclusive -16384 exclusive compressed -16384
Counts for qgroup id: 0/1849 are different
our:		referenced 60628021248 referenced compressed 60628021248
disk:		referenced 60628021248 referenced compressed 60628021248
our:		exclusive 107646976 exclusive compressed 107646976
disk:		exclusive 107663360 exclusive compressed 107663360
diff:		exclusive -16384 exclusive compressed -16384
Counts for qgroup id: 0/1853 are different
our:		referenced 60784132096 referenced compressed 60784132096
disk:		referenced 60784132096 referenced compressed 60784132096
our:		exclusive 137179136 exclusive compressed 137179136
disk:		exclusive 137195520 exclusive compressed 137195520
diff:		exclusive -16384 exclusive compressed -16384
Repair qgroup 0/256
Repair qgroup 0/257
Repair qgroup 0/1308
Repair qgroup 0/1327
Repair qgroup 0/1360
Repair qgroup 0/1379
Repair qgroup 0/1453
Repair qgroup 0/1807
Repair qgroup 0/1844
Repair qgroup 0/1849
Repair qgroup 0/1853
Repair qgroup status item
found 202853998592 bytes used, no error found
total csum bytes: 187146536
total tree bytes: 3096428544
total fs tree bytes: 2673704960
total extent tree bytes: 203653120
btree space waste bytes: 504208859
file data blocks allocated: 723639857152
 referenced 309573853184

That didn’t work either. So the problem is my btrfs partition is in read only mode i guess. Now the question is how to change that back to rw. Fstab look normal with rw option in the line:

# /dev/sda2
UUID=17243299-2eab-4c05-9d27-76b0968b95e5	/         	btrfs     	rw,noatime,compress=lzo,ssd,space_cache,commit=120,subvol=@	0 0

# /dev/sda2
UUID=17243299-2eab-4c05-9d27-76b0968b95e5	/home     	btrfs     	rw,noatime,compress=lzo,ssd,space_cache,commit=120,subvol=@home	0 0

I’ve managed to backup my home folder so (most) configs are saved. any ideas?

Since quota was the problem, then there could be set a limit while repairing. And if a limit is set, no file can be written.

Hmm - never heard that before.

One thing to understand is that clonezilla is an advanced form of dd.

How am I going to put this?

Do not run clonezilla within the environment to be cloned - based on how btrfs are handling the filesystem - I suspect you will get weird errors - no particular in mind.

To clone an entire device controlled by btrfs - I suggest you to boot an independent system - then use that system to do the bitwise cloning of the device using by clonezilla.

I have no intimate knowledge of the brrfs filesystem but I suspect that when you try to image the filesystem from within a booted system you are going to get into some issues - and because of my lack of intimate knowledge I would do some kind of cold-backup for a filesystem like btrfs.

After I ran a balance which resulted in a error message about balancing and read only filesystem, now the filesystem is stuck in read only. A new scan still gives errors:

    ~  sudo btrfs check --repair /dev/sda2                              ✔ 
enabling repair mode
WARNING:

	Do not use --repair unless you are advised to do so by a developer
	or an experienced user, and then only after having accepted that no
	fsck can successfully repair all types of filesystem corruption. Eg.
	some software or hardware bugs can fatally damage a volume.
	The operation will start in 10 seconds.
	Use Ctrl-C to stop it.
10 9 8 7 6 5 4 3 2 1
Starting repair.
Opening filesystem to check...
Checking filesystem on /dev/sda2
UUID: 17243299-2eab-4c05-9d27-76b0968b95e5
[1/7] checking root items
Fixed 0 roots.
[2/7] checking extents
No device size related problem found
[3/7] checking free space cache
cache and super generation don't match, space cache will be invalidated
[4/7] checking fs roots
[5/7] checking only csums items (without verifying data)
[6/7] checking root refs
[7/7] checking quota groups
Counts for qgroup id: 0/1809 are different
our:		referenced 61069459456 referenced compressed 61069459456
disk:		referenced 61069459456 referenced compressed 61069459456
our:		exclusive 190246912 exclusive compressed 190246912
disk:		exclusive 108769280 exclusive compressed 108769280
diff:		exclusive 81477632 exclusive compressed 81477632
Repair qgroup 0/1809
Repair qgroup status item
found 202921086976 bytes used, no error found
total csum bytes: 187146520
total tree bytes: 3108896768
total fs tree bytes: 2685730816
total extent tree bytes: 204095488
btree space waste bytes: 506857030
file data blocks allocated: 727317250048
 referenced 312141459456
   ~  sudo btrfs rescue chunk-recover -v /dev/sda2                   1 ✘ 
All Devices:
	Device: id = 1, name = /dev/sda2

Scanning: 100533354496 in dev0cmds/rescue-chunk-recover.c:130: process_extent_buffer: BUG_ON `exist->nmirrors >= BTRFS_MAX_MIRRORS` triggered, value 1
btrfs(+0x4eb62)[0x55e9ac8edb62]
btrfs(+0x4f9d7)[0x55e9ac8ee9d7]
/usr/lib/libpthread.so.0(+0x9259)[0x7f3a83017259]
/usr/lib/libc.so.6(clone+0x43)[0x7f3a82f405e3]
zsh: abort      sudo btrfs rescue chunk-recover -v /dev/sda2
   ~  sudo btrfs rescue zero-log /dev/sda2             ABRT ✘  3m 11s  
Clearing log on /dev/sda2, previous log_root 0, level
   ~  sudo btrfs rescue super-recover -v /dev/sda2                                               ✔ 
All Devices:
	Device: id = 1, name = /dev/sda2

Before Recovering:
	[All good supers]:
		device name = /dev/sda2
		superblock bytenr = 65536

		device name = /dev/sda2
		superblock bytenr = 67108864

		device name = /dev/sda2
		superblock bytenr = 274877906944

	[All bad supers]:

All supers are valid, no need to recover
   ~  sudo btrfs rescue fix-device-size /dev/sda2                                              1 ✘ 
No device size related problem found
   ~  sudo mount -w /dev/sda2                                                                 32 ✘ 
    ~  sudo btrfs quota disable /mnt/17243299-2eab-4c05-9d27-76b0968b95e5                         ✔ 
ERROR: quota command failed: Read-only file system

My best guess is that there is still a non recoverable problem with the filesystem, some corruption, that triggers a read only behavior. Have read somewhere that this is normal in such case to prevent further corruption. Bottom line is that my system is most likely non recoverable (hence the warning message at check --repair) and al the hours I have spend solving this problem would be better spend if I did a clean install of 21.1 with inproved btrfs support. Some feedback is still appreciated btw.

p.s. fsarchiver was unsuccessful restoring the image backup…

No I am not trying to image the boot drive from within the booted system. That is clearly said in the first sentences of the opening post. :slight_smile:

I clearly misunderstood something - my apology.

As clonezilla is the topic - have you checked with clonezilla to see if it is a bug?

I just did and with a few keystrokes I got this ticket at clonezilla

It appears to be a problem with clonezilla and btrfs filesystem.

There is an easy way out (i did this already):

  1. Dont do anything rw with the broken btrfs volume

  2. Boot a recovery system (DVD or USB …)

  3. mount your defect btrfs volume as ro (this can be handled already as a snapshot)

  4. List all subvolumes accesible on your volume (take @ or whatever is the subvolume with your data on it. You can browse it ro !!!)

  5. mount a empty backup-btrfs volume rw

  6. transfer the complete btrfs-volume with btrfs send to the new volume.
    This way you won´t loose any data :wink: Best of all, the backup volume can be smaller or larger than the original volume. Only the data actually used has to fit on it

  7. check your backup !
    6.If you think you can fix it. You can try now. But if it fails, you can move on to the next step

  8. reformat the original btrfs volume again with btrfs

  9. mount the backup ro

  10. transfer the backup back with btrfs send

btrfs send only transmits the data that is used and is therefore faster than most other backup programs
Tips for the future:

  • If you are afraid that your data may have been corrupted: use btrfs scrub
  • Please use btrfs balance very sparingly. This is mostly unnecessary and only loads the system. If, btrfs balance, then use it with limits e.g. 50%
  • Take automatic snapshots (e.g. with snapper) every hour if possible. At btrfs it doesn’t cost anything. Snapper can also automatically remove these snapshots after some time.

You can find good information at btrfs at kernel-org and in the arch-wiki

Viel Erfolg :sunglasses:

1 Like

Cool feature. How would the command look like in my case? We are are not trying it from a snapshot right? Those are the only tutorials I could find.

Yess have read multiple posts on that forum. Yesterday I tried to delete old snapshots. No luck because read only…

Ok another update.

I’ve decided to perform a clean install of 21.1. There was no insurance that things will keep on functioning correctly if I would have tried to repair it (assuming that I would get it working again). After almost a year it was time for a fresh start. Did a precausionary full error test with gnome disks utility and a full format of the boot ssd. Things are running fine now and I am have made a backup with clonezilla succesfully after snapshotting with timeshift. Thanks everybody for the help :smiley: