Btrfs Snapper, the Suse way with rollback

Hello,
I’ve tried to install Arch or Manjaro with or without the Architect Installer, which supports Snapper rollback and bootable snapshots though the default subvolume like Suse worked out. I wanted to provide this partition sheme also for unexpirienced users so i modified Manjaro Architect so everyone can easily install. I created a PKGBUILD for Grub with the patches from Suse, also I habe modified snap-pac-grub so the snapshots occur on grub because i dont use grub-btrfs because with the Suse patches in Grub there is a much simpler option. I’ll try to bring the PKGUILD in the Aur but I need help or time, so atm you have to download them from my Google Drive. I’ve only tried this with Uefi but maybe it would work with legacy too.

So here the short installation instructions
At first you have to create the subvolumes in your at /mnt mounted btrfs partition
“$(root_partition)” you have to replace.

This sheme is copied from Suse but you can change it if you keep the snapshot subvolume which you will obiosly need.

cd /mnt
btrfs subvolume create @
btrfs subvolume create @/.snapshots
mkdir ./@/.snapshots/0
btrfs subvolume create @/.snapshots/0/snapshot
btrfs subvolume create @/home
btrfs subvolume create @/opt
btrfs subvolume create @/root
btrfs subvolume create @/srv
btrfs subvolume create @/tmp
mkdir @/usr/
btrfs subvolume create @/usr/local
btrfs subvolume create @/var
chattr +C @/var
btrfs subvolume set-default $(btrfs subvolume list /mnt | grep "@/.snapshots/0/snapshot" | grep -oP '(?<=ID )[0-9]+') /mnt

cd /
umount /mnt
cd /mnt

# Mount root and create final mount-points 
mount "$(root_partition)" /mnt
mkdir /mnt/.snapshots
mkdir /mnt/home
mkdir /mnt/opt
mkdir /mnt/root
mkdir /mnt/srv
mkdir /mnt/tmp
mkdir -p /mnt/usr/local
mkdir /mnt/var

# Mount them in the right directorys
mount -o subvol=@/.snapshots "$(root_partition)" /mnt/.snapshots 
mount -o subvol=@/home "$(root_partition)" /mnt/home 
mount -o subvol=@/opt "$(root_partition)" /mnt/opt
mount -o subvol=@/root "$(root_partition)" /mnt/root 
mount -o subvol=@/srv "$(root_partition)" /mnt/srv
mount -o subvol=@/tmp "$(root_partition)" /mnt/tmp 
mount -o subvol=@/usr/local "$(root_partition)" /mnt/usr/local
mount -o subvol=@/var "$(root_partition)" /mnt/var 
'''
Make your installation yet without grub
from now on all commands have to be exicuded in chroot
'''
cd grub-suse-git
makepkg -sfCi # some there were problems with locales exspecily with the fish shell but a restart of the shell should help
sed -i '/SUSE_BTRFS_SNAPSHOT_BOOTING/s/^#//g' /etc/default/grub     # to enable bootable snapshots
grub-install
grub-mkconfig -o /boot/grub/grub.cfg
sed -i s/\#IgnorePkg   =/IgnorePkg = grub-suse-git/  /etc/pacman.conf # so it wont be overidden at update but i optimized the PKGBUILD so i dont know if this needed

Yet you can install and configure snapper:

pacman -S --noconfirm snapper
cp /etc/snapper/config-templates/default /etc/snapper/configs/root # Kopierung der Snapper konfiguration da dieser wenn schon das @/.snapshot Subvolume existiert keine erstellen kann.
sed -i "s/\=\"\"/\=\"root\"/"  /etc/conf.d/snapper # Aktivierung der Konfiguration
systemctl enable cronie
sed -i "s/done/done\ngrub-snapper-plugin -r/" /etc/cron.hourly/snapper # so the timeline snapshots occur in Grub

To enable snap-pac so that there pre and post snapshots:

pacman -S --noconfirm snap-pac # Pacman Hook which creates snapshots before and after transaction
cd snap-pac-grub-suse
makepkg -sfCi

To add and enable snapper-gui although i found it relativly useless:

pacman -S --noconfirm snapper-gui
sed -i s/ALLOW_GROUPS=\"\"/ALLOW_GROUPS=\"wheel\"/ /etc/snapper/configs/root # so snapper list works for every user which can call sudo

I hope this is enough to get it working without problems and I would be really happy about feedback.
Also my english is very bad also in school so sry for that I hope everyone understands everything.

here is a link to the PKGBUILDS but obviosly the https is missing because otherwise i couln’t provide it ://drive.google.com/drive/folders/1C-he8n8XPUpTrljA05eFRjEfk_KprGor?usp=sharing

Hi, i use snapper, snapper-gui, grub-btrfs, snap-pac and snap-sync.

some suggestions:

btrfs subvolume create @
btrfs subvolume create @/.snapshots
mkdir ./@/.snapshots/0
btrfs subvolume create @/var
chattr +C @/var
  1. your snapshot subvolume “@/.snapshots” for your root on @ is a child subvolume of @ self. Consider you have after some time many snapshots taken, all as child-subvolume into your root subvolme as parent at @. Now you want to make a rollback then you rollback @ to a new snapshot but not they included snapshot subvolumes from @/.snapshots. To restore even these snapshots you have these to move to the new @ subvolume.
  2. your @var subvolume with +C attribute set is the right way but not the full correct way. You forgot to exclude /var/lib. /var/lib must/should be contained in every snapshots that’s taken from @ your root. because into /var/lib the libraries and especially /var/lib/pacman should be snapshotted with the content of the snapshotted root system at @. On any later rollback, with included /var/lib content, we restore the full functional root-system. To get this configuration working i personally copy /var/lib to /usr/var/lib and in fstab i made a bind-mount from /usr/var/lib to /var/lib. In this configuration we exclude anything from @var → /var but include /var/lib = /usr/var/lib into a snapshot of @. We don’t break the dependencies this way.
  3. you need a patched version of Grub2. But with another, more easier, setup you don’t need that anyway.

Ok, my setup

ID 1372 gen 65296 top level 5 path @
ID 669 gen 65303 top level 5 path @home
ID 259 gen 65301 top level 5 path @var
ID 327 gen 43489 top level 5 path @snapshots
ID 504 gen 64264 top level 327 path @snapshots/root
ID 505 gen 65208 top level 327 path @snapshots/home

ID 1147 gen 34449 top level 504 path @snapshots/root/1/snapshot
ID 1148 gen 34450 top level 505 path @snapshots/home/1/snapshot
ID 1149 gen 34487 top level 505 path @snapshots/home/2/snapshot
ID 1211 gen 38203 top level 505 path @snapshots/home/48/snapshot
...

On top-level i create subvolumes @ → ‘/’, @home → ‘/home’, @var → ‘/var’ and @snapshots → no mount point. Then as childs of @snapshots i create subvolumes @snapshots/root → ‘/.snapshots’ and @snapshots/home → ‘/home/.snapshots’.

  1. i use a flat btrfs layout for the system @, @home, @var, @snapshots subvolumes. All have BTRFS-Root as parent, means parent = Top Level 5, as above shown. Snapshots for each configuration of snapper use a hiarchical btrfs layout and child-subvolumes of @snapshots.

  2. if we now rollback manually without snapper then we boot into a ro-snapshot and use

mv /btrfs/@ /btrfs/@snapshots/root/old_root
btrfs subvolume snapshot /btrfs/@snapshots/root/XXX/snapshot @

and reboot. Or we use

btrfs subvolume delete /btrfs/@
btrfs subvolume snapshot /btrfs/@snapshots/root/xxx/snapshot @

First case save the current non working @ subvolume to …/old_root or any other location, sencond case easily delete @ subvolume.

We working always without any default subvolume set, we ignore this feature fully no need for it. We don’t change anyting in GRUB or have to regenerate his configuration, because we exchange easily the @ subvolume on any rollback. That’s really more easier, because we do not change anything what is prior setup for GRUB to boot right.

I use snapper to take and manage my snapshots of root and home. I don’t use it to rollback and i don’t use his default setup of his snapshot layout. In my above configuration any snapshot take is’nt contained into the subvolumes from these we make snapshots. Thus we do n ot get any problems with rollbacks and the child-subvolumes which contains the own snapshots (on rollback these must be moved to the recorvered @,@home subvolume).

Now my fstab:

# Swap
UUID=06686a06-c069-49f7-86e4-7a962740b364       none                    swap            defaults                                                                        0 0

# UEFI
UUID=447C-E2BC                                  /boot/efi               vfat            noatime,codepage=437,iocharset=iso8859-1,shortname=mixed,utf8                   0 2

# System
UUID=26c8751d-2747-4a4d-b857-32c82d67b20a       /btrfs                  btrfs           noatime,ssd,compress-force=zstd:5,subvolid=5                                    0 0
UUID=26c8751d-2747-4a4d-b857-32c82d67b20a       /                       btrfs           noatime,ssd,compress-force=zstd:5,subvol=@                                      0 0
UUID=26c8751d-2747-4a4d-b857-32c82d67b20a       /home                   btrfs           noatime,ssd,compress-force=zstd:5,subvol=@home                                  0 0
UUID=26c8751d-2747-4a4d-b857-32c82d67b20a       /var                    btrfs           noatime,ssd,nodatacow,subvol=@var                                               0 0
UUID=26c8751d-2747-4a4d-b857-32c82d67b20a       /.snapshots             btrfs           noatime,ssd,compress-force=zstd:5,subvol=@snapshots/root                        0 0
UUID=26c8751d-2747-4a4d-b857-32c82d67b20a       /home/.snapshots        btrfs           noatime,ssd,compress-force=zstd:5,subvol=@snapshots/home                        0 0

# var/lib mount into subvol=@
/usr/var/lib                                    /var/lib                none            defaults,bind                                                                   0 0

# Backup
UUID=d49e1730-5137-473c-8e28-a76cf14e9830       /media/Backups          btrfs           nofail,noatime,ssd,compress-force=zstd:5,subvol=@backups                        0 0

As you see, i mount in /btrfs the top-level of BTRFS filesystem. If i work with any btrfs command i most use this /btrfs path to ensure the right place of as example a newly create subvolume.

ls -la /btrfs               
drwxr-xr-x 1 root root 302 24. Jan 18:50 @
drwxr-xr-x 1 root root  28 10. Jan 00:38 @home
drwxr-xr-x 1 root root  16 24. Jan 18:49 @snapshots
drwxr-xr-x 1 root root 126  5. Feb 18:54 @var

You see the flat btrfs layout. Only @snapshots contains subfolder/subvolumes @snapshots/root and @snapshots/home. The name is equal to each snapper configuration.

Then in my fstab you see the bind-mount of /usr/var/lib to /var/lib to maintain the correct dependencies of our root system and thery installed libraries on time we take a snapshot.
By the way: with this configuration you can boot from grub into ro-snapshots into running DE/GUI.

With snap-pac we get automatical snapshots of @,@home everytime before/after installing packages or AUR. With grub-btrfs we autom. update the grub menuentries to boot into ro-snapshots taken. With snap-sync we make daily backups to external btrfs formatted backup drive → above /media/Backups.

You don’t need using Suses-patched-GRUB, no need for setting btrfs default subvolume and don’t use snapper rollbacks, but you could do it with snapper if you want. I personaly made rollback by hand because there only 2 lines in CLI.

And: as my fstab above shows: we have to mount any UEFI partition into /boot/efi folder, not /boot as sometimes suggested into WEB (that’s worng). When we mount UEFI into /boot/efi anything contained into /boot/efi folder remains to our FAT32 UEFI Partition, obvousily. More important anything into /boot, especialy our kernel files remains to our btrfs subvolume @ our system root and snapshoted with it. If our pamac later deinstall some older kernel files eg. initramfs any older snapshot of @ contains his older, now deinstalled EOL kernels. grub-btfs is a clever piece of script which scans the taken older snapshots on every recreation of GRUBs menuentires of our snapshots. Means we never break our old snapshots and his contained kernel files, and we can even later boot these old snapshots readonly from GRUB with a kernel that’s old and on actual system long time deleted.

1 Like

There exists two traps of course:

  1. setup of btrfs layout when using manjaro-architect or manjaro installer.
  2. installation and setup of snapper

First we install with architect/installer our system, btrfs formatted with @,@home,@var. Nothing more. Then we have to change:

  1. @var → /var to set +C attribute to /var folder and any subfolder and files contained.
  2. we have with btrfs renewing this folder to ensure thaht the +C flag is used for any file into /var
  3. we have to create a folder /usr/var/lib and copy /var/lib into it
  4. create a folder /btrfs
  5. edit fstab and include mountpoint /btrfs with subvolid=5 and bind-mount from /usr/var/lib to /var/lib
  6. then we reboot
  7. we go into /btrfs/@var/lib end delete the old content we have copied to /usr/var/lib

Now btrfs is ready to install snapper, snap-pac,grub-btrfs and snap-sync if needed.
We use

sudo snapper -c root create-config / 
sudo snapper -c home create-config /home

to create configs for snapper. Snapper creates now his own layout of snapshots we have to change.
Delete his created subvolumes and his created folders.
Now

  1. create subvolume /btrfs/@snapshots
  2. create subvolume /btrfs/@snapshots/root
  3. create subvolume /btrfs/@snapshots/home
  4. mkdir /.snapshots
  5. mkdir /home/.snapshots
  6. edit fastab with above two mountpoints for @snapshots/root → /.snapshots and @snapshots/home → /home/.snapshots

reboot and after setup snapper configurations for root and home such as you needed. Make now yopur first snapshots of root and home → sudo snapper -c root create …

Now setup grub-btrfs, start his service grub-btrfs.path, setup snap-sync if needed.
Finish.

As you see above, any snapshot taken of @ or @home is stored outside of these subvolumes, not as child-subvolumes inside these subvols @/@home. On rollback we have no not to consider to move these snapshots to the right places, as with suses/snappers setup.

1 Like

it would be great if we could tell snapper on creation of any config how we want to created his subvolumes and how we want to store his snapshots. Then the complete setup where more easier.

In fact, snapper rollback is still not available by default, even if I follow the subvolume layout you provide.

Cannot detect ambit since default subvolume is unknown.
This can happen if the system was not set up for rollback.
The ambit can be specified manually using the --ambit option.

It can be used after executing the snapper rollback --ambit classic command.
However, it brings another problem. The snapshot rollback -- ambit classic command seems to set the read-write copy of the snapshot as the default subvolume, but the kernel boot parameter in grub seems to add the rootflags=subvol=@ parameter by default, which seems to prevent rollback from the snapshot.
I don’t know where to delete the rootflags=subvol=@ in the kernel boot parameters. :tired_face:

You are able to

  • move away “@”
  • move your new subvolume to “@”
  • set this subvolume as default (optional)

Then grub has no problems :wink:

https://forum.manjaro.org/t/howto-manual-rollback-with-btrfs/80230/2

But I don’t want to roll back manually. :tired_face:

To get Snapper rollback working you need three things:

  1. his nested subvolume layout
  2. using BTRFS default subvolume
  3. SUSE patched GRUB to boot this default subvolume

Last point is the hardest one, you have to install SUSE patched GRUB on a Manjaro system.

It is only 3 Steps (commands) :wink:
I also dont’t like manual rollback. But i found no good other solution for my needs.

  • Timeline only makes system snapshots (but i want /home also automatic. And making writeable snapshots for me is a NO go).
  • Snapper can make snapshots as you like, makes them readonly, and removes them automatically (but has a weird btrfs layout that is nested. Rollback is messy.)
  • So i do use snapper, make my own flat btrfs layout, and do easy manual rollbacks (once in a time)

ok, to clarify a litte bit more.

With above setup we don’t need the deafult subvolume feature of BTRFS when we use the right fstab setup. Thus a manual rollback are in most cases 2 lines into console.

sudo btrfs su de /@
sudo btrfs su sn /.snapshots/???/snapshot /@

or maybe saving it

sudo mv /@ /.snapshots/???/destroyed
sudo btrfs su sn /.snapshots/???/snapshot /@

u can do it directly in the running system, reboot or remount @.
In view of Grub and fstab our named @ subvolume is alyways our “default” subvolume. U need only in fstab to mount subvol=@ to /. Dont use subvolid=xxx mounts.

If it is a compromise, I know that there is a rollback-git tool to use.