Default btrfs mount options and subvolume layout

Manjaro Development Calamares
,
21

Yes I think is’nt the time to provide a BTRFS default setup for Manjaro, the needed toolchain is not ready yet.

  1. use of Snapper toolchain. We need then Suse patched GRUB and work like in Suse with nested Snapshots. Not the easiest solution to get it transparently explained to common users. The available Snapper-gui tool is even not comparable to the Timeshift GUI. Timeshifts GUI is more likely what common user know. With Snapper we chose a specific BTRFS setup, in my opinion not the best one. I use Snapper today, but the changes needed to setup a snapper configuration, removing his setup of nested snapshot subvolumes and installing new mountpoints, is a dirty trick to use snapper for a setup which is’nt designed for by his developer. And then we castrated snapper because we can’t use it for rollbacks. Not nice, but a way to go if the user know what he is doing.

  2. The Timeshift-way. A nicely GUI but limited to a specific BTRFS layout. But more badly, Timeshift create writeable snapshots and that contradicts the goal of snapshots. Hardcoded limitation to use only / and /home is even a serious limitation.

And there exists today no good toolchain with integrated snapshot and backup facility for BTRFS.
Only grub-btrfs, snap-pac works in a automatical setup system seamless and transparently for common users. Any toolchain setup, i know, need on some point hand made interaction from the user. For me it was’nt a hard way to learn BTRFS and use the CLI to get what i want, i’t not realy complicated i think. But that can’t be a scale for a linux distribution.

22

How about writing “a letter of frustration” to Snapper and Timeshift devs then?

23

LOL, why? In this thread we discus the objective possibility to get a btrfs default setup for manjaro installer and how should it be configured, the pro & cons. For my needs I go with snapper that’s a perfectly fit for me. I read already some issues, that covers above problems, from other peoples in snapper and timeshifts repository.

24

I mean as you consider Snapper and Timeshift’s defaults being not great, you could open an issue ticket in their repos, write them with rationale behind your thought on that and so on. I used “frustration” word only in order to make it sound funny.

25

Yes, i understand you. Have a look onto they issue list, there exists already and cover these questions.
But let us go more deeper. I think the need for a BTRFS tool that integrate the management (creation, synchronization, cleaning …) of snapshots with backups is growing. This tool should even have a nice and easy to understand GUI, to get accepted by most common users. The main points on developer view is to solve the problem with accessrights and how to translate the more complicated structure of BTRFS setups to a easily understanding GUI for common users. BTRFS, SSH and so on need most often root access, but the toolchain should be run in user space. These questions are even tried to answer from developer of snapper and timeshift and many others, each with his own answer. Now, we take a look on all them, try them and adopt it to our needs as today needed. I see this as one step of natural evolution of our toolchain.

26

There is no such limitation. Other layouts are unsupported, but work just fine. The only real requirement is subvolumes @ and @home, you can have others besides those.

I like this approach. Unfortunately, the existing gui tools don’t do it automatically. We would probably have to patch snapper gui to do something like this… I want to make btrfs accessible to new users, so gui tools are needed. Timeshift doesn’t handle this cleanly either.

Calamares can now handle swapfiles with btrfs automatically per user setting. It uses the @swap subvolume.

27

Thanks for the info, it’s long time ago as i have tested Timeshift, then one point of my above statement is irrelevant.

Another way i make rollbacks

sudo mv /btrfs/@ /btrfs/@snapshots/old_root
sudo btrfs su sn /btrfs/@snapshots/root/15/snapshot /btrfs/@

where /btrfs/@snpashots/root/15/snapshot is the currently booted ro-snapshot. This save the damaged subvolume, now as snapshot in @snapshots. You can even exchange @ from a running system (started from @), and reboot or remount into the new setup. Last case i tried self and it seemed to work properly, but would not trust it eighter :slight_smile:
The nice thing with this way is that any setup/config of FSTAB, GRUB, initramfs and kernels that’s stored into a snapshots remains correctly after rollback of @. You need only to know that in FSTAB we have to work only with subvol=@ mounts, not subvolid=xxx mounts. After a genfstab >> /etc/fstab we have to edit these mounts. There is only one point to consider: any later changes on partition, UUIDs, mountpoints and so on are forbidden, because we break the system configuration stored in our snapshots.

A small script started on boot from a ro-snapshot, selected in GRUB, could do this Rollback automatical and reboot the machine. Like we can when GRUB start the UEFI Shell.
This script maybe ask the User if he wish to start the CLI or Desktop Environment or want to rollback the system and reboot. I think thats a small patch on grub-btrfs script.

But you’re right, the problem are the toolchain, in any case some small changes are needed to get a system which can be presented to common users, with small knowledge of BTRFS. But, in longer timescale the user have to learn BTRFS. I remember my startup problems with BTRFS, the biggest one at beginning was: “how fu…ing should i setup my BTRFS layout ?”, “Have i to exclude /var or only partial like older suse setup with many subvolumes for /var ?” And on these question Manjaro Installer can give advise.

“Calamares can now handle swapfiles with btrfs automatically per user setting. It uses the @swap subvolume”,

good to know, i use architect or easily clone my systems with rsync. Last time i tried calamares i must overseen this feature, or is it new?

1 Like
28

Snapper is designed for SUSE, and the Timeshift developer can’t spend too much time on developing Timeshift.

What about having an in-house built snapshot manager?

I am currently designing one that

  • has simpler FS layout than Snapper
  • is more powerful than Timeshift
  • works with much less configuration as the above 2, and
  • gives less possibility to config errors (by finding out automatically as much setup details as possible)
  • can do rollbacks in a clean way, without messing up the entire FS like Snapper and Timeshift do
  • designed to work with subvolume name, not ID
  • can work with both read-only and read-write snapshots, based on user preference
  • doesn’t need the ext4 GRUB partition hack
  • supports custom tags for managing snapshots
  • compatible with Snapper/Timeshift GUI

I know developing a new GUI with a new tool is a problem, so it would be CLI argument compatible with Snapper or Timeshift (which GUI is better) for use with GUI, but it would provide an easier CLI interface when invoked from CLI. This could be possible with a wrapper script, or symlinking this tool to Timeshift, and finding out ARGV[0] (like Firejail does).

29

Sounds good to me, though I don’t have enough time to spend on it myself. I would maybe patch snapper-gui to alter the restoration method, but writing something from scratch gives you more flexibility.

If you code one and it is good, I can package it. What language are you using? With python/bash I can probably contribute, with compiled languages not so much. I’m also much more familiar with gtk than qt.

30

What did you mean with hack?
And where can i take a look on your tool?

31

I tried snapper-gui, and as I found out it doesn’t provide restore functionality. (Or it is hidden so well that the average user wouldn’t find it.)

I tried Timeshift GUI too, and in my opinion snapper GUI is more user-friendly.

But I guess it would be better to have the restore functionality accessible outside of these tools.
@Chrysostomus Could you develop a tool that checks, if the default subvolume or any other subvolume was booted, and if it’s not the default, it would display a systray icon with a toast? (with the text “To permanently restore your system to the currently running version, click here”) The system tray icon should remain accessible all the time when a non-default subvol is running.

32

Yes

No. I don’t have experience in doing systrays, and not all desktops have system trays anyway.

33

This is nearly the same layout that i do use nowadays. I had a few accidents with manjaro. So i had to rollback manually. I had to repair manjaro from LIVE-CD. I learned that a layout that is not flat gives a lot of trouble when you have to rollback permanently.
First it seams easy. but later i found that my snapshots where now taken in the “wrong” place. And even later i found that the old snapshots where not removed any more. To correct this i had to do a lot of reading on btrfs, cli-snapshoting(subvolumes), moving and deleting.

I wont suggest anything but a flat :point_up: btrfs-layout.

@ /
@home /home
@snapshots /.snapshots

additionally i do have

@home.snapshots /home/.snapshots
@nosnap /var/cache, /var/spool, /var/nosnap

At /var/nosnap are the virtual machines stored.

 mount |egrep btrfs                                                                                                                            [32]
/dev/sda2 on / type btrfs (rw,noatime,compress=zstd:3,ssd,space_cache,commit=300,subvolid=5959,subvol=/@)
/dev/sda2 on /.snapshots type btrfs (rw,relatime,compress=zstd:3,ssd,space_cache,commit=300,subvolid=3648,subvol=/@snapshots)
/dev/sda2 on /home type btrfs (rw,noatime,compress=zstd:3,ssd,space_cache,commit=300,subvolid=258,subvol=/@home)
/dev/sda2 on /var/nosnap type btrfs (rw,relatime,compress=zstd:3,ssd,space_cache,commit=300,subvolid=4455,subvol=/@nosnap)
/dev/sda2 on /var/spool type btrfs (rw,relatime,compress=zstd:3,ssd,space_cache,commit=300,subvolid=4455,subvol=/@nosnap)
/dev/sda2 on /var/cache type btrfs (rw,relatime,compress=zstd:3,ssd,space_cache,commit=300,subvolid=4455,subvol=/@nosnap)
/dev/sda2 on /home/.snapshots type btrfs (rw,relatime,compress=zstd:3,ssd,space_cache,commit=300,subvolid=5719,subvol=/@home.snapshots)
/dev/sda2 on /mnt/BTRFS_ROOT type btrfs (rw,relatime,compress=zstd:3,ssd,space_cache,commit=300,subvolid=5,subvol=/)

/mnt/BTRFS_ROOT is only used if i have to work with the btrfs filesystem itself. It is normally not mounted.

1 Like
34

How you got this ? manual partitioning while installing ? Can you make short guide what to change there and how ?

thx

35

I did this always after installing.
In my main-System i did this in the running system a few days after i had to rollback. When rolling back with a non-flat layout i found it to much work. So i changed my system afterwards.
In an new setup system on permanent USB-Stick i even installed first with ext4, then converted the filesystem to btrfs, and then setup a flat layout in btrfs. persistent manjaro usb with btrfs
On both machines i do automatic snapshots with snapper.

When changing the layout you have to watch grub.cfg and fstab. Yo do need both working to be able to rebbot. For safety you should have a live-CD. But i didn´t need it. You have to watch snapper, because snapper will not create new configs, when the subvolume/ dir /.snapshots or /home/.snapshots is already present. and it may be best to stop snapper while changing the layout :slight_smile:

I can´t create an real step by step howto, because i have no system to test the steps. My actual system is already converted. I am afraid that i do miss something an you get stuck in the middle of it. Maybe you could try this with an USB-Stick before you mess up your actual system. But when i have to do it the next time i will create a step-by-step you can follow. :thinking:
Tipp:
There is some safety in btrfs using subvolumes/snapshots. It is easy to make a readonly-snapshot before you start. When you mess something up, you are able to create a new writable snapshot of it, move it to /@ instead of the former /@ and define this snapshot as default. And you are saved.

cat /etc/fstab                                                 
PARTUUID=3ee1dfe1-19af-4102-945d-90d957d3c199	/         	btrfs     	rw,noatime,compress=zstd,commit=300,subvol=@	0 0
PARTUUID=3ee1dfe1-19af-4102-945d-90d957d3c199	/home     	btrfs     	rw,noatime,compress=zstd,commit=300,subvol=@home	0 0
PARTUUID=b1d3d562-88ff-4ac2-8326-9c5d82892379	/boot/efi 	vfat      	rw,relatime,fmask=0022,dmask=0022,codepage=437,iocharset=iso8859-1,shortname=mixed,utf8,errors=remount-ro	0 0
PARTUUID=3ee1dfe1-19af-4102-945d-90d957d3c199	/.snapshots	btrfs     	rw,relatime,compress=zstd,commit=300,subvol=@snapshots	0 0
PARTUUID=3ee1dfe1-19af-4102-945d-90d957d3c199	/home/.snapshots	btrfs     	rw,relatime,compress=zstd,commit=300,subvol=@home.snapshots	0 0
PARTUUID=3ee1dfe1-19af-4102-945d-90d957d3c199	/var/nosnap	btrfs   	rw,relatime,compress=zstd,commit=300,subvol=@nosnap	0 0
/var/nosnap/var/cache				/var/cache	none bind	0 0
/var/nosnap/var/spool				/var/spool	none bind	0 0
1 Like
36

Please keep “servers” in mind, some folks do use Manjaro.

37

On a server i would keep /, /home and /srv in separate btrsf-filesystems (when they can grow uncontrollably), and not just a subvolume. Maybe even on separate drives. Because:
When /srv (or /home) runs out of space it may strangle your system
If /srv will only grow in a controllable fashion (or you have lots of free space) /srv is a very good option for a subvolume. Maybe with:
@srv
@srv.snapshots

38

This topic dates from 2020.
I installed Manjaro twice from scratch, first time I selected wipe disk - btrfs - swapfile: no subvolume is created. Swapfile is on / (note this could break Timeshift, according to Manjaro wiki!).

Second time I selected “swap with hibernation”, again, no subvolume is created and there is no hibernation support. Worse, a swap partition is created, even though this is not recommended for SSDs and for BTRFS.

I know how to create a swapfile properly for BTRFS and to have it support hibernation (see this topic), but I am suprised Manjaro/Calamares does not take care of this by itself, especially after noticing this topic from last year?

It seems a lot of effort has been spend already.

39

When was this? Did you use automatic mode or manual partitioning? I haven’t tested the manual mode in a while, but with automatic partitioning it should definitely create a swapfile on a separate subvolume. If it doesn’t it’s a bug and needs to be fixed asap.

40

This was just last week. I did the same test a couple of times: 100% certain no swap subvolume is created. It is either a file on / or a partition.

I simply select wipe disk, select swap type and FS type.
I have not used manual partitioning.

For now, I simply select “no swap” and “BTRFS”.
After installation, I run the script (that I updated) from this topic:

This will calculate the amount of swap needed for hibernation, create a subvolume and create the swapfile, turn on swap and perform necessary tasks for hibernation to work.

1 Like