RPi4 with Manjaro on a SSD with btrfs

ARM General ARM Discussion
1

Well, I finally took the time to learn a little about btrfs. I have used it in a very basic fashion in the past, make file system, copy some files or run dd. I even took a basic btrfs file system and transferred my RPi4 image to it, it worked… sort of. I had a few odd errors and so that lasted about a week before going back to f2fs.

There is quite a bit of information on btrfs as it has been around for quite a few years now and has generated its’ fair share of web content. Some good, some bad, some old and outdated.

This time I decided to go “all-in”, to not just make a file system but to setup a rootfs with subvolumes and snaphots. With snaphots being the most compelling feature to me.

I booted on a SD which was updated against arm-unstable. Then I mounted two USB SSDs. One was the SSD of my daily driver image, also on arm-unstable, and the second being a new 60GB SSD.

On the new SSD, I created /boot, swap, and btrfs partitions with the following subvolumes:

$ sudo btrfs subvolume list /

ID 257 gen 316 top level 5 path @
ID 258 gen 317 top level 5 path @home
ID 259 gen 311 top level 5 path @snapshots
ID 260 gen 20 top level 5 path @srv
ID 261 gen 294 top level 5 path @tmp
ID 262 gen 257 top level 5 path @var_cache_pacman
ID 263 gen 314 top level 5 path @var_log

These are the “directories” which I do not want included in the rootfs snapshots. They are mounted like this:

$ cat /etc/fstab

UUID=92DA-BDF9                                  /boot                   vfat    defaults,nofail,noauto,x-systemd.device-timeout=10s                     0       0
UUID=001e31e7-d595-4fc0-9029-d0bbf9f582f2       none                    swap    defaults,nofail                                                         0       0
UUID=2b382e06-d18c-471c-81cf-493b76bca157       /                       btrfs   defaults,ssd,subvol=@                                                   0       0
UUID=2b382e06-d18c-471c-81cf-493b76bca157       /home                   btrfs   defaults,ssd,subvol=@home                                               0       0
UUID=2b382e06-d18c-471c-81cf-493b76bca157       /srv                    btrfs   defaults,ssd,subvol=@srv                                                0       0
UUID=2b382e06-d18c-471c-81cf-493b76bca157       /.snapshots             btrfs   defaults,ssd,subvol=@snapshots                                          0       0
UUID=2b382e06-d18c-471c-81cf-493b76bca157       /tmp                    btrfs   defaults,ssd,subvol=@tmp                                                0       0
UUID=2b382e06-d18c-471c-81cf-493b76bca157       /var/log                btrfs   defaults,ssd,subvol=@var_log                                            0       0
UUID=2b382e06-d18c-471c-81cf-493b76bca157       /var/cache/pacman       btrfs   defaults,ssd,subvol=@var_cache_pacman                                   0       0

Once mounted, I copied all the files from boot and root partitions from my daily driver to the btrfs file system. Modified the /etc/fstab as above and rebooted.

Finally, I installed snapper and configured it for the rootfs:

$ snapper -c rootfs list

 # | Type   | Pre # | Date                            | User | Cleanup  | Description | Userdata
---+--------+-------+---------------------------------+------+----------+-------------+---------
0  | single |       |                                 | root |          | current     |         
1  | single |       | Sun 17 Jan 2021 02:17:24 PM CST | root |          | Initial     |         
2  | single |       | Sun 17 Jan 2021 03:00:04 PM CST | root | timeline | timeline    |         
3  | single |       | Sun 17 Jan 2021 04:00:14 PM CST | root | timeline | timeline    |         
4  | single |       | Sun 17 Jan 2021 05:00:44 PM CST | root | timeline | timeline

Only time will tell if I end up regretting this. No doubt I will refine this more as I become more comfortable with btrfs… or back to f2fs.

Edit: To be able to reliably roll-back, I need the corresponding /boot files. So I copy the files from /dev/sda1 to /boot. They are then included in the snapshot and can be used to recreate the /boot partition.

2

I experimented with Btrfs on Raspberry in the past. I took a challenge from a friend that Rasbian Buster + python apps will fit on 512 MB SD card. And it was a success, thanks to Btrfs compression.
That said, Btrfs on Raspberry should work just fine - it’s Linux like any other, but compiled for ARM.

3

Create a folder /boot/efi, mount your UEFI (UUID=92DA-BDF9)on /boot/efi and install grub EFI with --efi-directory=/boot/efi. Then your /boot folder remains to your BTRFS subvolume @ and the kernels installed are contained into your snapshots.

Follow are my configuration:

UUID=650E-2DAD /boot/efi vfat rw,relatime,fmask=0022,dmask=0022,codepage=437,iocharset=iso8859-1,shortname=mixed,utf8,errors=remount-ro 0 2
UUID=7969b227-dc81-4540-979e-cf037bc4cc8e / btrfs rw,noatime,compress=zstd:5,ssd,space_cache,subvol=@ 0 0
UUID=7969b227-dc81-4540-979e-cf037bc4cc8e /btrfs btrfs rw,noatime,compress=zstd:5,ssd,space_cache,subvolid=5 0 0
UUID=7969b227-dc81-4540-979e-cf037bc4cc8e /home btrfs rw,noatime,compress=zstd:5,ssd,space_cache,subvol=@home 0 0
UUID=7969b227-dc81-4540-979e-cf037bc4cc8e /var btrfs rw,noatime,nodatacow,ssd,space_cache,subvol=@var 0 0
UUID=7969b227-dc81-4540-979e-cf037bc4cc8e /.snapshots btrfs rw,noatime,compress=zstd:5,ssd,space_cache,subvol=@snapshots/root 0 0
UUID=7969b227-dc81-4540-979e-cf037bc4cc8e /home/.snapshots btrfs rw,noatime,compress=zstd:5,ssd,space_cache,subvol=@snapshots/home 0 0

As you can see i mount subvolid=5 to /btrfs for easier access to BTRFS subvolumes. Then i mount the complete /var as subvolume, where /var have set +C attribute set, before i install anything. So you can boot readonly snapshots into GUI. I even use snapper but create two subvolumens @snapshots/root mountpoint /.snapshots and @snapshots/home mountpoint /home/.snapshots. This make things cleaner i think. Nowadays i use compress-force=zstd:5 instead of only compress=zstd:5 (M.2 SSD).

4

I look forward to being able to use grub, so far for a RPi4, it is not the best way to go. OpenSuse has grub working but they lose much, like sound and wireless. Once the RPi4 has UEFI completed, then grub will be more of an option. So in theory I can do as you suggest, but I would rather have a fully working RPi4 than be able to use grub to boot.

Compression is something I want to incorporate and may try adding that later today. I would have already added it, but to be honest, I did not think I was going to be successful on my first try at this. :slight_smile:

5

I have added compression to the mix by adding compress-force=zstd to the mount options for each of the btrfs subvolume mounts. Which results in the following actual mount options:

(rw,relatime,compress-force=zstd:3,ssd,space_cache,subvolid=257,subvol=/@)
6

The snapper.timer is cool, and no doubt very useful for other situations. However, how I setup the rootfs and subvolumes, it is not exactly what I am after. So I disabled the snapper.timer service and deleted all of the snapshots it made. I then installed the snap-pac package, it makes snapshots before and after pacman updates.

I also installed rsync and incorporated a slightly modified version of the pacman hook for the backup of /boot on kernel updates. Below are the snapshots from that installation:

$ sudo snapper -c rootfs list

  # | Type   | Pre # | Date                            | User | Cleanup | Description     | Userdata
----+--------+-------+---------------------------------+------+---------+-----------------+---------
 0  | single |       |                                 | root |         | current         |         
18  | pre    |       | Mon 18 Jan 2021 02:19:40 PM CST | root | number  | pacman -S rsync |         
19  | post   |    18 | Mon 18 Jan 2021 02:19:41 PM CST | root | number  | rsync xxhash    |

This is a pretty nice setup… so long as it all continues to operate error free. I am not very forgiving of file system issues.

7

My setup is very simliar. I use Snapper, Snapper-GUI, snap-pac, grub-btrfs and snap-sync. snap-pac to take snapshots before/after installing packages from repository and AUR. Grub-btrfs to update Grubs bootmenu if a snapshot of root is made and to boot into readonly Snapshots. And snap-sync to make daily/weekly differential backups to a external SSD formated with BTRFS trough SSH, for all my machines.This SSD contains a minimal Manjaro-Installation to access all the Backups and i use it to restore/recover a broken machine. A differential backup with snap-sync of root and home need only 2-10 seconds in the background. Snapper-GUI is simliar to Timeshifts-GUI.
All in one (i coming from Windows with over 25 years of experience) i never would go back to Windows without all these BTRFS features, and all my machines use BTRFS now.

8

I finally got around to looking into snapper-gui. No doubt it will be very helpful indeed. And snap-sync is also something I plan to look into, thanks for the tips.

9

I’m really interested in how to change file system on my rpi4 manjaro system from ext4 to f2fs, xfs or btrfs.

I use this guide from Darksky to create an backup image but when after restore to a microSD card, file system still ext4.

I used GPart to change file sistem for root partition to XFS before restore.

Thansk for your help!

10

Fsarchiver is supposed to be able to restore to a different file systems. Did you pre-format the destination partition with the desired file system first? Having said that it does not deal with a fat32 file system and have no clue about f2fs.

To avoid any partition labels being the same conflict I always make the fsarchiver backup image and then take the original source completely out of the picture before I prepare the new partition with the volume label and new file system.

11

Looks like fsarchive replace partition filesystem. So, I check fsarchive official guide here.

Option -F was used to convert a filesystem in old version. For instance, it allows to restore a filesystem which was ext2 when it was saved as reiserfs on the new partition. Now, you have to specify option mkfs=xxx with the destination partition.

I found that fsarchive unfortunately is not compatible with F2FS File System. I was able to use variable mkfs=xfs to conver from ext4 to XFS when restore, but got error when I try to boot manjaro on my rpi4:

image
image1008×756 330 KB

I tried to convert to btrfs using this command:
sudo fsarchiver -v -j 4 restfs root.fsa id=0,dest=/dev/sdb2,mkfs=btrfs

But I’m still troubleshooting this:

image
image1098×142 8.64 KB

I’ll continue to find a way to do it.

12

Wonder whether ext4 is the best choice for the root partition on Manjaro ARM. Other file systems may make provide a better fit - F2FS would be the simplest choice, maybe BTRFS. Not sure what’s going on in this regard with Manjaro x86_64. I remember reading this forum thread on implementing a default BTRFS setup in Calamares a while back:

13

Thanks @Feakster , I’ll see the guide with recommendations and give it a try. Now, I still trying to figure out how to migrate between File Systems (from EXT 4 to F2FS/BTRFS). :frowning:

14

I do not think there is a good migration. I copy the files from a SD to an SSD that has a btrfs /root. I use three drives to make it work easily.

15