Migrate from ext4 emmc to btrfs nvme

[airclay]

I reccently picked up a nvme drive for the pbp. I’d like to migrate, as topic reads, from emmc to nvme while switching to btrfs in the process.

My idea is to preserve BOOT_MNJRO on emmc and use nvme for root (mostly). Considering using the excess free space left on emmc for /@var, /@tmp and, /@.snapshots subvolumes

From my searching and reading it should go something like:

• create partition on nvme, label as ROOT_MNJRO
• format partition to btrfs
• create subvolumes (@,@home,@var,@tmp,@.snapshots)
• rsync -aAXv data from emmc by subvolume
• update fstab/extlinux.conf, mkinitcpio.conf and grub configuration
• IF all is well on reboot, format excess emmc space and migrate /@var, /@tmp and, /@.snapshots

Is there anything I’m missing? It seems a little simple for the task…

[andreas85]

have a look at:

[HowTo] transportables manjaro (2) btrfs

von ext4 auf btrfs umstellen ohne die Daten zu verlieren

Vorteile

Es schont den Stick weil btrfs mit Copy_on_Write arbeitet Es bietet die Möglichkeit automatische Snapshots mit snapper zu machen. Dadurch kann man das versehentliche Löschen oder Ändern von Dateien innerhalb eines vorgegebenen Zeitraums rückgängig machen. (Das ist kein Ersatz für ein gutes Backup) Es bietet die Möglichkeit die Dateien komprimiert zu speichern. Das erhöht die Kapazität, und schont den Stick zusätzlich Nach …

It is in german, but it describes all steps (google translate may help you) it describes all steps to do it “in place”. But the steps for you will be nearly the same (grub, fstab btrfs-layout …)

[winnie]

@airclay

Considering using the excess free space left on emmc for /@var, /@tmp and, /@.snapshots subvolumes

Might I ask why?

Wouldn’t you want var and tmp to live on the faster device? (And one that can handle more writes?)

As far as snapshots are concerned, they cannot traverse from one device to another with a block-based CoW file-system (Btrfs, ZFS). [1]

Unlike the traditional method (with rsync), snapshots created with Btrfs or ZFS are atomic and happen on the very subvolume / dataset itself.

Unless I am misreading something about your post?

---

[1] You can “send” snapshots to another device, but that is a different operation, akin to transferring to a backup drive / server.

[airclay]

/var and /tmp on other partitions simply for classic space/bottle neck reasons (put them on the hdd and everything else on ssd thinking), kinda unnecessary tbh

/.snapshots yes to send/store external copies of snapshots, maybe I used a common dir name for snapshot storage which led to the confusion. But yes to externally send and receive older snapshots for restoring, limiting the amount of space they use on nvme.

[winnie]

@airclay

/var and /tmp on other partitions simply for classic space/bottle neck reasons (put them on the hdd and everything else on ssd thinking), kinda unnecessary tbh

Not my system, so I can’t speak for you, but I will share that in my opinion I would keep everything on the faster, more robust device; not to mention it also simplifies installation, maintenance, and changing things later down the road.

---

@airclay

/.snapshots yes to send/store external copies of snapshots, maybe I used a common dir name for snapshot storage which led to the confusion. But yes to externally send and receive older snapshots for restoring, limiting the amount of space they use on nvme.

Have you used ZFS or Btrfs snapshots before? That’s not how they fundamentally work. Regardless of where and when you send the snapshots, they remain forever tethered to the original subvolume / dataset. (Won’t get into the exceptions of using “clones” and “promotions” as I’m only familiar with them on ZFS, and it’s still besides the point.)

The classical approach of using rsync (on Ext4, XFS, etc), whether through Timeshift or another method, is qualitatively different than block-based snapshots on Btrfs and ZFS.

Block-based snapshots only “consume space” in the amount of the differences between each other and the live file-system. In other words, not much at all, unless you’re holding on to some very old snapshots in which large amounts of data have been long since deleted, with the “deleted” (i.e, “unlinked”) blocks only existing in the older snapshots’ references and pointers.

---

EDIT: To be clear, if your idea is to “offload” these older snapshots to an external Btrfs volume/device, you’d still end up doing a full copy of everything if you ever need to revert to an older state if you ever destroy a common snapshot between the two subvolumes. (There will be no “base snapshot” for which to do an incremental/delta restore.) Also, lacking a “base snapshot” between the source and destination removes the ability to do an incremental send, which is time-consuming and inefficient.

[airclay]

I plan on keeping a 3mo, 1mo, weekly and, daily snapshot (and maybe a couple on demand). So I expect the latter will grow. It does make sense to keep copies of the snapshots on nvme for efficiency however for backup reasoning, it also makes sense to store external copies in the least. If taking snapshots why not use them as an added backup protection?

[winnie]

@airclay

If taking snapshots why not use them as an added backup protection?

You definitely can, and it’s good to not view snapshots as “backups” until the data also exists at a separate location.

---

Just bear in mind you cannot unlink / offload snapshots from their originating subvolume / dataset, and assume you can simply use the external location as a substitute.

@airclay

But yes to externally send and receive older snapshots for restoring, limiting the amount of space they use on nvme.

I was referring the the above which I highlighted in bold.

---

UPDATE: You can employ your idea with ZFS, leveraging its “bookmarks” feature. However, Btrfs has no such feature as far as I’m aware.

[airclay]

Thanks for walking me through that. Time/usage will tell in the efficiency vs size.

I’ll also run duplicati that stores to an s3 bucket. Plus syncing my personal data/projects to cloud/gitlab instances. The switch to btrfs is to add a final backup layer that has some easier restore options. (Prob can’t boot an external snapshot to your point). The ability to quickly revert files or dirs tbh, is an after thought. To explain how I was seeing it.

[airclay]

After reading these posts a few times through I decided to back up a little a start with a more default set-up and learn from there. So I used an iso w/ default btrfs install.

At the end of this process, I am stuck with a blank screen after boot loader (w/o ability to switch tty’s). I’ll try to walk through my process to identify what may help

Flashing img to nvme

Created img
sudo buildarmimg -d pbpro -e lxqt -b testing -p btrfs

Used Etcher to make bootable sd card and booted into live iso.

Used manjaro-arm-flasher to install to nvme.

Mounted nvme /boot and emmc /boot partitions
sudo mount /dev/nvme0n1p1 /mnt/tmp
sudo mount /dev/mmcblk2p1 /mnt/tmp2

Copied /boot/* from nvme0n1p1 to mmcbkl2p1
sudo rsync -avHAXS /mnt/tmp/ /mnt/tmp2

Edited extlinux.conf to reflect correct UUID of mmcblk2p1

Unmounted both /boot dir’s and mounted /home on nvme0n1p2
sudo mount -t btrfs -o subvol=@home,defaults /dev/nvme0n1p2 /dev/tmp

Edited /etc/fstab to reflect /boot on mmcblk2p1

Unmounted nvme completely and rebooted.

At this point I was able to boot and use lsblk to see everything was correct so far. /boot on emmc and /root on nvme. Feeling confident, I booted back into the live iso

Moving old install to nvme

Used gparted to remove nvme0n1p1 and expand nvme0n1p2 to whole disk.

Mounted my new nvme fs
sudo mount -t btrfs -o subvol=@,defaults /dev/nvme0n1p2 /dev/tmpRoot
sudo mount -t btrfs -o subvol=@home,defaults /dev/nvme0n1p2 /dev/tmpRoot/home
sudo mount /dev/mmcblk2p1 /mnt/tmpRoot/boot

Mounted my old emmc fs
sudo mount /dev/mmcblk2p2 /mnt/tmp

Move correct fstab to avoid mistakes
sudo cp -a /mnt/tmpRoot/\@/etc/fstab /mnt/tmp/etc/fstab

Copy data from emmc to nvme
sudo rsync -avHAXS /mnt/tmp/ /mnt/tmpRoot/\@ --exclude={'boot','dev','home','mnt','proc','run','sys','tmp'}
sudo rsync -avHAXS /mnt/tmp/home/ez/ /mnt/tmpRoot/\@home/ez

JIC, chroot into nvme fs and update
sudo manjaro-chroot /mnt/tmpRoot /bin/bash
sudo pacman -Syyu

Exit chroot and rebooted to blank screen. No blinking cursor or ability to change to other tty.

I chrooted back through the live iso and reinstalled the kernel, which ran mkinitcpio but, on reboot it had same effect.
sudo pacman -S linux

Second try chroot back into nvme fs, this time reinstalled uboot without any luck either.
sudo pacman -S uboot-pinebookpro
sudo dd if=/boot/idbloader.img of=/dev/mmcblk2 seek=64 conv=notrunc,fsync
sudo dd if=/boot/uboot.img of=/dev/mmcblk2 seek=16384 conv=notrunc,fsync

I know that’s quite the journey but, I can chroot in and update just fine so it shouldn’t be too far away…

lsblk/chroot/blkid/fstab/extlinux.conf output

[oem@manjaro-arm ~]$ lsblk
NAME         MAJ:MIN RM   SIZE RO TYPE MOUNTPOINTS
sda            8:0    1  14.5G  0 disk 
|-sda1         8:1    1 213.6M  0 part /boot
`-sda2         8:2    1  14.2G  0 part /
mmcblk2      179:0    0  58.2G  0 disk 
|-mmcblk2p1  179:1    0 213.6M  0 part 
`-mmcblk2p2  179:2    0    58G  0 part 
mmcblk2boot0 179:32   0     4M  1 disk 
mmcblk2boot1 179:64   0     4M  1 disk 
zram0        252:0    0   5.7G  0 disk [SWAP]
nvme0n1      259:0    0 238.5G  0 disk 
`-nvme0n1p2  259:1    0 238.2G  0 part 

[oem@manjaro-arm ~]$ sudo mount -t btrfs -o subvol=@,defaults /dev/nvme0n1p2 /mnt/tmpRoot/

[oem@manjaro-arm ~]$ sudo mount -t btrfs -o subvol=@home,defaults /dev/nvme0n1p2 /mnt/tmpRoot/home

[oem@manjaro-arm ~]$ sudo mount /dev/mmcblk2p1 /mnt/tmpRoot/boot/

[oem@manjaro-arm ~]$ lsblk
NAME         MAJ:MIN RM   SIZE RO TYPE MOUNTPOINTS
sda            8:0    1  14.5G  0 disk 
|-sda1         8:1    1 213.6M  0 part /boot
`-sda2         8:2    1  14.2G  0 part /
mmcblk2      179:0    0  58.2G  0 disk 
|-mmcblk2p1  179:1    0 213.6M  0 part /mnt/tmpRoot/boot
`-mmcblk2p2  179:2    0    58G  0 part 
mmcblk2boot0 179:32   0     4M  1 disk 
mmcblk2boot1 179:64   0     4M  1 disk 
zram0        252:0    0   5.7G  0 disk [SWAP]
nvme0n1      259:0    0 238.5G  0 disk 
`-nvme0n1p2  259:1    0 238.2G  0 part /mnt/tmpRoot/home
                                       /mnt/tmpRoot

[oem@manjaro-arm ~]$ sudo manjaro-chroot /mnt/tmpRoot /bin/bash

[root@manjaro-arm /]# cat /boot/extlinux/extlinux.conf 
LABEL Manjaro ARM
KERNEL /Image
FDT /dtbs/rockchip/rk3399-pinebook-pro.dtb
APPEND rootflags=subvol=@ initrd=/initramfs-linux.img console=ttyS2,1500000 root=PARTUUID=7b2e3695-1ad9-498d-b729-0d64f34150a1 rw rootwait quiet splash plymouth.ignore-serial-consoles

[root@manjaro-arm /]# cat /etc/fstab
# Static information about the filesystems.
# See fstab(5) for details.

# <file system> <dir> <type> <options> <dump> <pass>
PARTUUID=0c3b394c-c7da-4b06-86a6-aa2fe0db5f61  /boot   vfat    defaults        0       0
PARTUUID=7b2e3695-1ad9-498d-b729-0d64f34150a1 / btrfs  subvol=@,compress=zstd,defaults,noatime  0  0
PARTUUID=7b2e3695-1ad9-498d-b729-0d64f34150a1 /home btrfs  subvol=@home,compress=zstd,defaults,noatime  0  0

[root@manjaro-arm /]# sudo blkid
/dev/nvme0n1p2: LABEL="ROOT_MNJRO" UUID="a986a981-441a-41cd-b5ae-bd7c35d5c1c1" UUID_SUB="f882d0f9-4f2e-4c6c-b9ed-1d8055940760" BLOCK_SIZE="4096" TYPE="btrfs" PARTLABEL="primary" PARTUUID="7b2e3695-1ad9-498d-b729-0d64f34150a1"
/dev/mmcblk2p2: LABEL="ROOT_MNJRO_X" UUID="b6bfcf3e-3b70-413a-8021-870914aa76f2" BLOCK_SIZE="4096" TYPE="ext4" PARTLABEL="primary" PARTUUID="a6791224-0ddc-43d3-83df-7cf3a4abf9b8"
/dev/mmcblk2p1: SEC_TYPE="msdos" LABEL_FATBOOT="BOOT_MNJRO" LABEL="BOOT_MNJRO" UUID="A59A-BDF9" BLOCK_SIZE="512" TYPE="vfat" PARTLABEL="primary" PARTUUID="0c3b394c-c7da-4b06-86a6-aa2fe0db5f61"
/dev/sda2: LABEL="ROOT_MNJRO" UUID="1b509f2a-2757-4afb-847c-768cad1091dd" BLOCK_SIZE="4096" TYPE="ext4" PARTLABEL="primary" PARTUUID="fc5de836-8600-4753-870b-dfa59af1d21f"
/dev/sda1: SEC_TYPE="msdos" LABEL_FATBOOT="BOOT_MNJRO" LABEL="BOOT_MNJRO" UUID="EB96-8328" BLOCK_SIZE="512" TYPE="vfat" PARTLABEL="primary" PARTUUID="0dda2583-4800-450d-ae7b-45f5afb3f3fa"
/dev/zram0: LABEL="zram0" UUID="fda83a65-9868-45cd-87d4-63eb68c2e650" TYPE="swap"

[root@manjaro-arm /]# sudo pacman -Syyu
:: Synchronizing package databases...
 core                                                                                  242.4 KiB   182 KiB/s 00:01 [####################################################################] 100%
 extra                                                                                   2.4 MiB   803 KiB/s 00:03 [####################################################################] 100%
 community                                                                               6.2 MiB  1670 KiB/s 00:04 [####################################################################] 100%
:: Starting full system upgrade...
 there is nothing to do

UPDATE: It works!

Somehow by my own misunderstanding or less likely a malformed flash install the correct /boot partition was on the device I used for flashing. There may have also been an issue with the rsync command as well.

So copied the correct /boot partition (while chrooted)
cp -rf /run/media/user/000-000/BOOT_MNJRO/* /boot

On reboot I noted that some services failed (sysd-mods, zswap, etc) and it still did not even recognize a wifi device, which pointed me to a /etc/mkinitcpio.d/linux.preset.pacnew. This being the file for the 5.14.7 kernel, most likely created during a chrooted update.
sudo mv -f linux.preset.pacnew linux.preset
sudo pacman -U /var/cache/pacman/pkg/linux-5.14.7-1.aarch64.pkg.tar.zst
sudo systemctl reboot

Finally back together. /boot on emmc, /root on nvme, snapshots on root ,as suggested here, and duplicati back ups on both emmc and to s3

marking @andreas85 as solution to point others to that great guide/info