BackSnap: Call for Testers

andreas85 · 2 January 2023 18:45

BackSnap is a program for backing up btrfs snapshots to an external disk

Why should I use something like this? I already have a reliable backup. I don’t want to take any (!) risks with my backup.

Backsnap is based directly on btrfs. Therefore it can do the following:

backups are snapshots, and fully readable
backups are created differentially by btrfs send/receive
backups are created with CoW
compression is selectable as always with btrfs
all backup operations are performed by btrfs itself

Strictly speaking, BackSnap only works as a wrapper around the original btrfs command.

To the risk:

BackSnap is licensed under the GPL. It goes even further by containing its own source code in the binary
The backups have no special format, but are fully readable snapshots. So you can check at any time whether everything is there in the backup by mounting it. A restore from the backup is also possible with any file manager.
The only place BackSnap writes to, is the drive on which the backup needs to be written

Now I’m looking for “brave” testers for BackSnap

Development status:

I’ve been making my own backups with Backsnap for about 6 months from computers with snapper layout
It is sufficient to start Backsnap via CLI
I also use BackSnap with ssh to backup remote machines
In the meantime I have added a GUI that can make the backup transparently visible
You can now also clean up outdated backups via the GUI
and since June 19, 2023 there is a version supporting timeshift

Since I use snapper myself, everything was adapted to snapper. But I also adapted BackSnap for Timeshift. Therfore i did use a standard install of manjaro xfce in a vm.

In short:

You need to have java installed

version should be 17 or higher

pacman -S jdk-openjdk
sudo archlinux-java set java-20-openjdk

You then only need to download backsnap

sudo wget https://github.com/andreaskielkopf/BackSnap/raw/timeshift/backsnap -O /usr/local/bin/backsnap

And make it executable

sudo chmod a+x /usr/local/bin/backsnap

On my PC testing the program looks like this:

backsnap --version

Complete instruction can be found at:

github.com

andreaskielkopf/BackSnap/blob/timeshift/gallery/gallery.md

Sonntag, 14. Mai 2023 15:07 
# HowTo

## de
#### [Installation](install_de.md) 
#### [Backupmedium vorbereiten](device_de.md)
#### [CLI ausprobieren](clitest_de.md)
#### [CLI per script](backup_de.md)
#### ~~[GUI testen](guitest_de.md)~~
#### [GUI per script](backup_de.md)
####  [ssh einsetzen](backup_de.md)
#### [Restore vom Backup](restore_de.md)
Sie finden mich im [manjaro-forum](https://forum.manjaro.org/t/howto-hilfsprogramm-fur-backup-btrfs-snapshots-mit-send-recieve/130154) als "andreas85"

## en
#### [Install](install_en.md)
#### [Prepare backup medium](device_en.md)
#### [Try the CLI](clitest_en.md)
#### [CLI by script](backup_en.md)
#### ~~[Test GUI](guitest_en.md)~~

This file has been truncated. show original

Andreas
last updated on 2023/06/19

bedna · 9 June 2023 16:06

Cool!

Is it possible to have it NOT be called by snapper/Timeshift (only run it manually in the gui) OR create some kind of cleanup?
I know myself, I WILL forget this and the backup destination will become full.

andreas85 · 9 June 2023 16:23

It is never called automatically

You can indeed

create a backup script and connect it to cron
create a systemd-unit

I would suggest to NOT have the backup-disk connected permanently

Only connect it when you run your backup-script. (Or let the backup-script start automatically on mount). This way you will even be protected against some forms of evil programs.

After using it for 6 months with 5 computers, I can assure you that the backup volume will only grow slowly after the first backups.
Responsible for this are the differential backups and the compression. Somehow I suspect that btrfs is doing a mild form of deduplication in the background.

My volume with 3TB is only 53% full so far, even though manjaro as a “rolling release” is actually the enemy of every differential backup.

P.S.: I call it once a week

github.com

andreaskielkopf/BackSnap/blob/master/gallery/backup_en.md

# Example scripts for backup with backsnap
#### An example script for the local computer
The script must be started in a **graphical terminal** with **sudo** !
#### Don't forget to adapt!
The scripts must be adapted to the local conditions. In particular the following points:

`BS_ID=03417033`

  This must be a recognizable part of the UUID of the backup partition
  
`backup / manjaro18`

* The first parameter points to the mountpoint of the subvolume whose snapshots are to be backed up
* The second parameter names the name of the backup
### local

```
#!/bin/sh
# Back up all snapshots of the local machine to an attached drive

This file has been truncated. show original

bedna · 9 June 2023 16:38

Got snowed in on:

Desirable side effects

The backup strategy is already defined in snapper/Timeshift, and is involved here

This was what I was thinking.

Agree. Could possibly get that invoked in the systemd service, have to look into this. (my noob brain gets exited to learn stuff)

I have been running urbackup for a few years and as I understand it, btrfs kinda deduplicates “out of the box” this way. It checks the “parent” snapshot (subvlume) and links the new snapshot (new subvolume) to that with the exeption of changed files, ie, only a few kb if you haven’t changed anything on the source. With urbackup the changed files are also just patched, but that might be a urbackup thing more than a btrfs thing.
The paragraph above is me mostly speculating and have no basis in reality whatsoever. xD

I will def take a look at this. I also use clonezilla live and make backups once in a while keeping a few backups on a never mounted partition, since I start clonezilla from grub it takes a bit of effort If I “just want to make a quick backup before I start screwing around”, timshift is great, but it is not a backup, this is.
I already back up my home directory (separate partition) with urbackup, so I might have to look into how to exclude that, or not. I’ll just download it and see where I go from there.

Thank you for this!

andreas85 · 20 June 2023 12:50

BackSnap backups do not have include/exclude option as they always consist of a complete subvolume.

With BackSnap, you don’t have to exclude /home if your /home is already its own subvolume (e.g. subvolume /@home or @home). They create backups of the other subvolume (usually /@ or @). This way nothing from /home is included.

If you want to include /home, you need to make an extra backup of the subvolume of /home (/@home or @home).

Since June 19, 2023, backsnap also supports timeshift.

bedna · 20 June 2023 13:38

yeah, I noticed that, I have my home in btrfs but not as a subvol, sooooo… yeah…
I do backups on partition level with clonezilla as well so when I had /home in a subvol (but on other partition), that broke restoring my / partition So I opted to mount it “normally”.
Oddly timeshift works perfectly, I assume it actively excludes /home no matter if its a subvol or not when not clicking the home checkbox.

I am requesting support with things that are connected to storage on my computer but nobody seems to know why I get udisk errors every single boot and shutdown.
Until that issue is fixed for me, I wont play around too much with new storage things.

But I REALLY like the idea of your software!!

andreas85 · 24 June 2023 15:32

Together wit a improved version of BackSnap for Timeline, there is a new gallery of screenshots of BackSnap.

gui_free_some_space2

You can view the full set on:

gallery.md

andreas85 · 3 July 2023 17:03

Some small graphical improvements

are now in the master-branch.

andreas85 · 8 July 2023 15:24

As of today BackSnap does work for Timeshift and Snapper. It does show some differences in gui (like names for snapshots according to the tool you used to create the snapshots):

BackSnap used together with Timeshift:

BackSnap used together with Snapper:

andreas85 · 3 October 2023 13:06

There is a brand new version 0.6.6.21 which uses java 21

works with snapper and timeshift
uses compression for btrfs send/receive where possible
seems to be a little faster with java 21
uses configuration at /etc/backsnap/*.cfg
helps to prepare a btrfs-partition for backup
helps to create initial configuration
works with / without configuration
works with ssh

Usage:
sudo backsnap
or
sudo java -jar backsnap.jar

andreas85 · 30 October 2023 12:16

There is a new BackSnap Version 0.6.6.43 (2023/10/30)

The instructions for

Installing Backsnap
Preparing the backup disk
Use with the configuration in /etc/backsnap.d
have been revised in German and English.

Backsnap uses java 21 with -XX:+UseZGC -XX:+ZGenerational
needsJava21

P.S.:
I’m still looking for someone who would be willing and able to add BackSnap to the AUR.

bedna · 30 October 2023 14:55

This is great! (have a star from me)

I am in the process of doing something similar for my rpi backup tool soon, so I might glance a few times on your code, did so just now.

Did immediately see something though.
#!/bin/zsh
Not everybody has zsh installed by default. So unless Manjaro is the only target, maybe use bash or sh?
Or do you want me to do a pr? xD

Edit
Nvm, i checked the wrong file…
Checked the correct file and #!/bin/env -S bash
sry

cscs · 5 November 2023 03:01

Hey @andreas85
I could make a proper PR for you … but for now take my quick scribbles … seemed to work.

PKGBUILD:

# Project Author: Andreas Kielkopf
# with <3 from cscs

pkgname=backsnap-git
#_pkgname=${pkgname%-git}
_pkgname=BackSnap
pkgver=0.6.0.22.r29.gf338bef
pkgrel=1
pkgdesc="Backup btrfs snapshots with send/receive. Works with Timeshift and Snapper."
arch=('any')
url="https://github.com/andreaskielkopf/BackSnap"
license=('GPL3')
depends=('java-environment=21' 'pv')
source=("git+${url}.git")
sha256sums=('SKIP')

pkgver() {
    cd "${srcdir}/${_pkgname}"
    git describe --long --tags | sed 's/^v//;s/-/.r/;s/-/./'
}

package() {
    cd "${srcdir}/${_pkgname}"
    install -Dm755 "backsnap" "${pkgdir}/usr/local/bin/backsnap"
    install -Dm644 "readme.md" "${pkgdir}/usr/share/doc/${pkgname}/readme.md"
}

makepkg -sric >

$ backsnap -d
BackSnap Version 0.6.7.10 (2023/11/02)
args >  -d 
java [version=21, major=21, minor=null, patch=null]
using ThreadPerTaskExecutor
Doing a dry run ! 
no Pc & no Id
ende:X ready to exit java
254 ==>
BackSnap is made for btrfs snapshots on manjaro linux
but may work for others too

Usage:
------
/usr/local/bin/backsnap [OPTIONS]

 -h --help           show usage
 -x --version        show date and version
 -d --dryrun         do not do anything ;-)
 -v --verbose        be more verbose (-v=9)
 -s --singlesnapshot backup exactly one snapshot
 -g --gui            enable gui (works only with sudo)
 -a --auto           auto-close gui when ready
 -c --compressed     use protokoll version2 for send/receive (if possible)
 -i --init           init /etc/backsnap.d/local.conf (only with -g)
 -o --deleteold      mark old backups for deletion in gui (-o=500)
 -m --keepminimum    mark all but minimum backups for deletion in gui (-m=250)  
 
 -o,-m,        need  manual confirmation in the gui to delete marked snapshots
 -i            needs gui to confirm uuid of backup-medium
  
 For sources see@ https://github.com/andreaskielkopf/BackSnap and inside this file
 For help go to   https://forum.manjaro.org/t/howto-hilfsprogramm-fur-backup-btrfs-snapshots-mit-send-recieve

(sorry I cant actually test it because I dont use any of these things [btrfs, timeshift, snapper])