Best way to create system and files backups

Hi all. Backups are vital, both on servers and on a personal computer.

The fact is that in order to conveniently and efficiently make backups, we need certain tools for this. Personally, I’m only familiar with timeshift, but it doesn’t quite meet my needs:

  1. It is needed for backups of system files, not personal ones. It can be used for personal files, but I’m not sure if it’s correct.
  2. It does not archive files, which is why snapshots weigh quite a lot. Why not use 7-zip’s aggressive compression so that it uses less space?
  3. It cannot store files on a remote server. If we use a server, it would be good to use a separate FTP storage for backups.

Summarizing all the above, I have the following questions:

  1. Should I try to keep system and personal files separate? I believe that personal configurations, such as KDE settings, should also be included in the system files. The problem is that it is stored in the user’s local folder and is not considered part of the system.
  2. Is it possible to compress snapshots so that they take up less space?
  3. Is it possible to send snapshots to a remote FTP server?

I have a small script, as I imagine it, but I’m not at all sure that this is correct:

#!/bin/bash

# FTP storage settings
FTP_USER=user
FTP_PASSWORD=password
FTP_SERVER=example.com:21

# create snapshot
sudo timeshift --create &&

# zip all snapshots
7z a -t7z -m0=lzma -mx=9 -mfb=64 -md=32m -ms=on snapshots.7z /timeshift/ &&

# send (overwrite) snapshots archive to FTP server
# it will also delete source file (our archive) on success
lftp ftp://$FTP_USER:$FTP_PASSWORD@FTP_SERVER -e \
   "set ftp:ssl-allow no; put -e -E snapshots.7z; quit" &&

My idea is that to restore we just download the archive, unzip it to /timeshift/ and we can restore the snapshots as usual.

1 Like

That looks like you need BorgBackup that can offer all your points.

  1. Running BorgBackup in root for creating a root backup.
  2. Support different compression algorithm. e.g. zstd
  3. It can store backups/snapshots on a remote server via FTP or SSH.
  4. Support deduplication

Read Borg documentation:
https://borgbackup.readthedocs.io/en/stable/index.html


I personally use Restic, which is Borg’s alternative.
Both are similar, but Restic doesn’t support compression yet. Borg does not support multithreads
But Borg is written in Python, Restic is written in GO. That is why Restic’s benchmark is faster.

Both have no GUI.
If you need GUI, maybe Kopia would be interesting for you

4 Likes

Thanks for answer!

Actually, there are desktop client for BorgBackup. I don’t need it, since backups are needed mainly for the server.

Should I use Borg\Restic for system files backup? I find some information but it doesn’t look like timeshift at all. Using timeshift, I can simply restore a snapshot of the system with one command, but here it’s just an archive with data. If, for example, I took a snapshot before installing a dangerous program and this program broke everything for me, how can I “roll back” to the previous system snapshot?

Restic looks better I think.

That depends on your decision, which data is important, but a root backup requires root permission when running in root system to create a new root backup.

I use a weekly root backup inclusive some parts of root system and my important data, not whole system.
My root backup excludes some directories:

.snapshots/* # You do not need it when you are not using btrfs.
/home/**/.cache/*
/root/.cache/*
/var/log/*
/var/cache/*
/dev/*
/proc/*
/sys/*
/tmp/*
/run/*
/mnt/*
/media/*
/lost-found

I knew, Restic restoring is simpler than Borg restoring in CLI.

Define restic environment variables, no need to type “-r” for every command of restic.

  1. $ sudo vim /etc/environment to add: (For example, my restic-repo is in ZFS mirror filesystem)
RESTIC_REPOSITORY=/zfs/Zbackups/restic-repo

How to restore a snapshot / backup to your path

$ sudo restic  restore <Snapshot ID> --target </path>

restic restore requires target dir. If I restoring whole system, am I need to set “/” target? Will it overwrite files and delete files which are not in snapshot?

Yes, it overwrites files, but I do not think it deletes files.

This can be a problem, as snapshots must be a state of the system that can be completely restored.I’ll check if it deletes files

Check out the feature that isn’t implemented yet


There can be a workaround via restic mount and rclone sync:

$ sudo restic mount <mount_dir> 
$ rclone sync <mount_dir>/ids/<snapshot_id>/ <target_dir>/

rclone sync will delete local files that aren’t present in the snapshot.

There is rustic that is the successor of restic, but rustic is written in full rust language instead GO language.

Both have the same features, but rustic supports to restoring with the option --delete to delete files that aren’t present in the snapshot, but it is in beta

It is available in AUR

Make Backups:

  1. You could create different btrfs-subvolumes which produce different scopes for snapshots
  2. You could enable compression in btrfs. This is at least as efficient as 7zip
  3. You could use btrfs-send together with ssh :wink:

Please have a look at Btrfs in the wiki

1 Like

It’s a little different. I don’t need cold backups, something like “timeshift with ftp” is enough

clonezilla

i would never backup to the drive i’m using, it must be always a external device. i like the option to backup a real clone. once my disk will die i’ll replace the drive just with the clone and i’m back on track. less than 5 minutes of work and even the worst case (broken disk) is solved. using nvme’s gives acceptable speed and while backing up i’ll get a coffee and a smoke. what can be better ?

i had broken disks a couple of times in my life and that’s the moment when you have to have a real solution. clonezilla is the key

On my personal computer, I make backups to an external hard drive, it’s not a problem there. But on the server it is desirable to make a backup with sending to external storage

The difference between timeshift and borg is just that:

  • timeshift is made for hot backups
  • borg is made for cold backups

So think of timeshift like on Windows “System Recovery”.

I would suggest using timeshift for the root directory and borg for important files. Any configurations can go to a git repo.

If it is btrfs, you can increase the compression level.

You can use btrfs send to grab a raw stream of the snapshot and pipe it to a file. Also btrfs receive can get the file and put it back on the partition.
:notebook: A raw stream cannot be mounted and is not searchable.

that’s quite a more complex problem in fact, but in the end anything else than a mirror-server is pain in the butt. restoring a broken server is a no go

Also wasn’t specified if this is a VPS or a homeserver, you’ll get different options for each.

For a homemade backup you’ll probably only want to save the appropriate data in /var, and make sure you aren’t transferring it in in the plain if using FTP.

You should be careful, zip does not support unix permissions.
Many special permissions of system files will be lost after restoring.
AFAIK, tar supports unix permissions.

VPS\Dedicated

I have Synology NAS for this. One of the features of a NAS are exactly backups. My backups are set up as following:

  • system files specifically / partition (no home, no mounts): timeshift
  • /home partition: instant backup with Synology Drive Client
  • other partitions: not backed up

I recommend a similar setup, timeshift is great for undoing updates that went bad. That lowers my enxiety when updating because some things on the system absolutely need to work 100% of the time.

A NAS solution may be expencive, but it will always cost less than you personal files. I learned that once. Synology seems to work very well with its home cloud solution, but you can also build you own server and with some open source cloud solution like owncloud. A cloud solution is also better than others because files get synced instantly as soon as you save them and they can be easily configuret to be one way or both ways and also a good cloud solution will also handle versioning of your files.

I also have other partitions/folders that I don’t backup. Those contain unimportant files, like games and temporary downloads.

When it comes to saving backups, it a rather a deep rabit hole… :stuck_out_tongue: some crazy gurus say, for a real backup, you need one local copy then one copy on another location and finally another copy on another continent :stuck_out_tongue_winking_eye: For me a backup on a NAS with full redundancy is enough for me.

Unless you also wan’t to hack with rsync and whatnot along the way, I recommend timeshift + cloud solution.

My favorite method for backin up the system is CloneZilla. It can store the output file wherever you want, even on a remote computer. And it has different compression methods with my favorite compression being z9p which is designed for multicore systems and compresses 36 GB to a 16 GB file in less than 3 minutes. Decompression is even faster.

1 Like