Timeshift - Best way to create a backup/restore point for the future

Support
1

Hi, I found an old external hard drive and I was thinking of creating a backup/restore point of my system to use in case something fails after an update or any other problem. My HDD has 465GB of available space.

I was watching some tutorials and researching the best option, but I’m not sure how much space I need.

I ran sudo du -sh --exclude=/home / and got 86GB for / (I’m not sure why I got “Permission denied” since I used sudo).

du: cannot access '/run/user/1000/doc': Permission denied
du: cannot read directory '/proc/8365/task/8365/net': Invalid argument
du: cannot read directory '/proc/8365/net': Invalid argument
du: cannot access '/proc/52576/task/52576/fd/4': No such file or directory
du: cannot access '/proc/52576/task/52576/fdinfo/4': No such file or directory
du: cannot access '/proc/52576/fd/3': No such file or directory
du: cannot access '/proc/52576/fdinfo/3': No such file or directory
86G	/

I then ran sudo du -sh /home and the result was:

525G	/home

Seeing this, my HDD isn’t large enough. So, I was thinking it might be a good idea to create the backups/restore points while excluding unnecessary folders like these:

sudo du -sh --exclude=/home --exclude=/root/.cache --exclude=/tmp --exclude=/var/tmp --exclude=/var/cache /
sudo du -sh \
--exclude='/home/*/Projects/*/node_modules' \
--exclude='/home/*/Projects/*/build' \
--exclude='/home/*/VirtualBox VMs' \
--exclude='/home/*/.local/share/libvirt' \
--exclude='/home/*/Games' \
--exclude='/home/*/.steam' \
--exclude='/home/*/Videos' \
--exclude='/home/*/Music' \
--exclude='/home/*/Pictures' \
--exclude='/home/*/tmp' \
--exclude='/home/*/.local/share/Trash' \
--exclude='/home/*/.cache' \
--exclude='/home/*/Downloads' \
/home

After this, the result was:

58G     /
310G    /home

Perfect for my HDD.
So my final command:

sudo timeshift --create --comments "Backup / and home" --tags D --exclude "/home/*/Projects/*/node_modules,/home/*/Projects/*/build,/home/*/VirtualBox VMs,/home/*/.local/share/libvirt,/home/*/Games,/home/*/.steam,/home/*/Videos,/home/*/Music,/home/*/Pictures,/home/*/tmp,/home/*/.local/share/Trash,/home/*/.cache,/home/*/Downloads,/root/.cache,/tmp,/var/tmp,/var/cache
  1. Is excluding these folders a good idea?
  2. IIs there a better or alternative way that you would recommend?
  3. Are there any other unnecessary folders that I can exclude?

Thanks for your time! :slight_smile:

2

TimeShift is for the system, don’t manage your HOME with TimeShift, you’ll regret it. Use other tools MADE FOR THIS, like Deja-Dup So you can have your Home safely and intelligently backed up the way you want.

2 Likes
3

Thanks for your reply.
I didn’t know about Deja-dup, it’s the first time that I read about it. Always I was reading of Timeshift.

So you recommend my previous command only for / (excluding /home) and /home with Deja-Dup?

thanks!

4

I don’t recommend any command. Use the GUI if there is no reason blocking you from using it, you’ll find TimeShift or Deja-Dup in your Application Menu. My personal choice for TimeShift configuration is to disable all automatic snapshots, to exclude /home and /root (it should already be, by default) and I do snapshots manually when I feel like it (before big updates generally). It is only a click of a button to create a snapshot. Only a click of a button to delete old snapshots. I also like to clean the system before creating a snapshots (removing packages cache for example, vacuuming the logs).

For the Home there are many things you usually don’t want to backup, like your games, your downloads, things like that but only you will know. I use QDirStat to find where are things eating up my disk space it can help you discard what you do not want to backup in Deja-Dup (or similar tools made to backup your personal files).

1 Like
5

I will suggest some risk assesment.

Divide you data into categories - roughly something like

  • Invaluable - as in need to have
  • Casual - as in nice to have
  • I don’t care - everything else

This is a rough assement - you may want a more fine grained control - but using this rough assesment you have

  • Invaluable
    • source code
      • use bitbucket - public and private
      • use github - public and private
      • use gitlab - public and private
      • use codeberg - public
      • store local using a rpi and a gitea/forgejo instance
    • emails
      • imap account with trusted provider
    • documents
      • trusted cloud provider
  • casual
    • everything not included above - not easily produced by other means
    • store at trusted cloud provider
    • store at local device - removable storage
    • store with local service - raspberry pi with harddisk attached
  • I don’t care
    • anything that is easily reproduced by other means
    • e.g. programs stored as appimages, flatpak, snap

Use Timeshift to snapshot your system

5 Likes
6

Timeshift probably already does this, but these shouldn’t be included in a backup or system restore snapshot (or your du command).

/run
/tmp
/proc
/dev
/sys

If you add -x to your command, like so

sudo du -xsh /

it’ll miss out all other filesystems including the ones listed above (they’re temporary filesystems in RAM), and home if it’s on a separate partition.

As for backup utilities, here’s a list:

https://wiki.archlinux.org/title/Synchronization_and_backup_programs

2 Likes
7

Hi again, and thanks for your replies.
I was watching and researching a little bit and trying to follow your opinions.

I did an snapshot like you told me with no /root and no /home with Timeshift.
I didn’t use Deja-Dup because i’m using KDE and I have read it’s a gnome application but following the @linux-aarhus I did my backup of /home with rsync and excluding the folders that I commented in my first post:

sudo rsync -avh \
--exclude='/home/*/Projects/*/node_modules' \
--exclude='/home/*/Projects/*/build' \
--exclude='/home/*/VirtualBox VMs' \
--exclude='/home/*/.local/share/libvirt' \
--exclude='/home/*/Games' \
--exclude='/home/*/.steam' \
--exclude='/home/*/Videos' \
--exclude='/home/*/Music' \
--exclude='/home/*/Pictures' \
--exclude='/home/*/tmp' \
--exclude='/home/*/.local/share/Trash' \
--exclude='/home/*/.cache' \
--exclude='/home/*/Downloads' \
/home/ /mnt/

Sorry If I ask, but this is OK, right? So I have a problem for an update or something, I will be able to use the snapshot.

And in the worst scenario if my internal ssd is broken, I can reinstall a fresh system of Manjaro and after that replace the /home from my external hdd saved with rsync command.

Thanks again for your time and support always! :slight_smile:

8

It is not an issue it works as well on KDE.

1 Like
9

Thanks. Now I downloaded.
I will generate a backup using this tool and I will exclude the folders too.

I hope this is enough but you never known until the system is ■■■■ up and you need to use it xD

11

I assume this is btrfs since you are talking about snapshots? (Not ext4 /w rsync Timeshift snapshots, these are not snapshots.)

There are a few who will deter you from btrfs entirely, the reasoning is because it is too difficult.

I truly believe it is far from it, and your options really open up when you are backing up btrfs. And using it like an ext4 filesystem, there is not much more you need to know. But while it is there, you will find out eventually, there is some pretty cool stuff you can do that you couldn’t do before. And then on top of that, when the destination backup filesystem, local or network, is also btrfs; there are even more cool things you can do.

The point in time live snapshot, are rollback points, not backups. But they are perfect to use in conjunction with a backup, on a live system. And when your destination backup target is also btrfs, there are very unique ways you can backup, which is very efficient and very robust.

Timeshift is pretty simple, which is great! But there are a lot of hardcoded settings that would be nice to change, but you cannot. But for just taking snapshots before each upgrade, automatically with no configuration, along with easy manual snapshots with a nice clean GUI that works. This alone has value, and if you want, you can keep this. But it does not provide an additional copy of your data, unless you pick rsync over btrfs. And then treats it as a generic filesystem, and it does what rsync does, reads all the content on both sides one after another, reads the source and destination looking for the smallest change. With it already taking forever (especially on a HDD or over the network), it’s not the most robust solution. And I can spew many more paragraphs on the disadvantages of doing it this way. (Even knowing it creates hard links of files to save time and space.) It is still billions of more IOPS to do it this way.

With btrfs, these snapshots, rollbacks, individual file or partial restores, are near instant.

The cool thing you can do when backing up btrfs to btrfs, is send only the block level changes. With btrfs, It is very easy to tell exactly what it’s written since the last backup, and send only the changes. So a pacman upgrade that takes 1.5G before the (Y/n) prompt, could actually only send <100M for your backup. Rsync would be chugging along after a change like that, with just having way more ways to screw up. And the data is not even close to identical, it is guessing at files, and relying on timestamps. (You can’t even force a checksum type rsync, unless you run it yourself.)

Most people in these forums will just shout, “Use Snapper!” I have a love/hate relationship with Snapper, but it does everything and more. That is, once it is setup properly. I think this is where the learning curve needs to be the steepest, and deters most people.

But it looks like you’re now tinkering with DejaDup? I have never used, but you can use whatever you want. Although it looks like a fancy GUI that just copies data, not even rsync’d? The btrfs support that it could be using, is what many of these third party apps seem to lack.

Timeshift is automatically making these snapshots for you, which you can backup, just by finding a way to get these off to another filesystem/host. btrfs send and btrfs receive are very cool commands to send volumes/snapshots to other btrfs filesystems (the parent volume option is crucial for incremental sending of volumes). But programs like Snapper do all this for you. And also, I know @andreas85 has published his own app built for Manjaro to do this. And as much as I discourage Timeshift w/ rsync, I think it may actually be better, and easier, than Deja Dup in this case. Or even your own rsync command you were coming up with earlier, as you cannot change how it does rysnc in Timeshift.

But whatever you decide, it’s not a backup, unless you have tested the restore procedure.

These aren’t part of the filesystem you’re backing up, if you are only backing up btrfs volume/snapshots. Timeshift is one of these (and I would safely assume excludes them via rsync method.) In addition, with btrfs, /the volume @log is /var/log, and @cache is /var/cache, which are excluded in btrfs snapshots as they are their own volumes by default in a Manjaro btrfs install.

2 Likes
12

Thank you for the detailed information.

I’m learning about a lot of things.

When I open the Timeshift job wizard, the only option available is rsync;
I can’t select btrfs, and I don’t know why.

At the moment, I’m creating a backup with Deja Dup (or maybe it’s called “Backups” or “Duplicity”? Because that’s the name that appears, but in my HDD, the filenames are labeled as “Duplicity”), as recommended by @omano.

I don’t know which reply to mark as the solution because all of them were useful to me!

Thanks!!

13

Whilst they may be using btrfs, there’s no reason to think they’re using btrfs. Other than an assumption based on your interpretation of the word snapshot.

They’re not btrfs snapshots but they are timeshift snapshots.

btrfs is far more complicated than ext4. Not all the features are production ready either.

Robust is the wrong word, you’re equating “not the most robust solution” with slow, so the phrase you’re looking for is “not the fastest solution”. It’s more robust than a snapshot on the same filesystem.

You still need to transfer the data to another drive. It’s no more robust than using rsync.

No it’s not, timeshift is a system restore utility not a backup utility. Whilst it might be easier, it’s definitely not better. It may lead to data loss.

It’s just a matter of copying the data back. So more a case of verifying the data was backed up properly.

@ZeR0ByTe

Probably because you’re not using btrfs. However only you can answer that.

14

Yes, I’m using ext4. Stupid question, sorry :slight_smile:

15

No need to apologise. :smiley:

1 Like
16

Then all my information is nullified, you were talking about snapshots, and Timeshift backup, so I only assumed btrfs. The filesystem you were using would of been valuable information before I spewed all that. :wink:

It doesn’t sound like it, but check?
mount -t btrfs or
findmnt / or
cat /etc/fstab

A inxi -Fv8z in the original post would of have caught that to.

Ignore the rest of this @ZeR0ByTe…

=================================================================

The former true, but just to use it like your ext4 filesystem, it is very simple. You don’t have to use any of the advanced features of btrfs. The only thing you really have to know is how btrfs volumes work, really a 120 second read. Then just ignore the cool stuff if you want.

The latter is dead wrong. It’s been stable for purposes like these for a long time. (This comes from a very hard to reproduce bug in RAID 5 on power outages or the like.) But JBOD (or JBODs), RAID 0, RAID 1, RAID 10 have been stable for many many years.

It was standard practice backup for many people 20 years ago, I’ve been using it for almost 30. I can assure you, and I can even give you the numbers on how much more is sent with rsync over incremental btrfs sending. (Hell, even sending the whole volume not incrementally, but the differences aren’t as apparent.) Btrfs has a generation ID, and it sends only the new data blocks up until the current or requested generation. Technology has improved, and filesystems got more advanced in 30 years, go figure! You don’t seem to understand the drastic differences.

I use that word always, and I’ll keep using it. How many failed or screwed up rsyncs or fixing other peoples’ failed rsyncs have you done in the last 30 years? Have you watched it with strace?

I can also give you benchmarked numbers for each. The network throughput and size of the payload for one backup, and the total disk operations for each with is even much worse. There are bottlenecks everywhere with rsync and it does so much more work, that btrfs just doesn’t need to do. My SSD and CPU are the bottlebeck on my network for this. Even when I was running 2.5G Ethernet as opposed to 10G.

I don’t think you understand what block level synchronization actually entails. And it is far more robust.

I never said otherwise. I’m trying to understand what the OP wants to do. Much of what I said is using Timeshift (just for talking snapshots), then adding your own way to backup Timeshift’s snapshots. This is actually what I do on two of my systems. But I was mislead in his FS type. But I did not know he was not running btrfs.

I think rsync is great for specific purposes, but not for my root volume of my workstation I need everyday. (Which I’ve restored many times, over the network, in minutes to even seconds.)

17

I’m going off the docs, though there’s not much left.

https://btrfs.readthedocs.io/en/latest/Status.html

Still important to know if you’re going to use btrfs. Less so if you don’t plan on using any btrfs features, which begs the question why use it at all?

I removed this because it was pointless drivel, long before you posted.

None. The only time I have a failed rsync is if the destination runs out of space.

You were only talking about efficiency/speed earlier. Therefore the word robust was out of place. You’re still mostly talking about efficiency and speed.

I wasn’t talking about sending the snapshot to another drive and neither were you, though you did seem to mix btrfs snapshots on the same filesystem with sending rsync over the network.

Maybe something was lost before it got to the keyboard. :man_shrugging:

18

I think this happened before with you. I don’t know why it instantly turns into hostility.

OP did not post much info, even the FS type. When you talk snapshots in this forum, I assume btrfs. I use many other filesystems that support snapshots too. (But here you assume the Manjaro defaults, ext4 or btrfs. So I assume the one that snapshots!)

This all stemmed from when I quoted you that the memory filesytems don’t matter in this context. With it turning out being ext4, you were right and I was wrong. I did not mean any offence.

I still stand by what I said, if it was btrfs. But it’s all irrelevant.

19

We all do things differently - and when it comes safeguarding your data - and keeping your system stable - it all boils down to your workflow - and how you plan to recuperate after an accident.

I have been around a long time - and backup programs all have their pros and cons - I have over time developed a habit that keeps my data reasonably safe from disasters.

Personally I don’t use backup programs - I don’t even backup data - most of the data I have doesn’t change - media libraries - when you think about it - we are often hording data - data which we really don’t use.

  • I do not store documents on my system disk
  • I hae an old Synology 1010+ for long term storage
  • I have an account with Proton AG
    • for domain mail
    • for cloud storage
  • I utilize Github, Gitlab, Bitbucket, Codeberg
  • For my work code I have setup a gitea instance on a company server
  • For my scratchpads I use a raspberry pi using Manjaro Linux, nginx and forgejo
20

If so I apologize. It wasn’t meant to be hostile.

None was taken, nor did I mean to cause offence.

21

I have been using Timeshift for system back up and BackInTime for home directory for two years, can’t say enough about how well it works.

Also use Duplicati on some clients Windows machines, but have been meaning to try it out on Linux.

In the end, it is what ever works.

-John

4 Likes