There is the possibility of update-able (Live) USB-Sticks
(search for alma - where you can add what you want 
)
In the future you could use timeshift or another automatic backup
(But this needs space on an intern or extern device)
Make Backups:
Or you could use btrfs which is capable of snapshots
(where you can recover any deleted file/dir within reach of the snapshots)
You can find good Information about Btrfs in the wiki