Hello,
I hope i post this in the correct subforum. For a bit over a week I’m now observing a strange behavior with rsync. I’m using a little script to regularly back up my media files to a NAS using the following command. rsync -avzh --delete $drive1/tv /run/media/$user/MyCloudPublic/Shared\ Videos/Movies/
and that script has been working fine for about 2 years, and it is still doing the copying part fine. What I now do observe, is that despite the use of “-a” flag rsync is now repeatedly copying the files added within the last 2 weeks, despite there being no changes to it.
My question is do other rsync users have been observing a similar behavior, or is this something I need to look for the cause in my own system.
Changes to my system, have been the 2 recent stable updates, and changing to the 5.13.5 kernel last week.
If you have rsync create a log file (and specify the “info” to dump), you can view it after a successful sync and see “why” it transfered the file(s) over.
I recommend changing –delete to –delete-delay to improve performance.
Thank you for the sugggestion, I have been running this with a log on various levels of detail. Regrettably I don’t see the why: 2021/07/31 14:05:01 [153157] >f..tp.g... video/western...
making a bold assumption that
f stands for file
t indicating the transfer
of files. the p and g i have no idea, but saw those also on not transferred files. So i will clearly have to do more research on this.
If you have any further hints I would dearly appreciate them.
Based on that log entry, rsync’s rationale for transferring the file over is because there’s a difference with its “timestamp” (t), “permissions” (p), and “group ownership” (g).
Something might be up on the server and/or you should consider switching to a more “proper” method of using rsync over the network: either SSH or the rsyncd (daemon) method, if your NAS server supports it.
I don’t recommend rsync over an exported Samba share or GVFS.
All files populated on the log are transfers/modifications/deletions/errors. Under the presumption that “t” stands for “transfer” further adds confusion: “t” actually stands for "timestamp"
Thank you Winnie, your input is going to help me straighten this out.
I have to admit that the p ang does surprise me, as the archive function in rsync is meant to specifically maintain such information.
I was not able to figure out what suddenly changed to cause the difference in timestamp and permission and ownership. But including the -u flag satisfactorily returned this to working for me.
I think its best to understand why the timestamps and modifications are not being honored.
How are you transfering to the destination? Based on your initial post, it appears you’re using rsync over an SMB-shared folder over the network (not recommended.) This could potentially yield issues with permissions, ownership, and timestamps.
The preferable method is to either use rsync over ssh (default) or daemon (server-side) mode.
So, for example, using it over ssh (default) might look something like this:
You can use the server’s IP address (or hostname, if configured properly).
If you don’t want it to prompt for your user’s password (the user’s password of the server, not the local machine), then you just need to create an SSH keypair. (The user on the NAS also needs access to use SSH. It depends on your NAS and the default settings. Most allow it by default.)
If your NAS server suports Rsync over daemon mode (aka “modules”), it’s even faster than SSH mode, since the server generates the directory tree in parallel with the client.
The way you’re using it, rsync “believes” it’s being used locally (not over the network), since a network shared folder is abstracted as a local directory on your system.