Rsync with --exclude does not work

Support
1

I have also looked up for the example
alternative options for using --exclude
Trying to exclude a folder or a file from synchronising with rsync none of these command do work with me.

rsync -av --exclude=/path/foldertobeexcluded /origin/path/ /destination/path/
rsync -av --exclude={/path/foldertobeexcluded} /origin/path/ /destination/path/
rsync -av --filter=- /path/foldertobeexcluded /origin/path/ /destination/path/
rsync -av -f - /path/foldertobeexcluded /origin/path/ /destination/path/
2

These aren’t full (complete) commands, right?

3

The names of the paths are not the original ones but the command should be fine.

4

Nevermind. What does “do not work mean”? Post output if any.

Works fine with

rsync -av --exclude '*.whatever' source dest
1 Like
5

I have always used the option
--exclude-from
which takes a file name in which the patterns or the path/file names are written

In the manual page, the option is described as:
--exclude=PATTERN
or:
--exclude-from=FILE

I never needed to exclude just one file or folder - thus the --exclude-from=... seemed easier, because the file with the patterns is easy to maintain and change
while the command line doesn’t need to be changed.

6

Neither this works

Nor this

Always the excluded folder is also copied / synchronized to the destination, too.

7

You seem to be using an absolute path. It’s a pattern, so you’d need a relative path or a pattern.

rsync -avn --exclude={"Documents/*","Pictures","Videos"} ~/ /tmp/test/
1 Like
8

it’s a pattern - not a file name
…or: maybe you need to ‘escape’ it somehow

As I said:
I had difficulty with this as well and decided to use:
--exclude-from=FILE
and write the pattern there
since I never needed to exclude just one file or directory

here is the content of one such excludelist I used for system backups to exclude certain directories from the backup:

/home/*
/dev/*
/proc/*
/sys/*
/tmp/*
/run/*
/mnt/*
/media/*
/lost+found
/var/cache/pacman/pkg/*
/var/cache/squid/*

In this case, the file was called “excludelist-system” (which is totally irrelevant)
it was located in my /home/user directory (which is excluded from the backup)
The backup disk was mounted to somewhere in /run - which is, of course, also excluded,
else there will be infinite recursion …

2 Likes
9

Yes, this may be an issue. I am using absolute paths. But trying to use relative paths also does not work. I do not understand. Using this examples does not work.

rsync -av --exclude=foldertobeexcluded/* /origin/path/ /destination/path/

foldertobeexcluded/* is in the source folder /source/foldertobeexcluded/ like this.

Would you give a really step by step exact description, please?
Like the full command, the name of the Exclude-file and its path where you need to place it. Or can you place it anywhere on your system?

10

… anywhere - but it makes sense to just put it where you have easy access to - like I did: in my home directory

You already have the contents of the file - see above.

Filename: excludelist-system
Location: /home/nachlese/excludelist-system
pwd: /home/nachlese
Drive the backup goes to is mounted at /mnt

command (as root or with sudo - because this is a system backup, files and folders I don’t own):

rsync -a --progress --exclude-from=/home/nachlese/excludelist-system / /mnt/drive_name/system-backup-datum_von_heute

That was a full system backup excluding /home
and everything else I knew I didn’t need or want
… and: it makes no sense to try to back up /dev or /proc or /sys or /run … that is why these are excluded

11

Please describe/show the results, “does not work” is not very helpful.

Assuming /source is the same as /origin/path, the example should copy the directory but not the contents.

Does the directory name contain spaces or other special chars?
Error messages?

It would help if we could see the actual command, or at least a working example that exhibits the same problem. Perhaps create a test directory structure, so you don’t mind showing us what’s going on.

EDIT:

To clarify, I said relative paths because they can be simple patterns.

12

dmt was writing I should use relative paths but you are using absolute ones in your file excludelist-system?

Are you using a password or what means pwd?

13

it’s a command
it prints the working directory
it was to show you “where I was” when I ran that command.

I see no contradiction - in this case the pattern starts with the absolute path.
… it works like this :man_shrugging:

Try looking in man rsync for even some examples …
SHIFT+/ (that is: Shift and / at the same time) invokes the search function
“n” and “p” for next and previous result

14 
rsync -av --progress --exclude-from=/home/user/excludelist-system /run/media/omlinux/Daten/DDQT/Ausbildungsordnung/ /run/media/user/Daten/x-temp/

/home/user/excludelist-system has this content

/run/media/user/Daten/DDQT/Ausbildungsordnung/kollegiales-Gespraech/*

OR

rsync -av -exclude=kollegiales-Gespraech/ /run/media/user/Daten/DDQT/Ausbildungsordnung/ /run/media/user/Daten/x-temp/

OR

rsync -av -exclude-from=/run/media/user/Daten/DDQT/Ausbildungsordnung/kollegiales-Gespraech/ /run/media/user/Daten/DDQT/Ausbildungsordnung/ /run/media/user/Daten/x-temp/

They all copy also the folder /run/media/user/Daten/x-temp/kollegiales-Gespraech/ including its content. But this I do not want!!!

15

With relation to the topic title - that is not correct

You will need to use the correct syntax.

This snippet from my mirror sync script works just fine

rsync -rtlvH --safe-links 
    --delete-after --progress \
    --timeout=600 --contimeout=120 -p \
    --delay-updates --no-motd \
    --exclude='*.links.tar.gz*' \
    ${SOURCE} \
    "${TARGET}"
1 Like
16

… maybe later - experiment a bit

Does this work?
rsync -av -exclude='kollegiales-Gespraech/' /run/media/user/Daten/DDQT/Ausbildungsordnung/ /run/media/user/Daten/x-temp

or this?
cd /run/media/user/Daten/DDQT/Ausbildungsordnung/
and then:
rsync -av -exclude='kollegiales-Gespraech/' /run/media/user/Daten/DDQT/Ausbildungsordnung/ /run/media/user/Daten/x-temp

this:

can’t work, because the option --exclude-from needs a file name, not just the directory where the actual file is
…
and, as @dmt just noticed, but I didn’t - the syntax was incorrect :wink:

1 Like
17

That should be --exclude, using -exclude results in the behaviour you describe.

EDIT:
I saw it earlier, but @linux-aarhus got there first. However it’s easy to miss. I was still wondering if it was a typo on the forum, so I checked to see if the behaviour fit. :man_shrugging:

18

Thanks to ALL of you for your support and your patience. I really appreciate it.

You are right. I missed it. But using the correct --exclude still did not work.

THIS WORKS with --exclude=‘nameofthefolder’ !!!

I have checked also this way with a file for excluding several folders: --exclude-from=/pathwherefileislocated/nameoftheexcludefile

rsync -av --progress --exclude-from=/home/user/excludelist-system /run/media/omlinux/Daten/DDQT/Ausbildungsordnung/ /run/media/user/Daten/x-temp/

It works with file content of excludelist-system. The complete folder “kollegiales-Gespraech” will not be transferred at all.

kollegiales-Gespraech/

or

kollegiales-Gespraech

or

/kollegiales-Gespraech

or

/kollegiales-Gespraech/

but not with

kollegiales-Gespraech/*

This snippet I do not understand. I would need the whole “context” and explanation for understanding the correct use.

@Nachlese
How would be the path syntax if I would only like to exclude a sub-subfolder? E.g.

rsync -av --exclude='kollegiales-Gespraech/specialcontent/' /run/media/user/Daten/DDQT/Ausbildungsordnung/ /run/media/user/Daten/x-temp

where ‘kollegiales-Gespraech/specialcontent/’ in the sub-sub-folder should be excluded. I have tried it out but it did not work with such a path.

Working with --filter=merge functions may also be worth trying out!

19

rsync is a strange beast sometimes

But because of its power, it’s worth dealing with. I’ve been using it for well over 20 years. And whenever something didn’t work, it was “my fault” and not rsync’s. The pattern matching of rsync to the path names always required a series of detailed tests for me.

As I recall, rsync offers an option to disclose all of its decisions (which files to include and which not, and why) via “verbose” debug=FILTER. This helped me to find my mistakes myself.

But you need the appropriate data and directories to test it :man_shrugging:

Some excerpts from the manpage:

 --exclude-from=FILE
              This  option  is  related to the --exclude option, but it specifies a FILE that contains exclude patterns (one per line).  Blank lines in the file are ig‐
              nored, as are whole-line comments that start with ';' or '#' (filename rules that contain those characters are unaffected).

              If a line begins with "- " (dash, space) or "+ " (plus, space), then the type of rule is being explicitly specified as an exclude or an  include  (respec‐
              tively).  Any rules without such a prefix are taken to be an exclude.

              If a line consists of just "!", then the current filter rules are cleared before adding any further rules.

              If FILE is '-', the list will be read from standard input.

  --exclude=PATTERN
              This option is a simplified form of the --filter option that specifies an exclude rule and does not allow the full rule-parsing syntax  of  normal  filter
              rules.  This is equivalent to specifying -f'- PATTERN'.

              See the FILTER RULES section for detailed information on this option.

FILTER RULES

When  you  are  working  on  the  rules,  it can be helpful to ask rsync to tell you what is being excluded/included and why.  Specifying --debug=FILTER or (when
       pulling files) -M--debug=FILTER turns on level 1 of the FILTER debug information that will output a message any time that a file or directory is included or  ex‐
       cluded  and  which  rule  it matched.  Beginning in 3.2.4 it will also warn if a filter rule has trailing whitespace, since an exclude of "foo " (with a trailing
       space) will not exclude a file named "foo".

--debug=FILTER

Please read PATTERN MATCHING RULES carefully !

Be aware:

  • xxxx/ (exclude directory xxxx/ → A pattern that ends with a / only matches a directory not a regular file, symlink, or device. )
    is not the same as
  • xxxx/* (exclude files directly in xxxx/* but not in xxxx/yyyy/*)
    or
  • xxxx/** (exclude all files under xxxx/ )
    or
  • xxxx/*** (exclude all files under xxxx/ and xxxx itself )
    or
  • /xxxx (is anchored to the start of the transfer path (not at /(fs-root)) instead of the end )
    or
  • xxxx (excludes xxxx and everything inside (if xxxx is a dir))
If the pattern doesn't contain a (non-trailing) / or a "**", then it is matched only against the final  component  of  the filename or pathname.

or

  • /xxxx/ (find out yourself :wink: )

So some of your patterns do only work by chance, because you are in the right place when starting rsync

20

Seems eerily familiar… :grin: