others-how to copy(or cp) files recursively with exclusions in linux system ?

Mar 3, 2021

Problem

How to copy or cp files from source to destination recursively with some exclusions in linux operating system?

For example, I have a directory named A as follows:

➜  tree A
A
├── DEPENDENCIES
├── a.log
├── b.log
├── c.log
└── maven
    └── org.apache.httpcomponents
        └── httpclient
            ├── pom.properties
            └── pom.xml
3 directories, 6 files

Now I want to copy directory A to directory B recursively ,but I don’t want to copy the log files to B.

Environment

Linux or MacOS

Solution

We can use rsync to solve this problem:

	rsync -avr --exclude='*.log' A/* B

After run the above command, let’s check the destination directory B:

B
├── DEPENDENCIES
└── maven
    └── org.apache.httpcomponents
        └── httpclient
            ├── pom.properties
            └── pom.xml

3 directories, 3 files

It works!

How it works?

The ‘rsync’ command can not only sync with remote host’s directory but also sync with localhost directory.

Let’s check the three options used in above command:


-v, --verbose               increase verbosity

-a, --archive               archive mode; equals -rlptgoD (no -H,-A,-X)
                            This is equivalent to -rlptgoD. It is a quick way of saying you want recursion and want to preserve almost everything (with -H being a notable omission). 

-r, --recursive             recurse into directories

The ‘–exclude’ can be used with patterns:

--exclude=PATTERN

This option is a simplified form of the --filter option that defaults to an exclude rule and does not allow the full rule-parsing syntax of normal filter rules.

Aside from –exclude , you can also use –exclude-from / –include / –include-from:

--exclude-from=FILE
This option is related to the --exclude option, but it specifies a FILE that contains exclude patterns (one per line). Blank lines in the file and lines starting with oq;cq or oq#cq are ignored. If FILE is -, the list will be read from standard input.

--include=PATTERN
This option is a simplified form of the --filter option that defaults to an include rule and does not allow the full rule-parsing syntax of normal filter rules.
See the FILTER RULES section for detailed information on this option.

--include-from=FILE
This option is related to the --include option, but it specifies a FILE that contains include patterns (one per line). Blank lines in the file and lines starting with oq;cq or oq#cq are ignored. If FILE is -, the list will be read from standard input.

For example, if you want to exclude some types of files, just create a file named exclude.list

Content of exclude.list:

a
b.*
tmp/g

then run this command:

rsync -av --exclude-from=exclude.list /home/mysql/backup /home/mysql/backup2/