Rsync (Remote Sync) is the most commonly used command for copying and synchronizing files and directories remotely as well as locally in Linux/Unix systems.
With the help of the rsync command, you can copy and synchronize your data remotely and locally across directories, disks, and networks, perform data backups, and mirror between two Linux machines.
This article explains 10 basic and advanced usage of the rsync command to transfer your files remotely and locally in Linux-based machines. You don’t need to be a root user to run the rsync command.
Some Advantages and Features of Rsync Command
- It efficiently copies and sync files to or from a remote system.
- Supports copying links, devices, owners, groups, and permissions.
- It’s faster than scp (Secure Copy) because rsync uses a remote-update protocol which allows transferring just the differences between two sets of files. The first time, it copies the whole content of a file or a directory from source to destination but from next time, it copies only the changed blocks and bytes to the destination.
- Rsync consumes less bandwidth utilization as it uses compression and decompression method while sending and receiving data on both ends.
The basic syntax of the rsync command
# rsync options source destination
Some common options used with rsync commands
- -v : verbose
- -r : copies data recursively (but don’t preserve timestamps and permission while transferring data.
- -a : archive mode, which allows copying files recursively and it also preserves symbolic links, file permissions, user & group ownerships, and timestamps.
- -z : compress file data.
- -h : human-readable, output numbers in a human-readable format.
[ You might also like: How to Sync Files/Directories Using Rsync with Non-standard SSH Port ]
Install Rsync in Linux System
We can install the rsync package with the help of the following command in your Linux distribution.
$ sudo apt-get install rsync [On Debian/Ubuntu & Mint] $ pacman -S rsync [On Arch Linux] $ emerge sys-apps/rsync [On Gentoo] $ sudo yum install rsync [On Fedora/CentOS/RHEL and Rocky Linux/AlmaLinux] $ sudo zypper install rsync [On openSUSE]
1. Copy/Sync Files and Directory Locally
Copy/Sync a File on a Local Computer
The following command will sync a single file on a local machine from one location to another location. Here in this example, a file name backup.tar needs to be copied or synced to /tmp/backups/ folder.
[[email protected]]# rsync -zvh backup.tar.gz /tmp/backups/ created directory /tmp/backups backup.tar.gz sent 224.54K bytes received 70 bytes 449.21K bytes/sec total size is 224.40K speedup is 1.00
In the above example, you can see that if the destination is not already existed rsync will create a directory automatically for the destination.
Copy/Sync a Directory on Local Computer
The following command will transfer or sync all the files from one directory to a different directory in the same machine. Here in this example, /root/rpmpkgs contains some rpm package files and you want that directory to be copied inside /tmp/backups/ folder.
[[email protected]]# rsync -avzh /root/rpmpkgs /tmp/backups/ sending incremental file list rpmpkgs/ rpmpkgs/httpd-2.4.37-40.module_el8.5.0+852+0aafc63b.x86_64.rpm rpmpkgs/mod_ssl-2.4.37-40.module_el8.5.0+852+0aafc63b.x86_64.rpm rpmpkgs/nagios-4.4.6-4.el8.x86_64.rpm rpmpkgs/nagios-plugins-2.3.3-5.el8.x86_64.rpm sent 3.47M bytes received 96 bytes 2.32M bytes/sec total size is 3.74M speedup is 1.08
2. Copy/Sync Files and Directory to or From a Server
Copy a Directory from Local Server to a Remote Server
This command will sync a directory from a local machine to a remote machine. For example, there is a folder in your local computer “rpmpkgs” that contains some RPM packages and you want that local directory’s content sends to a remote server, you can use the following command.
[[email protected]:~]# rsync -avzh /root/rpmpkgs [email protected]:/root/ The authenticity of host '192.168.0.141 (192.168.0.141)' can't be established. ED25519 key fingerprint is SHA256:bH2tiWQn4S5o6qmZhmtXcBROV5TU5H4t2C42QDEMx1c. This key is not known by any other names Are you sure you want to continue connecting (yes/no/[fingerprint])? yes Warning: Permanently added '192.168.0.141' (ED25519) to the list of known hosts. [email protected]'s password: sending incremental file list rpmpkgs/ rpmpkgs/httpd-2.4.37-40.module_el8.5.0+852+0aafc63b.x86_64.rpm rpmpkgs/mod_ssl-2.4.37-40.module_el8.5.0+852+0aafc63b.x86_64.rpm rpmpkgs/nagios-4.4.6-4.el8.x86_64.rpm rpmpkgs/nagios-plugins-2.3.3-5.el8.x86_64.rpm sent 3.74M bytes received 96 bytes 439.88K bytes/sec total size is 3.74M speedup is 1.00
Copy/Sync a Remote Directory to a Local Machine
This command will help you sync a remote directory to a local directory. Here in this example, a directory /root/rpmpkgs which is on a remote server is being copied in your local computer in /tmp/myrpms.
[[email protected]:~]# rsync -avzh [email protected]:/root/rpmpkgs /tmp/myrpms [email protected]'s password: receiving incremental file list created directory /tmp/myrpms rpmpkgs/ rpmpkgs/httpd-2.4.37-40.module_el8.5.0+852+0aafc63b.x86_64.rpm rpmpkgs/mod_ssl-2.4.37-40.module_el8.5.0+852+0aafc63b.x86_64.rpm rpmpkgs/nagios-4.4.6-4.el8.x86_64.rpm rpmpkgs/nagios-plugins-2.3.3-5.el8.x86_64.rpm sent 104 bytes received 3.49M bytes 997.68K bytes/sec total size is 3.74M speedup is 1.07
3. Rsync Over SSH
With rsync, we can use SSH (Secure Shell) for data transfer, using SSH protocol while transferring our data you can be ensured that your data is being transferred in a secured connection with encryption so that nobody can read your data while it is being transferred over the wire on the internet.
[ You might also like: How to Secure and Harden OpenSSH Server ]
Also when we use rsync we need to provide the user/root password to accomplish that particular task, so using the SSH option will send your logins in an encrypted manner so that your password will be safe.
Copy a File from a Remote Server to a Local Server with SSH
To specify a protocol with rsync you need to give the “-e” option with the protocol name you want to use. Here in this example, We will be using the “ssh” with the “-e” option and perform data transfer.
[[email protected]:~]# rsync -avzhe ssh [email protected]:/root/anaconda-ks.cfg /tmp [email protected]'s password: receiving incremental file list anaconda-ks.cfg sent 43 bytes received 1.10K bytes 325.43 bytes/sec total size is 1.90K speedup is 1.67
Copy a File from a Local Server to a Remote Server with SSH
[[email protected]:~]# rsync -avzhe ssh backup.tar.gz [email protected]:/backups/ [email protected]'s password: sending incremental file list created directory /backups backup.tar.gz sent 224.59K bytes received 66 bytes 64.19K bytes/sec total size is 224.40K speedup is 1.00
[ You might also like: How to Use Rsync to Sync New or Changed/Modified Files in Linux ]
4. Show Progress While Transferring Data with rsync
To show the progress while transferring the data from one machine to a different machine, we can use the ‘–progress’ option. It displays the files and the time remaining to complete the transfer.
[[email protected]:/]# rsync -avzhe ssh --progress /root/rpmpkgs [email protected]:/root/rpmpkgs [email protected]'s password: sending incremental file list rpmpkgs/ rpmpkgs/httpd-2.4.37-40.module_el8.5.0+852+0aafc63b.x86_64.rpm 1.47M 100% 31.80MB/s 0:00:00 (xfr#1, to-chk=3/5) rpmpkgs/mod_ssl-2.4.37-40.module_el8.5.0+852+0aafc63b.x86_64.rpm 138.01K 100% 2.69MB/s 0:00:00 (xfr#2, to-chk=2/5) rpmpkgs/nagios-4.4.6-4.el8.x86_64.rpm 2.01M 100% 18.45MB/s 0:00:00 (xfr#3, to-chk=1/5) rpmpkgs/nagios-plugins-2.3.3-5.el8.x86_64.rpm 120.48K 100% 1.04MB/s 0:00:00 (xfr#4, to-chk=0/5) sent 3.74M bytes received 96 bytes 1.50M bytes/sec total size is 3.74M speedup is 1.00
5. Use of –include and –exclude Options
These two options allow us to include and exclude files by specifying parameters with these option helps us to specify those files or directories which you want to include in your sync and exclude files and folders with you don’t want to be transferred.
Here in this example, the rsync command will include those files and directory only which starts with ‘R’ and exclude all other files and directory.
[[email protected]:/]# rsync -avze ssh --include 'R*' --exclude '*' [email protected]:/var/lib/rpm/ /root/rpm [email protected]'s password: receiving incremental file list created directory /root/rpm ./ Requirename sent 61 bytes received 273,074 bytes 60,696.67 bytes/sec total size is 761,856 speedup is 2.79
6. Use of –delete Option
If a file or directory does not exist at the source, but already exists at the destination, you might want to delete that existing file/directory at the target while syncing.
We can use the ‘–delete‘ option to delete files that are not there in the source directory.
Source and target are in sync. Now create a new file test.txt at the target.
[[email protected]:~]# cd /root/rpm/ [[email protected]:~/rpm]# touch test.txt [[email protected]:~/rpm]# rsync -avz --delete [email protected]:/var/lib/rpm/ /root/rpm/ [email protected]'s password: receiving incremental file list deleting test.txt ./ .dbenv.lock .rpm.lock Basenames Conflictname Dirnames Enhancename Filetriggername Group Installtid Name Obsoletename Packages Providename Sha1header Sigmd5 Suggestname Supplementname Transfiletriggername Triggername __db.001 __db.002 __db.003 sent 445 bytes received 18,543,954 bytes 2,472,586.53 bytes/sec total size is 71,151,616 speedup is 3.84
Target has the new file called test.txt, when synchronizing with the source with the ‘–delete‘ option, it removed the file test.txt.
7. Set the Max Size of Files to be Transferred
You can specify the Max file size to be transferred or sync. You can do it with the “–max-size” option. Here in this example, the Max file size is 200k, so this command will transfer only those files which are equal to or smaller than 200k.
[[email protected]:~]# rsync -avzhe ssh --max-size='200k' /var/lib/rpm/ [email protected]:/root/tmprpm [email protected]'s password: sending incremental file list created directory /root/tmprpm ./ .dbenv.lock .rpm.lock Conflictname Enhancename Filetriggername Group Installtid Name Obsoletename Recommendname Requirename Sha1header Sigmd5 Suggestname Supplementname Transfiletriggername Triggername __db.002 sent 129.52K bytes received 396 bytes 28.87K bytes/sec total size is 71.15M speedup is 547.66
8. Automatically Delete source Files After Successful Transfer
Now, suppose you have the main web server and a data backup server, you created a daily backup and synced it with your backup server, now you don’t want to keep that local copy of backup in your web server.
So, will you wait for the transfer to complete and then delete that local backup file manually? Of Course NO. This automatic deletion can be done using the ‘–remove-source-files‘ option.
[[email protected]:~]# rsync --remove-source-files -zvh backup.tar.gz [email protected]:/tmp/backups/ [email protected]'s password: backup.tar.gz sent 795 bytes received 2.33K bytes 894.29 bytes/sec total size is 267.30K speedup is 85.40 [[email protected]:~]# ls -l backup.tar.gz ls: cannot access 'backup.tar.gz': No such file or directory
9. Do a Dry Run with rsync
If you are a newbie using rsync and don’t know what exactly your command going to do. Rsync could really mess up the things in your destination folder and then doing an undo can be a tedious job.
[ You might also like: How to Sync Two Apache Web Servers/Websites Using Rsync ]
Use of this option will not make any changes to the files and shows the output of the command, if the output shows exactly the same you want to do then you can remove the ‘–dry-run‘ option from your command and run on the terminal.
[[email protected]:~]# rsync --dry-run --remove-source-files -zvh backup.tar.gz [email protected]:/tmp/backups/ [email protected]'s password: backup.tar.gz sent 50 bytes received 19 bytes 19.71 bytes/sec total size is 267.30K speedup is 3,873.97 (DRY RUN)
10. Rsync Set Bandwidth Limit and Transfer File
You can set the bandwidth limit while transferring data from one machine to another machine with the the help of ‘–bwlimit‘ option. This option helps us to limit I/O bandwidth.
[[email protected]]# rsync --bwlimit=100 -avzhe ssh /var/lib/rpm/ [email protected]:/root/tmprpm/ [email protected]'s password: sending incremental file list sent 324 bytes received 12 bytes 61.09 bytes/sec total size is 38.08M speedup is 113347.05
Also, by default rsync syncs changed blocks and bytes only, if you want explicitly want to sync the whole file then you use the ‘-W‘ option with it.
[[email protected]]# rsync -zvhW backup.tar /tmp/backups/backup.tar backup.tar sent 14.71M bytes received 31 bytes 3.27M bytes/sec total size is 16.18M speedup is 1.10
That’s all with rsync now, you can see man pages for more options. Stay connected with Tecmint for more exciting and interesting tutorials in the future. Do leave your comments and suggestions.
Why I am getting 1 hour time difference from the original source file, whereas both target and source machine have the same times.
@Fasih,
It could be system time mismatch, set both server time to similar and see..
see whether executing “date -u” on both sides still show the same times.
I’m using rsync, talking to a rsyncd, to avoid encryption through ssh, as the rsyncing machines are low powered embedded boards. Consequently, the rsyncd provides modules to address the server directories, say, a module “backup”.
A client machine now attempts to sync some local directories to a corresponding path under the directory, specified by module on rsyncd machine, like:
But as long as $path doesn’t exist on the server, rsyncing $dir fails. I’m looking for a way to create $path on the server automatically when written to, akin to what mkdir -p does. Given that the canonical path on server is hidden behind the module name, letting client ssh to a server in order to create the directory prior to rsync is something I want to avoid.
I haven’t found an rsync option allowing the creation of destination directories on a needs base. Do you have any suggestion how to solve this case, short of manually creating destination directories prior to first use?
I was thinking of some intricate scheme of iterating through parent directories, rsyncing each level of parent directories while excluding all siblings until decided to the source directory. While this could possibly work (untested), I doubt that this effort is actually meant to be necessary for this presumably rather common use case.
Thank you for any suggestion.
I’m now considering a refinement to the thought of method above: Rather than iterating through directories, rsyncing each level separately for the creation of a single directory level, I’m now considering to try creating the whole path in a temporary directory, rsyncing all of those in one go, then removing the temporary directories again.
In consequence, the whole path has been created on the server, ready to commence the actual data transfer to its deepest level directory. Less effort already, but still rather Rube Goldbergish – more ideas still welcome.
I’ve now, at least for the time being, settled for the described approach:
* creation of temporary directory (mktemp -d)
* creation of wanted path in temp directory (mkdir -p)
* rsync top level directory of wanted path from temporary directory (rsync – this creates wanted the directory hierarchy on server)
* removal of temporary directory (rm -r)
* rsync $path dest::module/$path # as destination directory already exists, rsync proceeds without failure.
while probably not the most elegant method, the additional complexity is manageable: no iterating through path components, or other path decomposition. just plain use of path components already existing for scripting purposes. So unless a better (working) idea pops up, I’m happy for now. Thank you all for bearing with me.
The trick is to put trailing slashes on the paths:
Directory “source” will be named “target” at the receiving system. Provide the exact name you want.
Actually, there is a trailing slash in the source path, which got hidden behind using the variable $dir. A copy and paste of an actual and complete call are here (well, not quite actual, I simplified options a bit, but the result is the same):
Using or not using a trailing slash makes no difference here. rsync succeeds as soon the directory vpn/home/l/read is manually created under the directory representing module backup. This is the case regardless whether the destination is specified by module name or by canonical name over ssh.
Another thought came up which I may have to try out: Exclude everything by option, then include the topmost directory of the wanted path. Current “solution” is still to create destination directories manually when necessary.
Regarding about my previous comment: “then include the topmost directory” won’t do, as this transfers the unwanted siblings too. The whole path, down to deepest source directory, should be needed.
I don’t know why it differs for you. I have no trouble with this and I do it all the time (Linux). Here’s one I just used last weekend, worked like a charm.
Hi,
If I set rsync the folder of the files will get automatically change or need to set some cron for that? Please help me out.
@Vignesh,
To auto sync files/folders you must set cron for rsync.
You could have a look at lsyncd, which watches for modification on files through inotify, then transfers the modified files using rsync upon modification.
An alternative could be incrontab to do essentially the same. Latter is a more general approach, as incrontab can be used for other actions than rsync. Both methods will allow you to monitor changes and rsync those automatically shortly after they took place.
It seems comments get edited prior to being published. I noticed earlier that words were edited, unfortunately introducing errors that way. Same here: I originally wrote “incrontab“, not “in crontab”. While cron merely executes by time specification, incrontab allows inotify event specification, which is what I was actually referring to: https://linux.die.net/man/1/incrontab
@Paul,
Yes, comments were edited only if needed, but sorry I mistook it incrontab as “in crontab“. Thanks for clarification about incrontab, never heard before.
Thanks you for correcting my original message another time. Please change also:
“An alternative could be crontab to do essentially the same. Later is a more general approach, as in crontab can be used for other actions than rsync”
to
“An alternative could be incrontab to do essentially the same. Latter is a more general approach, as incrontab can be used for other actions than rsync”
In earlier messages, I was several times wondering already why I have overlooked evidential mistakes when rereading prior to sending – I initially thought that some autocorrection went wrong, and have the impression that editing introduces more mistakes than it fixes.
@Paul,
Sorry for trouble, corrected the line in the last comment as suggested..
Hello, is there any way to produce an exit code if it sees that source and destination are not the same while using the option –dry-run? Thanks
@Jeff,
I think you should read this article, that shows how to sync only new or changed files over rsync.
https://www.tecmint.com/sync-new-changed-modified-files-rsync-linux/
When rsyncing from local to local, compressing only slows it down. Skip the -z option if you’re backing a local disk up to a local disk, because it’s just reading the whole file, compressing it, decompressing it, and writing the whole file. No need for the compression middleman if the sync isn’t over a network.
@Bill,
Thanks for the tip, didn’t know this, actually I never used rsync for local backups, I always used to sync remote servers with local, anyway thanks hope this will help other users who used to backup files locally..
Hi Ravi,
I have tried as per your suggestion but it’s not working as my requirement. still, am able to copy the timestamp which has presented on the source .
Thanks for your reply.
As per your above command, it doesn’t meet my requirement. I guess ‘n’ argument is dry-run (not do any file transfers, instead it will just report the actions it would have taken.)
Please help me on this and I need only copy changed folders in source and don’t want copy the time stamp which is not copied and changed on destination.
Hi,
With Rsync command I want only copy/sync changed files/folder to the destination folder. I had some issue with Rsync-like whenever am executing the Rsync command am copied along with the time stamp as well.
For example, my destination folder ‘Linux’ had updated on 30th June and in my source folder there is no update info for the Linux folder, but when am performing the Rsync command my destination Linux folder time stamp has been updated with source folder time stamp. I don’t want to copy the timestamp as well. please suggest me on this and please glance on below command which I have used.
rsync -avh /source/Linux/ /destination/Linux/
@Raja,
You can use following command to only sync new or changed files over rsync to destination folder:
Thanks for your reply.
As per your above command, it does not work because the ‘n’ argument tells the dry-run (not do any file transfers, instead, it will just report the actions it would have taken)
For your info am doing this Rsync with two directories present on the same server. Please help me on this.
Simply, need to copy only latest changed files/folders from source to destination.(don’t want to copy the timestamp which has not changed in source)
@Raja,
Yes, the
-noption is used to check the files, and once you confirm that the files are listed correctly on the dry-run, remove the -n option and run:Hi Ravi,
I have tried as per your suggestion but it’s not working as my requirement. still, am able to copy the timestamp which has presented on the source .
Thanks for your reply.
As per your above command, it doesn’t meet my requirement. I guess ‘n’ argument is dry-run (not do any file transfers, instead it will just report the actions it would have taken.)
Please help me on this and I need only copy changed folders in source and don’t want copy the time stamp which is not copied and changed on destination.
Hi Ravi,
You mentioned in one of your examples that you could transfer the contents from source to destination securely over ssh when you use the “-e” option and specify ssh. May i know that over what protocols is the transfer done when not using ssh option?
@Gaurav,
Yes, I do mentioned that you can transfer files from source to another securely using SSH, but still it all depends on which protocol you configured under sshd_config file, I suggest to use Protocol 2 for more better security in SSH configuration file..
rsync can talk to remote rsync daemon without tunneling through ssh
rsync works great but how do I get it to keep using the entire network bandwidth for really large files, 1tb+? I’m hitting 200mbs a sec over a 10gb ethernet connection in the beginning but sometimes it slows to 20mbs for no apparent reason and stays there.
Is there a way to have it check or do an ack reset to re-negotiate the link speed through the switch?
@Daniel,
The slowness happens because of file encryption during transfer files over SSH, if you have that much of large data, you can reduce the encryption level or use other alternative tool like parsync (a rsync wrapper for larger data transfer).
@Daniel, for the same reason, do I now run rsync as daemon server side, launched through xinetd. Now rsync can talk to rsyncd without ssh, and throughput went up considerably, while CPU load dropped.
Security is of no concern in my case, as transfers take place in a trusted LAN, or go through a VPN – in which case I merely avoided double encryption, by SSH in addition to by VPN).
A new problem was then introduced though, for which I have currently no proper solution – I wrote about this in another post just a few minutes ago in another comment here, asking for suggestions.
@Daniel, regrettably I avoided addressing your actual issue: bandwidth limiting. Well, that’s possible with rsync running as daemon too, by use of the option –bwlimit=RATE. through this setting can you control the maximum amount of data transferred per time unit.