rsync : File Synchronization and File Transfer program
What is rsync command in Unix based system?
– Rsync, which stands for “remote sync”, is a remote and local file synchronization tool.
– It uses an algorithm that minimizes the amount of data copied by only moving the portions of files that have changed.
– Rsync is a widely-used utility to keep copies of a file on two computer systems the same. It is commonly found on Unix-like systems and functions as both a file synchronization and file transfer program.
– The rsync algorithm, a type of delta encoding, is used to minimize network usage. Zlib may be used for additional compression, and SSH or stunnel can be used for data security.
– Rsync can also operate in a daemon mode, serving files in the native rsync protocol (using the “rsync://” syntax).
– Due to its ubiquity on Linux and Unix-like systems and its popularity as a tool for system scripts, it is included on most Linux distributions by default.
How it works?
Rsync is typically used to synchronize files and directories between two different systems. For example, if the command rsync local-file user@remote-host:remote-file is run, rsync will use SSH to connect as user to remote-host. Once connected, it will invoke the remote host’s rsync and then the two programs will determine what parts of the file need to be transferred over the connection.
How to use?
Similar to rcp and scp, rsync requires the specification of a source and of a destination; either of them may be remote, but not both. Because of the flexibility, speed and scriptability of rsync, it has become a standard Linux utility, included in all popular Linux distributions. It has been ported to Windows (via Cygwin, Grsync or SFU) and Mac OS.
Generic syntax:
rsync [OPTION] … SRC [SRC] … [USER@]HOST:DEST rsync [OPTION] … [USER@]HOST:SRC [DEST]
…where SRC is the file or directory (or a list of multiple files and directories) to copy from, and DEST represents the file or directory to copy to. (Square brackets indicate optional parameters.)
By default, rsync uses the remote-shell program SSH for its communication. It can be configured to use a different remote-shell program, or to contact an rsync daemon directly via TCP, which per default then is via TCP port 873.
Explanation of rsync:
Let’s suppose we have two directory – dir1 and dir2.
To sync the contents of dir1 to dir2 on the same system, type:
rsync -r dir1/ dir2
The -r option means recursive, which is necessary for directory syncing.
We could also use the -a flag instead:
rsync -a dir1/ dir2
The -a option is a combination flag.
It stands for “archive” and syncs recursively and preserves symbolic links, special and device files, modification times, group, owner, and permissions.
It is more commonly used than -r and is usually what you want to use.
Note:-
– Always double-check your arguments before executing an rsync command.
– You may have noticed that there is a trailing slash (/) at the end of the first argument in the above commands:
rsync -a dir1/ dir2
This is necessary to mean “the contents of dir1“.
Rsync to Sync with a Remote System
Syncing to a remote system is trivial if you have SSH access to the remote machine and rsync installed on both sides.
Once you have SSH access verified on between the two machines, you can sync the dir1 folder from earlier to a remote computer by using this syntax (note that we want to transfer the actual directory in this case, so we omit the trailing slash):
rsync -a ~/dir1 username@remote_host:destination_directory
This is called a “push” operation because it pushes a directory from the local system to a remote system.
The opposite operation is “pull”. It is used to sync a remote directory to the local system. If the dir1 were on the remote system instead of our local system, the syntax would be:
rsync -a username@remote_host:/home/username/dir1 place_to_sync_on_local_machine
Like “cp” and similar tools, the source is always the first argument, and the destination is always the second.
If you are transferring files that have not already been compressed, like text files, you can reduce the network transfer by adding compression with the -z option:
rsync -az source destination
The -P flag is very helpful. It combines the flags –progress and –partial. The first of these gives you a progress bar for the transfers and the second allows you to resume interrupted transfers:
rsync -azP source destination
If you wish to exclude certain files or directories located inside a directory you are syncing, you can do so by specifying them in a comma-separated list following the –exclude= option:
rsync -a –exclude=pattern_to_exclude source destination
If we have specified a pattern to exclude, we can override that exclusion for files that match a different pattern by using the –include= option.
rsync -a –exclude=pattern_to_exclude –include=pattern_to_include source destination
