Of all the countless tools and utilities available in the Linux and UNIX world, few are as beloved as rsync. rsync is a utility that you can use to copy data from one place to another very easily, and there are many options available to allow you to be very specific about how you want the data transferred. Examples of its many use cases include copying files while preserving permissions, copying files while backing up replaced files, and even setting up incremental backups. If you don't already know how to use rsync, you'll probably want to get lots of practice with it, as it's something you'll soon see will be indispensable during your career as a Linux administrator, and it is also something that the Linux community generally assumes you already know. rsync is not hard to learn. Most administrators can learn the basic usage in about an hour, but the countless options available will lead you to learn new tricks even years down the road.

Another aspect that makes rsync flexible is the many ways you can manipulate the source and target directories. I mentioned earlier that rsync is a tool you can use to copy data from one place to another. The beauty of this is that the source and target can literally be anywhere you'd like. For example, the most common usage of rsync is to copy data from a directory on one server to a directory on another server over the network. However, you don't even have to use the network; you can even copy data from one directory to another on the same server. While this may not seem like a useful thing to do at first, consider that the target directory may be a mount-point that leads to a backup disk, or an NFS share that actually exists on another server. This also works in reverse: you can copy data from a network location to a local directory if you desire.

To get started with practicing with rsync, I recommend that you find some sample files to work with. Perhaps you have a collection of documents you can use, MP3 files, videos, text files, basically any kind of data you have lying around. It's important to make a copy of this data. If we make a mistake we could overwrite things, so it's best to work with a copy of the data, or data you don't care about while you're practicing. If you don't have any files to work with, you can create some text files. The idea is to practice copying files from one place to another; it really doesn't matter what you copy or to where you send it. I'll walk you through some rsync examples that will progressively increase in complexity. The first few examples will show you how to backup a home directory, but later examples will be potentially destructive so you will probably want to work with sample files until you get the hang of it.

Here's our first example:

sudo rsync -r /home/myusr /backup 

With that command, we're using rsync (as root) to copy the contents of the home directory for the myuser directory to a backup directory, /backup (make sure the target directory exists). In the example, I used the -r option, which means rsync will grab directories recursively as well (you should probably make a habit of including this if you use no other option). You should now see a copy of the myuser home directory inside your /backup directory.

However, we have a bit of a problem. If you look at the permissions in the /backup/myuser directory, you can see that everything in the target is now owned by root. This isn't a good thing; when you back up a user's home directory, you'll want to retain their permissions. In addition, you should retain as much metadata as you can, including things like timestamps. Let's try another variation of rysnc. Don't worry about the fact that /backup already has a copy of the myuser home directory from our previous backup. Let's perform the backup again, but instead we'll use the -a option:

sudo rsync -a /home/myuser /backup 

This time, we replaced the -r option with -a (archive), which retains as much metadata as possible (in most cases, it should make everything an exact copy). What you should notice now is that the permissions within the backup match the permissions within the user's home directory we copied from. The timestamps of the files will now match as well. This works because whenever rsync runs, it will copy what's different from the last time it ran. The files from our first backup were already there, but the permissions were wrong. When we ran the second command, rsync only needed to copy what was different, so it applied the correct permissions to the files. If any new files were added to the source directory since we last ran the command, the new or updated files would be copied over as well.

The archive mode (the -a option that we used with the previous command) is actually very popular; you'll probably see it a lot in the industry. The -a option is actually a wrapper option that includes the following options all at the same time:

-rlptgoD 

If you're curious about what each of these options do, consult the man page for rsync for more detailed information. In summary, the -r option copies data recursively (which we already know), the -l option copies symbolic links, -p preserves permissions, -g preserves group ownership, -o preserves the owner, and -D preserves device files. If you put those options together, we get -rlptgoD. Therefore, -a is actually equal to -rlptgoD. I find -a easier to remember.

The archive mode is great and all, but wouldn't it be nice to be able to watch what rsync is up to when it runs? Add the -v option and try the command again:

sudo rsync -av /home/myuser /backup 

This time, rsync will display on your Terminal what it's doing as it runs (-v activates the verbose mode). This is actually one of my favorite variations of the rsync command, as I like to copy everything and retain all the metadata, as well as watch what rsync is doing as it works.

What if I told you that rsync supports SSH by default? It's true! Using rsync, you can easily copy data from one node to another, even over SSH. The same options apply, so you don't actually have to do anything different other than point rsync to the other server, rather than to another directory on your server:

sudo rsync -av /home/myuser [email protected]:/backup 

With this example, I'm copying the home directory for myuser to the /backup directory on server 192.168.1.5. I'm connecting to the other server as the admin user. Make sure you change the user account and IP address accordingly, and also make sure the user account you use has access to the /backup directory. When you run this command, you should get prompted for the SSH password as you would when using plain SSH to connect to the server. After the connection is established, the files will be copied to the target server and directory.

Now, we'll get into some even cooler examples (some of which are potentially destructive), and we probably won't want to work with an actual home directory for these, unless it's a test account and you don't care about its contents. As I've mentioned before, you should have some test files to play with. When practicing, simply replace my directories with yours. Here's another variation worth trying:

sudo rsync -av --delete /src /target 

Now I'm introducing you to the --delete option. This option allows you to synchronize two directories. Let me explain why this is important. With every rsync example up until now, we've been copying files from point A to point B, but we weren't deleting anything. For example, let's say you've already used rsync to copy contents from point A to point B. Then, you delete some files from point A. When you use rsync to copy files from point A to point B again, the files you deleted in point A won't be deleted in point B. They'll still be there. This is because by default, rsync copies data between two locations but it doesn't remove anything. With the --delete option, you're effectively synchronizing the two points, thus you're telling rsync to make them the same by allowing it to delete files in the target that are no longer in the source.

Next, we'll add the -b (backup) option:

sudo rsync -avb --delete /src /target 

This one is particularly useful. Normally, when a file is updated on /src and then copied over to /target, the copy on /target is overwritten with the new version. But what if you don't want any files to be replaced? The -b option renames files on the target that are being overwritten, so you'll still have the original file. If you add the --backup-dir option, things get really interesting:

sudo rsync -avb --delete --backup-dir=/backup/incremental /src /target 

Now, we're copying files from /src to /target as we were before, but we're now sending replaced files to the /backup/incremental directory. This means that when a file is going to be replaced on the target, the original file will be copied to /backup/incremental. This works because we used the -b option (backup) but we also used the --backup-dir option, which means that replaced files won't be renamed, they'll simply be moved to the designated directory. This allows us to effectively perform incremental backups.

Building on our previous example, we can use the Bash shell itself to make incremental backups work even better. Here, we have two commands:

CURDATE=$(date +%m-%d-%Y) 
export $CURDATE 
sudo rsync -avb --delete --backup-dir=/backup/incremental/$CURDATE /src /target 

With this example, we grab the current date and set it to a variable (CURDATE). We'll also export the new variable so that it is available everywhere. In the rsync portion of the command, we use the variable for the --backup-dir option. This will copy replaced files to a backup directory named after the date the command was run. Basically, if today's date was 08-17-2018, the resulting command would be the same as if we run the following:

sudo rsync -avb --delete --backup-dir=/backup/incremental/08-17-2018 /src /target 

Hopefully, you can see how flexible rsync is and how it can be used to not only copy files between directories and/or nodes, but also to serve as a backup solution as well (assuming you have a remote destination to copy files to). The best part is that this is only the beginning. If you consult the man page for rsync, you'll see that there are a lot of options you can use to customize it even further. Give it some practice, and you should get the hang of it in no time.