Rsync backups are versatile. You can can create full, differential and incremental backups of a directory. Rsync can backup files locally or remotely.
Local Backups
If you are backing up to another directory, removable storage, or a locally mounted network file system, then you will be creating local backups.
Creating Local Backups
To create a local backup, run rsync with the relevant options, source directory, and destination directory.
The following example demonstrates backing up a directory named stuff to a directory named backup.
tyler@desktop:~/rsync$ rsync -avh stuff/ backup/The trailing slashes matter. If you leave off the slash after stuff, a directory named stuff will be created in backup. In the example, backup will be a clone of stuff.
Notice how stuff is copied into backup when the trailing slash is left off.
tyler@desktop:~/rsync$ rsync -ah stuff backup/
tyler@desktop:~/rsync$ ls backup/
stuffWhen the slash is included, just the files in stuff will be copied to backup.
tyler@desktop:~/rsync$ rsync -ah stuff backup/
tyler@desktop:~/rsync$ ls backup/
filea filebThe -h option displays sizes in a human readable format. I.e. it will display the size of the transfer in KB, MB, etc., instead of bytes.
The -a option is called archive mode. It is a shortcut for several other options. It instructs rsync to preserve time stamps, permissions, and a few other things. If you are interested, the man page describes all of the details.
Finally, -v turns on verbose output. With verbose output turned on, rsync will display the transfer speed, a list of files transferred, and the size of the transfer.
Restoring From Local Backups
To restore your backup, reverse the source and destination.
tyler@desktop:~/rsync$ rsync -ah backup/ stuff/Deleting Files
Sometimes you will want to backup or restore an exact copy of your data. If you wish to delete files in your destination that aren’t in your source, add the –delete option. For example, you may have an application that creates temporary or cache files.
Here is an example of restoring a backup with the –delete option.
tyler@desktop:~/rsync$ ls backup/
filea fileb
tyler@desktop:~/rsync$ rsync --delete -ah backup/ stuff/
tyler@desktop:~/rsync$ ls stuff/
filea filebUpdating a Backup
If you wish to update a backup, just run rsync again with the same arguments. Rsync will only copy the changes. Remember that –delete will delete any extra files that are in the destination directory, but not the source. This is an efficient way to maintain an exact copy of a directory.
Remote Backups
There are two methods for remote backups. You can either push the backup to a remote host, or you can pull the backups from the source host to the backup host. If you are planning on creating incremental or differential backups, you must use the pull method. Rsync can operate over rsh, ssh, or its own protocol. I recommend ssh because your data will be encrypted while it is being sent through the network.
I would avoid push backups. If the machine you are backing up is compromised, your backups could be compromised.
Pull example:
tyler@desktop:~/rsync$ rsync -e ssh -avh tyler@server:/important/ backup/Push example:
tyler@desktop:~/rsync$ rsync -e ssh -avh /important/ tyler@server:/backup/-e ssh instructs rsync to use ssh as the remote shell. Rsync logs into the remote host using the specified shell, and then transfers the data through the remote shell connection.
In the push example, the destination tyler@server:/backup/ means rsync will login to the remote host server as the user tyler and copy the files to the /backup/ directory on server. If you haven’t setup a key pair, you will be prompted for a password.
In the pull example, rsync logs into server as tyler and copies /important/ on server to the /backup directory on the local host.
Full Backups
There are a few approaches to full backups. You can maintain a single backup directory, or you can create a new directory each time. If you are just trying to maintain a mirror image of a directory, you probably want the former. If you are using rsync to create regular backups with the ability to go back to various points in time, you will want to create a new destination directory with each run.
Deduplication
If you are considering creating periodic full backups, consider a storage system with inline deduplication. Inline deduplication will allow you to make regular full backups without using an excessive amount of storage.
There are commercial products that offer inline deduplication as well as ZFS. If you don’t want to spend a lot of money, ZFS is natively supported on FreeBSD and FreeNAS. There is also the ZFS on Linux project if you prefer a Linux based system.
Incremental and Differential Backups
To create incremental and differential backups by use the –compare-dest option. The directory specified by –compare-dest is compared to the source directory. Any files in the –compare-dest that are identical to or newer than the file in the source are not copied to the destination.
Incremental and differential backups cannot handle files that have been intentionally deleted since the referenced full backup. Suppose you have a directory you want to regularly backup. You create a full backup. After creating the full backup, you delete a file. Now you create a differential backup. The restore will have the file that you deleted between the two backups.
There are a few approaches to solving the deleted file problem:
- You could accept that files that should have been deleted will be in your backups, and therefore restored.
- Manually keep a list of files you have deleted between each backup.
- Consider using only full backups.
- Use backup software, such as Amanda, instead of rsync.
- Write a script that compares which files are in your backup directories to the source directory and outputs a list of deleted files.
Differential Backups
To create a differential backup, use the absolute path to your most recent full backup as the value for the –compare-dest option:
rsync -avh --compare-dest=$(pwd)/full/ source/ diff1/If you are doing remote backups, you must use the pull method. The –compare-dest option will not accept a remote directory. If pulling isn’t an option, you can probably work around it by FUSE mounting the remote directory with sshfs. I haven’t written a guide about this yet, but nixCraft does a good job covering it.
Incremental Backups
Since incremental backups require multiple –compare-dest options, you must use rsync version 2.6.4 or newer.
Command to check your version:
tyler@desktop:~/rsync$ rsync --version
rsync version 3.1.2 protocol version 31
Copyright (C) 1996-2015 by Andrew Tridgell, Wayne Davison, and others.
Web site: http://rsync.samba.org/
Capabilities:
64-bit files, 64-bit inums, 64-bit timestamps, 64-bit long ints,
socketpairs, hardlinks, symlinks, IPv6, batchfiles, inplace,
append, ACLs, xattrs, iconv, symtimes, prealloc
rsync comes with ABSOLUTELY NO WARRANTY. This is free software, and you
are welcome to redistribute it under certain conditions. See the GNU
General Public Licence for details./Incremental backups are created the same way as differential backups, except multiple –compare-dest directories are used.
The following command will create an incremental backup of source containing only the files that have been created or modified since the latest backup, inc1, was created.
rsync -avh --compare-dest=$(pwd)/full/ --compare-dest=$(pwd)/inc1/ source/ inc2/