Complete Guide to Use Rsync in Linux

TL;DR: This complete guide to use rsync in Linux covers everything advanced users and sysadmins need to master efficient file synchronization and backups. From local copies to secure remote transfers, you’ll find the most essential rsync commands, verified real-world examples, and best practices for safe, production-grade operations. Learn how to use powerful flags like -a, -z, and --delete, avoid common disasters, and automate your workflows with cron.

Prerequisites

Before diving into the complete guide to use rsync in Linux, ensure your environment is ready for robust file synchronization. Rsync is available on all major Linux distributions but may not be installed by default on minimal servers.

Rsync installed: Confirm with which rsync or rsync --version.
SSH configured: Needed for secure remote transfers; public key authentication is highly recommended for scripting and automation.
User permissions: Ensure you have read access on the source and write permission on the destination.
Network access: Open the appropriate ports (usually TCP/22 for SSH; custom for rsync daemon if used).
Compatible versions: Rsync protocol version mismatches can cause subtle issues.

Examples

# Example 1: Check if rsync is installed
$ which rsync
/usr/bin/rsync

This checks if rsync is present in your system’s executable search path.

# Example 2: Check rsync version
$ rsync --version
rsync  version 3.2.3  protocol version 31
...

Useful for verifying compatibility, especially before syncing with remote systems.

# Example 3: Install rsync on RHEL/CentOS
$ sudo yum install rsync
...

On Red Hat-based systems, use yum. EPEL may be required for newer versions.

# Example 4: Install rsync on Debian/Ubuntu
$ sudo apt install rsync
...

On Debian/Ubuntu, simply use apt.

# Example 5: Test SSH connectivity
$ ssh web01.prod.example.com
...

Verifies that SSH is reachable, a must for secure remote rsync transfers.

WARNING: Always secure your SSH keys. Set permissions with chmod 600 ~/.ssh/id_rsa to prevent unauthorized access.

Distro Notes

RHEL/CentOS: Latest rsync may need EPEL repository.
Debian/Ubuntu: Rsync is in main repositories, typically up to date.
Arch: Install with pacman -S rsync.

TIP: On Arch Linux, use pacman -S rsync to quickly install the latest version. This ensures you have access to all modern rsync features.

Common Mistakes

Not verifying rsync version compatibility between client and server.
Forgetting to open firewall ports for SSH or the rsync daemon.
Using password-based SSH instead of keys for automated scripts.

Linux Security Hardening for Servers Tutorial

Introduction to Rsync

Rsync is the Swiss Army knife of file transfer and synchronization in Linux and Unix-like operating systems. Its robust, delta-transfer algorithm syncs only changed blocks or bytes, making it much faster than tools that copy whole files. Rsync preserves permissions, timestamps, ownerships, and can compress data during transfer. It works locally, over SSH, or (rarely) via its own daemon protocol.

For sysadmins, rsync is the backbone of reliable backups, server migrations, and disaster recovery. Its flexibility in excluding files, verifying integrity, and customizing transfer methods makes it a staple in every production toolkit.

NOTE: Rsync can operate in both push (send to remote) and pull (fetch from remote) modes, giving you flexibility in backup and migration strategies.

Examples

# Example 1: Local directory sync
$ rsync -a /var/log/nginx/ /backup/nginx/
...

The -a (archive) flag enables recursive copy and preserves metadata.

# Example 2: Remote pull (fetch from server)
$ rsync -avz web01.prod.example.com:/srv/data/ /local/data/
...

Pulls data from a remote server using archive (-a), verbose (-v), and compression (-z).

# Example 3: List files on remote server
$ rsync web01.prod.example.com:/etc/hosts
hosts

Lists a single file available on the remote host.

# Example 4: Use rsync daemon (rare in modern setups)
$ rsync -avz rsync://web01.prod.example.com/module/ /data/
...

Connects to a running rsync daemon. Not recommended unless you control the network.

# Example 5: Exclude certain files from sync
$ rsync -av --exclude '*.tmp' /tmp/ /backup/tmp/
...

Excludes files ending in .tmp from the transfer.

WARNING: Avoid using the rsync daemon in untrusted networks without authentication. SSH is far safer in nearly all environments.

Distro Notes

Rsync is broadly supported; daemon mode may need extra configuration.

Common Mistakes

Using the rsync daemon without authentication.
Not understanding trailing slash semantics (see below).
Assuming rsync is pre-installed on minimal systems.

Linux Performance Tuning: Real-World Scenarios Explained

Basic Rsync Commands

Rsync’s true power emerges with the right combination of flags. Here’s a breakdown of the most essential options:

-a (archive): Recursively copy, preserve permissions, timestamps, groups, and symlinks.
-v (verbose): Print detailed progress info.
-z (compress): Compress file data during transfer, useful over slow links.
-n (dry run): Show what would be done without making changes.
--progress: Display progress during transfer.
--exclude: Omit files/directories matching a pattern.

TIP: Combine -n (dry run) with other options to safely preview changes before committing to a sync, especially when using --delete.

Examples

# Example 1: Basic local sync with progress
$ rsync -av --progress /var/log/nginx/ /backup/nginx/
sending incremental file list
access.log
     1048576 100%   10.00MB/s    0:00:00 (xfr#1, to-chk=0/1)
...

This provides detailed output and preserves metadata.

# Example 2: Remote sync with SSH and exclude pattern
$ rsync -avz -e ssh --exclude '*.log' /srv/app/ ad***@****************le.com:/srv/app/
...

-e ssh secures the connection, and --exclude '*.log' skips log files.

# Example 3: Dry run before actual sync
$ rsync -avn /data/ /mnt/backup/data/
sending incremental file list
...

The -n flag previews what would change, with no files actually copied.

# Example 4: Sync a single file
$ rsync -av /etc/hosts web01.prod.example.com:/tmp/
...

Copies a single file, preserving attributes.

# Example 5: Mirror a directory, deleting extraneous files
$ rsync -av --delete /srv/media/ web02.prod.example.com:/srv/media/
...

--delete removes files from the destination that no longer exist at the source.

WARNING: Always use --dry-run before running --delete to avoid catastrophic data loss.

Distro Notes

All options are standard across major Linux distributions.

Common Mistakes

Forgetting -a (archive) when you want to preserve metadata.
Not using -z for large files or slow network links.
Misquoting or omitting exclude patterns, leading to unintended file transfers.

Advanced Rsync Features

For complex environments, rsync offers advanced options to fine-tune performance, backups, and redundancy:

--bwlimit=KBPS: Throttle bandwidth usage.
--partial: Keep partially transferred files for interrupted jobs.
--compress-level=NUM: Set compression level (1 = fastest, 9 = smallest).
--link-dest=DIR: Hard-link unchanged files for efficient incremental backups.
--checksum: Compare files using checksums rather than mod-time/size.

NOTE: Advanced features like --link-dest are invaluable for incremental backup schemes, allowing you to save space and time while maintaining full restore points.

Examples

# Example 1: Limit bandwidth to 1MB/s
$ rsync -avz --bwlimit=1024 /data/ web01.prod.example.com:/backup/data/
...

Prevents rsync from saturating your network.

# Example 2: Incremental backup with hard links
$ rsync -av --link-dest=/backup/prev/ /data/ /backup/current/
...

Unchanged files are hard-linked, saving space and speeding up backups.

# Example 3: Resume interrupted transfer
$ rsync -av --partial /largefiles/ web01.prod.example.com:/data/largefiles/
...

--partial allows resuming without retransferring already-synced data.

# Example 4: Use higher compression for slow links
$ rsync -avz --compress-level=9 /var/log/ web01.prod.example.com:/backup/logs/
...

Maximum compression reduces bandwidth for slower connections.

# Example 5: Use checksum for file comparison (slower, but accurate)
$ rsync -av --checksum /srv/data/ /mnt/backup/data/
...

Forces rsync to verify files by checksum, not just size and time—useful for critical data.

WARNING: --link-dest makes powerful, space-efficient backups, but deleting or moving hard-linked directories can cause massive data loss. Always test on sample data before production use.

Distro Notes

All features require rsync 3.x+ (standard on modern distros).

Common Mistakes

Using --checksum unnecessarily, significantly slowing transfers.
Neglecting to rotate or prune old incremental backups, filling disks.
Failing to test the impact of --bwlimit on production network traffic.

Linux Performance Tuning: Real-World Scenarios Explained Linux Performance Tuning: Real-World Scenarios Explained

Common Mistakes & Gotchas

Even experienced sysadmins can fall into rsync traps. Here are the most frequent:

TIP: Always double-check your source and destination paths, and use --dry-run for any new or potentially destructive operation.

Examples

# Example 1: Preview a destructive sync
$ rsync -av --delete --dry-run /empty/ /important/data/
deleting file1.txt
deleting file2.log
...

A dry run shows which files would be deleted—always verify before removing data.

# Example 2: Exclude multiple patterns
$ rsync -av --exclude '*.tmp' --exclude 'cache/' /srv/ /backup/srv/
...

Multiple --exclude flags can be combined for fine-grained control.

# Example 3: Forgetting trailing slash
$ rsync -av /data /backup/
# Results in /backup/data/ instead of merging into /backup/

No trailing slash on the source copies the directory itself, not just contents.

# Example 4: Overwriting files due to incorrect destination path
$ rsync -av /etc/hosts web01.prod.example.com:/etc/
# Overwrites /etc/hosts on remote!

Double-check your destination—rsync will overwrite if not careful.

# Example 5: Not preserving permissions
$ rsync /srv/app/ /backup/app/
# Permissions and ownership may not be preserved without -a

Omitting -a loses critical metadata.

WARNING: Always use --dry-run for unfamiliar or potentially destructive operations.

Common Pitfalls

Misunderstanding the meaning of trailing slashes on source/destination.
Running as root when not required, increasing risk.
Not validating exclude/include patterns, leading to missed or unwanted files.

Linux Security Hardening for Servers Tutorial

Security & Production Considerations

Production-grade rsync usage demands strong security hygiene and precise configuration.

SSH: Always use SSH for remote operations (-e ssh). Never expose the rsync daemon unprotected.
Alternate binaries: Use --rsync-path if the remote has a non-standard or custom-built rsync.
Permissions: Set file/directory permissions with --chmod to avoid open or world-writable backups.
SSH keys: For automation, restrict keys to specific commands to avoid compromise.

TIP: Restrict backup SSH keys in authorized_keys with a command="..." prefix to limit what the key can do, reducing your attack surface.

Examples

# Example 1: Secure transfer with SSH
$ rsync -avz -e ssh /data/ ad***@****************le.com:/srv/data/
...

SSH encrypts the session, protecting credentials and data.

# Example 2: Specify alternate rsync path (e.g., custom build)
$ rsync -avz -e ssh --rsync-path="/usr/local/bin/rsync" /data/ ad***@****************le.com:/srv/data/
...

Ensures the remote binary matches your local feature set.

# Example 3: Set permissions on destination files
$ rsync -av --chmod=Du=rwx,Dgo=rx,Fu=rw,Fgo=r /srv/app/ /backup/app/
...

Ensures directories (D) are rwx for user and rx for group/others, files (F) are rw for user and r for group/others.

# Example 4: Use SSH key for automation
$ rsync -avz -e "ssh -i /home/backup/.ssh/id_rsa" /data/ ba****@****************le.com:/srv/backup/
...

Automates backup securely using a dedicated key.

# Example 5: Use non-standard SSH port
$ rsync -avz -e "ssh -p 2222" /data/ ad***@****************le.com:/srv/data/
...

Useful if your SSH service isn’t on port 22.

WARNING: Never run rsync as root unless absolutely necessary. Limit your SSH keys’ abilities and use --rsync-path to point to a limited binary if possible.

Production Notes

Always test scripts with --dry-run.
Use dedicated backup users with minimal privileges.
Restrict SSH key usage with command= in authorized_keys.

Common Mistakes

Using password-based SSH in scripts, making automation insecure.
Not restricting SSH keys for backup accounts.
Not setting destination permissions, leading to open or unreadable files.

Linux Security Hardening for Servers Tutorial Tmux Cheat Sheet: From Beginners to Advanced Level

Integrating Rsync with Cron Jobs

Automation is critical for consistent backups and syncs. Cron, the Linux time-based scheduler, is the most common way to run rsync jobs regularly. Always use absolute paths, redirect output, and prevent overlapping jobs.

NOTE: When scheduling rsync with cron, always test your command interactively first to ensure paths and permissions are correct.

Examples

# Example 1: Daily backup at 2am
0 2 * * * rsync -avz /srv/data/ ba****@****************le.com:/srv/backup/

Runs a full backup every night at 2am.

# Example 2: Weekly log sync with exclude
0 3 * * 0 rsync -avz --exclude '*.gz' /var/log/ ba****@****************le.com:/backup/logs/

Syncs logs weekly, skipping compressed files.

# Example 3: Use SSH key for passwordless automation
0 1 * * * rsync -avz -e "ssh -i /home/backup/.ssh/id_rsa" /data/ ba****@****************le.com:/srv/backup/

Automates with a dedicated SSH key, ensuring unattended operation.

# Example 4: Log output to file
0 4 * * * rsync -avz /srv/media/ ba****@****************le.com:/srv/media/ >> /var/log/rsync.log 2>&1

Redirects both stdout and stderr to a log for monitoring and troubleshooting.

# Example 5: Use lockfile to prevent overlapping jobs
0 * * * * flock -n /tmp/rsync.lock rsync -avz /data/ ba****@****************le.com:/srv/backup/

flock ensures only one rsync runs at a time, avoiding simultaneous jobs.

WARNING: Never run cron rsync jobs as root unless absolutely necessary. Use restrictive SSH keys and log all output.

Distro Notes

flock is provided by util-linux in most distributions.

Common Mistakes

Not redirecting output—making it hard to debug failures.
Overlapping jobs causing corruption or network congestion.
Using relative paths, which may fail in cron’s minimal environment.

Linux Security Hardening for Servers Tutorial Linux Security Hardening for Servers Tutorial

Complete Guide to Use Rsync in Linux

Complete Guide to Use Rsync in Linux

Prerequisites

Examples

Distro Notes

Common Mistakes

Introduction to Rsync

Examples

Distro Notes

Common Mistakes

Basic Rsync Commands

Examples

Distro Notes

Common Mistakes

Advanced Rsync Features

Examples

Distro Notes

Common Mistakes

Common Mistakes & Gotchas

Examples

Common Pitfalls

Security & Production Considerations

Examples

Production Notes

Common Mistakes

Integrating Rsync with Cron Jobs

Examples

Distro Notes

Common Mistakes

Further Reading

Comments

Leave a Reply Cancel reply