rsync Incremental Backups with -link-dest Explained - 夜莺博客

rsync Incremental Backups with --link-dest Explained

rsync with --link-dest gives you something commercial backup products charge for: full, browseable snapshots for every backup run while storing only the changed blocks through hard links. The technique is decades old, entirely reliable, and still misconfigured in ways that quietly turn an incremental backup into a full copy or destroy data during a delete pass. This walkthrough builds a working rotation, covers the flags that matter, and explains what to verify after each run.

How hard-link snapshots work

The first run copies everything into a timestamped directory. Every later run compares against the previous one using --link-dest: unchanged files are hard-linked (no additional disk space, no copy), changed and new files are transferred normally. The result is a directory per run, each of which looks like a complete copy of the source at that moment, with the disk containing only one physical copy of each distinct file version.

#!/bin/bash
set -euo pipefail

https://www.linux.org/docs/man1/rsync.html=/srv/data/
DEST=/backup/srv-data
LATEST=$(readlink -f "$DEST/latest" 2>/dev/null || true)
STAMP=$(date +%Y-%m-%d_%H%M%S)

rsync -aHAX --numeric-ids --delete   --link-dest="${LATEST:-/nonexistent}"   "$https://www.linux.org/docs/man1/rsync.html" "$DEST/$STAMP/"

rm -f "$DEST/latest"
ln -s "$STAMP" "$DEST/latest"

The latest symlink is what makes the next run incremental; without it, every run is a full copy and your disk fills within days.

Flags that matter, and why

Flag Effect When to omit
-a Archive: recursion, symlinks, permissions, times, ownership Rarely — use it almost always
-H Preserve hard links within the transfer set Large trees where scan time dominates
-A / -X Preserve ACLs and extended attributes Sources with broken or irrelevant xattrs
--numeric-ids Transfer numeric UID/GID instead of names Never for cross-host backups — mapping errors corrupt ownership
--delete Remove files in the destination that no longer exist at source Never enable it before a dry run
--dry-run Show what would happen, change nothing Production runs only
--link-dest=DIR Hard-link unchanged files to DIR The first (seed) run
-z Compress in transit Fast LAN transfers or already-compressed data
--bwlimit=KBPS Cap throughput so backups do not saturate the link Dedicated backup networks
--exclude-from=FILE Skip caches, sockets, temp files Never — exclusions are what keep backups usable

Always dry-run first

rsync -aHAX --numeric-ids --delete --dry-run --stats   --link-dest="$(readlink -f /backup/srv-data/latest)"   /srv/data/ /backup/srv-data/2026-09-21_020000/ | tail -30

Read three numbers: files transferred, total transferred size, and the "deleting" list. A delete list that contains everything usually means the source path gained or lost a trailing slash — the most common rsync disaster, because /srv/data and /srv/data/ have different meanings.

Exclusions that keep backups sane

# /etc/rsync-backup.exclude
/proc/*
/sys/*
/dev/*
/tmp/*
/var/tmp/*
/var/cache/*
/var/lib/docker/overlay2/*
/home/*/.cache/*
**/*.sock
**/.gvfs/*

Backing up container layers or cache directories consumes enormous space and produces restores that are actively misleading. Exclude them and back up the declarative state (compose files, volumes, database dumps) instead.

Retention

# keep the last 14 timestamped snapshots
cd /backup/srv-data
ls -1dt 20* | tail -n +15 | xargs -r rm -rf

Because unchanged files are hard-linked across snapshots, deleting a snapshot only frees space for file versions unique to it. That is the property that makes daily retention affordable — and it also means a retention bug deletes far less than you might fear, while an --delete bug can destroy the current copy.

Verification after every run

rsync -aHAXn --delete --itemize-changes /srv/data/ /backup/srv-data/latest/ | head
df -h /backup
du -sh /backup/srv-data/*
find /backup/srv-data/latest -maxdepth 2 -type d | wc -l

An -n (dry run) against the latest snapshot should show almost nothing on a quiet system. Anything more means the source is changing faster than the schedule, or exclusions are missing. Also test a restore: pick a random file from a snapshot two weeks old and confirm it opens.

Turning it into a service

  • Run from a systemd timer or cron with logging to a file and to syslog.
  • Alert on non-zero exit status and on "no snapshot created today".
  • Add a pre-flight check that the backup filesystem has at least the expected free space.
  • Send the finished snapshots offsite — a link-based rotation protects against accidents, not against a lost server room.

Choosing where the backup writes

The destination determines how the backup behaves under load and how fast a restore is. Common options, with their real trade-offs:

Destination Advantage Disadvantage
Local disk on the source host Fastest, no network dependency Same failure domain as the data
NFS mount from a NAS Simple, central capacity Mount stalls can hang the run; use hard-mount timeouts carefully
iSCSI LUN Block-level performance Requires multipath and filesystem hygiene
Pull model (backup server initiates) Source host compromise cannot destroy the backup Requires credentials on the backup server
Push over SSH to a remote host Easy to script, works across sites Source holds the credentials and can delete the targets

The security argument is usually decisive: if the machine holding the data can also delete its own backups, ransomware or a careless administrator can remove both. The pull model, or at minimum an append-only target, is what actually protects production. Combine it with restricted SSH keys (command="rsync --server --sender ...") so the source can write but never delete.

Restricted key example

# ~/.ssh/authorized_keys on the backup server
command="rsync --server --sender -logDtpre.iLsfxC . /backup/",no-port-forwarding,no-pty,no-agent-forwarding ssh-ed25519 AAAA... backup-key

A key restricted this way cannot open a shell, cannot forward ports, and can only be used for the specific rsync transfer. Combined with --link-dest snapshots that are read-only on the target, this is enough to protect the archive from the source host itself.

Handling special cases

  • Databases: never back up open database files with rsync alone. Dump with the database's own tool, or snapshot the filesystem, or you will produce a backup that looks complete and will not start.
  • Files that change during the run: a file modified while rsync reads it transfers as a mix of old and new content. For config directories and code, that is acceptable; for state files, it is not.
  • Very large trees: split into multiple jobs so a failure only repeats one subset, and so runtime stays inside the backup window.
  • Sparse files and images: add -S to preserve sparseness; without it, a thin-provisioned image expands to its full logical size on the target.
  • Windows shares: mount at the appropriate level and disable permission mapping unless you specifically need ACLs preserved (--no-perms --no-owner --no-group).

Related articles

For transfer-level troubleshooting see Linux network troubleshooting with ss, netstat and tcpdump, and for automation wrappers see Ansible network facts and config backup examples. If you prefer filesystem-native snapshots, the equivalent approach on ZFS is covered in ZFS zpool, raidz and vdev administration; container disk growth is explained in Docker overlay2 disk full cleanup.

原文链接:https://www.linux.org/docs/man1/rsync.html