pgBackRest: Reliable PostgreSQL Backup and Restore - 夜莺博客

pgBackRest: Reliable PostgreSQL Backup and Restore

Backups that have never been restored are a hypothesis, not a backup. pgBackRest is the tool most PostgreSQL teams end up with because it solves the parts that make backups slow and fragile: compression is done in-stream with lz4 or zstd, work is parallelised across processes, backup consistency is guaranteed by waiting for the required WAL to reach the repository, and a delta restore makes large databases recoverable in minutes rather than hours. This guide covers a working configuration, the repository layout choices, retention, and the restore test that proves the whole thing works.

Stanza: the unit of configuration

A stanza is one PostgreSQL cluster with its own repository, retention and schedules. Directory layout matters less than the naming decision — use the cluster name, not the hostname, so a failover to a standby does not change the stanza.

# /etc/pgbackrest/pgbackrest.conf  (on the PostgreSQL host)
[global]
repo1-path=/var/lib/pgbackrest
repo1-retention-full=2
repo1-retention-diff=7
repo1-cipher-type=aes-256-cbc
repo1-cipher-pass=<strong-passphrase>
compress-type=zst
compress-level=3
process-max=4
log-level-console=info
start-fast=y

[main]
pg1-path=/var/lib/postgresql/16/main
pg1-port=5432

# stanzas must be created before the first backup
# pgbackrest --stanza=main stanza-create
# pgbackrest --stanza=main check

Architecture options

Layout Repository location Trade-off
Local only Same host Fast restores, no protection from host loss
Repository host + local Both Two repositories: fast local restore plus redundancy
Object storage (S3/Azure/GCS) Bucket Unlimited retention, ransomware-resistant if immutable
Backup from standby Any Moves backup load off the primary (backup-standby=y)

Multiple repositories are the feature to use deliberately: a local repository with short retention for quick restores, plus an object-store repository with long retention for disaster recovery. Because pgBackRest queries PostgreSQL through its own protocol rather than a client connection, remote operation does not require granting database access to the repository host.

Backups, WAL archiving and retention

postgresql.conf:
  archive_mode = on
  archive_command = 'pgbackrest --stanza=main archive-push %p'
  wal_level = replica

# full, differential and incremental backups
pgbackrest --stanza=main --type=full backup
pgbackrest --stanza=main --type=diff backup
pgbackrest --stanza=main --type=incr backup

# what do we have?
pgbackrest --stanza=main info
pgbackrest --stanza=main check

Compression and checksums are computed in stream, which is why a network repository is practical: the PostgreSQL host compresses, the repository host stores. Checksums are re-verified during restore, so a silent bit-rot in the repository is caught at restore time rather than six months later. Set archive-timeout thoughtfully for busy systems — a backup waits for the WAL needed to make it consistent, and a too-short timeout turns a slow archive into a failed backup.

The restore you must actually test

# time-line aware restore to a specific point, into a scratch directory
pgbackrest --stanza=main --type=time "--target=2026-10-14 02:00:00+08" \
  --target-action=promote --delta --pg1-path=/var/lib/postgresql/16/main restore

# verify then start PostgreSQL and check the recovery target was reached
pg_ctl -D /var/lib/postgresql/16/main start
psql -c "select pg_is_in_recovery(), now();"
psql -c "select count(*) from pg_stat_activity;"

Restore into a scratch path first whenever possible — --delta makes it fast by only changing files that differ, and a scratch restore cannot damage the live data directory. The delta mode has one gotcha documented by the project: exclusions are not honoured on delta restores, and files excluded by the backup will be removed by the delta restore. Restore the whole cluster, not selected tables, unless you are deliberately recovering data from a tablespace-aware backup.

Operational checklist

  • Run pgbackrest check on a schedule — it validates the archive and repository, and it is the only early warning you get.
  • Schedule a full backup weekly and differentials daily; verify retention by reading pgbackrest info, not by assuming the config took effect.
  • Encrypt the repository if it leaves the host; without a passphrase an object-store bucket full of WAL is still a database.
  • Restore into a test environment monthly and record the time it took — that number is your real recovery time objective.

Repository layout and encryption decisions

Where the backup lives determines what kind of disaster it survives. The sensible production pattern is two repositories with different purposes: a local one for fast operational restores, and a remote or object-store one that is out of reach of anything that compromises the database host.

Repository Retention Purpose
repo1 — local disk 2 full, 7 differential Fast restore of an accidental table drop
repo2 — object storage 4-12 weeks Host loss, ransomware, long-term compliance
repo2 with immutability As policy requires Protection against an attacker with credentials
[global]
repo1-path=/var/lib/pgbackrest
repo1-retention-full=2
repo2-type=s3
repo2-s3-bucket=pg-backups-prod
repo2-s3-region=ap-east-1
repo2-path=/pgbackrest
repo2-retention-full=8

[global:archive-push]
repo2-s3-key=<access-key>
repo2-s3-key-secret=<secret>
process-max=4

Encryption (repo1-cipher-type=aes-256-cbc) belongs on any repository that leaves the host, and the passphrase belongs wherever you keep disaster-recovery secrets — which is emphatically not the same place as the backup. A backup you cannot decrypt is a very expensive way to store noise.

Validation: proving the backup is restorable without restoring

Two commands do the heavy lifting, and they belong in a scheduled check rather than in a runbook nobody opens.

# repository and archive consistency, plus WAL delivery
pgbackrest --stanza=main check
pgbackrest --stanza=main info --output=json | jq '.[0].backup[] | {label, type, timestamp: .timestamp.stop}'

# verify the files of an existing backup against their checksums
pgbackrest --stanza=main --set=<label> verify

# and a periodic restore into a scratch path, timed
time pgbackrest --stanza=main --type=immediate --delta \
  --pg1-path=/var/lib/postgresql/16/main restore --stanza=main

check validates that the archive is being pushed and can be read back, which catches a broken archive_command long before anyone needs a restore. verify re-reads a backup's files and compares them with the stored checksums, which is the cheapest available defence against silent corruption in the repository. Neither replaces a real restore test, but together they make a monthly restore test a confirmation rather than an investigation.

Standby backups and the primary's load

Backups read the data directory and stream WAL; on a busy primary that competes with production queries. With a standby configured, backup-standby=y moves the backup to the standby host, and because pgBackRest talks to PostgreSQL through its own protocol, the backup host reads the files over the protocol rather than requiring filesystem access.

[global]
backup-standby=y          # prefer the standby
# require it: 'y' means backup only from standby, 'prefer' falls back
archive-timeout=5m
start-fast=y

# on the standby, verify the replication slot and that WAL is flowing
SELECT client_addr, state, sent_lsn, replay_lsn FROM pg_stat_replication;

Two cautions. A backup from a standby requires the standby to be healthy and caught up; if it is lagging, the backup waits for WAL and can exceed your window — which is what archive-timeout controls, and why a too-short value turns a slow standby into a failed backup. And the backup still needs the standby's data files, so on a large database the I/O moves rather than disappears; the win is that it moves off the primary.

FAQ

Does pgBackRest replace WAL archiving? No — it *is* WAL archiving, configured through archive_command. You need both WAL and a base backup for point-in-time recovery; they are two halves of one mechanism.

What about logical backups with pg_dump? Complementary, not alternative. A logical dump protects against a bad schema migration or a need to restore a single table into a live cluster; a physical backup protects the whole cluster and supports PITR. Keep both if the data matters.

How much space do I need? Plan for full backups plus the WAL generated between them, and remember that compressed differentials are small until they are not. Monitor pgbackrest info sizes as part of the routine rather than discovering the ceiling when a backup fails at 04:00.

How often should I test a restore? Monthly in a scratch location, plus after any change to the backup configuration, storage layout or PostgreSQL major version. Record the elapsed time — that is the number to quote when someone asks how long a recovery would take.

Quick reference: the five commands in the routine

pgbackrest --stanza=main check                     # daily: archive and repo health
pgbackrest --stanza=main --type=full backup        # weekly: base backup
pgbackrest --stanza=main --type=diff backup        # daily: changes since full
pgbackrest --stanza=main info                      # what exists and its size
pgbackrest --stanza=main --set=<label> verify       # periodic: checksum re-read

Put the first four in a scheduler and the fifth in a monthly maintenance window, with a real restore test alongside it. The common failure pattern is a schedule that has been running happily for months producing backups nobody has ever read back; the verify step is what converts that from an assumption into evidence, and it is cheap enough to run far more often than a full restore test.

Repository mode and permissions

Running the repository as the dedicated pgbackrest user with ownership of the repository path is the configuration that causes the fewest surprises across upgrades and distribution changes. Mixing the PostgreSQL operating-system user with the repository ownership produces permission failures at 03:00 that are trivially avoidable, and those failures happen on the night the retention policy expires the last good full backup.

Scheduling that matches the recovery objective

# crontab on the PostgreSQL host, with the stanza named explicitly
30 1 * * 0  pgbackrest --stanza=main --type=full backup
30 1 * * 1-6 pgbackrest --stanza=main --type=diff backup
15 7 * * *  pgbackrest --stanza=main check
0  3 1 * *  pgbackrest --stanza=main --set=$(pgbackrest --stanza=main info --output=json | jq -r '.[0].backup[-1].label') verify

Two rules make this sustainable. Never schedule a backup tighter than your retention and storage can absorb — overlapping full backups on a large database is how a repository fills up at 04:00 and takes the next backup down with it. And alert on the absence of a fresh backup rather than only on failures: a scheduler that silently stops running produces no error at all, just a backup set that quietly ages, which is the failure mode that hurts most.

Related Reading

Related reading: PostgreSQL Streaming Replication and Safe Failover, PostgreSQL 16 Streaming Replication with pg_basebackup, ZFS Send and Receive: Snapshot Replication Guide

原文链接:https://pgbackrest.org/configuration.html