MariaDB Galera Cluster: Three-Node Setup and Verification

Galera gives MariaDB synchronous multi-master replication: a write commits on all nodes
or on none, so every node has the same data and any node can accept writes. That property
simplifies failover enormously and introduces a new failure mode — the cluster stops accepting
writes when it loses quorum, which is the correct behaviour and always a surprise the first
time. This guide builds a three-node cluster, covers the bootstrap rule that prevents the most
common outage, and lists the status variables worth alerting on.

Prerequisites

• At least three nodes. Two nodes cannot form a majority after one failure.
  This is not a suggestion.
• Identical MariaDB and Galera versions across nodes.
• Low, reliable latency between nodes — Galera is synchronous, so a slow link is a slow
  database.
• A dedicated replication user for SST.

# On every node (example: Ubuntu/Debian)
sudo apt update
sudo apt install -y mariadb-server mariadb-client galera-4 rsync

# Firewall — the ports Galera needs
sudo ufw allow 3306/tcp   # client connections
sudo ufw allow 4567/tcp   # replication: multicast and unicast
sudo ufw allow 4567/udp   # replication: multicast
sudo ufw allow 4568/tcp   # IST — incremental state transfer
sudo ufw allow 4444/tcp   # SST — state snapshot transfer
sudo ufw reload

Step 1: The Galera Configuration File

# /etc/mysql/conf.d/galera.cnf
[mysqld]
binlog_format          = ROW
default_storage_engine = InnoDB
innodb_autoinc_lock_mode = 2
bind-address           = 0.0.0.0

# Galera provider
wsrep_on               = ON
wsrep_provider         = /usr/lib/galera/libgalera_smm.so
wsrep_provider_options  = "gcache.size=512M; gcache.page_size=128M"

# Cluster definition — identical on every node
wsrep_cluster_name     = "my_galera_cluster"
wsrep_cluster_address  = "gcomm://10.10.60.11,10.10.60.12,10.10.60.13"

# Node identity — unique per node
wsrep_node_name        = "node1"
wsrep_node_address     = "10.10.60.11"

# SST method — mariabackup is the modern choice
wsrep_sst_method       = mariabackup
wsrep_sst_auth         = "sst_user:SstPassword"
wsrep_sst_donor_rejects_queries = OFF

# Binary logging for SST and point-in-time recovery
log_bin                = mariadb-bin
log_bin_index          = mariadb-bin.index
server_id              = 1        # unique per node

# Consistency / conflict handling
wsrep_certify_nonpk    = ON
wsrep_max_ws_rows      = 131072
wsrep_max_ws_size      = 2147483648
wsrep_cert_index_size  = 32768

Node 2 and node 3 need only three values changed:

# node2 (10.10.60.12)
wsrep_node_name   = "node2"
wsrep_node_address = "10.10.60.12"
server_id         = 2

# node3 (10.10.60.13)
wsrep_node_name   = "node3"
wsrep_node_address = "10.10.60.13"
server_id         = 3

wsrep_cluster_address must list all nodes on every node. A
node whose list is missing a peer still works, but it becomes a silent inability to rejoin
after a restart, which is much worse than an error.

Step 2: Create the SST User

-- Run on whichever node you bootstrap first
CREATE USER 'sst_user'@'%' IDENTIFIED BY 'SstPassword';
GRANT RELOAD, LOCK TABLES, PROCESS, REPLICATION CLIENT ON *.* TO 'sst_user'@'%';
FLUSH PRIVILEGES;

Step 3: Bootstrap — and the Rule That Matters

The first node of a brand-new cluster is started with the galera bootstrap init system,
not with systemctl start mariadb:

# On node1 ONLY, first ever start
sudo galera_new_cluster

# Confirm
mysql -e "SHOW STATUS LIKE 'wsrep_cluster_size';"       # 1

Then start the other nodes normally — they discover the cluster from
wsrep_cluster_address and perform a state snapshot transfer:

sudo systemctl start mariadb        # node2
mysql -e "SHOW STATUS LIKE 'wsrep_cluster_size';"       # 2

sudo systemctl start mariadb        # node3
mysql -e "SHOW STATUS LIKE 'wsrep_cluster_size';"       # 3

The rule: if the whole cluster is down and its state is intact, bring up
exactly one node as the bootstrap node and then start the others normally. If you run
galera_new_cluster on two nodes, you create two independent single-node clusters
and split-brain becomes a restore operation. If the cluster state is not intact — you
lost all nodes — bootstrap the node with the most recent data and accept that the others must
resync.

Step 4: Verify the Cluster Is Healthier Than "It Has Three Nodes"

SHOW STATUS LIKE 'wsrep_cluster_size';        -- must be 3
SHOW STATUS LIKE 'wsrep_cluster_status';      -- Primary
SHOW STATUS LIKE 'wsrep_ready';               -- ON
SHOW STATUS LIKE 'wsrep_connected';           -- ON
SHOW STATUS LIKE 'wsrep_local_state_comment'; -- Synced
SHOW STATUS LIKE 'wsrep_flow_control_paused'; -- 0.00 when idle
SHOW STATUS LIKE 'wsrep_cert_deps_distance';
SHOW STATUS LIKE 'wsrep_local_recv_queue_avg';

-- Who is in the cluster, and what they think of each other
SHOW STATUS LIKE 'wsrep_incoming_addresses';
SHOW STATUS LIKE 'wsrep_gcomm_uuid';

Writes Stop and the Application Sees Errors

If you lose two of three nodes, the survivor has no quorum and rejects writes with
ERROR 1047 (08S01): Unknown command or
WSREP has not yet prepared node for application use. This is correct — it is
preventing divergence. Your options, in order of preference:

• Restore the missing nodes and let quorum return.
• If recovery is impossible, stop MariaDB on the survivor and restart it with
  --wsrep-new-cluster (or galera_new_cluster) after
  confirming it holds the newest data. Changing the cluster address temporarily to a single-node
  list is the safer variant.
• If you are unsure whether the data is complete, restore from backup. Guessing here is how
  clusters end up with two divergent histories.

# The safer single-node variant
# wsrep_cluster_address = "gcomm://"      # empty list = new cluster from local state
sudo systemctl restart mariadb

Design Notes

• Three nodes give you one failure of headroom. Five give you two. Never deploy two.
• Route writes through one node if you can — multi-master works, but conflict detection
  costs you latency and debugging time.
• Optionally run a garbd arbitrator on a light host to keep quorum in a two-datacentre
  design, but understand that garbd votes; it does not hold data and cannot help if the surviving
  site is the one without the data.
• Galera is synchronous and therefore sensitive to network latency and packet loss. Set a
  proper NTP baseline: Chrony NTP server configuration on Linux.
• Monitor wsrep_% variables through mysqld_exporter into Prometheus, and route
  alerts with Alertmanager routing and notification configuration.
• Galera is not a backup. Take logical dumps or physical snapshots on a schedule; if you
  want clustering for read scale-outs instead, compare with MySQL 8 GTID replication setup and troubleshooting and MongoDB replica set configuration and failover.

原文链接：https://mariadb.com/docs/galera-cluster/galera-cluster-quickstart-guides/mariadb-galera-cluster-guide