Keepalived VRRP for HAProxy: Virtual IP Failover Setup

A single HAProxy instance is a single point of failure. Keepalived runs the VRRP protocol on Linux to float one virtual IP (VIP) between two load balancers: the MASTER holds the VIP and answers traffic; when it dies — server crash, lost link, or HAProxy itself failing — the BACKUP takes the VIP over in one to three seconds and clients never notice. This guide builds a complete two-node keepalived + HAProxy failover pair on Ubuntu, then covers the tuning, health checks and diagnostics that separate a lab demo from something you can leave running.

Architecture

Clients -> VIP 10.0.0.10
             |
      +------+-------+
      |              |
  lb01 (MASTER)   lb02 (BACKUP)
  10.0.0.11       10.0.0.12
      +-------+------+
              |
        web1 / web2 backends

VRRP needs both nodes on the same L2 segment (failover relies on ARP and multicast 224.0.0.18, IP protocol 112). Keepalived does not load balance — only one node owns the VIP; HAProxy still balances across the backend pool. If your two nodes sit in different subnets or behind a cloud router that drops multicast, jump to the unicast VRRP section below before building anything else.

Step 1: HAProxy Configuration on Both Nodes

sudo apt install haproxy -y

Both nodes run the same config. HAProxy binds to the VIP, which the BACKUP does not own yet — allow binding to non-local addresses or HAProxy will refuse to start there:

frontend fe_main
    bind 10.0.0.10:80
    default_backend be_web

backend be_web
    balance roundrobin
    server web1 10.0.0.21:80 check
    server web2 10.0.0.22:80 check

echo "net.ipv4.ip_nonlocal_bind = 1" | sudo tee -a /etc/sysctl.conf
sudo sysctl -p

Also enable a stats socket so the health-check script and your monitoring can ask HAProxy questions directly:

global
    stats socket /run/haproxy/admin.sock mode 660 level admin
    stats timeout 30s

listen stats
    bind 127.0.0.1:8404
    stats enable
    stats uri /stats

Step 2: Keepalived Configuration — Master (lb01)

The vrrp_script is what makes failover react to HAProxy dying, not just the host. Without it, keepalived happily holds the VIP on a box whose proxy process is gone, and every client gets a connection refused while VRRP reports perfect health.

global_defs {
    router_id lb01
}
vrrp_script chk_haproxy {
    script "kill -0 $(cat /var/run/haproxy.pid)"
    interval 2
    weight -20
    fall 3
    rise 2
}
vrrp_instance VI_1 {
    state MASTER
    interface ens3
    virtual_router_id 51
    priority 100
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass Secret01
    }
    virtual_ipaddress {
        10.0.0.10/24
    }
    track_script {
        chk_haproxy
    }
}

Read the script parameters as a small state machine. weight -20 lowers the node's effective priority from 100 to 80 while the check fails, which is below the backup's 90 and therefore loses the election. fall 3 means three consecutive failures are needed before the state flips, so a single slow tick does not cause a flap; rise 2 requires two consecutive successes before the node tries to take the VIP back.

Step 3: Backup Node (lb02)

Identical except state BACKUP and a lower priority. Both nodes keep the same virtual_router_id and auth password — any mismatch and the pair never elects:

vrrp_instance VI_1 {
    state BACKUP
    interface ens3
    virtual_router_id 51
    priority 90
    ...
    virtual_ipaddress { 10.0.0.10/24 }
    track_script { chk_haproxy }
}

Keep every other parameter byte-identical between the two files: the interface name, the advert interval, the VRID and the authentication method. A single differing advert_int is enough to make one node declare the other dead on a timer.

Also enable IP forwarding for good measure on both nodes, then start the services:

echo "net.ipv4.ip_forward = 1" | sudo tee -a /etc/sysctl.conf
sudo sysctl -p
sudo systemctl enable --now keepalived haproxy

Kernel and Sysctl Tuning for the VIP

Two settings matter more than any others on a failover pair. net.ipv4.ip_nonlocal_bind = 1 lets HAProxy bind the VIP before the node owns it, and net.ipv4.conf.all.arp_ignore / arp_announce control how the host answers ARP, which becomes important when the VIP is on a sub-interface or when both nodes are behind the same switch port group.

net.ipv4.ip_nonlocal_bind = 1
net.ipv4.conf.all.arp_ignore = 1
net.ipv4.conf.all.arp_announce = 2
net.core.somaxconn = 4096
net.ipv4.tcp_tw_reuse = 1

Raising somaxconn matters because after a failover the surviving node absorbs the whole connection backlog at once, and a small accept queue turns a clean failover into a burst of resets. Widening TCP buffer defaults is the same trade-off as any busy proxy; the sysctl tuning guide linked below covers the buffer and backlog numbers in detail.

Unicast VRRP for Clouds That Block Multicast

VRRP relies on multicast 224.0.0.18, which many cloud and overlay networks drop. Keepalived supports unicast peering, and it is the only reliable option in AWS, Azure or any environment with proxy ARP in the path:

vrrp_instance VI_1 {
    state MASTER
    interface eth0
    virtual_router_id 51
    priority 100
    unicast_src_ip 10.0.0.11
    unicast_peer {
        10.0.0.12
    }
    virtual_ipaddress {
        10.0.0.10/24
    }
}

The peer block on each node lists only the other node's real address. With unicast there is no multicast advertisement to leak, so the pair is also easier to secure — but you lose the ability to detect a third, misconfigured VRRP speaker, which in multicast mode would be visible in the logs immediately.

Notify Scripts for Logging and Fast Convergence

A notify script turns state changes into something you can observe. Keepalived calls it with the transition name, so one script can cover every case:

notify "/etc/keepalived/notify.sh"

# /etc/keepalived/notify.sh
#!/bin/bash
TYPE=$1; NAME=$2; STATE=$3
echo "$(date '+%F %T') $TYPE $NAME -> $STATE" >> /var/log/keepalived-state.log
case "$STATE" in
  MASTER) logger -t keepalived "Took over the VIP";;
  BACKUP) logger -t keepalived "Released the VIP";;
  FAULT)  logger -t keepalived "Entered FAULT - both nodes may be down";;
esac

sudo chmod +x /etc/keepalived/notify.sh

The FAULT case is the one worth alerting on. If both nodes ever lose the VIP, every client is down and the only evidence is in this log line and in journalctl -u keepalived.

Verification and Failover Test

ip addr show ens3 | grep 10.0.0.10
systemctl status keepalived
journalctl -u keepalived -f
ip -s neigh show 10.0.0.10

Run the address check on both nodes at the same time: exactly one must show the VIP, and the neighbour entry on a client should point at that node's MAC. Then break it on purpose.

sudo systemctl stop haproxy      # on the MASTER

Within a few seconds the BACKUP's health-check script fails on the master, the master's VRRP priority drops below the backup's, and the VIP appears on lb02 — verify with ip addr on both nodes and by continuing to reach the VIP over HTTP. Start the old master and (with preemption, the default) the VIP returns to it.

Test the harder failure too: pull the network cable or shut the interface on the master rather than stopping the service. A process failure exercises the track script, while a link failure exercises VRRP's own advertisement timeout, and they fail in different ways at different speeds.

Common Failure Modes and How to Diagnose Them

• Both nodes claim MASTER - the classic split brain. Check that the VRID and auth password match, that multicast is not filtered between the nodes, and that advert_int is identical. With unicast, confirm each node's unicast_peer lists the other's real address.
• VIP moves but clients time out for 30 seconds - the switch's MAC address table still points at the old port, or the gratuitous ARP did not reach the clients. Confirm arp_announce and check whether the upstream device performs proxy ARP.
• The VIP is on the right node but nothing answers - HAProxy failed to start on the backup because ip_nonlocal_bind is missing, or the frontend bind address is wrong.
• Flapping every few seconds - fall is too small for a slow health check, or the two nodes are fighting over preemption with equal priorities. Raise fall and separate the priorities.
• Failover never happens - the track script is not actually testing the process. Test it by hand as the keepalived user; a script that depends on a PID file path that has changed will always return success.

Backend Health Checks That Mean Something

Keepalived keeps the VIP honest, and HAProxy keeps the backend pool honest — but only if the health check actually tests the service. The default TCP connect check passes as soon as something is listening on port 80, which is exactly what a web server with a broken application and a working socket looks like.

backend be_web
    balance roundrobin
    option httpchk GET /healthz HTTP/1.1\r\nHost:\ www.example.com
    http-check expect status 200
    default-server inter 2s fall 3 rise 2
    server web1 10.0.0.21:80 check
    server web2 10.0.0.22:80 check

With this configuration a backend is removed from rotation after three failed HTTP checks instead of only when the socket closes, and it is added back after two successes. Set inter no lower than the application's worst-case response time: a health endpoint that occasionally takes 500 ms behind a slow database will cause a healthy server to flap in and out of the pool, which looks exactly like an application problem and wastes a lot of debugging time.

For services that need a deeper test, use an agent check — a small script on the backend that exposes its own readiness on a separate port — so the load balancer can distinguish "the process is up" from "this node can serve requests".

Reloading HAProxy Without Dropping Connections

Editing the configuration and restarting with systemctl restart haproxy tears down every established connection. Reload instead, which starts a new process that inherits the listening sockets before the old one exits:

sudo systemctl reload haproxy
# or, when running HAProxy master-worker mode:
sudo kill -USR2 $(cat /var/run/haproxy.pid)

Run HAProxy in master-worker mode (-W or master-worker in the global section) if you want seamless reloads with zero listener downtime. On the keepalived side, note that a reload does not change the PID file semantics the track script depends on, so kill -0 $(cat /var/run/haproxy.pid) keeps working across reloads — but it stops working if you switch to a systemd socket-activation setup, in which case test the track script by hand.

Firewalling and Securing the Pair

Keepalived runs as root and manipulates addresses, so lock down both the VRRP traffic and the management surface:

sudo ufw allow from 10.0.0.12 to 10.0.0.11 proto vrrp
sudo ufw allow 80/tcp
sudo ufw deny 8404/tcp
sudo ufw enable

Keep the HAProxy stats listener bound to 127.0.0.1 and reach it through an SSH tunnel rather than exposing it. VRRP authentication in keepalived is a plain-text password in the configuration file with a maximum of eight characters; it prevents accidental cross-talk between unrelated clusters but is not a security control, which is another argument for unicast VRRP with a firewall rule limited to the peer address.

Related Guides on This Site

HAProxy keepalived pairs mirror the first-hop redundancy ideas in Cisco HSRP configuration; for active-active Layer 2 designs see Dell OS10 VLT peer routing. On the software side, pair this pair with the Nginx reverse proxy configuration guide if you terminate TLS in front of HAProxy, tune the kernel with Linux network sysctl tuning, and compare the concepts with VRRP active gateway on ArubaOS-CX.

原文链接：https://perlod.com/tutorials/keepalived-vrrp-haproxy-failover