Linux Network Bonding: LACP (802.3ad) and active-backup Modes

Bonding two NICs into one logical interface is the standard way to remove the server NIC as a single point of failure - and optionally to multiply throughput. The Linux bonding driver offers seven modes, but production boils down to two: 802.3ad LACP (mode 4) when you control the switch, and active-backup (mode 1) when you do not. This guide explains the modes, shows the switch side and the Linux side, and covers the verification and failover tests that prove the bond actually works.

Bonding Modes at a Glance

For new deployments pick mode 4 when you own the switch, mode 1 otherwise - a misconfigured LACP bond (one side negotiating, the other not) drops all traffic.

The modes differ along three independent axes: whether the switch must cooperate, how transmit frames are distributed across the slaves, and whether a failed link is detected by carrier state, link monitoring, or ARP. Balance-rr and balance-alb need no switch configuration at all, which makes them attractive in labs, but balance-rr reorders packets and can badly hurt TCP throughput while balance-alb depends on ARP replies arriving on the right slave - neither is something you want in a production server facing a switch you do not manage.

How Failover Actually Works in Each Mode

Mode 1 keeps exactly one slave active at a time. The kernel moves the active flag when the monitored link state says a slave has failed, so there is a short interruption measured in milliseconds rather than seconds. Because both slaves normally share the same MAC address, the switch sees the frame suddenly arriving on a different port and must relearn the MAC - or, on switches with port security or strict MAC tracking, must be told to accept the move. That is what fail_over_mac=active solves: each slave keeps its own burned-in MAC and only the active one transmits.

Mode 4 is not failover in the traditional sense. LACP forms an aggregation with the switch, and every slave that is successfully negotiated is a candidate for transmission. If a member link dies, the switch and the host both remove it from the aggregator and traffic continues over the remaining members. If the whole LACP negotiation fails - wrong switch config, wrong VLAN, LACP disabled - the bond goes down entirely rather than falling back. That is the single most common LACP incident: a working-looking interface that passes nothing.

Switch-Side Configuration (LACP only)

Mode 4 needs IEEE 802.3ad on both ends. Cisco IOS/XE example:

interface GigabitEthernet0/1
 channel-group 1 mode active
interface GigabitEthernet0/2
 channel-group 1 mode active
interface Port-channel1
 description bond0-server
 switchport mode access
 switchport access vlan 100

Use mode active when the Linux side sends LACP PDUs (it does by default); mode passive on one end is also fine, but mode on (static) will not negotiate. Juniper EX equivalent:

set interfaces ae0 aggregated-ether-options lacp active
set interfaces ge-0/0/0 ether-options 802.3ad ae0
set interfaces ge-0/0/1 ether-options 802.3ad ae0

Three switch-side details are worth checking before you blame Linux. First, the member ports must have identical configuration - speed, duplex, VLAN, MTU, and the same access or trunk mode. A port-channel refuses to bundle mismatched members, and the error appears only on the switch. Second, access ports carrying a server should have PortFast or an equivalent edge setting so a link event does not trigger a spanning-tree transition. Third, spanning tree still runs on the port-channel as a single logical port, so plan the VLAN and STP state deliberately rather than leaving it to chance. The Juniper side of the same design, including LACP periodic and aggregate interfaces, is covered in our Junos aggregated Ethernet configuration example.

Linux-Side Configuration with NetworkManager

# LACP (mode 4)
sudo nmcli connection add type bond con-name bond0 ifname bond0   bond.options "mode=802.3ad,miimon=100,xmit_hash_policy=layer3+4"

# active-backup (mode 1)
sudo nmcli connection add type bond con-name bond0 ifname bond0   bond.options "mode=active-backup,miimon=100,fail_over_mac=active"

# slaves
sudo nmcli connection add type ethernet con-name bond0-slave1 ifname enp3s0 master bond0
sudo nmcli connection add type ethernet con-name bond0-slave2 ifname enp4s0 master bond0

# address
sudo nmcli connection modify bond0   ipv4.addresses 192.168.1.50/24 ipv4.gateway 192.168.1.1   ipv4.method manual

sudo nmcli connection up bond0
sudo nmcli connection up bond0-slave1
sudo nmcli connection up bond0-slave2

miimon=100 polls link state every 100 ms for failover detection; xmit_hash_policy layer3+4 spreads flows across slaves for LACP; fail_over_mac=active in active-backup mode prevents both slaves presenting the same MAC during failover on switches that track port MACs strictly. Set connection.autoconnect-slaves 1 on the bond so slaves rejoin after reboot.

The full option set is worth understanding because the defaults are conservative:

• miimon versus arp_interval. miimon watches carrier state and is the right default. arp_interval watches reachability by sending ARP to arp_ip_target and catches failures the NIC driver does not report, but it needs a target that answers and can false-positive when the target is down.
• lacp_rate=fast asks for one LACP PDU per second instead of every 30 seconds. Use it when you want the switch to notice a dead member quickly, and pair it with lacp_rate fast on the switch side.
• updelay and downdelay in milliseconds avoid flapping during a slow link-training or switch-reboot window. updelay=30000 is a common starting point for LACP bonds that reshuffle during switch reloads.
• xmit_hash_policy values are layer2 (default), layer2+3, layer3+4, and encap2+3/encap3+4 for encapsulated traffic. For a server talking to many peers, layer3+4 gives the most even distribution; for a bond to a storage array that uses a single IP pair, no hash policy will help, because per-flow hashing pins one conversation to one slave.

Alternative Configuration Paths

NetworkManager is the right tool on most modern distributions, but the same bond options can be expressed elsewhere - useful when a host predates NetworkManager or uses a declarative config manager:

# systemd-networkd (30-bond0.netdev)
[NetDev]
Name=bond0
Kind=bond

[Bond]
Mode=802.3ad
MIIMonitorSec=100ms
TransmitHashPolicy=layer3+4
LACPTransmitRate=fast

# legacy ifcfg (RHEL)
BONDING_OPTS="mode=active-backup miimon=100 fail_over_mac=active primary=enp3s0"

One behavioural difference to remember: the primary= option in active-backup mode fixes which slave is preferred, and the bond reverts to it when the link comes back. Without a primary, the kernel keeps using whichever slave took over, which is usually fine but makes the interface state non-deterministic across reboots. The nmcli static IP guide covers the addressing side of these profiles in more detail.

Verification and Fault Testing

cat /proc/net/bonding/bond0      # mode, slaves, active slave, miimon
ip link show bond0
ethtool enp3s0                   # per-slave speed/duplex
sudo nmcli connection show bond0 | grep bond.options

# failover test
sudo ip link set enp3s0 down
cat /proc/net/bonding/bond0      # active slave should switch
sudo ip link set enp3s0 up

In LACP mode also verify the bundle on the switch (show lacp neighbor / show etherchannel summary on Cisco). Bonding pairs with the switch-side LAG concepts covered in our PAgP vs LACP guide and the Arista EOS port-channel guide; the per-vendor commands live in the multi-vendor CLI cheat sheet.

Read the bonding output carefully. In mode 4 you should see MII Status: up on every slave and an aggregator ID that is not zero; an aggregator ID of 0 means LACP never converged. In mode 1 the Currently Active Slave line tells you which NIC is carrying traffic, and Permanent HW addr is what the switch actually learned. Run the failover test during the maintenance window, not during an incident: ping -f or a continuous ping -i 0.2 against the gateway while you bounce the active slave shows you exactly how many packets a failover costs.

Choosing Between LACP and Active-Backup

• You control both ends and need more than one NIC's worth of throughput: mode 4, with layer3+4 hashing.
• The switch ports are not yours to configure, or are configured by an unresponsive team: mode 1. It needs nothing from the switch.
• You connect to two different switches for real path redundancy: mode 1 with one slave per switch, or mode 4 only if the switches are a single logical device (MLAG/vPC/stack). Two independent switches will not form one LACP bundle with a single host.
• Virtualisation hosts where VMs each need a VLAN: mode 4 with the bond as a bridge or OVS uplink, so the virtual switch can hash per-VM flow.
• Single-flow workloads such as iSCSI or a backup stream to one target: no mode multiplies that flow's bandwidth. Use multipath at the storage layer instead.

Common Failures and How to Diagnose Them

Frequently Asked Questions

Does bonding double my throughput? Only for multiple simultaneous flows. A single TCP connection is hashed to one slave and is capped at that slave's line rate. Bonding gives aggregate capacity and redundancy, not per-flow acceleration.

Can I mix a 1G and a 10G NIC in an LACP bond? Technically the kernel will accept it, but 802.3ad requires identical speed and duplex on all members to form an aggregator, so in practice you should not. Active-backup tolerates mixed speeds because only one slave is active.

Do I need to disable STP on the switch ports? No, but you should enable PortFast/edge mode so a link event does not cause a spanning-tree transition, and you must not run the bond into a loop. For Layer 2 topologies where the bond is a bridge uplink, review Proxmox VE VLAN-aware bridge and LACP bond for the virtualised case.

Monitoring a Bond in Production

A bond that works on day one can quietly degrade months later when a member link drops and nobody notices, because the bond keeps forwarding over the surviving slave. Alert on the number of active slaves, not on interface up/down events:

# per-slave counters - a silent dead member shows frozen or single-direction counters
ip -s link show enp3s0
grep -E "Slave Interface|MII Status|Speed" /proc/net/bonding/bond0

# LACP counters: nonzero rx_marker / dropped frames point at negotiation trouble
ethtool -S enp3s0 | grep -i lacp

# quick integrity check for a cron job or monitoring plugin
awk '/Slave Interface/{s=$0} /MII Status: down/{print s" -> "$0}' /proc/net/bonding/bond0

Two habits pay off. First, ship bond state into your monitoring system as a metric (active slave count, and the active slave's name) so a failover is an alert rather than a discovery made during the next outage. Second, after every switch firmware upgrade or cable change, re-read /proc/net/bonding/bond0 and confirm the aggregator still holds every member - an LACP bundle that silently loses a link halves your bandwidth while staying fully reachable.

原文链接：https://linuxjunkies.org/guides/linux-network-bonding