MC-LAG ICCP Failure Scenarios: Liveness and System ID

An MC-LAG pair is only as good as its control plane: the Inter-Chassis Control Protocol (ICCP) adjacency, the inter-chassis link (ICL/ICL-PL) that carries data between the two peers, and the optional out-of-band backup liveness channel. This article maps what actually happens to the multichassis aggregated Ethernet (MC-AE) interface and to the LACP system ID when one of those three pillars fails, then shows the Junos configuration and verification commands that keep a dual-homed server from tearing its bundle apart. It is written for engineers who already run MC-LAG in production and need the failure matrix rather than an introduction.

Why the LACP system ID is the whole ball game

A server connected to an MC-LAG sees the LACP system ID advertised by the switch. When ICCP is healthy, both peers advertise the same configured system ID, so the server forms one bundle with links landing on two different chassis. The failure mode to design against is split-brain: if the ICCP adjacency dies while both peers still think they are the active side, and they both fall back to their local default system ID, the server sees two different system IDs and keeps only the links from whichever it resolved first.

ICCP failure scenarios

The behaviour depends on the combination of ICCP state, ICL state and backup liveness peer status:

Read the third row carefully: with backup liveness detection configured and the peer confirmed dead, the surviving peer keeps the configured system ID and simply takes over. Without backup liveness, a lost ICCP link is indistinguishable from a dead peer, so the platform reverts to the default system ID and accepts a convergence hit.

Configuring backup liveness detection correctly

Backup liveness uses an out-of-band channel across the management network — not the ICL — so it survives a data-plane fault between the peers:

set interfaces ge-0/0/0 unit 0 family inet address 10.10.10.1/24
set interfaces ge-0/0/0 unit 0 family inet address 10.10.10.1/24 master-only
set protocols iccp peer 10.10.10.2 local-ip-address 10.10.10.1
set protocols iccp peer 10.10.10.2 redundancy-group-id-list 1
set protocols iccp peer 10.10.10.2 liveness-detection session-mode automatic
set protocols iccp peer 10.10.10.2 backup-liveness-detection backup-peer-ip 10.10.10.2
set protocols iccp peer 10.10.10.2 backup-liveness-detection liveness-detection failure-action hold

Configure master-only on the management IP address on both the primary and backup Routing Engines, otherwise the address can migrate between REs and the liveness check flakes.

Keeping the system ID stable with prefer-status-control-active

The single most valuable knob for split-brain avoidance is prefer-status-control-active, configured together with status-control standby. It stops the MC-AE system ID from reverting to the LACP default during an ICCP failure. Use it only when you are certain ICCP can go down only if the remote peer is genuinely gone:

set interfaces ae0 aggregated-ether-options lacp active
set interfaces ae0 aggregated-ether-options lacp system-id 00:11:22:33:44:55
set interfaces ae0 multi-chassis-protection 10.10.10.2 interface ae1
set switch-options service-id 1

Note that a prefer-status-control-active misapplied to a peer whose ICCP flaps for maintenance reasons is worse than the default behaviour: the surviving side never releases the standby links and traffic blackholes.

Liveness timer and peering best practices

• Peer ICCP over a loopback address, not a physical link, so a single link failure cannot tear down ICCP.
• If ICCP runs through an IRB interface, set the BFD liveness interval to at least 8 seconds; shorter intervals break graceful Routing Engine switchover (GRES).
• Default ICCP liveness uses multihop BFD in centralized mode; only a dedicated physical ICCP interface justifies single-hop BFD.
• Configure a session establishment hold time so a reboot does not immediately declare the peer dead.
• Keep the same service-id on both peers — it synchronises MAC learning, ARP and IGMP across MC-LAG members.

Verification and troubleshooting commands

show iccp
show iccp statistics
show lacp interfaces ae0
show interfaces ae0 mc-ae
show configuration protocols iccp
show log messages | match ICCP

Symptoms worth memorising: after a reboot or an ICCP configuration change, both peers using the default system ID simply means the peers have not synchronised yet — this is expected until ICCP comes up. MAC entries that persist on an MC-AE interface after both sides are disabled are also documented expected behaviour. Packets looping on the server when backup liveness packets are lost during a transient fault is likewise expected, and is the trade-off you accept for keeping the bundle up.

Relationship to our other MC-LAG notes

This page is the failure-scenario view. For the step-by-step build order see the MC-LAG ICCP liveness and LACP system ID run book, the detailed timer walk-through in backup liveness detection, and MTU-related session faults in ICCP session flap caused by MTU and keepalive mismatches.

原文链接：https://www.juniper.net/documentation/us/en/software/junos/mc-lag/topics/concept/best-practices-usage-notes.html