Juniper MC-LAG Design: ICCP Liveness, ICL and Hold Timers

MC-LAG lets two switches present one LACP bundle to a downstream server or switch, removing the chassis as a single point of failure - but the design details decide whether failures converge cleanly or cause flaps. The two control/data paths have different jobs: ICCP (Inter-Chassis Control Protocol) synchronizes state between the peers, while the ICL (inter-chassis link) forwards data traffic between them. When engineers tune one without the other, a link failure that should be invisible turns into a double failover. This article covers the design parameters that matter: ICCP transport and liveness, ICL sizing, and hold timers.

ICCP: Control over TCP, Failure Detection over BFD

MC-LAG peers exchange control information over ICCP, which runs over TCP/IP and therefore needs reachability between the two devices. The Junos configuration anchors ICCP on the loopback addresses and uses BFD for fast failure detection:

set protocols iccp local-ip-address 10.255.0.1
set protocols iccp peer 10.255.0.2 session-establishment-hold-time 50
set protocols iccp peer 10.255.0.2 redundancy-group-id-list 1
set protocols iccp peer 10.255.0.2 liveness-detection minimum-interval 60
set protocols iccp peer 10.255.0.2 liveness-detection multiplier 3

Three design points hide in these lines:

• Loopback as the ICCP source - a dedicated link between the peers is not strictly required for ICCP (it is for the ICL data path), but using loopback addresses makes the ICCP session independent of any single inter-chassis link, and BFD on top gives ~180 ms failure detection with a 60 ms interval and multiplier 3.
• session-establishment-hold-time 50 - recommended so the ICCP session comes up quickly and the redundancy group stabilizes early.
• redundancy-group-id-list must match on both peers and be referenced by the MC-AE configuration - mismatches here produce MC-LAG interfaces that never activate.

The ICL and the Hold-Time Rule

The ICL forwards data traffic between the peers (for example when a downstream server sends to a MAC learned only on the peer). Because ICCP and ICL are separate paths, a failure can hit them in either order - and the order decides whether the MC-LAG flaps:

set interfaces ae0 aggregated-ether-options hold-time up 1000 down 1000

Configure a hold-down timer on the ICL member links that is greater than the BFD timer for the ICCP session. The reasoning: if the ICL is reported down before ICCP detects the peer is gone, the status-control standby node sees its MC-LAG interface flap and convergence is delayed. Holding the ICL down announcement until ICCP/ BFD has already declared the peer dead lets the failure be handled as one deterministic event instead of two cascading ones. This is the single most commonly missed parameter in MC-LAG designs.

Failure Behavior: ICCP Down vs ICL Down

Knowing what the peers do in each scenario prevents "it worked in the lab" surprises:

• ICCP down: the backup MC-LAG peer changes its LACP system ID and brings down the MC-AE interfaces - the deterministic behavior that prevents the downstream device from seeing two different LACP partners and flapping between them (detailed in our ICCP failure: LACP system ID guide and the QFX-specific follow-up here).
• Double failover ordering: if ICCP goes down and then the active MC-LAG peer goes down, or an MC-LAG peer in active mode is lost, the sequence matters - Junos documentation describes these scenarios explicitly (secondary peer with status control set to standby becoming inactive, and L3 traffic to the standby peer not being redirected when the remote peer's status is unknown).
• Split-brain: if the inter-peer link fails but both peers stay up, both devices may believe they are primary; the ICCP/keepalive design and the LACP system-ID behavior above are what break the tie.

Design Checklist

• ICCP between loopbacks with BFD liveness (60 ms x 3 is a sane starting point).
• ICL sized for worst-case redirected traffic (it carries what the peer cannot switch locally).
• ICL hold timers longer than ICCP BFD detection.
• Identical redundancy-group-id-list and consistent MC-AE config on both peers.
• Understand the ICCP-down failure behavior before deploying - the underlying mechanism is explained in our MC-LAG/ICCP/BFD deep dive.

原文链接：https://packet-expert.org/2016/09/15/multi-chassis-lag/ · Juniper TechLibrary: Advanced MC-LAG Concepts