EVPN Multihoming vs MLAG: ESI, DF Election and Clos - 夜莺博客

EVPN Multihoming vs MLAG: ESI, DF Election and Clos

Every server rack needs its top-of-rack switches to look like one device, and there are now two standard answers. MLAG (also called vPC, VLT, VSX or MC-LAG depending on the vendor) is the incumbent: a proprietary peer link synchronises MAC and state between two switches. EVPN multihoming is the challenger: BGP advertises Ethernet Segment identifiers, and route types 1, 2 and 4 carry the state that MLAG used to tunnel across a peer link. This article compares the two on the three axes that matter in a Clos fabric - control plane, forwarding behaviour and failure modes - and ends with the selection criteria that decide between them.

How Each One Builds Redundancy

  • MLAG: two switches form a peer group over a dedicated peer link; a host LAG has one member on each switch. The peer link carries control traffic, synchronised MAC/ARP entries and any traffic that must cross to reach a member link on the other side. The pair must agree on the LACP system ID so the server sees a single aggregator.
  • EVPN-MH: both switches are independent PEs in the same EVPN fabric. Each Ethernet Segment gets a 10-byte ESI, advertised in a Type-4 Ethernet Segment route, and DF (designated forwarder) election decides which PE forwards BUM traffic for each segment. No peer link is needed: state travels in BGP.

The structural difference is worth stating plainly. MLAG makes two switches become one device, so the pair needs its own protocol, its own link and its own synchronisation logic. EVPN multihoming leaves two switches as two devices and instead standardises the information they must agree on - which segment the host is attached to, which MAC addresses live on it, and who forwards BUM traffic - as BGP routes that any PE in the fabric can process. Everything else in this comparison follows from that.

Control-Plane Pieces: ESI, Route Types and DF Election

The ESI is a ten-byte identifier that tells the fabric "this attachment circuit is the same Ethernet Segment as the one those other PEs are advertising". RFC 7432 defines several ESI types and each one encodes a different notion of identity: type 0 is manually configured, type 1 derives from LACP information (the switch's own system MAC plus the port key), type 2 comes from the spanning-tree topology, type 3 from the CE's MAC address, type 4 from a router identifier, and type 5 from an autonomous system number. In an all-active server design, type 1 with a deterministic LACP-backed ESI is the usual choice because it stays stable across a reload of either PE and is derived rather than typed.

Four route types carry the state that MLAG used to synchronise across its peer link:

  • Type-1 (Ethernet Auto-Discovery, per ES): advertised by every PE attached to the segment. It is what makes the ES visible fabric-wide and what allows a remote PE to withdraw reachability for the whole segment in a single update when the last local link goes down.
  • Type-2 (MAC/IP Advertisement): the MACs learned on the segment, with the ESI attached. The ESI is the hook that lets a remote PE understand "this MAC is reachable through several PEs" instead of treating each advertisement as a competing unicast path.
  • Type-3 (Inclusive Multicast Ethernet Tag): the PE's VTEP address and BUM reception intent, used for flooding and for building the ingress replication list.
  • Type-4 (Ethernet Segment route): carries the ESI plus the originating PE address, and it is the message the DF election algorithm consumes. A PE only considers itself multihomed onto a segment when it sees its own Type-4 route reflected back with the other PEs' addresses attached.

DF election itself has two algorithms that matter in practice. The default in RFC 7432 is the service-carving algorithm, which computes the designated forwarder from the VLAN identifier and the number of PE routers attached to the segment, spreading different VLANs across different PEs. RFC 8584 adds the Highest Random Weight algorithm, which produces a more stable assignment across membership changes and, crucially, adds the AC-influenced capability - the ability to weight the election result based on the state of the local attachment circuit. On a platform that supports AC-DF, a PE whose local links to the host are down can be excluded from the election, so BUM does not get sent to a PE that cannot deliver it.

Forwarding and BUM Traffic

In MLAG, an all-active pair both forward known unicast; BUM handling depends on vendor implementation and the peer link must absorb flooded traffic. In EVPN-MH, the DF election per ES and per VLAN is explicit: one PE is DF and forwards BUM, the others are non-DF and drop it, which gives deterministic behaviour and avoids loops without relying on a peer link. When the DF fails, election re-runs and another PE takes over.

Two EVPN mechanisms deserve separate names because they are the ones that replace MLAG's internal logic:

  • Aliasing. Because the Type-2 advertisement carries the ESI, a remote PE knows that a MAC reachable through one PE of the segment is equally reachable through the others. It can therefore load-share toward the multihomed host across all PEs of the segment rather than pinning the flow to the PE that happened to advertise the MAC. This is the equivalent of MLAG's ability to use both chassis for the same host.
  • Split-horizon. The PE that receives a BUM frame from the segment must not send it back onto the same segment, which is expressed by the ESI in the forwarding path. Without split-horizon, multihoming would loop between PEs. In MLAG the equivalent guarantee comes from the peer-link design rules; in EVPN it is a protocol property.

Comparison

Aspect MLAG / vPC / VLT EVPN multihoming
Control plane Proprietary peer protocol BGP EVPN (RFC 7432, RFC 8584 DF election)
Peer link required Yes, must be sized for failure No
Number of PEs per host Exactly two Two or more
MAC/state sync Over the peer link Fabric-wide via BGP
Convergence Peer link or protocol specific Mass withdrawal and ES routes
Failure isolation Peer link failure can split the pair Underlay failure is just a BGP event
Loop prevention for BUM Peer-link design rules and primaries DF election, split-horizon
Load sharing to the host Hash across both chassis Aliasing across all PEs of the segment
Vendor interoperability Vendor-specific, no standard Standards-based, multi-vendor
Typical scale limit One pair per host rack ESI count bounded by BGP, not by pairs

Failure Modes Side by Side

The comparison above is architectural; the failure modes are what people actually operate. In an MLAG pair, the failure that hurts is the one that takes out the peer link or the state synchronisation while both switches stay up. The two chassis then disagree about what they own, and depending on the vendor's rules you get a dual-primary state in which both advertise the same LACP identity and both forward for the host. Recovery usually means one chassis giving up its role, which is a manual or timer-driven decision rather than a protocol one.

In EVPN multihoming, the equivalent failures decompose into BGP events. Lose one PE and the Type-4 route for its ESI is withdrawn, the election re-runs, and the remaining PE takes over the BUM forwarding for the VLANs it wins. Lose the underlay path between two PEs and the session between them goes down, which is a routing problem rather than a Layer 2 problem. There is no peer link that can split, because there is no peer link.

Two EVPN-specific failure modes are worth knowing anyway. If the ESI is misconfigured on one PE - a single digit different - then the two PEs advertise two different segments, DF election runs separately on each, and the host effectively has two independent single-homed attachments. Traffic continues, which is why this fault survives casual testing. If the DF election capability is inconsistent across the fabric, some PEs use service carving while others use HRW, and two PEs can both believe they are DF for the same VLAN. Both faults are configuration-consistency problems, and both are detectable by reading the ES and DF state on every PE of the segment rather than only on two of them.

Where MLAG Still Wins

MLAG remains the pragmatic choice when the fabric is not EVPN: a small campus or a standalone pair with no BGP underlay, an environment where the operational team already knows the vendor tooling, and legacy platforms without ESI support. It also requires less design work - two switches, one peer link, one system ID.

Add two more reasons that are usually omitted. First, tooling maturity: peer-link diagnostics, consistency checkers and vendor runbooks for MLAG are extremely well developed, and in a two-switch deployment there is nothing for a BGP policy to get wrong. Second, migration cost: if the fabric is not already carrying EVPN, then choosing multihoming means building an EVPN control plane - address families, route targets, route reflectors - before the first host is dual-homed, which is a much larger project than a peer link.

Where EVPN-MH Wins

  1. Clos fabrics: no extra peer link cabling and no peer-link bandwidth reservation inside the leaf pair.
  2. Scale: 2+ multihoming (a host can be dual-homed to more than two leafs) and per-host ESIs that BGP can carry across pods.
  3. Consistency: the same mechanism that provides Layer 2 extension (EVPN) also provides redundancy, so there is one control plane to learn, monitor and automate.
  4. Convergence: failure is handled by route withdrawal, which is already the fabric's fastest signal.
  5. Multi-vendor attachment: because the ESI, the route types and the election algorithm are standards, a host can be dual-homed to leafs from different vendors without either side implementing the other's peer protocol.

Migrating Without an Outage

The practical migration path from MLAG to EVPN multihoming keeps the two mechanisms apart until the fabric is ready. Build the EVPN control plane first - EVPN address family on the underlay sessions, route targets for the tenant VRFs, Type-3 reachability between all VTEPs - and confirm that a test host attached to a single leaf works before any host is multihomed. Then move one rack at a time: remove the MLAG peer configuration from a leaf pair, convert the existing LACP port-channel to an ESI-based attachment by adding the ESI and the ES-import route target, and verify the Type-4 route appears on the other PEs. The host-side configuration does not change, because the host still sees a standard LACP bundle; that is the property that makes this migration tractable.

Operational Checks Before You Choose

! verify ESI and DF state
show evpn ethernet-segment detail
show bgp l2vpn evpn route-type 4
show evpn ethernet-segment interface Ethernet1 designated-forwarder

! MLAG equivalent, for comparison
show mlag
show mlag interfaces
show vpc

Read three things out of that output: the ESI value itself, which must be identical on every PE attached to the segment; the list of PEs the local switch believes are attached to the segment, which comes from the Type-4 routes it has received; and the DF state per VLAN, which must show exactly one designated forwarder for each VLAN of the segment. A missing PE in the second list means a BGP or policy problem, and two DFs in the third means inconsistent election capability.

Whichever you deploy, test three scenarios in the lab before go-live: single member-link failure, one switch power failure, and (for MLAG) peer link failure with the "liveness" feature both enabled and disabled. The third scenario is the one that produces surprises in production.

Related reading: Juniper EVPN ESI-LAG multihoming configuration, BGP in EVPN data center fabric design and MLAG dual-primary and peer link troubleshooting.

原文链接:NVIDIA: Comparing solutions for boosting data center redundancy