Leaf-Spine vs Three-Tier: 2026 Data Center Design Guide - 夜莺博客

Leaf-Spine vs Three-Tier: 2026 Data Center Design Guide

Choosing between leaf-spine and traditional three-tier is the first architecture decision in any data center refresh, and the answer changed as workloads changed: three-tier was built for client-server (north-south) traffic, while virtualization, containers, distributed storage and AI clusters generate mostly server-to-server (east-west) traffic that the old model was never designed to carry. This guide compares both topologies on the metrics that actually matter — latency, bandwidth, scalability and operations — and gives a decision checklist for 2026 networks.

This article deliberately does not re-run the historical comparison. The physics of Clos versus hierarchy, the ECMP-versus-STP argument and the 80/20 rule have been settled for years and are covered in our leaf-spine vs three-tier design article. What matters in 2026 is that four things changed at once — switch silicon crossed 100 Tbps, 800G became the default leaf port, copper started running out of reach, and racks started running out of air — and each of those shifts moves the decision in a specific direction.

How the Traditional Three-Tier Model Works

The classic hierarchy has three layers: access switches connect servers, aggregation (distribution) switches collect access traffic and provide services, and core switches interconnect the aggregation layer and route north-south. Redundancy relies on Spanning Tree Protocol, which blocks redundant links — wasting up to half the uplink capacity — and every server-to-server conversation crosses the aggregation and core, adding hops and latency. Growth means buying bigger core and aggregation switches, which is expensive and requires careful planning.

The Leaf-Spine (Clos) Alternative

Leaf-spine is a two-tier Clos topology: every leaf switch connects to every spine switch, and no leaf ever connects to another leaf. Because every path is exactly two hops — leaf to spine to leaf — latency is deterministic and independent of where the two servers live. Equal-Cost Multipath (ECMP) routing replaces STP, so all uplinks actively forward traffic instead of sitting blocked. Scaling is incremental: add a spine for more bandwidth, add a leaf for more ports, with no topology redesign.

New Variable 1: Switching Silicon Doubled to 102.4 Tbps

The single biggest change since the last refresh cycle is the switch ASIC itself. The previous generation topped out at 51.2 Tbps per chip — enough for 64 ports of 800G. The current generation doubles that: Broadcom’s Tomahawk 6 (BCM78910) and NVIDIA’s Spectrum-6 both deliver 102.4 Tbps on a single die, built on a 3 nm process, with sample I/O configurations of 64×1.6T, 128×800G, 256×400G or 512×200G. Tomahawk 6 reached production volume in March 2026, and systems are shipping now.

Two consequences matter for topology choice:

  • Radix, not bandwidth, is what collapses tiers. A 102.4 Tbps chip with 512 lanes of 200G PAM4 SerDes can attach a very large number of hosts per leaf. Broadcom’s own framing is a two-tier Clos supporting a 128,000-XPU cluster at 200 Gbps per link — previously a three- or four-tier design.
  • Fewer tiers means fewer optics and less latency. Analysts put it bluntly: two-tier at 128K endpoints means simpler load balancing, simpler congestion control and far fewer optical modules. Every tier you remove removes a whole stage of buffering, queueing and failure domain.

Three-tier topologies cannot exploit this density. The hierarchy exists to concentrate bandwidth into a small, expensive core; when a single leaf-class ASIC can hold 128×800G, the core becomes an unnecessary bottleneck rather than a scaling mechanism.

New Variable 2: 800G Leaf Ports Make Oversubscription Cheap

800G is now the standard leaf-to-server port and the standard leaf-to-spine uplink. What that changes is the achievable oversubscription ratio. In a 2026 leaf-spine, a 1RU 32×800G switch gives 25.6 Tbps of front panel capacity; pair it with 8×800G uplinks to two spines and you have a 4:1 oversubscription ratio with a two-hop path. Achieving 4:1 in a three-tier design requires careful aggregation sizing and usually a 2:1 access-to-aggregation plus 2:1 aggregation-to-core plan with blocked STP links eating the reserve.

# Arista EOS: verify ECMP width and per-flow hashing on a 2026 leaf
switch# show ip route 10.20.0.0/16
switch# show ip hardware fib ecmp 10.20.0.0/16
switch# show port-channel load-balance
switch# show interfaces Ethernet1-8 counters rates | include Ethernet

The practical rule: choose the ratio from your workload, not from a legacy template. AI training back-end fabrics want 1:1 and will pay for it; general compute and storage consolidation are comfortable at 3:1 to 5:1. What changed in 2026 is that a mid-range ratio no longer forces a three-tier design — ECMP across 800G uplinks gives you the same result with fewer devices.

New Variable 3: Copper Extends, Optics Enters

The “optics in, copper out” narrative oversimplifies what actually happened. At 200G per lane copper is genuinely near its limit: passive direct-attach copper (DAC) at 112G PAM4 reaches about 2 metres, active electrical cable (AEC) with retimers reaches roughly 4–7 metres, and 800G OSFP DAC remains the cheapest option for links under about 2.5 metres with essentially zero incremental power. Beyond that, optics wins.

The 2026 reality is coexistence, and it splits by domain:

  • Scale-up (intra-rack, XPU to switch): copper still dominates today. Active copper with retimers is extending the reach ceiling at 224G/lane rather than surrendering to optics.
  • Scale-out (rack to rack, leaf to spine): optics is already mandatory and is where the power argument now plays out.
  • The handoff point: once a cluster grows past a single rack, every link crossing a row boundary is optical. That is a leaf-spine decision, not a three-tier one — the three-tier core concentrates all cross-building traffic into a small number of very long, very expensive optical links.

This matters to the topology debate because it changes the cost model. In leaf-spine every leaf-to-spine uplink is optical, so optical cost scales linearly with ports. In three-tier the aggregation-to-core links are also optical but must be sized for peak aggregation — and, critically, half of them are blocked by STP at any moment, so you buy optical capacity you do not use.

New Variable 4: Optics Power Is Now a Design Constraint

Power per bit has become a first-class procurement metric because rack density hit a wall. A single AI rack now draws 120–150 kW, and in some regions the binding constraint is not GPU allocation but the utility power quota for the site. Optics, which used to be a rounding error, is now one of the densest heat sources in the rack.

The numbers explain why the interconnect conversation changed:

  • Conventional 800G DSP pluggable: roughly 14 W per 800G, or about 17 pJ/bit.
  • Linear-drive pluggable optics (LPO): removes the DSP and runs the host SerDes directly into the optics. Second-generation 800G DR8 LPO parts announced at OFC 2026 operate near 1 pJ/bit at under 1 W per 800G module for short intra-data-centre links.
  • Co-packaged optics (CPO): moves the optical engine onto the switch package. Broadcom’s published figure for its CPO implementation is about 5 W of optical interconnect power per 800 Gbps including lasers, against roughly 14 W for a DSP pluggable — a 60–70% cut — and package-level insertion loss drops from roughly 22 dB through a pluggable path to about 4 dB.
  • The switch ASIC itself: Tomahawk 6 is rated under 0.1 nJ/bit, a roughly 40% improvement on the previous node, and the move to chiplets is what let port density rise without a linear power increase.

For the leaf-spine-versus-three-tier decision, the implication is direct: a topology that minimises the number of optical endpoints per unit of delivered bandwidth wins on both CapEx and OpEx. Two-tier leaf-spine with 800G uplinks uses fewer, higher-rate optics than a three-tier design delivering the same east-west throughput.

New Variable 5: Liquid Cooling Changes Where Switches Can Live

Air cooling reached its practical limit around the 30–40 kW rack, which is exactly why liquid cooling moved from exotic to standard in 2026. Once racks are liquid-cooled, the network equipment in the same row must fit the same envelope.

  • Direct-to-chip cold plate designs now follow the OCP cold plate specification, and vendors are shipping liquid-cooling-compatible optical modules with integrated cold plate interfaces rated for several hundred watts of thermal dissipation per module.
  • This creates an operational split inside a single fabric: leaf switches in the liquid-cooled GPU row and spine switches in an air-cooled network cabinet have different thermal headroom, so the same optics may be derated differently at each end.
  • Three-tier topologies concentrate heat in fewer, larger core chassis. A liquid-cooled fabric spreads it across many small leaf switches, which is friendlier to heterogeneous cooling but requires the facilities team to plan coolant distribution to far more locations.

The practical takeaway: if the compute is liquid-cooled, the top-of-rack network must be liquid-cooled, and that favours the leaf-spine model where networking is distributed into the row rather than concentrated in a dedicated cooled core room.

Head-to-Head Comparison (2026 Terms)

  • Traffic pattern: three-tier suits north-south (client to server); leaf-spine suits east-west (server to server).
  • Latency: three-tier hop count varies by location; leaf-spine is always exactly two hops with predictable latency.
  • Bandwidth use: STP blocks redundant links in three-tier; leaf-spine uses ECMP across all links.
  • Scalability: three-tier scales up by replacing core hardware; leaf-spine scales out by adding spine/leaf switches.
  • Failure domain: a core switch failure in three-tier is a major event; a spine failure in leaf-spine only degrades capacity slightly.
  • Operational model: leaf-spine pairs naturally with VXLAN/BGP EVPN overlays, automation and modern telemetry.
  • Silicon fit (new): a 102.4 Tbps leaf-class ASIC replaces the entire aggregation tier; three-tier cannot use the density it paid for.
  • Power per bit (new): fewer tiers means fewer optical endpoints; LPO and CPO savings compound with topology savings.
  • Cooling fit (new): liquid-cooled rows want distributed liquid-cooled leaf switches, not a remote air-cooled core.

The classic “80/20” rule that justified three-tier is largely obsolete in virtualized environments — measure your actual east-west share before designing. For the topology math, oversubscription ratios and a migration path, our leaf-spine vs three-tier design article and the migration guide go deeper. The mechanics of how STP blocks half the links and why ECMP does not are covered in our STP blocking and bandwidth article.

Congestion Control: The Thing Tier Count Actually Buys

For AI fabrics, the tier argument is really a congestion-control argument. RDMA over Converged Ethernet (RoCEv2) is the production workhorse for Ethernet-based training fabrics, and it depends on a lossless path: PFC to prevent drops and ECN with DCQCN to signal congestion before buffers fill. Every additional tier is another set of buffers to tune, another set of ECN thresholds to align and another place for PFC storms to start.

Two tiers mean one spine hop to tune. Three tiers mean access, aggregation and core each with independent thresholds and a much harder hairpin and incast problem. This is why high-radix 102.4 Tbps switches are described by vendors as making large clusters feasible with fewer tiers rather than simply faster. If you are building a RoCEv2 fabric, read our RoCEv2 lossless tuning guide before finalising the topology, and see our AI cluster fabric comparison if InfiniBand is still on the table.

When Three-Tier Is Still the Right Answer

Three-tier remains reasonable for small campus networks, offices and environments with minimal east-west traffic where the CapEx of a full Clos redesign cannot be justified. Leaf-spine is the default for anything hosting virtualized workloads, Kubernetes, NVMe storage or AI training clusters. Many organizations keep a three-tier campus and deploy leaf-spine only in the data center — the architectures are not mutually exclusive.

What has genuinely changed in 2026 is the middle ground. There is now a large class of deployments — 200 to 2,000 servers, a few hundred kilowatts — where a single pair of 102.4 Tbps spines and a handful of 800G leaves deliver everything a three-tier core used to, at lower device count and lower optical module count. In those designs there is no core tier left to justify.

Decision Checklist for a 2026 Refresh

  1. Measure east-west traffic share and the ratio of collective operations to unicast flows.
  2. Size the fabric so the whole design fits in two tiers if possible: count endpoints, divide by leaf radix, multiply by required per-link rate.
  3. Pick leaf port speed (800G today, 1.6T planning) and uplink speed together — mismatched rates force breakout cabling and confuse capacity planning.
  4. Compute power per bit end to end: ASIC, optics, retimers, fans or pumps. Compare DSP pluggable, LPO and CPO for each link class separately.
  5. Decide the cooling domain first, then place leaf switches inside it and spines where facilities allow.
  6. Choose the control plane: eBGP underlay with an EVPN overlay is the default; confirm your team can operate it before committing. Multi-site designs add a border gateway layer — see our VXLAN EVPN multi-site guide.
  7. Budget spares by optics part number, not by link. A single ZR+ or 800G module type can cover several link budgets.
  8. Plan the refresh cadence around silicon, not around chassis end-of-life. The data rate upgrade cycle has compressed from roughly five years to one or two, which makes long chassis commitments riskier than they used to be.

Migration and Vendor Reality

Moving from three-tier to leaf-spine usually starts with the data center core: replace the aggregation pair with leaf switches, introduce two spines, and shift routing to BGP (optionally EVPN/VXLAN) so the fabric is standards-based rather than vendor-locked. Cisco (ACI/Spine-Leaf), Arista and Juniper all ship validated leaf-spine designs; choose based on your automation maturity, not just port pricing. A related comparison of platform vendors is in our architecture comparison article.

One 2026-specific warning about that migration: if you are moving to 800G leaf ports at the same time as changing topology, do them as separate projects. A topology change and a port-speed change in the same window means every problem you hit has two possible causes, and the rollback is twice as expensive. Change the topology at your existing port speed, stabilise, then upgrade the optics.

原文链接:https://network-switch.com/blogs/switches/leaf-spine-vs-3-tier-architecture