Port-Channel Load Balancing Hash Algorithm Tuning - 夜莺博客

Port-Channel Load Balancing Hash Algorithm Tuning

You bundled four links, the port-channel came up cleanly and STP is nowhere in sight — yet one member carries 90% of the traffic while the others idle. Link aggregation does not balance load by itself; it hashes each flow onto a member, and the fields used in that hash decide whether your traffic spreads or piles onto one cable. This guide covers the available hash methods, how to choose one for a given traffic pattern, and how to prove the distribution is actually even.

How the hash works

The switch computes a hash over selected packet header fields and uses the result to pick a member link. The algorithm is deterministic, which is the feature, not a bug: packets of the same flow always land on the same member so they stay in order. Distribution is therefore only as good as the variability of the hashed fields.

On classic Catalyst hardware the hash produces a value in the 0–7 range, so eight members distribute perfectly and, for example, three members receive a 3:3:2 split of the hash buckets. Where you cannot control which flow maps to which member, you can only influence the load balance by choosing a method with the greatest variety.

The available methods

Switch(config)# port-channel load-balance ?
  dst-ip                 Dst IP Addr
  dst-mac                Dst Mac Addr
  dst-mixed-ip-port      Dst IP Addr and TCP/UDP Port
  dst-port               Dst TCP/UDP Port
  mpls                   Load Balancing for MPLS packets
  src-dst-ip             Src XOR Dst IP Addr
  src-dst-mac            Src XOR Dst Mac Addr
  src-dst-mixed-ip-port  Src XOR Dst IP Addr and TCP/UDP Port
  src-port               Src TCP/UDP Port
Method Use it when Watch out for
src-dst-mac Pure Layer 2 access with many distinct MACs Routed traffic has one router MAC pair — distribution collapses
src-dst-ip Routed traffic between many hosts Few host pairs (e.g. one server to one storage array) still collide
src-dst-port Many parallel TCP sessions between few IP pairs Non-TCP/UDP protocols hash poorly
src-dst-mixed-ip-port Modern platforms, mixed east-west traffic Needs platform support

The default on many Catalyst platforms is effectively source XOR destination IP for IPv4 and source XOR destination MAC for non-IP frames — check your own platform rather than assuming.

Configure it

# Global (applies to all port-channels on classic platforms)
Switch(config)# port-channel load-balance src-dst-ip
Switch(config)# end
Switch# show etherchannel load-balance
EtherChannel Load-Balancing Configuration:
        src-dst-ip

# Some platforms allow per-port-channel tuning
Switch(config)# interface port-channel 10
Switch(config-if)# port-channel load-balance src-dst-mixed-ip-port

Prove the distribution instead of guessing

Switch# show etherchannel 10 summary
Switch# show etherchannel 10 port-channel
Switch# show interfaces port-channel 10 | include packets/sec

# Per-member utilisation
Switch# show interfaces Gi1/0/1 | include rate
Switch# show interfaces Gi1/0/2 | include rate

For a lab or a maintenance window you can often ask the switch directly which link a given flow will use, using the platform's load-balance test command (for example test etherchannel load-balance interface port-channel 10 ip 10.0.0.1 10.0.0.2 in privileged mode). Feed it a handful of real source/destination pairs and you will see immediately whether they all hash to the same member.

The arithmetic you are looking for: with four members carrying four large elephant flows, the best possible distribution is 25% per member. If you measure 70/20/5/5, the flows are colliding and the fix is a different hash, more flows, or a different topology.

The design answer when hashing is not enough

  • Few, large flows are fundamentally hard to balance. A single TCP session cannot exceed one link's bandwidth. More sessions (or per-packet/adaptive load balancing on platforms that support it) is the only real remedy.
  • ECMP at Layer 3 does the same job with better granularity in spine-leaf designs; a host with a 4×10G bond to a single switch pair is far more constrained than four routed paths.
  • LACP hashing is per-flow, always. Any expectation of "the aggregate is 40G for one server" is wrong unless the application opens many concurrent flows.
  • Mismatched hash on the two ends is not a problem — each direction hashes independently. What must match is the bundle configuration (mode, speed, VLANs, MTU).

Checklist before you change the hash

  1. Record per-member utilisation over a busy period — baseline first.
  2. Identify the dominant flow pattern: many-to-many (routed), many-to-one (backup job), or one-to-one (replication).
  3. Pick the hash with the most variability for that pattern.
  4. Change it in a window: changing the hash re-shuffles flows and can cause brief reordering on affected sessions.
  5. Re-measure. If the distribution is still skewed, the problem is flow count, not the algorithm.

Hash tuning is not a silver bullet, but it is the cheapest 10-minute change in the L2 toolkit — and measuring before and after is what turns it from folklore into engineering.

Related Reading on This Site

原文链接:https://www.cisco.com/c/en/us/support/docs/lan-switching/etherchannel/12023-4.html (Cisco - Understand EtherChannel Load Balance and Redundancy on Catalyst Switches)