MTU, MSS and PMTUD: Fixing Black-Hole Connections

The classic MTU black hole looks like this: ping works, small web pages load, and any large transfer stalls forever. Nothing is broken at layer 1 or 2 - a path element has a smaller MTU, the ICMP "fragmentation needed" message is filtered, and PMTUD never learns the real limit. This article explains the relationship between MTU and MSS, why PMTUD fails in practice, and how to fix it deterministically.

MTU, MSS and the Arithmetic

IPv4: MSS = MTU - 20 (IP header) - 20 (TCP header)   = MTU - 40
IPv6: MSS = MTU - 40 (IP header) - 20 (TCP header)   = MTU - 60

MSS is advertised in the SYN and is a promise about what the receiver can handle. It is computed from the interface MTU, not from the path MTU - that is the whole problem. A router interface set to 1500 will advertise MSS 1460 even when a tunnel downstream adds 60 bytes of overhead.

Why PMTUD Fails

• Firewalls and load balancers routinely drop ICMP type 3 code 4 (fragmentation needed).
• Asymmetric paths mean the ICMP message returns via a device that does not know about the flow.
• Tunnels (GRE, IPsec, VXLAN) change the effective MTU but the endpoints are unaware.
• Security policies that block all ICMP break the mechanism at the first hop.

Because of this, PMTUD cannot be the only control. Treat it as an optimisation and use MSS clamping as the deterministic fallback.

Fix 1 - Clamp MSS at the Tunnel or Edge

interface Tunnel0
 ip tcp adjust-mss 1360
! IPsec with ESP overhead:
interface Dialer1
 ip tcp adjust-mss 1360
! For an 8-byte GRE plus IPsec overhead, 1360-1400 is a common safe value

Clamping rewrites the MSS option in passing SYNs, so both endpoints negotiate a segment size that fits the narrowest link. It is applied where the tunnel terminates or on the WAN edge, and it costs nothing when the path is already correct.

Fix 2 - Verify Per-Interface MTU

show interfaces GigabitEthernet1/0/1 | include MTU
show ip interface Tunnel1 | include MTU
show platform software fed switch active ifm interfaces tunnel
ping 10.0.0.1 size 1472 df-bit
ping 10.0.0.1 size 1500 df-bit

On Catalyst 9000 there are two MTU views to keep straight: the software IP MTU seen by the routing stack and the hardware MTU programmed into the forwarding ASIC. A DF-bit ping sweep - binary searching the largest size that succeeds - is still the fastest way to establish the real path MTU between two points.

Fix 3 - Tunnel and Overlay MTU Planning

For overlay designs, set the underlay MTU high enough to carry the encapsulated frame without fragmenting, and document the resulting guest MTU. Data-centre designs commonly run 9214-byte jumbo frames on the underlay; the interaction with host NICs and switches is covered in jumbo frames and MTU mismatch troubleshooting. For IPv6, remember the minimum 1280-byte MTU every node must support, and the smaller MSS that follows - see IPv6 neighbour discovery and router advertisements. Where NAT is in the path, MSS clamping is usually still needed because NAT rewrites do not change advertised MSS - the commands are in IOS NAT/PAT overload and port forwarding.

原文链接：https://www.cisco.com/c/en/us/support/docs/switches/catalyst-9500-series-switches/217233-troubleshoot-mtu-on-catalyst-9000-series.html