MTU, MSS and PMTUD: Fixing Black-Hole Connections

The classic MTU black hole looks like this: ping works, small web pages load, and any large transfer stalls forever. Nothing is broken at layer 1 or 2 - a path element has a smaller MTU, the ICMP "fragmentation needed" message is filtered, and PMTUD never learns the real limit. This article explains the relationship between MTU and MSS, why PMTUD fails in practice, and how to fix it deterministically.

MTU, MSS and the Arithmetic

IPv4: MSS = MTU - 20 (IP header) - 20 (TCP header)   = MTU - 40
IPv6: MSS = MTU - 40 (IP header) - 20 (TCP header)   = MTU - 60

MSS is advertised in the SYN and is a promise about what the receiver can handle. It is computed from the interface MTU, not from the path MTU - that is the whole problem. A router interface set to 1500 will advertise MSS 1460 even when a tunnel downstream adds 60 bytes of overhead.

Why PMTUD Fails

• Firewalls and load balancers routinely drop ICMP type 3 code 4 (fragmentation needed).
• Asymmetric paths mean the ICMP message returns via a device that does not know about the flow.
• Tunnels (GRE, IPsec, VXLAN) change the effective MTU but the endpoints are unaware.
• Security policies that block all ICMP break the mechanism at the first hop.

Because of this, PMTUD cannot be the only control. Treat it as an optimisation and use MSS clamping as the deterministic fallback.

Fix 1 - Clamp MSS at the Tunnel or Edge

interface Tunnel0
 ip tcp adjust-mss 1360
! IPsec with ESP overhead:
interface Dialer1
 ip tcp adjust-mss 1360
! For an 8-byte GRE plus IPsec overhead, 1360-1400 is a common safe value

Clamping rewrites the MSS option in passing SYNs, so both endpoints negotiate a segment size that fits the narrowest link. It is applied where the tunnel terminates or on the WAN edge, and it costs nothing when the path is already correct.

Fix 2 - Verify Per-Interface MTU

show interfaces GigabitEthernet1/0/1 | include MTU
show ip interface Tunnel1 | include MTU
show platform software fed switch active ifm interfaces tunnel
ping 10.0.0.1 size 1472 df-bit
ping 10.0.0.1 size 1500 df-bit

On Catalyst 9000 there are two MTU views to keep straight: the software IP MTU seen by the routing stack and the hardware MTU programmed into the forwarding ASIC. A DF-bit ping sweep - binary searching the largest size that succeeds - is still the fastest way to establish the real path MTU between two points.

Fix 3 - Tunnel and Overlay MTU Planning

For overlay designs, set the underlay MTU high enough to carry the encapsulated frame without fragmenting, and document the resulting guest MTU. Data-centre designs commonly run 9214-byte jumbo frames on the underlay; the interaction with host NICs and switches is covered in jumbo frames and MTU mismatch troubleshooting. For IPv6, remember the minimum 1280-byte MTU every node must support, and the smaller MSS that follows - see IPv6 neighbour discovery and router advertisements. Where NAT is in the path, MSS clamping is usually still needed because NAT rewrites do not change advertised MSS - the commands are in IOS NAT/PAT overload and port forwarding.

The Overhead Arithmetic Behind a Safe MSS Value

Every encapsulation eats bytes from the payload MTU, and the smaller the payload the smaller the TCP segment that will fit. These are the overheads that cause the majority of black holes in production networks:

• GRE without key or sequence numbers: 24 bytes (20-byte outer IP header plus 4-byte GRE header). Adding key and sequence fields takes it to 32 bytes.
• IPsec ESP in tunnel mode with AES-GCM: 56 to 73 bytes, depending on IV and ICV sizes plus block padding. When you tunnel GRE inside IPsec, add the two overheads together rather than picking the larger one.
• VXLAN: 50 bytes for the new outer Ethernet, IP, UDP and VXLAN headers; 36 bytes if the outer Ethernet header is already accounted for by the underlay MTU budget.
• WireGuard: 60 bytes of overhead for the outer IP header, the 32-byte WireGuard header, the 16-byte Poly1305 tag and padding to a 16-byte boundary.
• PPPoE: 8 bytes on top of the 14-byte Ethernet header, which is where the familiar 1492-byte MTU and 1452-byte MSS come from.
• 6in4, SIT and other IPv6 transition tunnels: 20 bytes when the outer header is IPv4 and 40 when it is IPv6.

You never have to memorise the list if you apply the rule instead: find the smallest MTU on the path, subtract every header the packet will accumulate, and set the tunnel interface MTU to the result. Clamp MSS to that value minus 40 for IPv4 or minus 60 for IPv6. Prescriptive configuration beats arithmetic guesswork, but only when the numbers are written down in the design document rather than rediscovered during an outage.

# Establish the real path MTU between two hosts using a DF-bit ping sweep
for size in 1472 1452 1400 1372 1350 1300; do
  ping -M do -s $size -c 1 10.0.0.1 >/dev/null 2>&1 \
    && echo "OK   $size" || echo "FAIL $size"
done
# 1472 = 1500 - 28 bytes (20-byte IP header + 8-byte ICMP header)

Read the sweep from the largest size downwards: the first successful size tells you the payload MTU, and 28 bytes above that is the path MTU. A sweep that fails everywhere below the interface MTU, while a plain ping succeeds, is the signature of a policy that drops ICMP fragmentation-needed messages rather than a genuinely narrow link.

Reading the Advertised MSS from the Wire

The fastest way to confirm what two hosts actually negotiated is to capture the SYN and read the MSS option. If the advertised value is larger than the path can carry, you have found the bug without touching a single counter.

sudo tcpdump -ni eth0 -vv 'tcp[tcpflags] & tcp-syn != 0' -c 5
# Look for: options [mss 1460,sackOK,TS val ...],nop,wscale 7
# A value of 1460 on a path that only carries 1400 is the black hole

# Watch for the ICMP that never arrives
sudo tcpdump -ni eth0 'icmp[icmptype] == icmp-unreach'

Capture on both ends if you can. The sender shows what it offered, the receiver shows what it would accept, and the difference between the two is either normal asymmetry or a middlebox that is rewriting options it should not touch.

Clamping on the Host: nftables and iptables

Routers are not the only place MSS is clamped. Any Linux host acting as a NAT gateway, a VPN concentrator or a container node can rewrite the MSS option with a mangling rule, and this is the standard fix for Kubernetes and Docker environments where pod-to-pod traffic crossing VXLAN or WireGuard silently truncates large transfers.

# nftables: clamp TCP MSS on forwarded traffic to the route MTU
nft add table inet mangle
nft add chain inet mangle forward '{ type filter hook forward priority mangle; }'
nft add rule inet mangle forward tcp flags syn tcp option maxseg size set rt mtu

# iptables: the classic OpenVPN fix, applied to the outbound path
iptables -t mangle -A FORWARD -p tcp --tcp-flags SYN,RST SYN \
  -j TCPMSS --clamp-mss-to-pmtu
iptables -t mangle -A OUTPUT -p tcp --tcp-flags SYN,RST SYN \
  -j TCPMSS --set-mss 1360

The --clamp-mss-to-pmtu target reads the route MTU of the outgoing interface, so the rule keeps working when you change the tunnel MTU later. --set-mss is blunt but correct when the path MTU cannot be derived from the routing table, for example on a load balancer fronting clients from the whole internet. On modern distributions prefer the nftables rule; iptables and nftables cannot both own the same hook without confusing results.

PMTUD versus PLPMTUD: Two Different Betting Strategies

Classic PMTUD depends on ICMP type 3 code 4 messages coming back, which is exactly the traffic that firewalls, load balancers and security appliances drop by default. RFC 4821 defines Packetization Layer Path MTU Discovery, which probes with progressively larger segments and infers failure from the transport layer instead of trusting ICMP. Linux exposes the behaviour per socket with IP_MTU_DISCOVER, and globally through a small set of sysctls.

sysctl net.ipv4.ip_no_pmtu_disc    # 0 = follow ICMP, 1 = always fragment, 2 = use local MTU only
sysctl net.ipv4.tcp_mtu_probing    # 1 = probe after a black hole, 2 = always probe
sysctl net.ipv4.tcp_base_mss       # starting MSS for probing (default 1024)
sysctl net.ipv4.route.min_pmtu     # floor for PMTU, 552 by default

tcp_mtu_probing = 1 is the pragmatic middle ground: the host behaves normally until it detects a black hole through repeated retransmissions with no progress, then switches that connection into probing mode. It repairs the symptom for that host but does nothing for the path, so treat it as a diagnostic crutch and a safety net rather than the permanent answer. Raise min_pmtu only if you understand that it also sets the floor for the MSS the kernel will ever use.

Platform Clamp and Verify Commands

! Cisco IOS / IOS XE
interface GigabitEthernet0/0/0
 ip tcp adjust-mss 1360
show ip interface GigabitEthernet0/0/0 | include MTU

! Cisco Nexus / NX-OS
interface Ethernet1/1
 ip tcp mss 1360
show running-config interface Ethernet1/1 | include mss

! Juniper Junos - the clamp lives under the security flow stanza
set interfaces ge-0/0/0 unit 0 family inet mtu 1400
set security flow tcp-mss all-tcp mss 1360
show interfaces ge-0/0/0 extensive | match MTU

! Arista EOS
interface Ethernet1
 tcp mss ceiling ipv4 1360
show interfaces Ethernet1 | include MTU

Note the interface-level keyword differences: IOS XE uses ip tcp adjust-mss, NX-OS uses ip tcp mss, and EOS uses a tcp mss ceiling that also takes an IPv6 form. On Junos the clamp lives in the security flow configuration, which regularly surprises engineers who expect it underneath the interface. None of these keywords change the arithmetic; they only decide where the rewrite happens.

Common Mistakes and a Working Checklist

1. Clamping the wrong direction. MSS is a receiver-side promise about what that host can receive. To control what the other end sends, you clamp the SYN arriving from it, which on a stateful firewall means the inbound direction of that flow.
2. Clamping only TCP. ICMP, UDP-based overlays such as VXLAN and QUIC, and anything with DF set will still black-hole. Fix the tunnel MTU, not only the MSS.
3. Forgetting IPv6. The same path carries 20 bytes more header on IPv6, so an MSS that is safe for IPv4 is exactly 20 bytes too large for a v6 connection.
4. Trusting the CLI over the ASIC. The software IP MTU and the hardware forwarding MTU are programmed by different code paths and can disagree. Verify both.
5. Blocking all ICMP for policy reasons. Allow ICMP type 3 code 4 inbound and ICMPv6 type 2 (packet too big); without them PMTUD cannot function, and no amount of clamping will make it.

The verification loop is always the same: DF-bit ping sweep to learn the path MTU, compare it against the configured interface MTU, capture the handshake to read the advertised MSS, then clamp on the narrowest device. When the same path also carries an overlay, the underlay budget is covered in VXLAN MTU and underlay jumbo frames, and the WireGuard-specific handshake failure mode is documented in WireGuard MTU handshake troubleshooting. For host-side buffer and retransmission counters that mask a black hole, see Linux network tuning with sysctl.

原文链接：https://www.cisco.com/c/en/us/support/docs/switches/catalyst-9500-series-switches/217233-troubleshoot-mtu-on-catalyst-9000-series.html