MTU Probing, TCP MSS Clamping & Path MTU Discovery (PMTUD) Troubleshooting
Diagnose and eliminate cross-border PMTUD blackholes where ping succeeds but HTTPS, SSH, or WireGuard tunnels hang, using ICMP DF probing, tcp_mtu_probing, and iptables MSS clamping.
One of the most perplexing networking symptoms on international links or encrypted overlay tunnels (WireGuard, IPsec, GRE, VXLAN) is the PMTUD (Path MTU Discovery) Blackhole: ICMP
pingworks with 0% loss, initial TCP three-way handshakes succeed, and small SSH commands respond instantly—yet runninghtop,git clone, or loading an HTTPS certificate hangs indefinitely. This guide explains how MTU & TCP MSS work and how to permanently resolve fragmentation blackholes.
1. Understanding MTU, TCP MSS, and PMTUD Blackholes
- MTU (Maximum Transmission Unit): The largest Layer-3 IP packet (in bytes) that a network interface can transmit without fragmentation. Standard Ethernet defaults to
1500bytes. - MSS (Maximum Segment Size): The maximum TCP payload inside a single IP packet, calculated as
MTU - 20 bytes (IPv4 Header) - 20 bytes (TCP Header). For a1500MTU link, standard IPv4MSS = 1460bytes (or1440bytes over IPv6 due to the 40-byte IPv6 header).
Common Network MTU & MSS Reference Table:
| Link / Tunnel Type | Encapsulation Overhead | Optimal Interface MTU | Optimal IPv4 TCP MSS | Optimal IPv6 TCP MSS |
|---|---|---|---|---|
| Standard Datacenter Ethernet | 0 Bytes |
1500 |
1460 |
1440 |
| Residential PPPoE Broadband | 8 Bytes |
1492 |
1452 |
1432 |
| WireGuard over IPv4 | 60 Bytes |
1420 |
1380 |
1360 |
| WireGuard over IPv6 | 80 Bytes |
1400 |
1360 |
1340 |
[!WARNING] Why Do Connections Hang? Modern TCP packets set the
DF(Don’t Fragment) bit. When a 1500-byte packet hits a residential PPPoE router (1492MTU) or tunnel interface (1420MTU), the router drops the oversized packet and returns an ICMPFragmentation Needed (Type 3, Code 4)message. If an intermediate firewall drops that ICMP notification, the sender keeps retransmitting 1500-byte packets forever—resulting in a hung connection (PMTUD Blackhole).
2. Probing Exact Path MTU with ping (Don't Fragment)
You can determine the exact maximum unfragmented packet size supported between your server and any target IP by setting the DF (Don’t Fragment) flag.
Note that ping -s specifies the ICMP payload size, which excludes the 20-byte IPv4 header and 8-byte ICMP header (28 bytes total overhead):
# Test if standard 1500 MTU (1472 payload + 28 header = 1500) passes cleanly
ping -M do -s 1472 -c 4 target_ip
# Test if PPPoE 1492 MTU (1464 payload + 28 header = 1492) passes cleanly
ping -M do -s 1464 -c 4 target_ip
If ping -M do -s 1472 returns message too long, mtu=1492 or 100% packet loss, while -s 1464 or -s 1392 succeeds immediately, you have confirmed a Path MTU bottleneck along the route.
For IPv6 (where the header overhead is 40 bytes IPv6 + 8 bytes ICMPv6 = 48 bytes):
# Test 1500 MTU over IPv6 (1452 + 48 = 1500)
ping6 -M do -s 1452 -c 4 2001:4860:4860::8888
3. Solution 1: Enable Kernel RFC 4821 Smart MTU Probing
The Linux kernel includes built-in Packetization Layer Path MTU Discovery (PLPMTUD, RFC 4821). When enabled, if the kernel detects repeated TCP retransmissions indicative of an ICMP blackhole, it automatically reduces the TCP segment size on that connection without relying on external ICMP packets:
cat << 'EOF' | sudo tee /etc/sysctl.d/98-pmtud.conf
# Enable smart TCP Path MTU Probing when blackholes are detected (1 = on blackhole, 2 = always)
net.ipv4.tcp_mtu_probing = 1
# Set safe fallback base MSS (1024 bytes) for blackhole recovery
net.ipv4.tcp_base_mss = 1024
EOF
sudo sysctl --system
4. Solution 2: Automatic TCP MSS Clamping (iptables / nftables)
If your FlawlessNode cloud server operates as a WireGuard / IPsec gateway, router, or container host, applying TCP MSS Clamping on the mangle table automatically rewrites the MSS field in TCP SYN handshake packets to match the outgoing interface’s Path MTU:
4.1 Applying MSS Clamping via iptables (IPv4 & IPv6)
# Automatically clamp TCP MSS to Path MTU for forwarded and outbound traffic
sudo iptables -t mangle -A POSTROUTING -p tcp --tcp-flags SYN,RST SYN -j TCPMSS --clamp-mss-to-pmtu
sudo ip6tables -t mangle -A POSTROUTING -p tcp --tcp-flags SYN,RST SYN -j TCPMSS --clamp-mss-to-pmtu
4.2 Applying MSS Clamping via nftables
sudo nft add table inet mangle
sudo nft 'add chain inet mangle postrouting { type filter hook postrouting priority mangle; policy accept; }'
sudo nft 'add rule inet mangle postrouting tcp flags syn / syn,rst tcp option maxseg size set rt mtu'
4.3 Setting Explicit MTU in WireGuard Configurations
In /etc/wireguard/wg0.conf, always specify an explicit MTU = 1420 (or 1400 if transporting over IPv6 or PPPoE):
[Interface]
PrivateKey = <server_private_key>
Address = 10.66.66.1/24
ListenPort = 51820
MTU = 1420
[!TIP] Combining
MTU = 1420on tunnel interfaces withnet.ipv4.tcp_mtu_probing = 1in sysctl eliminates 99.9% of cross-Pacific TLS handshake stalls and large-packet drops.