tc netem: Simulating Latency and Packet Loss

2026年10月14日 LOG admin

Applications that work perfectly on a lab network fail in surprising ways over a WAN: timeouts that only appear at 150 ms round-trip, retransmission storms that need 0.5% loss, reordered packets that break a naive protocol implementation. tc netem is the standard way to reproduce those conditions on demand, and it is one of the few tools that lets you test the failure mode deliberately instead of discovering it in production. This guide covers the options that matter, the command patterns, and the placement rules that determine whether your measurements are meaningful.

The options

Option	Parameter	Notes
`delay`	time [jitter [correlation]]	Applied to egress packets; jitter adds variation
`loss`	percent [correlation]	Refines patterns: can use Gilbert-Elliot models
`duplicate`	percent [correlation]	Duplicates packets before queueing
`reorder`	percent [correlation] [gap]	Requires delay; gap N reorders every Nth packet
`corrupt`	percent [correlation]	Flips random bits, tests checksums and error paths
`rate`	rate [overhead]	Adds a bandwidth limit to the emulation

Correlation is the parameter people ignore: it biases each subsequent value by a percentage of the previous one, giving bursty behaviour instead of statistically independent events. A loss 0.3% 25% profile loses a few packets in a burst, which is what real congestion looks like; independent random loss is often easier for TCP to handle and flatters the application.

Command patterns

# 100 ms delay with 10 ms jitter, 25% correlation, 0.3% bursty loss
tc qdisc add dev eth0 root netem delay 100ms 10ms 25% loss 0.3% 25%

# change it in place without tearing the interface down
tc qdisc change dev eth0 root netem delay 150ms 20ms

# a satellite-like path: 600 ms delay, 2% loss, 0.1% reorder
tc qdisc change dev eth0 root netem delay 600ms loss 2% reorder 0.1% 50%

# periodic loss: every nth packet via the Gilbert-Elliot-ish model
tc qdisc change dev eth0 root netem loss gemodel 1% 10% 70% 0.1%

# inspect and remove
tc qdisc show dev eth0
tc -s qdisc show dev eth0
tc qdisc del dev eth0 root

Always check the counter output (tc -s) after a test: it reports how many packets were delayed, dropped or reordered, which is how you confirm the profile you typed is the profile that was applied. A profile that reports zero dropped packets after a loss test is a misplaced qdisc, not a perfect network.

Placement: the rule most people get wrong

netem affects egress traffic from the interface it is attached to. That constraint has a consequence the documentation spells out: because of mechanisms like TCP small queues, realistic TCP performance measurements require netem to be placed on the ingress side of the receiver host — in practice, on the sending side of the path into the host you are testing. If you attach netem to the sender and measure throughput at the receiver, TCP's own send buffering can mask much of the delay. For a lab, the clean arrangement is netem on the sender for both directions of a two-host test, or on the receiver's ingress using an IFB device.

# shaping ingress: redirect to an ifb device, where netem can be applied
modprobe ifb numifbs=1
ip link set ifb0 up
tc qdisc add dev eth0 handle ffff: ingress
tc filter add dev eth0 parent ffff: protocol ip u32 \
   match u32 0 0 action mirred egress redirect dev ifb0
tc qdisc add dev ifb0 root netem delay 80ms loss 0.5%

What to test with it

The profiles worth keeping in a script: a long-haul WAN (100–150 ms with jitter), a congested link (0.5–2% bursty loss with a rate limit), a satellite or intercontinental path (600 ms), and a mobile access network (variable delay plus reordering). Run each against the same workload and compare error rates, timeout behaviour and retry counts — the interesting bugs are usually in timeout constants and retry logic, not in the data path. Also test with an MTU-aware profile: combining netem loss with a full-size packet stream is how you discover PMTU black holes before a customer does.

Building a repeatable network profile suite

Ad-hoc netem commands produce ad-hoc conclusions. The professional version is a small script that applies named profiles, runs the test, collects evidence and cleans up — so that two engineers comparing results are comparing the same conditions.

#!/bin/bash
# netem-profiles.sh   
DEV=${2:-eth0}; SECS=${3:-30}
case "$1" in
  wan)        PROFILE="delay 100ms 15ms 25% loss 0.1% 25%" ;;
  congested)  PROFILE="delay 30ms 10ms loss 1% 25% rate 20mbit" ;;
  satellite)  PROFILE="delay 600ms 50ms loss 0.5% 25%" ;;
  mobile)     PROFILE="delay 60ms 40ms reorder 1% 50% loss 0.5%" ;;
  corrupt)    PROFILE="delay 40ms corrupt 0.1%" ;;
  *) echo "unknown profile: $1"; exit 2 ;;
esac
cleanup() { tc qdisc del dev "$DEV" root 2>/dev/null; }
trap cleanup EXIT INT TERM
cleanup
tc qdisc add dev "$DEV" root netem $PROFILE
echo "applied: netem $PROFILE on $DEV"
tc -s qdisc show dev "$DEV" | head -20
sleep "$SECS"
tc -s qdisc show dev "$DEV" | sed -n '/Sent/,/backlog/p'

The trap for cleanup is not cosmetic: leaving a 600 ms delay attached to an interface after a test is a memorable way to cause the next incident. Pair the profile with the workload under test and store the qdisc counters with the result — they prove the emulation was actually applied.

Correlating symptoms with causes

Symptom under test	Likely application cause	What netem shows you
Timeouts only at high delay	Timeout constants sized for a LAN	Fails at 150 ms, passes at 50 ms — the constant is the bug
Throughput collapses with tiny loss	Congestion control and window sizing	0.5% loss halves goodput on a long-fat path
Random protocol errors	Reassembly or ordering assumptions	reorder or corrupt reveals it immediately
Slow but functional transfers	Serial request/response design	Latency multiplies round trips; pipelines would not
Connection resets under load	Queue or buffer exhaustion combined with loss	rate plus loss reproduces the reset

This table is the reason to run the profile suite before a release rather than after a customer complaint: the diagnosis is the same, but the cost is a test run instead of an incident call. Extend it with the workload's own metrics — retry counts, connection pool wait times, error rates — because a profile that changes nothing observable is a profile your application is already robust to, which is worth knowing explicitly.

Combining netem with policing and shaping

Real WAN links do not just add delay; they also constrain bandwidth and police bursts. Combining netem with a hierarchy is what makes a test realistic.

# HTB class limit + netem on the class: a 20 Mbit link with 100 ms delay
tc qdisc add dev eth0 root handle 1: htb default 20
tc class add dev eth0 parent 1: classid 1:1 htb rate 100mbit
tc class add dev eth0 parent 1:1 classid 1:10 htb rate 20mbit ceil 20mbit
tc qdisc add dev eth0 parent 1:10 handle 10: netem delay 100ms 10ms
tc filter add dev eth0 protocol ip parent 1:0 prio 1 u32 match ip dst 10.0.0.0/8 flowid 1:10

# verify: the class limit and the delay should both be visible
tc -s class show dev eth0
tc -s qdisc show dev eth0

Add queue depth to the model to reproduce bufferbloat — the buffer size in the qdisc determines whether a constrained link adds latency under load, and reproducing a 500 ms bufferbloat spike is often the fastest way to explain a user complaint about "the network being slow" that never shows up in a ping test.

FAQ

Why do my results differ from my colleague's? Almost always placement or MTU. Confirm netem is on the intended interface and direction, check that both tests used the same packet size, and make sure no residual qdisc from an earlier test is still attached (tc qdisc show).

Does netem affect TCP retransmissions accurately? It drops or delays packets as instructed, which is what TCP sees; the resulting retransmission behaviour is genuine. Where it flatters the result is when the sender's buffers mask the induced delay, which is the TSQ effect described in the manual.

Can I use netem in a container? Yes, with the capability to manipulate the network namespace (NET_ADMIN). Inside a container it acts on that namespace's interfaces, which is convenient for testing a specific pod's connectivity without affecting the node.

How do I test IPv6 paths? The same qdisc applies; add a filter matching the IPv6 destination if you want to affect only one flow, and make sure PMTU-related tests use full-size packets so the ICMPv6 path is exercised.

Quick reference: apply, inspect, remove

# apply a WAN profile
tc qdisc add dev eth0 root netem delay 100ms 10ms 25% loss 0.3% 25%

# change it without touching the interface
tc qdisc change dev eth0 root netem delay 150ms 20ms

# inspect: counters prove the emulation is live and at the intended rate
tc -s qdisc show dev eth0

# remove — always, and confirm it is gone
tc qdisc del dev eth0 root
tc qdisc show dev eth0

Treat the removal as part of the test, not as cleanup you might forget: a persistent qdisc is invisible to almost every monitoring tool and will quietly distort every measurement made on that interface afterwards, including the ones you use to argue that problem is solved. If you routinely test on shared lab hardware, add the removal to a login banner or a monitoring check that alarms on any netem qdisc present outside a test window.

What netem cannot reproduce

It is worth being clear about the boundaries. netem shapes egress traffic with probability models; it does not reproduce the dynamics of a congested link's buffer, where latency builds and collapses as queues fill and drain, unless you model the queue explicitly with a shaping class. It does not reproduce asymmetric routing, middlebox interference, or a path whose characteristics change over time — the profile you applied is static, and real WANs are not.

Where that matters, combine netem with an additional tool: use a shaping class with a shallow queue to induce realistic bufferbloat, and vary the profile over time with a script to approximate a path that degrades and recovers. For anything needing genuine path emulation with multiple hops and distinct per-hop characteristics, a network emulator in a lab is the right instrument — netem is the single-link approximation that answers most test questions at a fraction of the cost, and knowing which questions it cannot answer keeps you from over-reading its results.