Linux tc netem: Simulate Latency, Packet Loss and Jitter

Before an application meets a real WAN, it should meet tc netem - the Linux kernel's built-in network emulator. netem is a queuing discipline (qdisc) that deliberately delays, drops, corrupts, duplicates or reorders packets on an interface, turning a healthy host into a convincing stand-in for a congested transatlantic link or a flaky mobile connection. It is the standard tool for testing retry logic, timeouts, TCP behaviour and failover before users find the bugs, and it is available on every modern Linux distribution with no extra packages beyond iproute2. This guide covers the core netem commands, the class-based way to scope them so you do not break your own SSH session, and how to verify and clean up afterwards.

Adding Latency and Jitter

The basic form adds a fixed delay to every packet leaving the interface. Adding a second value produces jitter - a random variation around the base delay - and a distribution keyword makes the variation statistically realistic instead of uniform:

# Fixed 100ms delay on all egress traffic
sudo tc qdisc add dev eth0 root netem delay 100ms
# 100ms + 20ms jitter (uniform)
sudo tc qdisc add dev eth0 root netem delay 100ms 20ms
# Normal distribution, more realistic
sudo tc qdisc add dev eth0 root netem delay 100ms 20ms distribution normal
# 25% correlation: consecutive packets get similar delays (bursty)
sudo tc qdisc add dev eth0 root netem delay 100ms 20ms 25%

netem is egress-only and one-directional: 100ms of egress delay on one host adds ~100ms to ping RTT (which sums both directions), not 200ms. That is why a 50ms delay applied on both ends of a link yields a 100ms round trip. The correlation value (the trailing percentage) is what makes a test convincing for real-time traffic: with high correlation, delay changes slowly, mimicking an intercontinental path, while zero correlation produces the twitchy behaviour of a congested Wi-Fi link. For very realistic tests, combine a normal distribution with 20-40% correlation.

Packet Loss, Corruption, Duplication, Reordering

Loss is the impairment most likely to expose application bugs, and netem offers both a simple random model and the bursty Gilbert-Elliott model that matches real wireless and satellite links:

# 5% random loss
sudo tc qdisc add dev eth0 root netem loss random 5%
# Bursty loss with the Gilbert-Elliott model (realistic for wireless)
sudo tc qdisc add dev eth0 root netem loss gemodel 1% 10% 70% 0.1%
# Corrupt 5% of packets (single bit flip)
sudo tc qdisc add dev eth0 root netem corrupt 5%
# Duplicate and reorder
sudo tc qdisc add dev eth0 root netem duplicate 3%
sudo tc qdisc add dev eth0 root netem delay 50ms reorder 25% 50%

The gemodel parameters (p, r, 1-h, 1-k) let you control how often loss periods start and end, producing clumps of drops rather than isolated ones - which is what actually breaks retransmission timers. Corruption flips a bit inside the packet, so checksums fail and the packet is treated as lost upstream. Duplication sends a copy of the packet, useful for testing how a stack handles unexpected retransmissions. Reordering requires a delay to work: with delay 50ms reorder 25% 50%, 25% of packets are reordered and half of those are delayed by the base amount, so a portion of the stream arrives out of order. Adjust the percentage to keep reordering below the "reordering is just loss" threshold that most stacks assume.

Scope It: Don't Break Your Own Session

A root qdisc shapes everything, including your SSH replies. Add 300ms of delay on the interface you are connected through and your terminal becomes unusable, while any monitoring agent also sees the impairment. The fix is to scope the impairment to one destination, port or protocol using an intermediate class (htb or prio) and a filter so that only the traffic under test is affected:

# Class-based: impair only traffic to the database server's port
sudo tc qdisc add dev eth0 root handle 1: htb default 10
sudo tc class add dev eth0 parent 1: classid 1:1 htb rate 1gbit
sudo tc class add dev eth0 parent 1:1 classid 1:20 htb rate 1gbit
sudo tc qdisc add dev eth0 parent 1:20 handle 20: netem delay 100ms loss random 5%
sudo tc filter add dev eth0 parent 1: protocol ip prio 1 u32 match ip dport 5432 0xffff flowid 1:20

Now PostgreSQL traffic to port 5432 experiences 100ms delay and 5% loss while your SSH, DNS and monitoring traffic stay clean. The filter matches on destination port here, but you can just as easily match on source or destination IP, on a DSCP value, or on a whole subnet. If you only need to impair a single IP, a prio qdisc with a match filter is enough for most labs and avoids the extra class hierarchy. A useful pattern for client-side testing is to match on the destination address of the service under test, so the impairment applies to exactly one conversation.

# Impair only traffic to one host
sudo tc qdisc add dev eth0 root handle 1: prio
sudo tc qdisc add dev eth0 parent 1:3 handle 30: netem delay 80ms
sudo tc filter add dev eth0 parent 1: protocol ip prio 1 u32 match ip dst 10.20.30.40/32 flowid 1:3

Verify and Clean Up

ping -c 100 TARGET | tail -5          # check added latency/loss
sudo tc -s qdisc show dev eth0        # drop/delay counters
watch -n1 'tc -s qdisc show dev eth0'
# remove everything
sudo tc qdisc del dev eth0 root

The -s flag is the one that matters: it prints the number of packets dropped, delayed and reordered by the qdisc, so you can prove the impairment is actually being applied to the traffic you intended. If the counters stay at zero while you generate load, your filter is not matching. Combine the qdisc counters with ss -ti to watch TCP's retransmission and cwnd behaviour under loss, and with a packet capture to confirm which side is dropping. Remember to remove the qdisc when the test is over - a forgotten 300ms delay on a production interface is a classic self-inflicted incident, and it is invisible unless you know where to look.

Combined with iperf3 (see our bandwidth testing guide), netem lets you prove how throughput and retransmissions degrade under loss, and with tcpdump you can watch TCP react. If you also need to cap bandwidth rather than impair quality, pair netem with an HTB class - the rate and ceil mechanics are explained in our Linux tc HTB traffic shaping guide.

Building a Realistic WAN Test Profile

Single impairments are easy to apply and easy to reason about, but real links degrade in combination. A convincing "bad WAN" profile stacks latency, jitter, loss and bandwidth limits so that the application meets the conditions it will actually face. A transatlantic path, for example, looks like 80-120ms of latency with modest jitter, near-zero loss, and a few hundred Mbit/s of capacity; a congested mobile link looks like 60ms latency with heavy jitter, bursty loss and a highly variable rate. Build the profile as a small class tree: an HTB parent to cap the rate, a netem child to add delay and loss, and filters to direct only the traffic under test into it.

# A "bad WAN" class: cap at 5mbit, add 120ms + 40ms jitter, 2% loss
sudo tc qdisc add dev eth0 root handle 1: htb default 10
sudo tc class add dev eth0 parent 1: classid 1:1 htb rate 1gbit
sudo tc class add dev eth0 parent 1:1 classid 1:20 htb rate 5mbit ceil 5mbit
sudo tc qdisc add dev eth0 parent 1:20 netem delay 120ms 40ms distribution normal loss gemodel 1% 10% 70% 0.1%
sudo tc filter add dev eth0 parent 1: protocol ip prio 1 u32 match ip dst 10.20.30.0/24 flowid 1:20

Keep a copy of the exact command lines in your runbook so that the same profile can be replayed later - reproducibility is the entire point of using netem. A profile that only lives in your shell history is worthless six months later when you need to reproduce a bug.

Impairing Both Directions

Because netem shapes egress only, a two-way link needs the impairment applied on both hosts (or both ends of a network namespace pair). Applying delay on just one side produces an asymmetric path, which is useful for testing but not representative of most production links. If you cannot log into the remote host, apply netem on the router or on an intermediate VM between the two endpoints; a lightweight Linux bridge with two veth interfaces makes an excellent inline impairment device, and it also lets you impair traffic you do not control at either endpoint. A common mistake when testing a client-server protocol is to impair only the server's egress and then wonder why the client's uploads are unaffected - both directions of the conversation carry data, and both need to be shaped for the result to be believable.

netem in Containers and Network Namespaces

For testing microservices or isolating a client, netem is often more useful inside a network namespace than on a physical interface. Create a namespace pair, cable them together with a veth pair, and apply the qdisc to the veth inside the namespace - the impairment then applies only to that container's traffic and never touches the host's real connectivity. Docker and Kubernetes pods can be impaired the same way by entering the pod's network namespace with nsenter and applying the qdisc to the pod's interface. This is the cleanest way to reproduce a "slow upstream service" in a CI pipeline without affecting the test runner itself.

Common Pitfalls

• Forgetting the root qdisc already exists. Some distros attach a default qdisc; a second add ... root fails with "File exists" - use replace instead.
• Applying netem to the wrong interface. On a host with several NICs, the default route interface is where egress impairment belongs.
• Expecting loss to be symmetric. Egress loss on one host means that host drops its own outgoing packets; the peer's drops are separate.
• Leaving the qdisc in place. Always tc qdisc del dev eth0 root when the test finishes.
• Using huge percentages. 50% loss is almost never realistic and mostly tests your patience; 1-5% is where real bugs live.

Choosing Realistic Impairment Values

Good netem testing is a question of picking numbers that match a real path, not of making the network as bad as possible. Start from measurements: run ping against a real target to get baseline RTT and jitter, and use mtr to see where loss appears along the route. A domestic fibre link is typically 5-20ms with under 5ms of jitter and effectively zero loss; a mobile network is 30-80ms with 10-30ms of jitter and occasional bursts of loss; a satellite hop is 500-700ms with near-zero jitter but a higher corruption rate from the radio link. For application timeouts, the interesting zone is usually where the RTT approaches the configured timeout - that is where retry storms and duplicate requests appear. Add loss in small increments (0.1%, 0.5%, 1%, 2%) and watch where behaviour changes, rather than jumping straight to a number that simply breaks everything.

原文链接：https://oneuptime.com/blog/post/2026-03-04-simulate-network-latency-packet-loss-tc-netem-rhel-9/view