iperf3 Bandwidth Testing Guide: TCP/UDP Commands Explained

iperf3 is the standard command-line tool for measuring real network throughput, jitter and packet loss between two machines. It works on a simple client-server model — one host listens, the other generates traffic — and the numbers it produces tell you immediately whether a link is running at line rate or where the bottleneck is. This guide covers installation, TCP and UDP testing, parallel streams, reverse tests, JSON output for automation and the tuning knobs that make results accurate on fast links.

Two habits separate a useful measurement from a misleading one. First, test both directions: many paths are asymmetric, and a one-way test on a load-balanced or rate-limited link hides half the story. Second, record the exact command line with every result. A throughput number without its flags, duration and stream count is anecdote, not evidence, and the next person to look at the graph will have no way to reproduce it.

Installation and Server Setup

# Debian/Ubuntu
apt install iperf3
# RHEL/CentOS
yum install iperf3

Start the server on the receiving host (listens on TCP 5201 by default):

iperf3 -s
iperf3 -s -p 9000        # custom port
iperf3 -s -1             # accept one test, then exit
iperf3 -s -D             # run as daemon

Open the port in the firewall on the server side before testing: ufw allow 5201/tcp and ufw allow 5201/udp when you also test UDP. Confirm the listener exists before blaming the path:

ss -lntp | grep 5201
ufw status | grep 5201

When the far side is a device you cannot install software on — a switch with no embedded iperf, an appliance — the server may already be running as a management utility. Check the vendor documentation before assuming the feature is absent; many platforms expose an iperf or iperf3 mode once enabled.

Running Client Tests

iperf3 -c 192.168.1.10            # basic 10-second TCP test
iperf3 -c 192.168.1.10 -t 30      # longer test
iperf3 -c 192.168.1.10 -i 2       # report every 2 seconds
iperf3 -c 192.168.1.10 -P 8       # 8 parallel streams

On 10G/25G links a single TCP stream often tops out at 3-5 Gbps because one CPU core cannot keep up; use -P parallel streams to saturate the link. To measure the opposite direction add -R (server sends to client), useful on asymmetric paths, or --bidir (both directions at once, iperf3 3.7+) for simultaneous throughput.

RAM matters as much as CPU at high rates. The default TCP window is small enough that a receiver with plenty of bandwidth but a shallow socket buffer becomes the bottleneck; -w raises the window and -l changes the individual write size. For long-distance paths where the bandwidth-delay product is large, both are needed to get anywhere near the theoretical limit.

Reading the Output

An iperf3 summary is compact and worth decoding once:

[ ID] Interval           Transfer     Bitrate         Retr  Cwnd
[  5]   0.00-10.00  sec  1.09 GBytes   938 Mbits/sec  312   1.42 MBytes

• Transfer / Bitrate — total bytes moved and the resulting rate over the interval. Bitrate is decimal (Mbits/s), so a 1 Gbps link reporting 938 Mbits/sec is close to line rate after framing overhead.
• Retr — TCP retransmissions. A handful on a fast link is normal; large numbers mean loss, a saturation point, or a buffer problem.
• Cwnd — the sender's congestion window at the end of the test. A cwnd that is still growing means the test was too short to reach steady state.
• Sender versus receiver lines — when they differ, packets were lost or dropped between them, and the receiver figure is the one that matters.

For a rate that looks suspiciously close to a round number — 100, 200, 500 Mbit — suspect a policer or shaper rather than the hardware. Synthetic limits are the signature of policy, not of physics.

Parallel Streams and CPU Limits

iperf3 -c 192.168.1.10 -P 4 -t 20
iperf3 -c 192.168.1.10 -P 16 -t 20 -w 4M

Each stream on a given side is handled by its own thread, and the whole process is still bounded by the sum of what one process can push. When even parallel streams plateau, run several iperf3 instances on different ports and add their results:

# server side
iperf3 -s -p 5201 & iperf3 -s -p 5202 &
# client side, run in separate terminals
iperf3 -c 192.168.1.10 -p 5201 -t 20
iperf3 -c 192.168.1.10 -p 5202 -t 20

Pin the process if the host has multiple cores and a busy scheduler — taskset -c 2 iperf3 -c ... — and verify during the test that no core is at 100% with top or mpstat -P ALL 1. If one core saturates while the link is not full, the number is a CPU benchmark, not a network measurement.

UDP Tests: Jitter and Packet Loss

TCP hides loss through retransmission, so UDP is the better test when you want jitter and loss numbers — important for VoIP, video and real-time traffic. Always set the target bandwidth with -b, otherwise iperf3 defaults to a useless 1 Mbps:

iperf3 -c 192.168.1.10 -u -b 100M
iperf3 -c 192.168.1.10 -u -b 1G -t 30
iperf3 -c 192.168.1.10 -u -b 500M -l 1400    # 1400-byte packets like real traffic

[ ID] Interval           Transfer     Bitrate         Jitter    Lost/Total Datagrams
[  5]   0.00-10.00  sec   596 MBytes   500 Mbits/sec  0.025 ms  0/427225 (0%)

Jitter near 0 ms with 0% loss means the path is clean; any loss above 0% at your target rate means the link cannot sustain that bandwidth.

A UDP test with loss is more informative than one without, because the loss rate climbs abruptly at the point the path saturates. Sweep the offered rate — 100M, 200M, 500M, 1G — and note where loss begins. That knee is the real capacity for that packet size and those QoS markings, and it is far more useful for capacity planning than a single pass or fail at one rate. For VoIP realism, test with -l 200 to mimic G.711 payloads and watch jitter rather than aggregate throughput.

The Flags Worth Memorising

iperf3 -c HOST -P 8 -t 30 -i 1 -O 3 -R        # skip 3s warm-up, then reverse test
iperf3 -c HOST -Z                             # zero-copy (sendfile), lower CPU on client
iperf3 -c HOST -4 / -6                        # force IPv4 or IPv6
iperf3 -c HOST -B 10.0.0.5                    # bind to a specific source address
iperf3 -c HOST -b 0                           # UDP: unlimited rate (3.9+)

-O (omit) is the most under-used of the group: TCP slow start means the first seconds never represent steady state, so discarding them gives a cleaner number. -Z reduces client CPU load by using zero-copy send, which matters when the client itself is the bottleneck. -B is essential on a host with several addresses, such as one running VLAN sub-interfaces or policy routing, where the kernel would otherwise pick a source you did not intend.

JSON Output for Automation and Monitoring

iperf3 -c 192.168.1.10 -J > result.json

The -J flag emits structured JSON that scripts can parse with jq — for example .end.sum_received.bits_per_second — and feed to Grafana or a log file for trend analysis. Combine with -P 2 and -t 20 for a repeatable daily check, and alert when throughput drops below a threshold.

iperf3 -c 192.168.1.10 -J -P 4 -t 20 \
  | jq -r '.end.sum_received.bits_per_second / 1000000 | "\(.) Mbit/s"'

For UDP the useful fields live under .end.sum: jitter_ms, lost_percent and lost_packets. Storing one line per run in a time series database turns "the network feels slow" into a visible trend, and a threshold-based alert on a scheduled test catches degradation long before users notice it.

Methodology: Making Results Repeatable

• Test both directions with a plain run and one with -R; asymmetric paths are common on WAN circuits.
• Compare like with like. Same packet size, same duration, same stream count, same time of day.
• Watch the hosts, not just the tool. CPU, NIC queue drops and interrupt distribution all affect the result; mpstat and ethtool -S tell you which one moved.
• Bypass the middle once. Test host-to-host on the same switch to establish the ceiling, then insert hops one at a time.
• Record the command line alongside the number. A result nobody can reproduce is a rumour.

Testing Through Tunnels, VLANs and Overlay Networks

When the path under test is not a plain L2 link, a bare throughput number can hide a large correction. Two adjustments keep the result honest.

First, account for the encapsulation overhead. A tunnel adds bytes to every packet, so a 1500-byte payload becomes a larger frame on the wire and the reported bitrate at the application layer will always trail the interface counter. Compare the iperf3 figure with the interface statistics to see the gap:

# before and after the test, on each side
ip -s link show eth0
ethtool -S eth0 | grep -E 'tx_bytes|rx_bytes'

Second, set the MTU correctly before measuring. If the tunnel's MTU is smaller than the payload iperf3 sends, fragmentation or silent drops appear as unexplained loss at high rates. A ping with the DF bit set finds the real ceiling quickly:

ping -M do -s 1420 -c 3 192.168.1.10   # 1420 payload + 28 bytes of IP/ICMP = 1448 MTU

On an overlay network — VXLAN, WireGuard, GRE — test with -l 1400 or smaller so the encapsulated packets stay under the underlay MTU, and run both directions because overlay paths are often asymmetric. A throughput test that passes with small packets and collapses with large ones is a fragmentation problem, not a bandwidth problem.

iperf3 and Its Alternatives

iperf3 covers throughput, jitter and loss, which is enough for most work. Knowing the edges of the tool prevents misdiagnosis:

# a rough sanity check with no server required
nc -l 5001 > /dev/null                       # receiver
dd if=/dev/zero bs=1M count=2000 | nc 192.168.1.10 5001   # sender

• netperf — richer TCP semantics (request/response, connection rates) and per-connection tuning; useful when the question is about latency under load rather than raw throughput.
• tcpdump plus iperf3 — the combination that actually explains a result, since the capture shows retransmissions and window behaviour behind the number.
• tc netem — used on the far side to inject latency and loss so that iperf3 can demonstrate how a real application will behave rather than how an idealized TCP stream performs.
• Host-side tools — file transfer and storage tools measure the stack end to end; a network result that looks fine while an application is slow points away from the network and toward storage, DNS or the application itself.

The general rule: use iperf3 to bound what the path can do, then use capture and per-hop tools to explain why it will not do it.

Tuning and Troubleshooting

• Throughput lower than expected: CPU bottleneck (add -P), small TCP window (-w 4M for high-BDP paths), or NIC offloads; verify the interface first with ethtool diagnostics.
• Unable to connect: firewall blocking 5201; check with ss -lntp | grep 5201 on the server.
• UDP 100% loss: firewall dropping UDP or the path cannot sustain the configured -b.
• Server busy: kill stale instances with pkill iperf3.
• 40G+ links: run several server instances on different ports to work around the single-threaded CPU limit.
• Current and projected throughput warnings: iperf3 refuses some combinations on Linux; lower the rate slightly or use the --forceflush behaviour of a newer build rather than ignoring the message.
• Numbers that match a round policed rate: look for a shaper or a cloud instance's bandwidth cap before replacing hardware.

iperf3 pairs well with the packet-level view from tcpdump BPF filters when you need to confirm retransmissions, and with the host-side checks in our Linux network troubleshooting guide. When a test shows loss on a long path, the per-hop view from mtr for packet loss and latency usually identifies the hop that is dropping, and for a fast MTU discrepancy that quietly caps throughput, check the packet size with a ping sweep before trusting any single number.

原文链接：https://linux-shell.org/linux-shell/iperf3