ethtool Ring Buffers and Coalescing Tuning

Packet drops that appear in ethtool -S but nowhere else in the stack are usually a driver-level problem: ring buffers too small for the burst, or coalescing parameters that batch interrupts too aggressively for a latency-sensitive workload. This guide covers the tuning surface that actually matters on Linux - ring sizes, coalescing timers, queue counts, RSS, and the offload flags that interfere with measurement and shaping - plus the commands that show whether a change helped. It is aimed at servers doing 10G and above, where defaults are frequently sized for 1G-era traffic.

Read the current state first

ethtool -g eth0          # ring buffer sizes
ethtool -c eth0          # coalescing settings
ethtool -l eth0          # channel / queue counts
ethtool -k eth0          # offload features
ethtool -S eth0 | grep -i drop
ethtool -i eth0          # driver and firmware version

Always capture the "before" state. Ring and coalescing changes are driver, firmware and even port-speed dependent, and a value that helps one NIC family can be rejected by another.

Ring buffers

ethtool -G eth0 rx 4096 tx 4096

A NIC receives packets into a descriptor ring; when the ring fills, further packets are dropped in hardware before the kernel ever sees them. Larger rings absorb bursts but increase the worst-case latency a packet can spend queued in the adapter. If rx_dropped or rx_missed_errors climbs while the interface shows no CPU saturation, raise rx first. The maximum available is reported by ethtool -g, and the setting is rarely persistent across reboot without a network manager hook or udev rule.

Interrupt coalescing

ethtool -C eth0 rx-usecs 50 rx-frames 32 tx-usecs 50 tx-frames 32
# or let the adapter adapt
ethtool -C eth0 adaptive-rx on adaptive-tx on

Coalescing decides how long the adapter waits, or how many packets it accumulates, before raising an interrupt. Higher values mean fewer interrupts and lower CPU load, at the cost of added latency. For throughput-oriented storage and backup nodes, tolerating 50-100 microseconds is worthwhile; for trading or telemetry workloads, adaptive mode or very small rx-usecs values are the correct starting point. This is the single knob most often blamed for "the network is slow" when the real problem is a CPU-bound softirq path.

Queues, RSS and flow steering

ethtool -L eth0 combined 16
ethtool -x eth0                    # show RSS indirection table
ethtool -X eth0 equal 16           # distribute flows evenly
ethtool -n eth0                    # ntuple filters (accelerated RFS)

Multi-queue NICs spread flows across rings and CPUs by hashing. If one CPU shows 100% softirq while others idle, the hash distribution or the queue count is wrong. Verify with mpstat -P ALL 1 next to /proc/interrupts, and check that IRQ affinity has not pinned every queue to core 0.

Offloads that change your measurements

ethtool -K eth0 tso off gso off gro off lro off

Segmentation and aggregation offloads bundle packets before the kernel sees them, which distorts tcpdump output, breaks tc shaping accuracy, and can hide per-packet behaviour. Turn them off on interfaces used for shaping or deep packet inspection, and leave them on for pure throughput paths where CPU efficiency matters more. Disable EEE on latency-sensitive links as well:

ethtool --set-eee eth0 eee off
ethtool -m eth0     # module EEPROM and optical diagnostics, if supported

Where this fits

NIC tuning is one layer of a larger picture. Pair it with tc HTB traffic shaping when the bottleneck is your own egress, and with DAC, AOC and optics selection when the errors are physical rather than driver-level.

原文链接：https://techdocs.broadcom.com/us/en/storage-and-ethernet-connectivity/ethernet-nic-controllers/bcm957xxx/adapters/software-installation/installing-the-linux-driver/linux-ethtool-commands.html