TRex Traffic Generator: Stateless and Stateful Testing

Testing a firewall, a load balancer or a DDoS mitigation platform with iperf tells you almost nothing. Real traffic is a mix of many short flows, many hosts, realistic packet sizes and a mixture of TCP and UDP - and the platform's forwarding performance depends heavily on all of those. TRex is a low-cost, high-speed traffic generator built on DPDK that produces that kind of traffic, in both a stateless mode for raw packet blasting and a stateful mode for real TCP connection behaviour.

This guide covers the two modes, how to get it running on a bare-metal host, and the details that decide whether your numbers are meaningful.

Stateless vs Advanced Stateful

The distinction matters for interpreting results. A firewall that survives 40 Gbps of stateless UDP may fall over at 50,000 new TCP connections per second, because the cost is in state creation, not bytes.

Host Preparation

# hugepages - TRex allocates from here
echo 8192 | sudo tee /proc/sys/vm/nr_hugepages
sudo mkdir -p /mnt/huge && sudo mount -t hugetlbfs nodev /mnt/huge

# CPU isolation and turbo off for repeatable results
# kernel cmdline: isolcpus=2-13 nohz_full=2-13 rcu_nocbs=2-13
sudo cpupower frequency-set -g performance

# bind the test NICs to a DPDK driver
sudo modprobe vfio-pci
sudo dpdk-devbind.py --bind=vfio-pci 0000:03:00.0 0000:03:00.1
sudo dpdk-devbind.py --status

# /etc/trex_cfg.yaml - the port order here is the order TRex sees them
- version: 2
  interfaces: ["03:00.0", "03:00.1"]
  port_info:
    - dest_mac: "00:11:22:33:44:55"   # the DUT interface facing port 0
      src_mac:  "aa:bb:cc:dd:ee:00"
    - dest_mac: "00:11:22:33:44:56"
      src_mac:  "aa:bb:cc:dd:ee:01"
  platform:
    master_thread_id: 1
    latency_thread_id: 2
    dual_if:
      - socket: 0
        threads: [4,5,6,7]

Getting the MAC addresses wrong is the most common first failure: TRex transmits to the dest_mac you specify, and if that is not the interface of the device under test, the frames go nowhere and you conclude the DUT is dropping everything.

Running

# interactive stateless console
sudo ./t-rex-64 -i --astf

# headless, port 0 to port 1, with a profile
sudo ./t-rex-64 -f stl/simple.yaml -m 10 -d 60

# in the console
TRex> start -f stl/udp_1pkt_simple.yaml -m 10mpps -d 60
TRex> start -f stl/imix.py -m 40gbps -d 120
TRex> pause
TRex> stat -a    # aggregate counters
TRex> tui       # live charts

A Stateless Profile

- duration: 60.0
  generator:
    distribution: "seq"
    clients_start: "16.0.0.1"
    clients_end:   "16.0.255.255"
    servers_start: "48.0.0.1"
    servers_end:   "48.0.255.255"
    clients_per_second: 50000
    mac:
      - "00:11:22:33:44:55"
      - "00:11:22:33:44:56"
    ip:
      - "16.0.0.1"
      - "48.0.0.1"
  cap_info:
    - name: base
      type: "raw"
      isg: 100.0
      cps: 0
      ipg: 0
      rtt: 1000000
      w: 0
      cap_interface: server
      packet: base.pcap

Using a real capture for the packet payload is what makes the test match reality - application-aware devices classify traffic by inspection, and a synthetic all-zeros UDP payload is trivially optimised away by some platforms.

An ASTF Profile

# simple HTTP-like workload
from trex.astf.api import *

class Prof1:
    def get_profile(self):
        ip_gen_c = ASTFIPGenDist(ip_range=["16.0.0.1", "16.0.0.254"], distribution="seq")
        ip_gen_s = ASTFIPGenDist(ip_range=["48.0.0.1", "48.0.0.254"], distribution="seq")
        ip_gen   = ASTFIPGen(glob=ASTFIPGenGlobal(ip_offset="1.0.0.0"),
                             dist_client=ip_gen_c, dist_server=ip_gen_s)
        return ASTFProfile(default_ip_gen=ip_gen,
                           cap_list=[ASTFCapInfo(file="../avl/delay_10_http_browsing_0.pcap",
                                                 cps=1000)])
def register():
    return Prof1()

Connection rate (cps) is the number that exposes most firewalls: scale it until the device drops sessions or the session table overflows, and record both the rate and the resulting latency. Reported connection rates should always be paired with a look at the device's own counters - a firewall that reports "success" while its session table is at 100% will fail at the next spike in production.

Test the Tester

Before you publish a number, run the same profile with the two TRex ports connected directly to each other. That gives the loopback baseline: the maximum the generator can produce on this host with these NICs. Any subsequent number higher than the baseline, or suspiciously close to it, is a measurement artefact rather than a device result.

# baseline: back-to-back, no DUT
sudo ./t-rex-64 -f stl/udp_1pkt_simple.yaml -m 100gbps -d 30 --lo

# watch for drops and out-of-order at the generator itself
TRex> stat -a

Results That Hold Up

• Layer of the figure. L1, L2 and L3 bit rates differ by the framing overhead. Always state which one you mean - a 40 Gbps result is 33% short of 40 Gbps line rate depending on frame size.
• Frame size mix. IMIX (about 7:4:1 of 64/594/1518 byte frames) is the honest default; a pure 1518-byte test flatters every device.
• Latency, not just throughput. TRex measures per-flow latency; the interesting number is the tail, at the load level where the device is still forwarding without loss.
• State. A stateful test that runs for 60 seconds does not exercise the same code paths as one that runs for an hour with established flows being torn down. Long-run and churn tests find leaks that short tests never see.

For pure bandwidth characterisation of a link, a simpler tool is often enough - the methodology in this iperf3 testing guide covers single-stream and parallel tests properly. TRex is for the cases where the flow model itself is the thing under test. If your target is a containerised CNF data plane, the same DPDK and SR-IOV plumbing that matters for VPP applies, as described in this SR-IOV, Multus and DPDK guide, and when you push a high-BDP path to its limits the socket-level considerations in this TCP tuning guide will be the limiting factor rather than the generator.

原文链接：https://trex-tgn.cisco.com/