NGINX stream Module: TCP and UDP Load Balancing - 夜莺博客

NGINX stream Module: TCP and UDP Load Balancing

The http block only speaks HTTP. Databases, DNS, syslog, MQTT, SMTP and SIP all need something lower-level, and that is what the stream module is for: a TCP/UDP proxy and load balancer built from the same configuration language. This guide builds a working configuration from an empty file — listeners, upstream groups, balancing methods, timeouts and buffers — and covers the details that catch people out during a rollout.

stream vs http: What Changes

The stream context is a top-level block, sibling to http. TCP is the default protocol, so there is no tcp parameter on listen; UDP must be declared explicitly. Inside a stream server you use proxy_pass, not location blocks, and the directives are connection-oriented rather than request-oriented. Make sure the module is compiled in (nginx -V 2>&1 | grep -- --with-stream); on distro packages it usually ships in the same binary.

Minimal Proxies: One TCP Port, One UDP Port

stream {
    server {
        listen     12345;
        proxy_pass backend.example.com:12345;      # TCP by default
    }

    server {
        listen     53 udp;
        proxy_pass dns_backend;                    # UDP needs the udp parameter
    }
}

Upstream Groups and Balancing Methods

Once more than one backend exists, define an upstream and choose how connections are spread. Round-robin is the default; least_conn is usually the better default for long-lived database or application sessions because it accounts for connections already open:

stream {
    upstream stream_backend {
        least_conn;
        server backend1.example.com:12345 weight=5;
        server backend2.example.com:12345 max_fails=2 fail_timeout=30s;
        server backend3.example.com:12345 max_conns=3;
        server backup1.example.com:12345 backup;      # used only when others fail
    }

    upstream dns_servers {
        least_conn;
        server 192.168.136.130:53;
        server 192.168.136.131:53;
        server 192.168.136.132:53;
    }

    server {
        listen              12345;
        proxy_pass          stream_backend;
        proxy_timeout       3s;      # idle timeout for the proxied connection
        proxy_connect_timeout 1s;    # how long to wait for a backend to accept
    }

    server {
        listen     53 udp;
        proxy_pass dns_servers;
    }
}
Directive Effect
weight Relative share of new connections for that server.
max_fails + fail_timeout Passive health checking: after N failures within the window, the server is removed for fail_timeout.
max_conns Hard cap on simultaneous connections — protects a fragile backend.
backup Only used when all primary servers are unavailable.
proxy_timeout Idle timeout. Too low kills long-lived sessions (SQL, SSH, MQTT keepalive); too high pins resources on dead peers.

Note that active health checks (health_check) are an NGINX Plus feature. In open-source NGINX you rely on passive checks (max_fails/fail_timeout), so size fail_timeout against how quickly you need a dead backend out of rotation.

Binding, Buffers and Client IP Preservation

stream {
    server {
        listen            127.0.0.1:12345;
        proxy_pass        backend.example.com:12345;
        proxy_bind       127.0.0.1:12345;    # source address for outbound connections
        proxy_buffer_size 16k;               # or disable with proxy_buffering off
        proxy_protocol    on;                # forward original client address to the backend
    }
}

Two of these matter more than they look. proxy_buffer_size affects the first chunk of data from the backend — too small and you add latency on every connection. proxy_protocol on is how the backend learns the real client IP; without it, every session appears to come from the load balancer, which breaks access logs, per-IP rate limits and auditing.

Deploy and Verify

nginx -t                                  # syntax check before reload
nginx -T | sed -n '/stream {/,/^}/p'      # show the effective stream config
systemctl reload nginx

ss -ltnp | grep -E '12345|:53'            # listeners exist and belong to nginx
tail -f /var/log/nginx/error.log          # upstream connect errors appear here

Then test end-to-end rather than trusting the config: connect through the proxy to each backend, stop one backend and confirm connections still succeed (failover), and for UDP use dig @proxy-ip example.com repeatedly to see responses from more than one server.

Pitfalls

  • Port conflicts: a stream listener on 443 and an http listener on 443 cannot coexist — decide which layer terminates TLS.
  • Proxy timeouts too aggressive: a 3-second proxy_timeout is right for short queries and wrong for a database session that idles.
  • No client IP at the backend: add proxy_protocol and configure the backend to expect it.
  • Plaintext by default: stream proxying is transparent — if the protocol is not encrypted, neither is the hop through nginx.

相关阅读:NGINX 反向代理配置:location 与 proxy_passLinux 网络内核参数调优:TCP 缓冲与 backlog 以及 TCP MSS 钳制与 PMTUD 排障

原文链接:NGINX Documentation - TCP and UDP Load Balancing