kube-proxy Modes: iptables vs IPVS vs nftables - 夜莺博客

kube-proxy Modes: iptables vs IPVS vs nftables

Every Service IP in a cluster is implemented by kube-proxy pushing rules into the kernel, and the mode it uses determines how those rules scale: linear rule walks, hash-table load balancing, or a modern nftables ruleset with sets and maps. IPVS – long the "performance" answer – is now deprecated in favour of the nftables backend. This article explains what each mode does under the hood, how to check which one your nodes are running, and when switching actually matters.

What kube-proxy actually does

The daemon watches Services and EndpointSlices through informers, diffs the new desired state against the kernel state, and calls its sync routine to program rules. It does not proxy packets itself in any of the Linux modes; the kernel does the forwarding. The mode only changes the data structures used.

iptables mode (default)

For every Service, kube-proxy writes a KUBE-SERVICES match that jumps into a per-Service chain (KUBE-SVC-xxxx), which contains one rule per endpoint with declining probability values, each jumping to a KUBE-SEP-xxxx chain that performs the DNAT. The lookup is a linear scan of netfilter chains, O(n) per new connection, and the sync cost grows with the number of Services and endpoints. Tuning knobs include:

iptables:
  minSyncPeriod: 1s
  syncPeriod: 30s

IPVS mode

IPVS is an in-kernel L4 load balancer. kube-proxy creates a dummy interface kube-ipvs0, adds every ClusterIP to it, and programs virtual services with real servers behind them; the scheduler (rr, lc, sh, mh, ...) picks a backend with a hash lookup, O(1) rather than O(n).

ip addr show kube-ipvs0
ipvsadm -Ln

Two caveats are worth remembering: IPVS mode still uses iptables for masquerade, source rewriting and external traffic policy, and it requires the IPVS kernel modules to be present (kube-proxy exits with an error if they are missing). Kubernetes deprecated the IPVS backend in v1.35 – the nftables mode is intended as its replacement.

nftables mode

Introduced as alpha in 1.29, beta in 1.31 and GA in 1.33, the nftables backend builds a single table (kube-proxy) using nftables sets and maps so that endpoint selection is a set/map lookup rather than a chain walk. It is the recommended replacement for both iptables and IPVS on current kernels, with better sync and forwarding performance than either. On kernels too old for it, the guidance is to stay on iptables rather than move to IPVS, because iptables performance has improved considerably since IPVS mode was introduced.

Checking and switching modes

# what is running now
kubectl -n kube-system get cm kube-proxy -o yaml | grep -i mode
kubectl -n kube-system logs -n kube-system ds/kube-proxy --tail=50 | grep -i "Using .* Proxier"
iptables-save | grep -c KUBE
nft list tables

# switching mode (kubeadm clusters)
kubectl -n kube-system edit cm kube-proxy     # set mode: "nftables"
kubectl -n kube-system rollout restart ds/kube-proxy

After the restart, confirm the old structures are gone: no KUBE-* chains in iptables-save and the expected nftables table present. Rolling the change node by node keeps the API reachable while rulesets are rebuilt.

Do you need to change anything?

  • Small clusters (hundreds of Services) run fine on iptables; the sync cost only becomes visible at thousands of Services/endpoints or with very high connection churn.
  • Clusters already using eBPF datapaths (Cilium in kube-proxy replacement mode) bypass this entirely.
  • Latency-sensitive workloads benefit more from the nftables mode than from IPVS, and it avoids the deprecated path.

Related: CNI comparison: Calico vs Cilium vs Flannel, MetalLB Layer2 vs BGP, and NetworkPolicy default deny patterns.

原文链接:https://kubernetes.io/docs/reference/networking/virtual-ips