Proxmox VE Networking: VLAN-Aware Bridge and LACP - 夜莺博客

Proxmox VE Networking: VLAN-Aware Bridge and LACP

Proxmox VE uses the plain Linux network stack, which is why its networking is both flexible and easy to get subtly wrong. The three building blocks are a bridge (a virtual switch), an optional bond (NIC aggregation) and optional VLAN tags - and most "VLANs do not work" reports trace back to a mismatched switch port or an IP address placed on an untagged bridge while the traffic is actually tagged. This guide covers the two designs that cover almost every deployment, plus the safe-apply workflow that keeps the management path reachable.

The Three Building Blocks

Component Role Typical name
Physical NIC Connects the host to the network; gets no IP when it is a bridge port enp1s0, eno1
Linux bond Combines NICs for failover or aggregate capacity bond0
Bridge Virtual switch guests attach to vmbr0

The bridge normally holds the host IP and the physical NIC behaves as a port on that virtual switch. An IP in the 192.168.10.0/24 range on an untagged bridge does not put the host on VLAN 10 - the VLAN tag does, and nothing else.

Everything is described declaratively in /etc/network/interfaces (and any files pulled in from /etc/network/interfaces.d/), and Proxmox applies it through ifupdown2. That matters because ifupdown2 is what makes ifreload -a possible: it computes the difference between the running configuration and the file and applies only the deltas, instead of tearing down every interface as a traditional ifup/ifdown cycle would.

The bridge is a real Layer-2 switch and it does learn MAC addresses - unlike an ESXi standard vSwitch, which floods unknown destinations. That is why bridge fdb show is a genuinely useful troubleshooting command on Proxmox and largely useless on ESXi. Guests connect through a tap interface (tap100i0 for VM 100's first NIC, veth100i0 for a container), and each tap appears as a port on the bridge.

Design 1: VLAN-Aware Bridge (one bridge, many VLANs)

iface enp1s0 inet manual

auto vmbr0
iface vmbr0 inet static
        address 192.168.1.10/24
        gateway 192.168.1.1
        bridge-ports enp1s0
        bridge-stp off
        bridge-fd 0
        bridge-vlan-aware yes
        bridge-vids 10 20 30

Set the VLAN tag on each VM or container NIC and the traffic is tagged transparently. The switch port must be a trunk carrying the same VLANs - if the switch side is an access port, nothing will work no matter what the host does.

Three details decide whether this works in practice:

  • The bridge's own IP lives in the native VLAN. With bridge-pvid 1 (the default when you do not specify one), the host's management address in the example above is untagged VLAN 1. Put the host address on a tagged VLAN only by setting bridge-pvid to that VLAN, and only when the upstream port untags it for the host.
  • bridge-vids is an allow-list. A tagged frame for a VLAN that is not in bridge-vids is dropped silently on ingress. Adding a new VLAN for a guest therefore means editing bridge-vids, not just setting a tag on the VM.
  • Do not mix the two tagging designs. On a VLAN-aware bridge the tag belongs on the guest NIC (tag=20). Creating vmbr0.20 on top of a VLAN-aware bridge as well double-tags the traffic and produces frames nobody receives.

Setting the tag on a guest is a single line in its configuration - tag=20 in qm config for a VM, or in pct config for a container - and the GUI exposes it as the VLAN Tag field on the network device. VLAN 0 means untagged, and VLAN 4095 is not available here: unlike ESXi's VGT, Proxmox does not offer a per-guest trunk mode, so a guest that must see several VLANs does its own tagging on top of an untagged NIC.

Design 2: LACP Bond Feeding the Bridge

iface enp1s0 inet manual
iface enp2s0 inet manual

auto bond0
iface bond0 inet manual
        bond-slaves enp1s0 enp2s0
        bond-mode 802.3ad
        bond-miimon 100
        bond-lacp-rate fast
        bond-xmit-hash-policy layer3+4

auto vmbr0
iface vmbr0 inet static
        address 192.168.1.10/24
        gateway 192.168.1.1
        bridge-ports bond0
        bridge-stp off
        bridge-fd 0
        bridge-vlan-aware yes

Two expectations to set correctly: LACP gives aggregate capacity across many flows and link resilience, but a single TCP stream still tops out at one member link's speed unless the hash spreads it. Configure the switch LAG before applying the host change, otherwise the bond comes up with only one member active.

The full option set that matters for a production bond:

Option Value Effect
bond-mode 802.3ad Dynamic LACP; requires a matching LACP LAG on the switch
bond-mode active-backup No switch configuration required; usable across two different switches
bond-miimon 100 Link monitoring interval in ms; the default and usually the right value
bond-lacp-rate fast LACPDUs every second instead of every 30 s; faster failure detection
bond-xmit-hash-policy layer3+4 Hashes on IP and port - spreads flows best, but not supported by every NIC driver
bond-updelay / bond-downdelay 200 Debounces link flapping before the link is declared up or down
bond-primary enp1s0 Preferred active member for active-backup mode

Choose active-backup when the two NICs are patched to different switches and those switches are not MLAG/VLT peers - a LAG across two independent switches simply will not form, and pretending otherwise produces a bond with one active member and half its capacity. LACP is the right answer only when both members land on the same logical switch (one chassis, or an MLAG/VLT/VSX pair). The hashing trade-offs are identical to the ones on any Linux bonding setup: LACP spreads flows, not packets, and a policy that hashes on layer 3 and 4 addresses gives up the ability to spread protocols that carry neither. The mode-by-mode comparison of LACP against active-backup, balance-tlb and the rest is in Linux bonding modes: LACP and active-backup.

Design 3: Many Bridges, One Per VLAN (the older pattern)

Before VLAN-aware bridges became the default, most hosts used one bridge per VLAN with an 802.1Q sub-interface as the bridge port:

auto enp1s0.20
iface enp1s0.20 inet manual

auto vmbr20
iface vmbr20 inet static
        address 192.168.20.10/24
        bridge-ports enp1s0.20
        bridge-stp off
        bridge-fd 0

This still works and is easy to reason about, but it scales badly: every VLAN needs a bridge, a sub-interface and a matching guest configuration, and the guest NIC no longer carries a tag at all. It is worth recognising because you will meet it in existing installations - and because mixing it with a VLAN-aware bridge on the same uplink is a reliable way to create loops and duplicate frames.

Applying Changes Without Locking Yourself Out

# the GUI writes to a staging file: /etc/network/interfaces.new
ifreload -a                    # apply staged config live (ifupdown2)
ip -brief link
ip -brief address
ip route
bridge link
cat /proc/net/bonding/bond0

Proxmox stages changes and lets you apply them in one transaction precisely because a bad bridge definition cuts your management access. Prefer the GUI's Apply Configuration button or ifreload over bouncing interfaces by hand, and always keep a console or IPMI path open when changing the bridge that carries the management IP.

The verification commands above answer four different questions, and running them in this order is the quickest diagnosis path:

  • ip -brief link - is the bridge, bond and each member NIC actually UP?
  • ip -brief address - which interface ended up holding the management IP? If it is on the physical NIC instead of the bridge, the host is about to lose connectivity.
  • bridge link - are the expected ports (bond0 plus each guest tap) members of the bridge?
  • cat /proc/net/bonding/bond0 - which members are active, and what is the aggregated link status?

Two more commands complete the picture: bridge vlan show prints per-port PVID and the allowed VLAN list, which is the only way to prove that the bridge is passing VLAN 20 as an egress-untagged or tagged member; and ethtool enp1s0 confirms the negotiated speed - a "gigabit" bond whose members negotiated 100 Mb/s because of a bad patch cable is a surprisingly common cause of "slow after the change" reports.

Proxmox SDN as an Alternative

Proxmox VE also ships an SDN stack (Zones, VNets, subnets) configured under Datacenter > SDN, with state stored in /etc/pve/sdn/. A Simple zone with a VNet gives guests isolated Layer-2 segments with no per-host configuration drift, and a VLAN zone maps VNets onto VLAN tags on an existing bridge. For clusters where the same segment must exist identically on twenty nodes, the SDN abstraction removes the per-node bridge definition entirely. It is not a replacement for understanding bridge-vids and trunks - under the hood a VLAN zone still tags on a VLAN-aware bridge - but it removes the copy-paste step where mistakes are introduced.

MTU and Guest-Side Consistency

Jumbo frames are decided once and must then be honoured everywhere. On Proxmox that means setting MTU on the physical NIC, the bond, the bridge and (implicitly) every guest whose traffic needs it:

iface enp1s0 inet manual
        mtu 9000

iface bond0 inet manual
        mtu 9000

iface vmbr0 inet static
        mtu 9000
        # ... rest of bridge configuration

A guest configured for 9000 whose path crosses a single 1500-byte hop will see its larger frames dropped with no ICMP back - the failure looks like an application timeout, not a network error. The physical switch ports, including every trunk in the path, must be set to the same MTU or raised to a value that accommodates it.

Common Failures and Their Cause

  • VLANs do not reach other devices: the switch port is access, not trunk; or the VLAN is missing from bridge-vids.
  • Bond has one active member: the switch LAG is missing or in the wrong mode, or member NICs are on different switches without MLAG.
  • Guests lose network after a change: the bridge's port list or PVID changed; verify with bridge vlan show.
  • Host unreachable after adding a bond: the management IP was on the physical NIC, which is now a port of the bond - move the IP to the bridge.
  • Frames seen twice or a loop reported: two tagging designs in parallel (a VLAN-aware bridge plus a vmbr0.NN sub-interface), or bridge-stp off combined with a cabling loop between two bridges.
  • New VLAN visible from the host but not from the VM: the tag is set on the VM NIC but the VLAN was never added to bridge-vids, so ingress is dropped before it reaches the tap.
  • Guest cannot reach the gateway after a NIC change on the switch: the upstream port was reset to access mode by a configuration management run - the host-side configuration is rarely the culprit when only one guest VLAN breaks.

Documenting the intended VLAN list in an IPAM before touching the host pays for itself the first time a VLAN is added: keeping the switch trunk, bridge-vids and the VLAN database in sync is exactly what NetBox IPAM is for. The same VLAN-aware bridge pattern applied outside Proxmox is covered in our Linux bridge VLAN configuration guide, and the equivalent concepts on the hypervisor side in ESXi vSwitch, port groups, VLAN and teaming.

原文链接:Proxmox VE wiki: Network Configuration