Cisco Nexus vPC Failover Scenarios and Troubleshooting Checklist - 夜莺博客

Cisco Nexus vPC Failover Scenarios and Troubleshooting Checklist

A vPC lets two Nexus switches appear as a single port channel to a downstream device, giving Layer 2 multipathing — but the failover behavior depends entirely on which link fails and in what order. Understanding the failure reactions is what separates a well-run vPC domain from one that melts down during maintenance. This article explains the three canonical vPC failure scenarios and provides the troubleshooting checklist used to verify a healthy domain, based on the Cisco Community knowledge-base article.

The reason this matters is that a vPC domain does not "converge" the way an STP or routing topology does. There is no blocked port to unblock and no shortest-path recalculation. Instead, a small number of role decisions — primary or secondary, peer reachable or not, peer-link up or down — determine which member ports stay forwarding. Get those decisions wrong, or lose the control channel that carries them, and the domain will happily keep forwarding traffic into a black hole while every interface still reports up/up.

Background: What Actually Holds a vPC Domain Together

Before looking at failure reactions, it helps to be precise about the three components that make vPC work:

  • Peer-link — a Layer 2 port-channel trunk between the two peers. It carries Cisco Fabric Services (CFS) control traffic, synchronises MAC and IGMP state, and forwards traffic for orphan ports and for traffic that ingresses one peer and must egress the other. It must never carry routed VLANs or act as a normal data path for routed traffic.
  • Peer-keepalive — a lightweight Layer 3 heartbeat sent over a separate path (management VRF, a dedicated switch, or a routed link). It is roughly 64 bytes at 1–2 packets per second, and the peer is declared dead after three missed heartbeats.
  • vPC member ports — the downstream port-channels that share a common vPC number on both peers and form one logical LAG toward the access device.

The critical design insight is that the peer-link is a data and control path while the peer-keepalive is a liveness path only. Detection of a peer-link failure therefore has two possible sources: local link/UDLD state on the port-channel members, or failure of CFS messages to be acknowledged. Detection of peer death comes only from the keepalive.

Failure Scenario 1: Peer-Link Failure

When the vPC peer-link fails, each peer checks the remote peer's status over the peer-keepalive (pk-link) heartbeat — 1–2 packets per second at ~64 bytes; failover triggers after 3 missed heartbeats. If both peers are alive, the secondary switch disables all its vPC ports to prevent loops. Data keeps forwarding down the remaining active port-channel ports, with failover gated on CFS message failure or UDLD/link-state detection.

The operational consequence is asymmetric. The primary switch continues to forward on every vPC member port and every orphan port. The secondary suspends all vPC member ports, and any orphan port on the secondary is a topology-dependent liability — if it can reach a VLAN that also exists behind the primary, you have just built a loop. This is why orphan-port design review belongs in the same conversation as peer-link redundancy.

If both member links of the peer-link port-channel can fail at once — for example, both links terminate on the same line card and that card resets — the entire domain loses its control path in a single event. Cisco's guidance is to spread peer-link members across line cards or fabric modules so that no single hardware failure removes the whole link. Verify with:

switch# show port-channel summary interface port-channel 10
switch# show interface port-channel 10 | include members
switch# show vpc

A healthy output shows Peer-link status: Up and, on the secondary, vPC peer-link is up. Immediately after a peer-link flap you will see the secondary report suspended vPC member ports and a message such as %VPC-2-VPC_PEER_LINK_DOWN in the log.

Failure Scenario 2: Peer-Link and Keepalive Both Fail

This double failure is the hardest to discover: HSRP/PIM/STP fail to become active on the secondary, data continues forwarding but learning breaks for new flows. Using the management port as the keepalive link at least alerts the NOC via loss of management access — a valuable side effect.

Why does traffic still flow? Because with no keepalive the two switches cannot prove each other dead, so both keep their vPC member ports forwarding — a split-brain, active-active state. Existing flows with already-programmed MAC entries continue to be forwarded by the hardware. What breaks is everything that requires control-plane coordination: new MAC addresses learned on one peer are not synchronised to the other, so return traffic can be delivered to the wrong peer and dropped. HSRP/VRRP active election cannot complete, so the virtual gateway address may exist on neither peer or on both without arbitration. PIM DR election, IGMP snooping sync, and STP root coordination all degrade in the same way.

In practice the symptoms reported by users are "some new connections fail, existing ones are fine" and "it works after a retry". The diagnostic fingerprint is:

switch# show vpc brief
switch# show vpc peer-keepalive
switch# show logging logfile | include VPC
switch# show system internal vpcm info global

If the peer-keepalive shows peer is not reachable or messages are failing while the peer-link is also down, you are in a split-brain and must not simply reload a switch — decide which peer should forward, shut the offending vPC ports on the other, restore connectivity, and then recover. Enabling peer-gateway and vPC auto-recovery is not a substitute for a dedicated, diverse keepalive path.

Failure Scenario 3: One Peer Switch Fails

The remaining switch keeps forwarding data for the vPC — the cleanest failure mode and the one the design is built for.

Because both switches were programmed with identical forwarding state for the vPC member ports, the survivor needs no state transfer to keep forwarding. The downstream device sees the LAG lose half its members and continues on the rest. The gate on how fast this is perceived is the downstream device's LACP timeout (fast mode, 1 second) and any BFD sessions riding the LAG. The survivor promotes itself to operational primary, takes over HSRP/VRRP active role, and remains the STP root if it was primary. Total data-plane impact is usually limited to the seconds it takes the access device to rehash its own LAG hashing and re-converge its own control protocols.

The trap here is a "clean" primary failure where the primary was also the only BGP/OSPF next hop for a routed VLAN. vPC protects Layer 2 multipathing, not Layer 3 reachability — routing adjacency must be designed to survive the loss of one peer separately.

How Each Control Plane Reacts

Component Peer-link down, peer alive Peer-link and keepalive down One peer powered off
vPC member ports Suspended on secondary Forward on both (split-brain) Survivor forwards
HSRP/VRRP Secondary does not become active Election cannot coordinate Survivor becomes active
PIM / IGMP snooping No sync, DR stays on primary Independent DR per peer, duplicate flows Survivor takes over
STP Root unchanged, no blocking on vPC ports Potential loop if orphan topology is unsafe Survivor root
MAC learning Primary only, secondary ports down Broken for new flows Survivor learns normally

Type-1 and Type-2 Consistency Checks

Before bringing up a vPC, the two peers exchange configuration to verify compatibility. Type-1 parameters (e.g. VLANs, STP mode, LACP mode) must match exactly — a mismatch drops the vPC on the secondary (graceful consistency check default). Type-2 parameters are less critical and do not bring the vPC down.

In practice the Type-1 set you must keep identical across peers includes STP mode and MST region/instance configuration, the STP port type, the peer-link VLAN list, the LACP rate and suspend-individual setting, MTU and QoS priority-flow entries, and the set of VLANs allowed and created on both sides. Type-2 parameters include things like MAC aging timers and certain interface-level knobs where a mismatch only produces inconsistent behaviour rather than a suspension.

Because a Type-1 mismatch only suspends the secondary, the first symptom is often "half the port-channels stopped passing traffic after a maintenance window" rather than a total outage — which is easy to misattribute to the downstream device. Always run the consistency check after any change that touches VLANs, MST, or LACP:

switch# show vpc consistency-parameters global
switch# show vpc consistency-parameters interface port-channel 20
switch# show vpc consistency-parameters vpc 20

Prerequisites Before You Touch a vPC Domain

  • Peer-keepalive must be out-of-band: a separate VRF, a dedicated physical link, or a management switch — never ride the peer-link.
  • The peer-link must be a Layer 2 trunk that carries only vPC VLANs and the peer-link VLAN; routed VLANs must not be allowed on it.
  • Peer-link members should be spread across line cards and, where possible, across fabric modules.
  • The vPC number must be identical on both peers for each downstream port-channel; the system MAC and system priority must also match.
  • Role priority should be set deliberately so you know which peer will be primary, and auto-recovery should be configured for peer-link failure events.
  • Graceful consistency check should be left enabled so a Type-1 mismatch suspends rather than breaks forwarding unpredictably.
switch(config)# vpc domain 100
switch(config-vpc-domain)# peer-keepalive destination 10.0.0.2 source 10.0.0.1 vrf management
switch(config-vpc-domain)# role priority 100
switch(config-vpc-domain)# auto-recovery reload-delay 240
switch(config-vpc-domain)# peer-gateway

Troubleshooting Checklist

switch# show vpc
switch# show vpc brief
switch# show vpc consistency-parameters
  • Is the peer-keepalive link in a separate VRF (or a management switch on the mgmt ports)?
  • Are the keepalive source/destination IPs reachable from that VRF? (ping vrf)
  • Is the peer-link a Layer 2 port-channel trunk carrying only vPC VLANs?
  • Is the vPC number identical on both peers for each downstream port-channel?
  • Are Type-1 parameters identical? Check with show vpc consistency-parameters.
  • Does show vpc role show the intended primary, and does the operational role match the configured role priority?
  • After a peer-link flap, did the secondary recover its vPC ports automatically, or did it require manual intervention?
  • Are any ports flagged as orphan ports, and does the orphan topology survive a peer-link failure without creating a loop?

Interpreting the Output

Three fields in show vpc brief carry most of the diagnostic value. vPC peer-link status must be up. vPC status per vPC must read up — down* with an asterisk means the peer-link is down but the vPC is configured, and suspended means a consistency check failed or the peer-link is down. Consistency Status reading success means all Type-1 parameters match. Any field showing failed is a Type-1 mismatch you can localise with show vpc consistency-parameters.

switch# show vpc role
switch# show vpc peer-keepalive
switch# show port-channel summary
switch# show spanning-tree vlan 100 | include Root

Common Mistakes and Fixes

  • Keepalive on the peer-link. If the keepalive path shares fate with the peer-link, a peer-link failure looks identical to a peer death and you get split-brain instead of a clean secondary shutdown. Fix: dedicated path.
  • Routed VLANs on the peer-link. Causes traffic to take a suboptimal path and can interact badly with the secondary role. Fix: prune to vPC VLANs only.
  • Type-1 drift during maintenance. A VLAN added on one peer but not the other suspends every vPC. Fix: configure both peers from one source of truth, and run the consistency check afterwards.
  • No auto-recovery. After a peer-link comes back, the secondary can stay suspended indefinitely without it. Fix: enable and tune the reload delay.
  • Orphan ports across both peers. A device single-homed to each peer on the same VLAN creates a loop the moment the peer-link fails. Fix: keep orphan ports on one peer, or dual-home them.

FAQ

How long does a vPC failover take? There is no single timer. Once the peer-link is declared down, the secondary suspends its vPC ports in milliseconds to low seconds. The downstream device's own LAG and BFD timers usually dominate the end-to-end recovery time, which is why fast LACP and BFD are recommended on vPC member ports.

Can I reload the vPC primary? Yes — a rolling reload is one of the supported operations, because one-peer failure is the clean scenario. Confirm the secondary holds operational primary, verify Type-1 consistency, then reload. Never reload both peers at once.

Does vPC remove the need for STP? No. STP still runs and remains the safety net for topology errors and orphan ports. It simply is not the mechanism that provides redundancy for the vPC member ports.

What is peer-gateway for? It lets each peer forward traffic destined to the other peer's router MAC, avoiding a detour across the peer-link for gateway traffic. It does not change failure behaviour, but it does reduce dependence on peer-link bandwidth.

vPC is the Nexus answer to the same problem MLAG solves on Arista — see Arista EOS MLAG 详解 for the comparison, and Cisco EVPN/VXLAN Multisite 排障 for EVPN integration context. For the initial build, see Cisco NX-OS vPC Configuration: Step-by-Step with Commands, and for the design details that most often cause these failures, Nexus vPC: Peer-Gateway, Orphan Ports and Health Checks. If you are comparing multi-chassis technologies across vendors, Dell OS10 VLT Troubleshooting: show vlt Commands and Failures covers the equivalent failure semantics.

原文链接:https://community.cisco.com/t5/data-center-and-cloud-knowledge-base/vpc-failover-scenarios-and-troubleshooting-checklist/ta-p/3143104