Unexpected MAC Learning on Catalyst 9000 Switches

When the gateway MAC address suddenly shows up on a user-facing access port instead of the uplink, the entire VLAN can lose connectivity — and on a network running 802.1x with MAC authentication bypass, the problem can stick until someone manually recovers the port. This article reconstructs a real Cisco TAC case on Catalyst 9300 switches where an endpoint was silently reflecting traffic sourced from the gateway back into the switch, and shows exactly how the engineers proved it using embedded packet capture (EPC) and SPAN, and why the security feature set made the MAC 'stick' to the wrong port. It is a masterclass in MAC learning fundamentals and endpoint-driven anomalies.

How Catalyst Switches Learn MAC Addresses

Catalyst switches learn MAC addresses on ingress based on the source MAC address (SMAC) of the received frame. The MAC address table is normally a trustworthy map of where an address lives — if a MAC is learned on an unexpected interface, the switch received a frame with that SMAC on that port. In very rare cases internal forwarding-plane reflection can also cause this, but the endpoint explanation should always be ruled out first.

The Problem: Gateway MAC Learned on Random Access Ports

In the reported case, endpoints in the data VLAN lost connectivity to hosts outside their subnet. The VLAN 2 gateway MAC was learned on a user interface (Gi1/0/2) instead of the expected uplink (Te1/1/1). The symptom appeared randomly across a multi-campus network, but a trend emerged: the same endpoint model was involved in every occurrence.

Because the access ports ran 802.1x with MAB fallback, the reflected gateway MAC triggered an authentication session and was programmed as a static entry. The security implementation then blocked MAC movement, so the switch could neither age out the MAC on the user port nor re-learn it on the uplink.

Sequence of Events

1. MACs are learned on the expected interfaces — normal state.
2. The endpoint reflects traffic sourced from the gateway back into its switch port.
3. Port security treats the reflected MAC as a new host: it authenticates and the MAC is programmed as STATIC.
4. When the correct entry ages out on the uplink, security prevents re-learning.
5. Traffic for the whole local VLAN is impacted; the port needs shut/unshut to recover.

Proving It with Packet Capture

Use the Embedded Packet Capture (EPC) on Catalyst to catch inbound frames from the suspect endpoint:

Switch# monitor capture TAC interface gi1/0/2 in match mac host aaaa.bbbb.cccc any
Switch# monitor capture TAC start
Switch# monitor capture TAC stop
Switch# show monitor capture TAC buffer brief

Physical SPAN with a MAC filter is equally reliable:

Switch(config)# monitor session 1 source gi1/0/2 rx
Switch(config)# monitor session 1 filter mac access-group MACL
Switch(config)# monitor session 1 destination gig1/0/48

Resolution and Key Takeaways

The ultimate fix was an endpoint firmware update — the reflection behavior was already known to the vendor. The Catalyst hardware, software and configuration behaved entirely as expected. The takeaway: if a MAC appears on an unexpected interface, the switch is almost certainly telling the truth about what it received. Also review our EVPN MAC-VRF validation on ACX7000 and the Arista EOS MLAG run book for related MAC/forwarding troubleshooting on other platforms.

原文链接：https://www.cisco.com/c/en/us/support/docs/switches/catalyst-9300-switch/222771-understand-unexpected-mac-learning-on-ca.html