Cisco IOS-XE Embedded Packet Capture: EPC Runbook - 夜莺博客

Cisco IOS-XE Embedded Packet Capture: EPC Runbook

Embedded Packet Capture (EPC) puts a capture engine inside IOS-XE: you define a buffer and a capture point, filter what you want, then export a PCAP for Wireshark. It is the right tool when you need the packet on the device that is actually dropping it and no SPAN destination is available. This runbook covers configuration, monitoring and export, plus the limitations that make people prefer SPAN when they have the choice.

When to Use EPC, SPAN or ERSPAN

All three give you packets, and picking the wrong one wastes an hour before you even start looking at frames. The decision is about where the problem lives and how much traffic you can afford to carry.

Method Where it captures Best for Cost
EPC Inside the router or switch, on the CEF or punt path Proving what the device itself sees, receiving or dropping CPU and buffer memory on the device, no cabling required
SPAN Mirrors a port to a local analyser Deep, long captures with full tooling A dedicated destination port and a capture host on site
ERSPAN Mirrors a port over an IP tunnel to a remote analyser No local capture host, or many switches to one collector Bandwidth on the transport path, and encapsulation to unravel

The distinguishing question is whether you need to know what the router received versus what the endpoint sent. If a customer insists the traffic left their host but the service never saw it, EPC on the ingress interface settles it in one command. If you need twenty minutes of a busy 10G link with Wireshark's expert analysis over the top, mirror to a real host: no EPC buffer on any platform holds that. And if the suspect device is in a remote data centre with no capture host, ERSPAN is usually cheaper than flying a laptop there. The transport details for mirroring are in SPAN, RSPAN and ERSPAN mirroring transport and the older multi-vendor procedure in port mirroring and SPAN.

One caveat up front: EPC is a troubleshooting tool, not a monitoring tool. Leaving a capture running for days consumes memory and finds nothing. Capture narrowly, capture briefly, turn it off.

Sizing the Buffer

A capture buffer that is too small gives you the last 200 packets of a ten-minute incident; one that is too large costs the device memory it may need. Do the arithmetic before you configure it.

# buffer size is in kilobytes
# example: 100 Mbps flow, want ~10 seconds of history
# 100 Mbps = 12.5 MB/s => 10 s = 125 MB => size 128000
monitor capture buffer CAP1 size 128000 max-size 1518 circular

The variables that actually matter are packet rate and average packet size, not link speed. A 1 Gbps link carrying small Voice-over-IP packets at 500 pps fills a buffer far more slowly than a link delivering a 9000-byte-MTU bulk transfer. Estimate from live counters rather than assuming:

show interfaces GigabitEthernet0/0/1 | include rate|packets
show policy-map interface GigabitEthernet0/0/1 | include rate

max-size matters too: 1518 truncates jumbo or encapsulated frames, while 9000 on a 1500-byte link just reserves more memory per slot than necessary. The linear-versus-circular choice interacts with size — a linear buffer guarantees the first N packets after it starts, which suits a fault you can trigger on demand; a circular buffer guarantees the last N before you stopped it, which suits an intermittent fault you cannot reproduce. Buffers also share a memory pool, so several running captures add up, and exceeding the pool produces a configuration error rather than a silent failure.

Define the Capture Buffer

monitor capture buffer CAP1 size 8192 max-size 1518
monitor capture buffer CAP1 circular

Size in kilobytes, and max-size sets the largest packet stored. Circular mode keeps the newest packets and is almost always what you want for intermittent faults; a linear buffer stops when full. Add a filter to avoid filling the buffer with noise:

ip access-list extended CAP-FILTER
 permit ip host 10.10.10.10 any
 permit ip any host 10.10.10.10
monitor capture buffer CAP1 filter access-list CAP-FILTER

Capture Point Types and What They Actually See

A capture point is a hook in the packet path. Which hook you attach to determines whether you see every packet, some packets, or none at all — and this is where most "EPC captured nothing" cases begin.

  • CEF (ip cef) — packets in the hardware-accelerated forwarding path. This is the normal data path, and for traffic being routed it is what you want. Packets that are CEF-switched may be handled in hardware on some platforms, so a CEF capture point sees what the software path is given.
  • Process-switched (ip process-switched) — packets punted to the CPU: the device itself as source or destination, ICMP, TTL expiry, and anything that requires software handling. If you are debugging a routing protocol, a management access problem, or a control-plane issue, this is the point to use, and CEF will show nothing.
  • Interface — attaches to the interface itself, closest to the wire. On many platforms only one interface capture point may exist at a time, and an interface capture point sees traffic before routing decisions are applied.
! control plane / punted traffic
monitor capture point ip process-switched POINT_CP both

! data plane on a specific interface
monitor capture point ip cef POINT_CEF GigabitEthernet0/0/1 both

! list what exists
show monitor capture point all

Traffic that is hardware-forwarded on an ASIC-based platform may genuinely not reach the capture point at all — this is the single most important limitation to know before you spend an afternoon on a PCAP that is missing exactly the flow you care about. If a capture returns nothing while counters show the packets arriving, suspect the forwarding path rather than your filter, and consider whether the platform documents a hardware-accelerated exception. When you need proof from a specific point in the hardware path, a mirror to a capture host is often the only reliable answer.

Define the Capture Point

monitor capture point ip cef POINT1 GigabitEthernet0/0/1 both
monitor capture point associate POINT1 CAP1

Capture points can be ip cef (hardware-accelerated, CEF path), ip process-switched (traffic punted for software processing) or interface-based. Use cef for normal forwarding path traffic, and remember that a point matched to one direction (in, out or both) has to be chosen deliberately: capturing both on a busy link fills the buffer twice as fast.

Filters That Keep the Buffer Useful

An unfiltered capture on a production uplink fills its buffer with the noisiest chatter on the link. Three filter mechanisms exist and they behave differently, which matters when you are trying to explain why a capture is empty.

! 1. ACL-based filter (most common)
ip access-list extended CAP-FILTER
 permit ip host 10.10.10.10 any
 permit ip any host 10.10.10.10
monitor capture buffer CAP1 filter access-list CAP-FILTER

! 2. match / class-map based filter
class-map match-all CAP-CLASS
 match access-group name CAP-FILTER
monitor capture buffer CAP1 filter class-map CAP-CLASS

! 3. verify what is bound before you start
show monitor capture buffer CAP1 parameters

The ACL is evaluated with implicit deny, so only traffic explicitly permitted is stored. That catches people out: a filter written for one direction silently discards the return traffic, and the resulting capture looks like a one-way conversation that never happened. Write both directions, and when the flow involves a translated address, permit the before- and after-NAT forms if the capture point sits near the NAT boundary.

Combine the filter with the direction on the capture point. Capturing both on a busy trunk doubles your buffer consumption and makes the PCAP harder to read; another option is to filter on one side and use the direction to halve the volume. If you are chasing a specific application, filter on the port and the host tuple rather than on the host alone — a single busy server can carry hundreds of flows, and a host-only filter buries your transaction in the middle of them.

The on-device capture filter is coarser than a Wireshark display filter — it cannot match TCP flags, sequence numbers or payload. Capture slightly more than you need and filter afterwards: capture filter versus display filter draws the boundary, and retransmission versus out-of-order analysis is the next step.

Start, Inspect, Stop

monitor capture point start POINT1
show monitor capture point POINT1
show monitor capture buffer CAP1 dump
show monitor capture buffer CAP1 parameters
monitor capture point stop POINT1

Reproduce the fault while the capture runs. The dump command prints a summary of what is in the buffer - packet count, timestamps, and the first packets - which is often enough to confirm whether the traffic is arriving at all before you export anything.

Running the Capture: Monitoring and Multiple Points

Starting a capture is one command; running it well is about knowing what the device is doing while it runs. Capture state is per association, and a point can be started without a buffer association — which produces a nicely configured capture that stores nothing.

monitor capture point start POINT1

show monitor capture point all            # state, buffer, filter per point
show monitor capture buffer CAP1 dump     # packet count + first packets
show monitor capture buffer CAP1 parameters
show monitor capture buffer CAP1 detailed # per-packet view

monitor capture point stop POINT1

Use the dump output as a checkpoint before exporting. It shows how many packets matched, whether they are arriving at all, and enough of the headers to confirm the flow is the one you expected. Discovering that the capture was empty at the device saves you the transfer, the open in Wireshark and the disappointment.

Multiple capture points let you compare ingress and egress on two interfaces of one chassis, localising a one-way-forwarding drop to the device. Each point needs its own buffer, and only a limited number can run at once — check the platform limits before designing the comparison.

monitor capture point associate POINT1 CAP1
monitor capture point start POINT1
! ... reproduce ...
monitor capture point stop POINT1
monitor capture buffer CAP1 clear         # always clear between runs

Clearing matters more than it sounds: stopping a capture does not empty the buffer, and a stale buffer mixes two incidents with only timestamps to separate them.

Export to Wireshark

monitor capture buffer CAP1 export tftp://10.0.0.50/cap1.pcap
! or to flash, then copy off-box
monitor capture buffer CAP1 export flash:cap1.pcap

Export produces a standard PCAP; open it in Wireshark or feed it into tshark for scripted analysis. Also clear the buffer between runs - an uncleared buffer produces captures that mix two incidents, which is a subtle way to waste an hour.

Export and Analysis Workflow

The export itself has a few failure modes worth knowing. EPC writes a standard libpcap file, so anything that reads PCAP will read it, but the round trip is where time is lost.

! TFTP is simple but slow and unauthenticated
monitor capture buffer CAP1 export tftp://10.0.0.50/cap1.pcap

! FTP is faster and can be authenticated
monitor capture buffer CAP1 export ftp://user:pass@10.0.0.50/cap1.pcap

! or stage locally, then pull over SCP
monitor capture buffer CAP1 export flash:cap1.pcap
copy flash:cap1.pcap scp://lab@10.0.0.50//home/lab/

Validate the file after transfer: an interrupted TFTP transfer often still opens and simply ends early, silently truncating your incident. Compare the final packet timestamp against the count reported by show monitor capture buffer CAP1 dump.

capinfos cap1.pcap
tshark -r cap1.pcap -q -z io,stat,1
tshark -r cap1.pcap -Y 'tcp.analysis.retransmission' | wc -l
tshark -r cap1.pcap -T fields -e frame.time_relative -e ip.src -e ip.dst -e tcp.flags | head

For scripted triage, tshark beats a GUI: packet rate per second, a retransmission count and a filtered field dump usually identify loss, misdirection or a protocol rejection before you open a window. Keep the raw PCAP regardless — there is no way to re-capture an incident that has passed.

This is also the point to bring other evidence alongside the capture. Flow records in NetFlow v9 on IOS with Flexible NetFlow show which flows traversed the device even when the packets were not captured, and sFlow versus NetFlow versus IPFIX explains when each export format is the better instrument. A capture tells you what one packet did; a flow record tells you how often.

Troubleshooting EPC Itself

Before debugging the network, make sure the capture tool is doing what you think it is.

Symptom Likely cause Fix
Buffer has 0 packets Point attached to the wrong path (CEF where traffic is punted, or vice versa) Switch between ip cef and ip process-switched and retest
Buffer has 0 packets, counters increment Filter too narrow, or one-directional ACL Permit both directions; test with no filter first
Point will not start No buffer associated, or the point limit is reached show monitor capture point all; remove unused points
Buffer full of noise No filter, or filter matching too broadly Add an ACL for the exact host and port tuple
Export fails or hangs Unreachable TFTP host, wrong VRF, or ACL blocking the transfer Replace with a flash export, then copy with SCP
Capture misses hardware-forwarded traffic Platform offloads the flow in the ASIC Mirror to a capture host instead
Mixed incidents in one PCAP Buffer not cleared between runs monitor capture buffer CAP1 clear

Two habits prevent most of these. Prove the capture works before you need it — a two-minute capture of a known flow on a healthy link, exported and opened — so that during an incident you are testing one thing rather than two. And record the exact commands, buffer size and filter in the ticket, because the next engineer will want to reproduce them.

Practical Limits and Alternatives

  • EPC captures what the router sees at the chosen point, not what the endpoint sent: VLAN tags and encapsulation are as the interface presents them, so captures from trunks need to be read with that in mind.
  • Performance impact is real on software capture points; always use filters and stop the capture when finished.
  • For long or high-rate captures, use SPAN to a real capture host instead - the multi-vendor procedures are in port mirroring and SPAN.
  • Pair EPC with counter and flow views for a complete picture: flow records in NetFlow v9 configuration, alternative flow exports in sFlow versus NetFlow versus IPFIX, path-level evidence in mtr for packet loss and latency and reachability probing in IP SLA with track objects.

Case Study: An Intermittent One-Way Audio Drop

A branch reports that calls between two sites lose audio in one direction roughly twice a day, for a few seconds. Nothing appears in the router's logs, interface counters are clean, and the call platform blames the network while the network blames the platform. This is the archetypal EPC case: brief, intermittent, and disputed.

  1. Narrow the filter first. Capture only the RTP port range used by the media gateway, in both directions, on the interface facing the branch. A host-only filter would capture the signalling as well and fill the buffer with SIP.
  2. Use a circular buffer sized for the event. Two minutes of the peak media rate is enough; a buffer that holds ten minutes costs memory and makes the analysis slower for no benefit.
  3. Start the capture and leave it running during the fault window. Because the buffer is circular, the packets before the drop survive. A linear buffer would be full and stale by then.
  4. Stop, then inspect the dump. If packets are present inbound and absent outbound, the device is dropping or misrouting; if they are absent inbound, the problem is upstream or at the source.
  5. Export and compare timestamps with the call platform's CDR. A RTP stream that simply stops, with no ICMP and no TCP reset, points at a path change or a stateful device in the middle — a firewall's session timeout is a frequent culprit, and the evidence is silence rather than error.
  6. Correlate with routing. If the outage coincides with a routing event, the same moment will show in the logs of the peer device. Path-level probing with mtr for packet loss and latency and synthetic reachability tests via IP SLA with track objects turn a twice-daily mystery into a monitored condition.

The lesson is about narrowing scope: a filtered, circular, two-minute capture answers a question a week of unfiltered data cannot. Junos monitor traffic packet capture examples shows the same discipline in different syntax.

原文链接:https://www.cisco.com/c/en/us/td/docs/ios-xml/ios/epc/configuration/xe-16/epc-xe-16-book.html