Cisco ASR 9000 Troubleshooting Commands Guide

The ASR 9000 series runs IOS XR, where interface state lives in the Interface Manager (IM) database and memory thresholds are configurable knobs that prevent silent failures. This article walks through the troubleshooting commands from the official ASR 9000 advanced system command reference — what show im database actually tells you, how to read its views, and how to configure watchdog memory thresholds that keep the router healthy under pressure.

What separates IOS XR from classic IOS is the split between the database-driven control plane and the hardware forwarding plane. Each line card, RSP and fabric module runs its own distributed process set, and administrative state is replicated through redundant databases rather than read directly from hardware registers. That architecture is why a link can be physically present on a port ASIC while the routing stack still refuses to use it — and why tools such as the IM database and the watchdog subsystem exist at all. Troubleshooting on the ASR 9000 is therefore less about running a single magic command and more about walking the state chain from physical layer to protocol layer in a fixed order.

Understanding the cisco-support Command Context

Several ASR 9000 troubleshooting commands carry the cisco-support task ID and are intended for use during fault-finding with Cisco TAC. They require your user group to be associated with the matching task group; if the command is blocked, your AAA administrator must adjust the assignment. In practice, a lab or break-glass account should belong to a user group whose task list includes cisco-support alongside the usual basic-services, interface and monitoring tasks, otherwise the CLI reports an authorization error even for a user with full read access to configuration.

RP/0/RSP0/CPU0:router# show aaa user-group cisco-support tasks
RP/0/RSP0/CPU0:router# task read cisco-support

The task read command is available inside the context of commands that accept task declarations, and it is the mechanism TAC engineers use to temporarily grant read access to a diagnostic command set. Treat it as a change-controlled action: record who ran it and when, and remove the grant once the incident is closed.

show im database: The Interface State Database

RP/0/RSP0/CPU0:router# show im database brief location 0/0/CPU0
Handle   | Name        | State | MTU  | Views
0x01080060 Gi0/0/0/0   up      9212   GDP|LDP|L3P|OWN
0x010800c0 Gi0/0/0/3   down    1514   GDP|LDP|L3P|OWN

The IM database is the authoritative record of every interface's state, MTU and ownership. Use brief for a fast overview, interface <type> to drill into a specific interface, and verbose for the full ownership and state-transition detail — the Views column (OWN, L3P, LDP, GDP...) tells you which subsystem owns the interface at each layer, invaluable when routing sees a link that the line card does not.

Three invocation patterns cover almost every case. To list everything the node knows about, run the brief form against a specific location; to understand one link end to end, query by interface; and to see the raw ownership tree with timestamps, add verbose.

RP/0/RSP0/CPU0:router# show im database interface GigabitEthernet 0/0/0/3 detail
RP/0/RSP0/CPU0:router# show im database interface GigabitEthernet 0/0/0/3 verbose
RP/0/RSP0/CPU0:router# show im database brief location 0/1/CPU0

Reading the IM Views: OWN, L3P, LDP, GDP and BGP

Each view is a registered consumer of the interface object, and the interface is only fully usable when every required view reports up. Understanding the acronyms shortens diagnosis dramatically.

• OWN — the owner view, representing the interface itself. If OWN is down, nothing above it can be up.
• L3P — the Layer 3 protocol view; tracks whether IPv4/IPv6 is enabled and whether the address family is operational.
• LDP — the Label Distribution Protocol view used by MPLS. A stuck LDP view commonly blocks label binding advertisement even though the link passes traffic.
• GDP — the Generic Data Path view; reflects whether traffic can actually be pushed to the hardware forwarding path.
• BGP — an upper-layer consumer that appears when BGP is bound to the interface, useful when a session stays idle despite a working link.

A classic asymmetry looks like this: an interface reports up in show interfaces, but the IM database shows the L3P view down because the address was never applied or was removed by a configuration rollback. Comparing the two outputs immediately isolates a software-state problem from a physical one.

watchdog threshold memory: Proactive Memory Monitoring

IOS XR can alarm on memory pressure per node instead of failing unpredictably. The recommended thresholds scale with total memory (minor 20%, severe 10%, critical 5% of available memory):

RP/0/RSP0/CPU0:router# configure
RP/0/RSP0/CPU0:router(config)# watchdog threshold memory location 0/RP0/CPU0 minor 30 severe 20 critical 10
RP/0/RSP0/CPU0:router(config)# commit

Set global defaults without a location to cover all nodes, and override per-node where needed. A global statement of watchdog threshold memory minor 30 severe 20 critical 10 applies to every card installed in the chassis, which is usually the safer baseline; the per-location form then raises or lowers limits on a card that has a known higher steady-state footprint, such as a route processor running large BGP tables.

If a node sits at the critical threshold, collect show processes memory and show watchdog memory-state before assuming a leak. Threshold crossings generate syslog messages with the severity matching the configured tier, which is precisely why they belong in your NMS alarm set: a memory alarm that fires hours before a crash is worth far more than a post-mortem.

RP/0/RSP0/CPU0:router# show watchdog threshold memory all
RP/0/RSP0/CPU0:router# show watchdog memory-state location 0/RP0/CPU0
RP/0/RSP0/CPU0:router# show processes memory location 0/RP0/CPU0 sorted

Interface-Level Diagnosis Beyond the IM Database

Once the IM database tells you which layer is unhappy, drop one level and read hardware counters. Physical-layer faults — a dirty optic, a marginal fibre run, a speed mismatch — surface as CRC or input errors that the IM database never reports, because the database tracks administrative state while counters track data-plane quality.

RP/0/RSP0/CPU0:router# show interfaces GigabitEthernet 0/0/0/3
RP/0/RSP0/CPU0:router# show controllers GigabitEthernet 0/0/0/3 internal
RP/0/RSP0/CPU0:router# clear counters GigabitEthernet 0/0/0/3

Clear the counters, wait a defined interval, and re-read them. Errors that stop after a reseat point at the physical connector; errors that continue at a constant rate point at the far end or at the optic; errors that scale with traffic point at a congestion or forwarding-path issue rather than a hardware fault.

Verifying Forwarding and NP Counters

The ASR 9000 forwards in dedicated network processors, and drops there are invisible to interface counters. When applications report loss but every interface is clean, examine the NP and forwarding statistics on the ingress card.

RP/0/RSP0/CPU0:router# show controllers np ports all location 0/0/CPU0
RP/0/RSP0/CPU0:router# show ppa interface location 0/0/CPU0
RP/0/RSP0/CPU0:router# show mpls forwarding counters

Elevated input drops with clean interface counters almost always mean either a policer or QoS policy discarding traffic, or a punting path that cannot keep up with exception traffic. Correlate the timestamps of the drop counters with the reported outage before you touch configuration.

Building a Repeatable Triage Order

Experienced TAC engineers work top-down from the most objective evidence to the least. A practical order for the ASR 9000:

1. Confirm the scope — is this one interface, one line card, or the whole chassis?
2. Read show im database brief for the affected location to see which views are down.
3. Read show interfaces and show controllers for physical and counter evidence.
4. Check show watchdog memory-state and process memory for resource exhaustion.
5. Examine NP and QoS statistics if counters are clean but traffic is lost.
6. Collect a targeted command set for TAC before making changes that destroy evidence.

This order matters because each step narrows the hypothesis. Restarting a process or clearing counters early can erase the exact state that would have identified the root cause, turning a fifteen-minute diagnosis into a multi-day investigation.

Collecting the Right Evidence for TAC

When an SR is opened, TAC will ask for structured output rather than a narrative. On the ASR 9000 the fastest way to produce a complete evidence set is the platform-specific tech-support collection, scoped to the affected card where possible to keep the file manageable.

RP/0/RSP0/CPU0:router# show tech-support routing bgp location 0/RP0/CPU0
RP/0/RSP0/CPU0:router# show tech-support interfaces
RP/0/RSP0/CPU0:router# show logging | include WATCHDOG|MEMORY|IM_

Always capture the running configuration, the install inventory and the timestamps of the first failure. A log excerpt without an absolute time reference is nearly useless, because correlating a crash with a configuration change or a traffic shift depends entirely on precise timing.

Pairing with the TAC Collection Set

For full-platform triage, combine these with the IOS XR TAC command collection list (platform, install, BGP/OSPF, MPLS and drops) and our multi-vendor CLI cheat sheet for IOS XR/Junos/SR OS/VRP cross-reference. Two further guides make a useful pair with this one: IOS XR input drops and NP counter troubleshooting for the forwarding-plane side of the workflow, and IOS XR line card diagnostics with show diag and safe OIR for hardware-level faults on a single card. If the symptom is physical rather than logical, IOS XR interface CRC errors covers the counter patterns in detail.

Common Failure Patterns and Their Signatures

Most ASR 9000 incidents fall into a small number of recognizable shapes, and matching the symptom to the pattern saves the most time of any single skill.

• Interface up, no traffic — check the GDP view. If the generic data path is down while OWN is up, the forwarding ASIC has not been programmed and the fault is internal to the line card, not the cable.
• Interface flaps at long intervals — read the state-transition history in the verbose IM output and compare against optical receive power. Random multi-hour flaps usually track temperature or a degrading optic.
• Gradual memory growth with no leak report — capture show processes memory twice, an hour apart, sorted by growth. A single process climbing steadily identifies the culprit far faster than a full tech-support file.
• Control-plane sessions reset under load — correlate with watchdog and NP punt counters. Exception traffic exceeding the punt policer resets sessions while leaving interface counters pristine.
• Card not reachable after OIR — verify the card reached IOS XR RUN in show platform before investigating anything logical; a card stuck in a boot loop will never register views.

Key Takeaways

The IM database is your state oracle: it tells you which layer owns an interface and which view is failing, and it does so without touching the hardware. Watchdog memory thresholds convert a silent, unpredictable failure mode into a logged, actionable alarm, so configure them on every node and wire the resulting syslog into monitoring. Everything else — interface counters, NP statistics, tech-support files — exists to confirm or eliminate the hypothesis the first two commands generate. Run them in a fixed order, capture evidence before you change state, and the ASR 9000 stops being opaque.

原文链接：https://www.cisco.com/c/en/us/td/docs/routers/asr9000/software/adv-sys/command/reference/b-advsys-cr-asr9000/b-advsys-cr-asr9000_chapter_01.html