Cisco IOS-XR Line Card Diagnostics: show diag and Safe OIR - 夜莺博客

Cisco IOS-XR Line Card Diagnostics: show diag and Safe OIR

When an IOS-XR line card misbehaves the router rarely tells you why in one command. The evidence is spread across diagnostics, logging, per-location context dumps and hardware-module error counters. This runbook collects the sequence Cisco documents for a suspect card: prove the slot state, capture what the card logged, dump the process context while the fault is live, then decide between reset and replacement.

Two things make this harder than it should be. First, IOS-XR is deliberately distributed: each line card runs its own copy of many processes, so a fault on card 0/2 is invisible from the RSP unless you ask a location-aware command for it. Second, most of the useful evidence is transient - a context dump disappears after a reload, and error counters often reset when the card comes back up. The order of operations below therefore matters as much as the commands themselves: capture first, reset second, replace last.

Step 1 - Slot and Card State

show platform
show platform | include "0/[0-9]"
show diag slot 0/2
show inventory

show platform gives the node state for every card (IOS XR RUN, FAILED, UNPOWERED, MDR_RELOAD). show diag slot returns the card's hardware details and internal status; show inventory confirms the PID and serial against your asset records - do this before opening a TAC case, since RMA needs the exact PID/SN.

Reading the Node States

The single line of output from show platform is the fastest triage signal you have, but only if you know what each state implies:

  • IOS XR RUN - the card is up and its processes are running. A card in this state can still be faulty at the forwarding layer, which is why the counter checks later in this runbook exist.
  • FAILED - the card failed to boot or a critical process died. This is the state that justifies pulling the context dump.
  • UNPOWERED / PWRDN - the slot has no power, either because it was administratively shut down or because the chassis power budget is exhausted. Check show environment before assuming hardware failure.
  • MDR_RELOAD - the card is reloading as part of a minimum disruption restart, initiated by the platform itself or by a manual command.
  • IMAGE LOAD FAIL / BOOT LOOP - the card cannot load its software image. This is usually an FPD or bootflash problem, not a component failure, and it is worth checking the FPD levels on the slot before you order a replacement.
  • REL / RELOADING - a transient state during a normal bring-up; wait for it to settle before drawing conclusions.

Where the platform exposes a faster view of interfaces, it is worth combining the platform check with an interface sweep so you know exactly which ports are affected before touching anything:

show interfaces summary | include "0/2"
show ipv4 interface brief | include "0/2"
show interfaces counters errors | include "0/2"

Making Sense of show diag Output

show diag slot 0/2 is dense, and the fields that matter depend on the fault. The card descriptor block at the top gives you the PID, the hardware revision and the board serial - the three values that RMA and TAC both require. Below it, each port or sub-block carries operational state, and a port that reports a non-operational status while the card itself claims IOS XR RUN is a strong hint that the problem is at the port or optics level rather than the card level.

show diag slot 0/2 detail
show diag subslot 0/2/0
show diag summary

Use show diag summary when you need a fast inventory sweep of the whole chassis, and the per-slot detail form when you are building the evidence bundle for a vendor case. Capture a copy of the detail output and store it with the ticket: the card's own view of itself is the first thing TAC will ask for, and it is much easier to collect while the card is still installed and powered.

Step 2 - What the Card Logged

show logging | include 0/2
show hw-module subslot 0/2/0 errors
show context location 0/2 summary
show uidb data location 0/2

Hardware-module errors reveal parity, error-correction and fabric-facing counters on the card, while show context lists the crash contexts. If the card produced a context dump, retrieve it while the fault is still present - after a reload the dump is gone.

On a busy chassis, the plain logging view buries the useful lines. Three refinements make it tractable:

show logging | include 0/2
show logging | include 0/2 (Error|Critical|Warning)
show logging last 200 | include 0/2
show alarms brief card location 0/2

The pattern to watch for is repetition. A single parity error on a card that has been running for three years is noise; the same error appearing every few hours, or a burst of them after a specific interface event, is a hardware trend. Recording the counters at the time of the incident and again after a reload is what turns that observation into a defensible replacement case.

Hardware Module Error Counters Explained

show hw-module subslot 0/2/0 errors is the closest thing IOS-XR has to a per-card fault counter. The counter families to look for:

  • Parity and ECC errors - single-bit errors are corrected silently but their rate is the leading indicator of a memory component on its way out. Rising correctable counts are a trend to track; an uncorrectable count usually means the card is already metastable.
  • Fabric or backplane CRC errors - errors between the card and the fabric cards point at the slot, the connectors or the fabric ASIC rather than at the line card itself. If the same counter fires on a replacement card in the same slot, you have localised the fault to the chassis.
  • NPU and TCAM counters - these are forwarding-plane diagnostics rather than component health, and they are covered in detail in IOS-XR input drops and NP counters.
  • Transceiver and optics alarms - an optical fault usually surfaces as interface errors before it shows up as a card-level counter, so pair this check with IOS-XR interface CRC errors before blaming the line card.

Step 3 - Reset Before You Remove

hw-module subslot 0/2/0 reload
show platform
show logging | include 0/2

A subslot reload is the IOS-XR equivalent of a card power cycle, and it is usually faster and safer than a physical reseat in a chassis you cannot touch. Watch the node come back to IOS XR RUN and confirm the interfaces return with the expected configuration.

Before you issue it, check three things. Is the slot carrying traffic that cannot be interrupted - a single-homed customer link, the only path to a management network, a routing adjacency you cannot afford to lose? Is the running configuration saved, so that a card that comes up with a different configuration does not leave you with mismatched state? And is there a load interval worth waiting for, such as a maintenance window that starts in ten minutes?

show configuration commit list | head
show redundancy summary
show platform | include "0/2"
hw-module subslot 0/2/0 reload
show platform | include "0/2"

The reload is far less disruptive than an OIR because the card never leaves the chassis and the fabric path is not physically disturbed. If a card recovers cleanly under a subslot reload, the problem is frequently a stuck process rather than failing silicon - useful evidence when you are deciding whether to escalate.

Step 4 - Online Insertion and Removal, Done Properly

  • Confirm the platform supports OIR for that slot, and that the slot is not carrying the only path to a critical neighbour.
  • Verify the running configuration has been saved - an OIR of a card that holds unique configuration can still surprise you if the config lives only in the RSP's volatile memory.
  • Physically remove the card, then seat the replacement fully and tighten the ejector screws.
  • Let the router bring the card up; do not power-cycle the chassis to "help" it.
show platform
show interfaces | include "0/2"
show controllers npu stats | include 0/2

A workable OIR checklist, in the order you actually need it:

show redundancy summary
show platform | include "0/2"
show running-config interface | include "0/2"
admin show diag slot 0/2
show inventory | include "0/2"
show logging | include 0/2
show alarms brief card location 0/2
copy running-config harddisk:/pre-oir-backup.cfg   ! if the platform supports file copies from config
show hw-module subslot 0/2/0 errors

Note the administrative commands. On many IOS-XR chassis the diagnostic and inventory commands you want during a maintenance activity are in the admin plane rather than the exec plane, and an engineer who has only ever run show commands from the normal prompt will find half the output missing. Confirm which plane your card's diagnostics live in before the window opens, not during it.

Step 5 - If the Card Fails Again

Repeat failures usually mean a slot, fabric or firmware problem rather than a card problem. Compare the error counters you captured before and after the reload, verify the FPD and SMU levels on the slot, and check the interface counters for evidence of a physical-layer trigger. Packet-level and counter-level isolation for the ingress path is covered in IOS-XR input drops and NP counters, optical faults in IOS-XR interface CRC errors, and the firmware side in ASR 9901 FPD firmware recovery.

Checking FPD and SMU Levels Before Replacing Hardware

Field-programmable device versions are the most frequently missed cause of repeat line card failures, and they are the cheapest thing to fix. A card whose FPDs are behind the chassis baseline can fail to boot, boot into an inconsistent state, or fail intermittently in ways that look exactly like bad hardware.

show hw-module fpd location 0/2/0
show hw-module fpd
show install active summary
show install committed summary
admin show hw-module fpd location 0/2/0

If the slot reports FPDs in a "Not Upgraded" or "Needs Upgrade" state while the rest of the chassis is current, upgrade the slot before you RMA the card. The same logic applies to SMUs and package installs: a card that is running an older committed package set than its peers is a configuration defect, not a hardware failure. The recovery procedure for a card stuck in an inactive FPD state is documented in the ASR 9901 case study linked above and follows the same pattern across the 9000 series.

Building the Evidence Bundle for TAC or RMA

When you decide the card needs replacing, the difference between a case that moves and a case that stalls is the evidence you attached at the start. Collect all of the following while the card is still installed:

show tech-support
show tech-support linecard  ! where supported on the platform
show platform
show diag slot 0/2 detail
show inventory
show hw-module subslot 0/2/0 errors
show logging | include 0/2
show context location 0/2 summary
show version
show install active summary

Add the PID and serial number from show inventory, the date and time of the first observed fault, and the exact sequence of symptoms. Attach the error counter values captured both before and after the reload if you performed one, because the delta is what distinguishes a one-off from a trend. show tech-support output is large, so capture it to a file on the router or export it rather than pasting it into a ticket tool.

Deciding Between Reset and Replacement

A simple decision order keeps the process honest:

  • Card in IOS XR RUN but carrying error counters - monitor the rate. Escalate if correctable errors rise steadily or fabric CRCs appear; otherwise record and re-check at the next maintenance window.
  • Card in FAILED or reloading repeatedly - pull the context dump and logging first, then try a subslot reload. If it comes back clean and stays clean for a full week, treat it as a software event.
  • Card fails again after a successful reload - check FPD and package versions. Only after those are confirmed current does the card itself become the prime suspect.
  • A replacement card fails in the same slot - stop replacing cards. The slot, the fabric cards or the chassis midplane is the fault domain, and further RMAs will not help.

Post-Replacement Validation

After a successful OIR, verify more than the green light on the card:

show platform | include "0/2"
show interfaces | include "0/2" | include "line protocol"
show interfaces counters errors | include "0/2"
show hw-module subslot 0/2/0 errors
show logging | include 0/2
show install active summary

Confirm that every port returns to line protocol up, that the interface configuration matches what the card ran before removal, that the error counters start at zero, and that the software versions on the new card match its neighbours. A card that comes up running a different FPD set is a ticking problem that will resurface at the least convenient moment, so reconcile versions while you still have the maintenance window open.

Finally, update your asset records with the new serial number and close the loop on the old card - a spare that never gets returned to the vendor quickly becomes a spare nobody can find when the next card fails.

原文链接:https://www.cisco.com/c/en/us/td/docs/iosxr/asr9000/hardware-install/ethernet-line-card-installation-guide/b-asr9k-ethernt-line-card-install-guide/b-asr9k-ethernt-line-card-install-guide_chapter_010.html