Cisco NX-OS ISSU: Impact Checks and Upgrade Workflow

An in-service software upgrade on Nexus is not one command, it is a sequence of checks whose entire purpose is to answer one question: will this upgrade be hitless, and if not, why? Teams that skip the impact check because “the same upgrade worked on another switch” are the ones who discover a kernel uprev or a vPC incompatibility at 02:00. Below is the check-first workflow, what each output means, and how enhanced ISSU and graceful insertion/removal change the blast radius.

Step 1: ask the installer what it thinks

switch# show install all impact nxos bootflash:nxos64.10.2.1.169.F.bin
Verifying image bootflash:/nxos64.10.2.1.169.F.bin for boot variable "nxos". 100% -- SUCCESS
Verifying image type. 100% -- SUCCESS
Compatibility check is done:
Module  bootable  Impact   Install-type  Reason
------  --------  -------  ------------  ------
1       yes       disruptive  reset      default upgrade is not hitless
27      yes       disruptive  none       default upgrade is not hitless

Read the Impact and Reason columns, not the percentage bars. disruptive with default upgrade is not hitless tells you the platform or the release pair cannot do a hitless upgrade — usually because the kernel version changes between releases. In that case plan a maintenance window, or upgrade in two hops through an intermediate release whose kernel matches.

Step 2: check feature incompatibilities

switch# show incompatibility system bootflash:nxos64.10.2.1.169.F.bin
show install all failure-reason
show install all status

This lists features that must be disabled before the upgrade (classic example: vPC or distributed CoPP on older trains). If the output names a feature, disable it, complete the upgrade, then re-enable it — do not hope the installer ignores it.

Step 3: check the control-plane impact on spanning tree / vPC

switch# show spanning-tree issu-impact
switch# show install all impact nxos bootflash:nxos64.10.2.1.169.F.bin

For a non-disruptive ISSU in an STP topology, verify there is no active topology change in any instance and that bridge assurance is not enabled on ports other than MCT/vPC peer-links. On platform switches that run STP, the switch should be the root for the VLANs it serves; show spanning-tree issu-impact answers that directly.

Step 4: choose the right upgrade mechanism

• Standard ISSU (redundant supervisor / dual-supervisor chassis): images are loaded on the standby, a switchover happens, then each module is upgraded — no data-plane downtime, minimal control-plane downtime.
• Enhanced ISSU (container based) on TOR switches: the new NX-OS version is booted in a second container and the active container is destroyed after the switchover. Control-plane traffic is impacted for roughly three seconds. If a kernel uprev is detected, the installer automatically falls back to a normal (non-enhanced) ISSU.
• SMUs for targeted bug fixes: hot-restart SMUs change no data plane; some SMUs trigger a cold box reload, which the README for each SMU states explicitly. Read it before you install.
• GIR / maintenance mode for disruptive events: system mode maintenance drains traffic from the device using routing metrics before you reload it, so the maintenance window costs you capacity, not reachability. show system mode confirms the state.

Step 5: execute and verify

switch# copy running-config startup-config
switch# install all nxos bootflash:nxos64.10.2.1.169.F.bin
switch# show version
switch# show install all status
switch# show module
switch# show vpc brief
switch# show interfaces status err-disabled

After the upgrade: confirm the running image, confirm all modules are ok, confirm vPC peer-link and vPC roles are restored, and check for EPLD upgrades that may still be pending (they sometimes need a second install all run and a copy run start beforehand). Cisco also recommends a Nexus Health and Configuration Check before the window — it catches field notices and missing recommended configuration while you can still reschedule.

What Actually Makes an Upgrade Hitless

Hitless ISSU depends on the platform having two control planes that can run different images — a redundant supervisor, or the container-based enhanced ISSU on TOR switches — and on the release pair sharing a compatible kernel and hardware-programming model. When the kernel version changes between the current and target NX-OS releases, the installer cannot keep forwarding while it swaps the control plane, and it reports a disruptive upgrade. That is a property of the release pair, not of your configuration, so the only remedies are to accept a maintenance window or to upgrade in two hops through an intermediate release whose kernel matches both ends.

The Pre-Upgrade Checklist

Run every check while the switch is healthy, days before the window, so a failure becomes a schedule change instead of an outage:

switch# show version
switch# show install all impact nxos bootflash:nxos64.10.2.1.169.F.bin
switch# show incompatibility system bootflash:nxos64.10.2.1.169.F.bin
switch# show boot
switch# show module
switch# show environment
switch# show system redundancy status
switch# show vpc brief
switch# show run | inc feature
switch# copy running-config startup-config

Record the outputs for comparison after the upgrade, and confirm the target image on bootflash has been verified (show boot shows the current boot variable). If the switch is part of a vPC domain, upgrade the peer first or follow your vendor's order so the domain never has both members down at once.

EPLD and BIOS: The Second Upgrade Pass

An NX-OS image upgrade does not always upgrade the microcode underneath it. EPLDs (Electronically Programmable Logic Devices) and the BIOS on the supervisor or line cards can lag the running image, and the installer tells you when they do — usually at the end, with a message that EPLDs are pending. Those updates need a second pass, often followed by a reload, and each one is documented in its own README. Budget time for a second install all run and a reload after the software upgrade, because skipping it leaves the platform in a mixed state that later bugs get blamed for.

vPC and STP Specifics During ISSU

In a vPC domain the ISSU changes how the control plane restarts on each peer. Verify before the window that the peer-link is healthy, that no orphan ports are flapping, and that the switch is not the STP root in a way that would trigger a topology change while a peer restarts. show spanning-tree issu-impact and show vpc brief answer these directly; if a peer-link or orphan port is degraded, fix it first, because ISSU will not repair a broken vPC. The orphan-port and peer-gateway checks are covered in depth in our Nexus vPC peer-gateway, orphan ports and health checks guide.

Rollback and Recovery When It Goes Wrong

There is no single "undo" for an ISSU, so recovery is designed in two layers. Before the upgrade, take a configuration checkpoint so you can roll the config back to a known state. If the new image fails to boot or a module does not come up, the standard remedy is to boot the previous image from bootflash and restore the checkpoint. This is why keeping the old image on bootflash — not deleting it to save space — is not optional on a production switch.

switch# checkpoint before-upgrade
switch# show boot
switch# boot nxos bootflash:nxos64.10.1.155.F.bin
switch# rollback running-config checkpoint before-upgrade

Post-Upgrade Verification in Depth

Once the switch is back, verification is more than show version. Confirm the running image and the boot variable match, all modules are ok and none is in a power-up or failed state, and the control and data planes are healthy. On a dual-supervisor chassis, re-check the redundancy status and confirm the standby is in sync:

switch# show version | inc image
switch# show module
switch# show system redundancy status
switch# show vpc brief
switch# show spanning-tree summary
switch# show interfaces status err-disabled
switch# show feature | inc enabled

Compare interfaces, routes and vPC roles against the pre-upgrade recording; anything that was up before and is down now is a finding, not noise.

Change Window and Scheduling

Even a hitless upgrade deserves a window, because the plan changes the moment the installer reports a disruptive result. Schedule ISSU during a low-traffic period, upgrade the peer switch in a vPC or vPC/MLAG pair in a separate window, and keep an out-of-band path (console) available in case the switch returns without management reachability. If the impact check says disruptive, either accept a full maintenance window or schedule the two-hop path — do not start a potentially disruptive upgrade next to a change freeze.

The Most Common Failure Reasons

• Kernel or hardware uprev between releases: the most common reason an upgrade is disruptive rather than hitless.
• Unsupported upgrade path: jumping several releases without installing an intermediate image.
• Feature incompatibility left unresolved: a listed feature (a vPC or CoPP feature, for example) was never disabled before install.
• Pending EPLD or BIOS update ignored: the software upgraded, the microcode did not.
• Not enough bootflash space: the new image cannot be staged because the old one was never cleared — check with dir bootflash: before you start.

Fabric Upgrade Order: One Switch at a Time

A fabric of leaf and spine switches is upgraded one member at a time, never all at once. The usual sequence is spines first, draining and reloading each before moving to the next, then leaves; in a vPC pair the two peers are upgraded in separate windows so the pair is never both rebooting. Verify after each switch that its routing adjacencies came back and its vPC roles are correct before you touch the next device, and keep traffic flowing on the rest of the fabric. This order keeps capacity available throughout and turns a fleet-wide upgrade into a series of small, reversible steps.

Before You Start: Space, Images and Licensing

switch# dir bootflash:
switch# show file bootflash:nxos64.10.2.1.169.F.bin md5sum
switch# show license usage
switch# show boot

Confirm there is room for the new image alongside the current one, verify the file's checksum against the published value so a corrupt copy does not fail mid-window, and check that any licensed features are unaffected by the target release. These three checks cost a few minutes and prevent the most preventable maintenance-window surprises.

Pick a Target Release You Can Support

The safest upgrade is often not the newest image available. NX-OS releases are organized into trains, and which train you land on decides how long the code stays supported and how often you have to do this again. Innovation and Fast trains introduce features quickly; Extended and Long-Lived trains hold a stable feature set for years and receive fixes without new functionality. For a production fabric, pick a train your team can stay on, cross-check the release against the Nexus hardware you actually run, and prefer a release the vendor marks as recommended for that platform. A switch running an image with an open field notice is a problem you inherit at the next audit — it is not a problem the upgrade solves.

Two compatibility axes are easy to forget. First, the management stack around the switch: if you use a dashboard, a fabric controller or a telemetry collector, confirm the target release is supported by that version before the switch moves. Second, the BIOS and EPLD matrix in the release notes. Some releases require a minimum microcode level, and the installer’s end-of-run message about pending EPLDs is the only warning you get.

Clock, Logging and Monitoring Across the Reload

A reload that takes fifteen minutes should not take your monitoring with it. Before the window, confirm the clock is synchronized — if the switch is not an NTP client, the post-upgrade logs carry a timestamp that makes correlating them with anything else impossible:

switch# show ntp status
switch# show ntp peers
switch# show logging server
switch# show snmp host
switch# show feature | inc enabled

Then decide what the reload does to your visibility. Syslog destinations, trap hosts and telemetry exporters live in the configuration and return with it, but a reload clears counters, resets interface error statistics and discards the syslog buffer. Capture show logging logfile to a file first, and afterwards confirm the paths are actually working rather than assuming configuration equals function: show logging server lists the configured targets, while a test message proves the path.

Watching Progress and Finding the Logs

An install all run can take twenty minutes on a chassis with several line cards, and the console scrolls faster than anyone reads. Status can be polled independently, which is what you want on a slow serial link:

switch# show install all status
switch# show install all failure-reason
switch# show logging logfile | inc -i "install\|EPLD\|reboot"
switch# dir bootflash: | inc nxos64

The installer writes to the switch’s own log file, so a session that drops mid-upgrade does not lose the evidence: reconnect and run show logging logfile to reconstruct what happened, including which module finished and which one stopped. If the install halts partway, show install all failure-reason is the first command to run, not the last — it names the module and the reason the installer gave up, which is usually enough to choose between retrying and rolling back.

Related reading: NX-OS checkpoint and rollback, vPC failover troubleshooting and IOS-XE install mode upgrades.

原文链接：https://www.cisco.com/c/en/us/td/docs/switches/datacenter/nexus9000/sw/93x/upgrade/guide/b-cisco-nexus-9000-nx-os-software-upgrade-downgrade-guide-93x/b-cisco-nexus-9000-nx-os-software-upgrade-downgrade-guide-93x_chapter_0111.html