Cisco EEM Applets: Automate with event syslog Triggers - 夜莺博客

Cisco EEM Applets: Automate with event syslog Triggers

Cisco Embedded Event Manager (EEM) lets the router react to events by itself — no external monitoring server required. An EEM applet pairs an event detector (a syslog pattern, an SNMP OID crossing a threshold, a timer, an interface state) with a list of actions (run CLI commands, write syslog messages, send mail). It is the built-in automation layer for the kind of "watch for X, then do Y" operations that otherwise keep engineers on call. This article shows the applet structure and the most useful production examples.

Applet Anatomy

event manager applet NAME
 event <detector>
 action <sequence> <action type> <parameters>

Actions run in sequence-number order; numbering by decimals (1.0, 1.5, 2.0) lets you insert steps later. Applets support environment variables, and $_cli_result holds the output of the last CLI action so it can be embedded in a message.

Example 1: Auto-Recovery on Interface Shutdown

When an interface is shut down (or flaps), this applet re-enables it and emails the current users:

event manager applet interface_Shutdown
 event syslog pattern "Interface FastEthernet1/0, changed state to administratively down"
 action 1.0 cli command "enable"
 action 1.5 cli command "configure terminal"
 action 2.0 cli command "interface fa1/0"
 action 2.5 cli command "no shutdown"
 action 3.0 cli command "end"
 action 3.5 cli command "who"
 action 4.0 mail server "192.168.1.1" to "netops@example.com" from "eem@example.com" subject "ISP1 interface recovered" body "Users: $_cli_result"

Use event syslog pattern with a regex for finer matching, e.g. event syslog pattern ".*UPDOWN.*Ethernet1/0.*" occurs 4 to fire only after the link has bounced four times.

Example 2: Capture Data When CPU Spikes

An SNMP-polled applet that snapshots diagnostics to flash when CPU exceeds 60 percent:

event manager applet HIGH-CPU
 event snmp oid 1.3.6.1.4.1.9.9.109.1.1.1.1.3 get-type next entry-op gt entry-val 60 poll-interval 5
 action 0.1 cli command "enable"
 action 0.3 cli command "term length 0"
 action 1.1 cli command "show process cpu sorted | append flash:EEM_CPU"
 action 1.4 cli command "show ip traffic | append flash:EEM_TRAFFIC"
 action 1.6 cli command "show logging | append flash:EEM_LOGS"
 action 4.2 syslog msg "High CPU captured to flash:"

The same pattern — event triggers, CLI snapshots appended to flash — turns a router into a self-documenting device for post-mortems.

Example 3: Load-Based ACL toggling

EEM can also poll interface load and apply or remove an access-group when rxload crosses a threshold, giving lightweight congestion control on a WAN edge without QoS redesign.

Verification and Debugging

show event manager policy registered
show event manager history events
debug event manager action cli

show event manager policy registered lists every applet with its event type and maxrun; debug event manager action cli shows each CLI action as it executes, which is the fastest way to see why an applet fails. Remember the default applet run time is 20 seconds (maxrun extends it), and cli actions in an applet always need an enable step first. EEM complements SNMP-based platforms like Zabbix network monitoring for detection, and pairs with the troubleshooting workflow in Catalyst unexpected reload troubleshooting.

Event Detectors Beyond Syslog

Syslog pattern matching is the workhorse detector, but EEM ships several others and choosing the right one is the difference between a targeted applet and one that fires on noise:

  • event syslog pattern - a regex (or plain string) matched against log output. Add occurs N to require N matches, and period N to set the window they must occur within.
  • event snmp oid - polls a single OID on a timer and compares it against a threshold (entry-op, entry-val, poll-interval). Ideal for CPU, memory or interface counters.
  • event timer - fires on a watchdog interval or a cron-like schedule; the tool for scheduled housekeeping and self-check tasks.
  • event interface - reacts directly to an interface state change without depending on a matching log line.
  • event cli - fires when an operator types a specific command pattern, useful for log-and-warn guards around disruptive commands.
  • event rpc and event none - the latter registers an applet with no detector at all, which is how you build something you trigger manually.

A single applet may register more than one event block, so one policy can react to either a syslog line or an SNMP threshold breach without duplicating the actions.

Registration, maxrun and Built-In Variables

Every applet is a registered policy made of a name, one or more events and a numbered list of actions. Two mechanics matter before you write anything. maxrun sets how many seconds the applet may execute, defaulting to 20; an applet that greps a full BGP table or writes several show outputs to flash will need more. Actions run in the order of their sequence numbers, and decimal numbering (1.0, 1.5, 2.0) leaves room to insert a step later without renumbering everything.

EEM also injects a set of built-in variables you can embed in a message. $_cli_result holds the trimmed output of the previous CLI action, the event's syslog line is available for pattern-matched applets, and a set of $_event_* values describe the trigger. You can also define your own with a set action and reference it later in the same applet, which is how a long applet avoids repeating the same string five times.

Example 4: React When a BGP Neighbour Drops

BGP resets are the classic "the route vanished at 3 a.m." event. This applet fires when the adjacency to a specific neighbour goes down, captures routing state to flash immediately, and pages the team:

event manager applet BGP-PEER-DOWN
 event syslog pattern "%BGP-5-ADJCHANGE.*Down.*198.51.100.1" occurs 1 period 60
 action 0.5 cli command "enable"
 action 0.8 cli command "term len 0"
 action 1.0 cli command "show ip bgp summary | append flash:EEM_BGP"
 action 1.2 cli command "show ip bgp neighbors 198.51.100.1 | append flash:EEM_BGP"
 action 1.4 cli command "show ip route summary | append flash:EEM_BGP"
 action 2.0 syslog msg "BGP peer 198.51.100.1 down - state captured to flash:EEM_BGP"
 action 2.5 mail server "192.168.1.1" to "netops@example.com" from "eem@example.com" subject "BGP peer down" body "Peer 198.51.100.1 went down. Routing snapshot written to flash:EEM_BGP."

Because the applet writes the routing tables at the instant the log fires, the postmortem has evidence from the failure window rather than from ten minutes later - which is when the routes have already reconverged and the interesting state is gone.

Example 5: Scheduled Housekeeping with event timer

EEM is not only reactive. A watchdog timer runs recurring maintenance without needing a scheduler on an external host:

event manager applet NIGHTLY-SAVE
 event timer cron cron-entry "0 2 * * *"
 action 1.0 cli command "enable"
 action 1.5 cli command "write memory"
 action 2.0 syslog msg "Nightly running-config archived to startup-config"

event timer watchdog does the same for interval-based tasks, taking a simple time 3600 seconds argument instead of a cron entry. Both are a lightweight substitute for a cron job driving expect scripts, and they keep running whether or not the jump host is reachable - which is exactly the situation when a scheduled backup matters most.

Example 6: Guard Against Configuration Changes

A config-log event fires whenever the running configuration is modified, including by another engineer or by an automation tool. Use it to build an audit trail and to catch changes made outside change control:

event manager applet CONFIG-CHANGE-ALERT
 event syslog pattern "%SYS-5-CONFIG_I" occurs 1
 action 0.5 cli command "enable"
 action 1.0 cli command "show configuration | append flash:CONFIG_AUDIT"
 action 1.5 cli command "show users | append flash:CONFIG_AUDIT"
 action 2.0 syslog msg "Configuration change detected - see flash:CONFIG_AUDIT"
 action 2.5 mail server "192.168.1.1" to "change-audit@example.com" from "eem@example.com" subject "Config changed" body "A configuration change was detected. Users online: $_cli_result"

Pair this with a syslog collector so the alert survives a device reload - the Cisco IOS logging and syslog configuration guide covers levels, buffers and trap forwarding, and the rsyslog central log server guide covers the receiving end. A log aggregator such as Grafana Loki then makes the flash files and the syslog stream searchable together.

Sending Alerts: syslog, mail and SNMP Traps

Three action types carry an alert off the device, and they have different reliability characteristics:

  • action ... syslog msg "text" writes to the local log; a normal logging configuration forwards it onward. Cheap and reliable as long as the logging pipeline is, which is why it is the default choice.
  • action ... mail server X to Y from Z subject ... body ... sends SMTP directly from the router. Convenient, but the device needs a reachable relay and a resolvable DNS name, and there is no retry if delivery fails.
  • action ... snmp-trap strdata "text" raises a trap that the SNMP platform already monitors, which is the cleanest way to reuse existing alert routing and on-call rules rather than inventing a second notification path.

For anything that must be recorded for compliance, prefer the syslog action plus central collection and reserve mail for the small number of events genuinely worth waking someone for.

Controlling, Editing and Removing Applets

show event manager policy registered
show event manager policy registered summary
event manager applet CONFIG-CHANGE-ALERT
 action 1.0 cli command "enable"
no event manager applet CONFIG-CHANGE-ALERT

Editing an applet means re-entering the event manager applet block and adding or replacing actions by sequence number; removing the event line leaves the applet registered but inert, which is a useful way to park one without deleting it. no event manager applet NAME deletes it outright. The summary form is the one to run before a maintenance window, because it lists every policy and its event type in a scrollable form.

Testing Without Waiting for the Event

The biggest annoyance with EEM is that you normally have to cause the event to test the applet. Two tricks shorten the loop. Register the applet with event none and trigger it by hand:

event manager applet TEST-APPLET
 event none
 action 1.0 cli command "show version"
 action 2.0 syslog msg "manual test ran"

event manager run TEST-APPLET

Then watch execution live with debug event manager action cli and read the history with show event manager history events and show event manager history traps. Together these tell you whether the applet failed to fire at all (an event-detector problem) or fired and failed partway through (an action problem) - the split that every EEM debugging session starts from.

Best Practices and Cautions

  • Always start CLI actions with enable; applet CLI runs at privilege level 1 by default.
  • Set term len 0 before any show command whose output matters, so paging cannot truncate it.
  • Keep applets small and single-purpose. A ten-action applet that reconfigures the device is impossible to reason about at 3 a.m.
  • Bound the blast radius. An applet that issues no shutdown on every UPDOWN log will flap a genuinely failed link forever; use occurs N and a period so it acts on a pattern rather than a single blip.
  • Raise maxrun deliberately for applets that generate real output, and remember that a long-running applet blocks subsequent instances of itself.
  • Document each applet inline or in the device description field. The next engineer will not know why it exists, and deleting it will look harmless right up until the event it was guarding against occurs.

原文链接:https://community.cisco.com/t5/networking-knowledge-base/cisco-eem-basic-overview-and-sample-configurations/ta-p/3148479