From gNMI to Kafka: A Streaming Telemetry Pipeline

Polling every device every five minutes gives you a picture of the network that is always a few minutes out of date and costs CPU on devices that would rather be forwarding. Streaming telemetry inverts the model: the device pushes its own counters on a subscription, the collector forwards them to a message bus, and consumers subscribe to whatever they need. The architecture is well understood but the details — subscription types, collector-side filtering, and what happens when a device reboots — decide whether the pipeline is useful during an incident or just noisy. This article builds a gNMI-to-Kafka pipeline and covers those details.

Dial-in versus dial-out

Dial-in is usually the faster way to start because nothing changes on the switches: gNMI does not require prior sensor path configuration on the target, merely that gRPC/gNMI is enabled. Dial-out becomes attractive at scale or when the device must initiate for security reasons.

Collecting with gNMIc

# one-off subscribe to interface counters every 10 seconds
gnmic -a 192.0.2.10:57400 -u admin -p secret --insecure \
  subscribe \
  --path '/interfaces/interface[name=Ethernet1]/state/counters' \
  --sample-interval 10s

# store the subscription in a file for reuse
cat > subs.yaml <<'EOF'
subscriptions:
  ifcounters:
    paths:
      - /interfaces/interface[name=*]/state/counters
    mode: sample
    sample-interval: 10s
  bgp-neighbors:
    paths:
      - /network-instances/network-instance[name=default]/protocols/bgp/neighbors
    mode: on-change
EOF

gnmic --config subs.yaml subscribe --name ifcounters

Subscription modes matter for volume. Sample sends the value at a fixed interval; on-change sends only when the value changes, which is ideal for state that rarely moves but matters greatly when it does (BGP session state, interface oper status). A pipeline that samples everything every second produces far more data than most teams can store usefully, and the cost is paid twice — on the device and in the storage layer.

Fan-out to Kafka

# gnmic.yaml -- multiple outputs from one collector
username: admin
password: secret
insecure: true
encoding: json_ietf

targets:
  192.0.2.10:57400:
    subscriptions:
      - ifcounters

subscriptions:
  ifcounters:
    paths:
      - /interfaces/interface[name=*]/state/counters
    mode: sample
    sample-interval: 10s

outputs:
  kafka-out:
    type: kafka
    address: kafka-1:9092,kafka-2:9092,kafka-3:9092
    topic: network-telemetry
    format: json
    num-workers: 4

Kafka decouples collection from consumption: one producer, many consumers — a time-series database, a stream processor for anomaly detection, and an alerting path that only cares about a handful of paths. Because Kafka retains the topic, a consumer that restarts can replay rather than lose the window. Partition by device or by site so that consumers can scale horizontally without reordering data within a device's stream.

Filter, don't dump

A collector that forwards every leaf of every interface wastes bandwidth and forces every consumer to filter. Do the filtering once, at the collector or in the stream processor: select the counters you actually alert on, drop the administrative noise, and enrich with metadata (site, role, device name) that consumers would otherwise have to look up. The pipeline discipline is the same as in OpenTelemetry Collector pipelines: collect once, filter early, enrich centrally, deliver everywhere.

Failure modes worth designing for

• Device reboot: subscriptions must be re-established automatically. Verify that the collector reconnects and that no gap in data goes unnoticed by adding a "device stopped reporting" alert.
• Collector restart: with dial-in, a collector outage means data is simply absent for that period. Run collectors in pairs and distribute targets, as gNMIc supports clustering for exactly this reason.
• Broker loss: buffer at the collector where the output supports it, and monitor producer errors rather than assuming Kafka absorbed everything.
• Path churn: YANG paths differ between vendors and releases. Test each path against a real device from every platform and record the supported versions next to the subscription definition.

Streaming telemetry complements rather than replaces classic polling. SNMP still covers the metrics no YANG path exposes, and platforms that support neither still need to be monitored — the exporter model in Prometheus SNMP exporter monitoring handles that gap. And on IOS-XE specifically, the subscription options and their configuration are covered in gNMI streaming telemetry on IOS-XE. Start with a handful of high-value paths, prove the pipeline end to end, and then expand the subscription list — migrating everything at once produces a data lake nobody queries.

原文链接：https://gnmic.openconfig.net/