DGARM Nagios Core - Network Switch Onboarding Summary

Source video: DGRAM-Infosys-Nagios-Core-Deployment-Network-switch.mp4 (3h 14min)

Summary

Continues directly from the Deployment Process session. Two parts:

Part 1 — Windows host cleanup + onboarding (~0–40 min)

• Found a misconfiguration: host ELKMAS01 (a Linux server) had been accidentally placed into the windows-servers host group in NagiosQL. Caught and corrected — worth a sanity sweep of host-group membership across the environment, since this kind of mistake silently skews which template/checks apply.
• Onboarded a Windows host the same clone-and-edit way established in the Deployment Process video: duplicate an existing working Windows host, change IP/name, save, "write all config files," wait ~5 min for status to move off pending.

Part 2 — Automated network device (switch/router) onboarding (~40 min–end)

This is the important part: manually onboarding network devices one interface at a time doesn't scale (one switch alone had 92 interfaces), so the team built and used a custom automation script to bulk-generate Nagios config from live SNMP data.

Plugin used: check_nwc_health (the well-known Consol Labs network-component-health plugin), confirmed directly from a generated config file: <syntaxhighlight lang="text"> define service {

   host_name         DCPRDLSWT01
   service_description   1/1/1 TO A01-SR01 Status
   check_command      check_nwc_health_ifstatus!1/1/1
   ...

} define service {

   host_name         DCPRDLSWT01
   service_description   1/1/1 TO A01-SR01 Bandwidth
   check_command      check_nwc_health_ifusage!1/1/1!80!90
   ...

} </syntaxhighlight> i.e. two services per interface — link status and bandwidth-usage (with 80%/90% warning/critical thresholds) — auto-named per port/description.

Workflow demonstrated:

1. Prerequisite: confirm SNMP is reachable and working on the target device first — this environment uses SNMPv3 (not v2c), and the SNMP username/password/auth credentials must match between the device and the discovery script — mismatches here just silently fail ("credential does not match", "unknown user"). Sanity-check with a manual snmpwalk-style command before trusting the automation.
2. Discovery/generation script (custom-built, described as Python + shell): lives under /opt on dcprdlmonitor02 (referred to as "OPD" in speech — almost certainly this /opt script directory). Takes a target IP/device list as input (switches.txt-style file), runs SNMP walk against it, and generates two files per run under /opt/generated/:

• • a host config file (one host entry)
  • a services config file (one status + one bandwidth service per discovered interface, named after each interface's description) — confirmed output header: # Created by: Nagios QL Version 3.5.0 / # Nagios 4.x config file / # --- DO NOT EDIT THIS FILE BY HAND ---
  • Applies to any device with ports/bandwidth — switch, router, or firewall, not switches specifically.

1. Import into Nagios: copy (cp, not just create) the generated host + services files into /usr/local/nagios/etc/objects/import/ (must empty that import directory first — leftover files from a previous device caused confusion/wrong imports), then in NagiosQL: Tools → Data import, select both files, import. Confirmed via NagiosQL's Tools → Check configuration files afterward — one real run reported 12 hosts, 349 services total after imports (79 + 92 + others), validating nothing broke.
2. Write config + restart: Tools → "write all config files," then allow a few minutes for Nagios to pick up and move new services off "pending."
3. Router/firewall caveat hit live: attempted to onboard a router the same way and hit "router has no SNMP" / SNMP not reachable — a reminder that this whole pipeline depends entirely on SNMP being enabled and reachable on the target device first; if it isn't, that's a networking-team blocker, not a Nagios config problem.
4. Session ends with the plan to let the client's team (Chala/Narendra) continue onboarding the remaining routers/switches/Windows hosts independently the next day, using this same script + import workflow, with the vendor lead available only if they hit something new.

Additions to the runbook's fix/issue log

Symptom	Root cause	Fix
Host shows correct checks but wrong template behavior (e.g. missing a check that should apply)	Host was placed in the wrong host group (e.g. a Linux box under windows-servers)	Audit host-group membership in NagiosQL; move host to the correct group
SNMP discovery script fails with "credential does not match" / "unknown user" against a switch	SNMPv3 username/auth/priv values in the script don't match what's configured on the device	Re-verify SNMPv3 credentials on both sides before assuming the script is broken
NagiosQL import brings in wrong/duplicate services after using the discovery script repeatedly	Leftover files from a previous device still sitting in /usr/local/nagios/etc/objects/import/	Clear the import directory before copying in the newly generated host/services files each time
Router won't onboard via the switch automation script at all	SNMP not enabled/reachable on that router — not a Nagios-side issue	Escalate to network team to enable/open SNMP before retrying onboarding
Switch port bandwidth graphing incomplete (ties back to the Deployment Process video's MRTG gap)	check_nwc_health_ifusage reports usage numerically, but MRTG-based historical graphing per-port needs its own log file set up separately	Treat MRTG log setup as a separate prerequisite step per device, not assumed to come free with check_nwc_health

Source recording