Jump to content

DGARM Nagios Core - Storage Onboarding Summary

From TetraWiki
Revision as of 09:04, 26 July 2026 by Maintenance script (talk | contribs) (Embed source videos)
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)

Source videos:

  • DGRAM-Infosys-Nagios-Core-Deployment-.storage-1.mp4 (35 min)
  • DGRAM-Infosys-Nagios-Core-Deployment-storage-2.mp4 (75 min)
Note on this summary: audio across both recordings is sparse/degraded (mostly
silent screen-work, with Part 2 particularly affected by background noise the
transcription model couldn't handle). This summary is built primarily from clear
on-screen terminal/browser evidence.

Summary[edit]

Two-part, same-day session onboarding NetApp storage monitoring, working from the existing Nagios XI server (dcprodnagios01) to carry storage plugins and configs across to the new Core server (dcprdlmonitor02 / 10.150.24.167) — the "copy templates from XI" plan described back in the kick-off meeting, now executed for the storage category. Also continues network switch onboarding using the automation from the Network Switch session.

Part 1 — plugin and config transfer[edit]

  • Plugin identified: check-netapp-ng.pl (Perl), at /usr/local/nagios/libexec/check-netapp-ng.pl on the XI server. Handles multiple check types via SNMP against NetApp filers, including:
    • SHELFINFO — disk shelf status/count
    • FSSTATUS — overall filesystem/volume status, read via SNMP OID snmpfsOverallStatus (and its _text companion), mapped to OK/WARNING/CRITICAL
  • Existing NetApp config files copied across via scp from XI (/usr/local/nagios/etc/services/) to the Core host's /opt staging area: Netapp_cpu.cfg, Netapp_disksummary.cfg, Netapp_fan.cfg, Netapp_shelf.cfg, NetApp_top_volumes.cfg, Netapp_uptime.cfg — six standard NetApp health checks reused as-is rather than rebuilt.
  • Scale context: the XI server's services directory showed dozens of per-device config files for switches, routers, and NetApp filers (e.g. KOLKATA-AIU-SWITCH.cfg, MIAL-AGGREGATOR-SWITCH.cfg, NDC_Netapp_*.cfg, NDCPRDLSWT02.cfg) — this is a large, long-running XI estate being incrementally extended into Core, not a greenfield setup.
  • XI already has notification glue scripts: xi_service_event_handler / xi_service_notification_handler commands call /usr/local/nagiosxi/scripts/handle_nagioscore_event.php and handle_nagioscore_notification.php — worth evaluating reuse of these for Core rather than writing new notification handling.
  • Brief mentions (not demonstrated in detail): Grafana as a possible alternative/supplement to PNP4Nagios for storage graphing, and confirmation that PowerShell-based checks are still in active use for some Windows-side checks.

Part 2 — going live[edit]

  • NetApp storage host added: NDC_NetApp_Storage (10.50.18.100), confirmed live in Thruk with service imp_NDC_NetApp_Storage - NetApp CPU (the imp_ prefix matches the same NagiosQL import workflow used for switches).
  • Same transient "pending" pattern seen elsewhere: newly imported host showed PENDING (obj has not been checked yet) immediately after import — clears after the first active check cycle, not a fault.
  • More switch interfaces onboarded on a second switch (dcprdlswt02), following the same LAG (link aggregation) naming convention from the Network Switch session: dcprdlswt02_lag7_a01_sr04_status, ..._lag8_a02_sr04_bandwidth, ..._lag9_a01_sr05_status, etc. — confirms the discovery-and-import automation is now the standard process, not a one-off.
  • Only one storage device onboarded so far, with the intent that the client's team repeats the process for the rest independently — same pacing pattern as other device categories. Session closed out for the day at the end.

Additions to the runbook[edit]

Area Note
Storage checks Reuse check-netapp-ng.pl + the six standard Netapp_*.cfg service definitions from XI rather than rebuilding NetApp monitoring from scratch
Notifications Check whether XI's handle_nagioscore_event.php / handle_nagioscore_notification.php scripts are meant to be reused for Core's notification handling before writing new ones
Graphing Grafana was mentioned as a possible alternative/addition to PNP4Nagios for storage metrics — needs a decision, not yet implemented
Storage host onboarding Same NagiosQL import pipeline as network devices: generate/prepare host + service config, copy into the import directory, import, write config, wait for pending → active
"Pending" status after import Expected transient state, not a bug — clears after the first scheduled active check completes
LAG interfaces The switch discovery script's naming convention extends to link-aggregation (LAG) interfaces, not just physical ports — confirmed on a second switch (dcprdlswt02)

Source recordings[edit]

Part 1[edit]

Part 2[edit]