Jump to content
Main menu
Main menu
move to sidebar
hide
Navigation
Main page
Recent changes
Random page
Help about MediaWiki
TetraWiki
Search
Search
Appearance
Create account
Log in
Personal tools
Create account
Log in
Pages for logged out editors
learn more
Contributions
Talk
Editing
DGARM - Engagement Report - 20260508v1
Page
Discussion
English
Read
Edit
View history
Tools
Tools
move to sidebar
hide
Actions
Read
Edit
View history
General
What links here
Related changes
Special pages
Page information
Appearance
move to sidebar
hide
Warning:
You are not logged in. Your IP address will be publicly visible if you make any edits. If you
log in
or
create an account
, your edits will be attributed to your username, along with other benefits.
Anti-spam check. Do
not
fill this in!
''Auto-generated from the uploaded PDF [[:File:DGARM_-_Engagement_Report_-_20260508v1.pdf|DGARM_-_Engagement_Report_-_20260508v1.pdf]]. This is an extracted-text rendering for searchability; see the original PDF for exact formatting, diagrams, tables, and images.'' <pre> <nowiki> 2 | elastic.co | Β© 2026 Elasticsearch B.V. All Rights Reserved. Table of Contents Table of Contentsβ 3 Document Propertiesβ 4 Version Historyβ 4 Referencesβ 4 Executive Summaryβ 5 Overviewβ 5 Participantsβ 5 Business Valueβ 6 Outcomesβ 6 Next Stepsβ 7 Cluster Reviewβ 8 Cluster Architecture & Topologyβ 8 Elasticsearch Configuration Files and Bootstrap Checksβ 9 Version & Licenceβ 11 Kibana Reviewβ 11 Findingsβ 11 Recommendationsβ 12 Security and Access Control Reviewβ 12 Findingsβ 12 Recommendationsβ 13 Snapshot, Backup, and Disaster Recovery Reviewβ 13 Current Stateβ 13 Snapshot Lifecycle Management (SLM)β 13 Findingsβ 14 Recommendationsβ 14 Data Lifecycle, Storage, and Retention Reviewβ 15 Findingsβ 15 Recommendationsβ 16 Monitoring and Alerting Reviewβ 16 Current Stateβ 16 Findingsβ 16 Recommendationsβ 17 Data Source Inventory & Integration Statusβ 18 Out-of-Scope Dashboard Assistanceβ 24 Action Plan - Next Stepsβ 28 3 | elastic.co | Β© 2026 Elasticsearch B.V. All Rights Reserved. Document Properties Version History Version Date Author Description 1.0 2026 May 8th Salm Alaeddine Initial Release References Ref. No Document Name Location Description DGARM Design Document - 1 Email Design Document 20250829 v1.2 4 | elastic.co | Β© 2026 Elasticsearch B.V. All Rights Reserved. Executive Summary Overview Elastic conducted an advisory review of DGARMβs Elastic environment to assess the current cluster deployment, validate the implementation against the available design documentation, and identify any gaps, risks, or improvement opportunities. The review focused on the deployed Elasticsearch and Kibana clusters, including cluster configuration, monitoring, alerting, snapshot and restore readiness, security controls, data lifecycle management, and operational alignment with Elastic best practices. The engagement was structured around the following areas: ββ Current state discovery. ββ Cluster configuration review against the documented design. ββ High-level health and operational review. ββ Alignment to Elastic best practices. ββ Monitoring, alerting, backup, and disaster recovery readiness. ββ Identification of risks, findings, recommendations, and next steps. Elastic guidance referenced during the review includes production readiness, dedicated monitoring, index lifecycle management, shard sizing, snapshot and restore, and dedicated master node design. Elastic recommends using ILM to automate rollover and retention, using snapshots as the supported backup method, and using a separate monitoring cluster for production monitoring where applicable. Participants Org Name Role/Position mail Biswajit Banerjee Project Manager biswajit@tetrain.com Mukul Mahajan mukul@tetrain.com Muralidharan Tetra / Lead Architect muralidharan_S12@infosys.com Subramanian Infosys Bhasha Sisodia Operations Manager bhasha.sisodia@infosys.com Karan Neelkanth Platform Lead karan.neelkanth@infosys.com 5 | elastic.co | Β© 2026 Elasticsearch B.V. All Rights Reserved. Divya Dhyani divya_dhyani@infosys.com Bimlesh Kumar bimlesh.kumar@infosys.com Juhu Sajwar juhu.sajwar@infosys.com Prashant Kashyap prashant.kashyap@tetrain.com Manish Singh Manish@tetrain.com Salim Alaeddeine Consulting Architect salim.alaeddine@elastic.co Elastic Sri Suba Selvachamy Delivery Manager sri.selvachamy@elastic.co Business Value The cluster review provides DGARM with a clear understanding of the current Elastic platform state and highlights the actions required to improve platform stability, resilience, governance, and operational maturity. The key business value includes: ββ Improved confidence in Elastic platform health and recoverability. ββ Early identification of risks that may impact service availability. ββ Better alignment between the deployed environment and the approved design. ββ Improved operational visibility through monitoring, alerting, and audit logging. ββ Reduced risk of data loss through validated snapshot and restore processes. ββ Clearer prioritisation of remediation and optimisation activities. Outcomes The engagement produced the following key outcomes: ββ Reviewed the current Elastic deployment at a high level. ββ Compared the current environment against the available design documentation. ββ Identified critical areas requiring immediate attention, including: ββ Production license status. ββ Snapshot repository functionality. ββ Backup and disaster recovery readiness. ββ Monitoring architecture. 6 | elastic.co | Β© 2026 Elasticsearch B.V. All Rights Reserved. ββ Alert notification actions. ββ Reviewed the current monitoring and alerting setup. ββ Identified that Stack Monitoring alert issues related to API keys were resolved after updating the API key. ββ Reviewed the current audit logging approach. ββ Identified that Elasticsearch audit logs are currently consumed by Imperva but not ingested into Elastic. ββ Captured recommended actions to improve alignment with Elastic best practices. Next Steps The recommended next steps are: ββ Apply the correct production license for the Search Cluster before 8 May 2026. ββ Review and remediate the snapshot repository issue. ββ Validate Snapshot Lifecycle Management policies and snapshot success history. ββ Perform a controlled restore test to confirm recoverability. ββ Move monitoring to a dedicated monitoring cluster where aligned with the design and governance requirements. ββ Configure alert actions for critical Stack Monitoring alerts. ββ Enable logs collection for Elasticsearch, Kibana, and relevant ingestion components. ββ Review audit logging requirements and consider enabling Kibana audit logging. ββ Back up non-Elasticsearch configuration artefacts. 7 | elastic.co | Β© 2026 Elasticsearch B.V. All Rights Reserved. Cluster Review Cluster Architecture & Topology The following clusters were reviewed or identified as part of the engagement. At the time of the review, no dedicated monitoring cluster was deployed. Search Cluster Node Role Hostname IP CPU RAM (GB) Disk (GB) Master + Data dcplelkschmdh01 10.150.25.70 16 64 1024 Node Master + Data dcplelkschmdh02 10.150.25.71 16 64 1024 Node Master + Data dcplelkschmdh03 10.150.25.72 16 64 1024 Node Machine dcplelkschml01 10.150.25.73 8 64 200 Learning Node Kibana Node dcplelkschkib01 10.150.25.74 8 16 200 Kibana Node dcplelkschkib02 10.150.25.75 8 16 200 Logstash Node dcplelkschlgst01 10.150.25.76 8 16 300 Logstash Node dcplelkschlgst02 10.150.25.77 8 16 300 Observability Cluster Node Role Hostname IP CPU RAM (GB) Disk (GB) Master Node dcplelkobsmas01 10.150.25.78 8 16 200 Master Node dcplelkobsmas02 10.150.25.79 8 16 200 Master Node dcplelkobsmas03 10.150.25.80 8 16 200 Hot Node dcplelkobshtdt01 10.150.25.81 16 64 1024 Hot Node dcplelkobshtdt02 10.150.25.82 16 64 1024 Warm Node dcplelkobswmdt01 10.150.25.83 16 64 3072 Warm Node dcplelkobswmdt02 10.150.25.84 16 64 3072 Warm Node dcplelkobswmdt03 10.150.25.85 16 64 3072 8 | elastic.co | Β© 2026 Elasticsearch B.V. All Rights Reserved. Warm Node dcplelkobswmdt04 10.150.25.86 16 64 3072 Machine dcplelkobsml01 10.150.25.87 16 32 200 Learning Node Machine dcplelkobsml02 10.150.25.88 16 32 200 Learning Node Kibana Node dcplelkobskib01 10.150.25.89 8 16 200 Kibana Node dcplelkobskib02 10.150.25.90 8 16 200 Fleet Server dcplelkobsflt01 10.150.25.91 8 16 200 Node Fleet Server dcplelkobsflt02 10.150.25.92 8 16 200 Node APM Node dcplelkobsapm01 10.150.25.93 8 16 200 APM Node dcplelkobsapm02 10.150.25.94 8 16 200 Logstash Node dcplelkobslgst01 10.150.25.95 8 16 300 Logstash Node dcplelkobslgst02 10.150.25.96 8 16 300 Elasticsearch Configuration Files and Bootstrap Checks As part of the cluster review, the Elasticsearch configuration files were reviewed against Elastic production guidance and bootstrap requirements. The review identified several configuration items that should be cleaned up or hardened to improve operational clarity, production stability, and resilience. Elasticsearch.yml Finding Recommendation Elastic Reference cluster.initial_master_nodes entry was Remove the entry from the Bootstrapping a found in the Elasticsearch configuration file to avoid cluster configuration file. This setting is only confusion during future required during the initial cluster maintenance, restarts, or bootstrap and should not remain in the node replacement activities. configuration after the cluster has formed. 9 | elastic.co | Β© 2026 Elasticsearch B.V. All Rights Reserved. bootstrap.memory_lock: true is β’ Disable swap at the Disable swapping / currently commented out. This means operating system level, set Bootstrap checks Elasticsearch memory may be vm.swappiness = 1 where swapped to disk, which can cause applicable, OR long garbage collection pauses and β’Enable node instability. bootstrap.memory_lock: true, and validate that memory locking is active. No shard allocation awareness Configure shard allocation Shard allocation configuration was found. awareness based on the awareness node.attr.rack, node.attr.zone, or physical topology, such as similar awareness attributes are not rack, zone, or data centre, to configured. help distribute primary and replica shards across failure domains. Kibana.yml Finding Recommendation Elastic reference Plaintext password in kibana.yml - Remove from file. Add to Secure Settings elasticsearch.password: "" Kibana keystore: bin/kibana-keystore add elasticsearch.password All three encryption keys exposed in Move all three keys to Security plaintext - xpack.*.encryptionKey Kibana keystore. Configuration Audit logging not enabled - Enable Audit Logging xpack.security.audit.enabled not xpack.security.audit.enable configured d: true Elastic Documentation reference: ββ Elastic Production Guidance β Running Elastic Stack in production requires planning for resilience, performance, and scalability. ββ Elasticsearch Important Settings β Production configuration items include cluster name, node name, network host, discovery, heap, logging, and backups. ββ Elasticsearch Bootstrap Checks β In production mode, failed bootstrap checks prevent Elasticsearch from starting. 10 | elastic.co | Β© 2026 Elasticsearch B.V. All Rights Reserved. Version & Licence The Elasticsearch clusters are currently deployed on version 9.3.1. The Observability cluster has the correct production licence applied, and no immediate action is required. The Search cluster is currently running on a trial licence, which is due to expire on 8 May 2026. The trial licence should be replaced with the appropriate licence before the expiry date to avoid any loss of licensed features or potential service impact. The severity of this issue has been communicated to the relevant team, and it should be addressed promptly to avoid loss of licensed features or potential service impact. The DR cluster currently has a full production licence applied. This should be reviewed, as the DR cluster was scoped to remain under the appropriate non-production licence unless it is actively being used during a disaster recovery scenario. During normal operations, the production licence should remain assigned to the production cluster. In the event of a DR activation, the production licence can be moved to the DR cluster and then reverted back once the production environment has been restored. Kibana Review The Kibana review covered: ββ Kibana availability and sizing. ββ Spaces and saved object organisation. ββ User access and role-based access control. ββ External authentication integration. ββ Role mappings. ββ Alerting and connector configuration. ββ Kibana audit logging. Findings Finding Severity Observation Kibana audit logging is Medium Kibana audit logging is not currently enabled. not enabled 11 | elastic.co | Β© 2026 Elasticsearch B.V. All Rights Reserved. Alert actions are not High Critical monitoring alerts may not notify operational configured teams. Role mappings require Medium Access model should be reviewed against the validation documented security design. Recommendations ββ Validate spaces and saved object organisation. ββ Review RBAC design and role mappings. ββ Enable Kibana audit logging if required by governance or compliance requirements. ββ Configure alert actions for critical operational alerts. Security and Access Control Review The security review covered: ββ Authentication model. ββ External authentication or SSO integration. ββ Role-based access control. ββ Role mappings. ββ TLS and certificate configuration. ββ Audit logging. Findings Finding Severity Observation Elasticsearch audit logs are not Medium Audit logs are consumed by Imperva but not ingested into Elastic available in Elastic for investigation or correlation. Kibana audit logging is not Medium Kibana user activity visibility is limited. enabled RBAC requires validation Medium Roles and mappings should be reviewed against the security design. 12 | elastic.co | Β© 2026 Elasticsearch B.V. All Rights Reserved. Certificate lifecycle Medium TLS and certificate management should be management requires reviewed. validation Recommendations ββ Review security design against the live implementation. ββ Validate SSO or external authentication if available. ββ Review role mappings against least privilege principles. ββ Enable Kibana audit logging where required. ββ Confirm whether Elasticsearch audit logs should be ingested into Elastic. ββ Validate TLS configuration and certificate expiry management. Snapshot, Backup, and Disaster Recovery Review Current State Snapshot configuration is not confirmed as functional. The snapshot repository appears to have issues that require further investigation. Backup and disaster recovery readiness therefore remain a critical risk area. Elastic snapshots are the supported way to back up Elasticsearch data. Elastic explicitly states that filesystem-level copies of node data directories are not a supported backup and restore method. Snapshot Lifecycle Management (SLM) Cluster snapshots can be managed using Snapshot Lifecycle Management (SLM). Note this is different from the searchable snapshots which are managed as part of ILM actions. When configuring long term snapshot retention, it is important to consider the frequency of the snapshots as well as the retention of a snapshot. Each snapshot has some overhead on the master nodes, so having an appropriate snapshot lifecycle management strategy is important for cluster stability. 13 | elastic.co | Β© 2026 Elasticsearch B.V. All Rights Reserved. The recommended strategy is to apply multiple SLM policies with varying intervals and retention to provide flexibility in both snapshot granularity as well as retention length. The following three policies could be used in conjunction: Snapshot Policy Frequency Retention Approximate Snapshot Count Short-term snapshots Every 15 minutes 24 hours 96 Daily snapshots Daily 1 month 31 Monthly snapshots Monthly 1 year 12 Combining the above three policies will ensure that snapshots are taken every 15 minutes, minimising the potential data loss and/or reingestion required if recent data needs to be restored. In addition, it reduces the overall number of snapshots required by taking snapshots at longer intervals. This process is described in detail in the following documentation. Findings Finding Severity Observation Snapshot repository issue Critical Repository health requires investigation. Snapshot success history not Critical Snapshot reliability is not yet validated. confirmed Restore test not confirmed Critical Recoverability is not proven. Non-Elasticsearch artefacts require Medium Elastic snapshots do not cover all deployment separate backup and configuration files. Recommendations ββ Review and fix the snapshot repository issue. ββ Validate Snapshot Lifecycle Management policies. ββ Confirm snapshot success and failure history. ββ Perform a controlled restore test. ββ Monitor snapshot failures and configure alert actions. 14 | elastic.co | Β© 2026 Elasticsearch B.V. All Rights Reserved. ββ Back up non-Elasticsearch configuration artefacts separately. Configuration artefacts to back up outside Elasticsearch snapshots include: ββ elasticsearch.yml ββ kibana.yml ββ Logstash pipeline files. ββ Operating system configuration. ββ Certificates and private keys. ββ Other deployment artefacts. Data Lifecycle, Storage, and Retention Review The review covered: ββ ILM policy design. ββ Rollover conditions. ββ Retention periods. ββ Data tier allocation. ββ Shard sizing and shard count. Elastic ILM is designed to automate the lifecycle of time-based indices, including rollover, retention, and deletion, helping optimise performance, reliability, and storage cost. Elastic also provides shard sizing guidance to avoid excessive shard and index counts, including guidance around the number of indices per GB of heap on master nodes. Therefore enforcing a strict shard strategy is required. The following guidelines should be followed for shard sizing: 1.β Primary Shards should be between 10GB and 50GB in size. 2.β In cases where logsdb data stream type is used, 10-30GB is an ideal size. 3.β Shards should contain no more than 200 million documents. 4.β No more than 3000 indices per 1 GB of master heap. Findings Finding Severity Observation 15 | elastic.co | Β© 2026 Elasticsearch B.V. All Rights Reserved. ILM alignment requires High ILM policies need to be compared with business validation retention requirements. Shard sizing requires High Shard count and shard sizing should be reviewed as per review design Storage tier alignment Medium Data placement should align with data value, search requires review frequency, and retention requirements. Recommendations ββ Review all ILM policies against business retention requirements. ββ Confirm rollover conditions are appropriate for data volume. ββ Validate delete phases to avoid uncontrolled data growth. ββ Review data tiers and ensure data is stored on the correct tier. Monitoring and Alerting Review Current State The current monitoring state appears to be self-monitoring for each cluster, with metrics collection only. Logs are not fully collected into Elastic for monitoring and troubleshooting. Stack Monitoring alerting had an API key related issue, which was resolved by updating the API key. However, alert actions are not currently configured. Elastic recommends using a separate monitoring cluster for production environments. This helps ensure monitoring data remains available if the production cluster is unavailable, and it also prevents monitoring activity from impacting production performance. Findings Finding Severity Observation Self-monitoring is High Monitoring data may be unavailable if the currently used production cluster is impacted. 16 | elastic.co | Β© 2026 Elasticsearch B.V. All Rights Reserved. Logs are not fully High Troubleshooting visibility is limited. collected Alert actions are not High Critical alerts may not notify administrators. configured API key issue was Closed / Updating the API key fixed the Stack Monitoring resolved Informational alert issue. Recommendations ββ Move monitoring to a dedicated monitoring cluster in line with the design documentation. ββ If a dedicated monitoring cluster is not feasible, assess AutoOps suitability based on governance and compliance requirements. ββ Collect both metrics and logs for Elasticsearch, Kibana, Logstash, Elastic Agent, and Beats where applicable. ββ Configure notification actions for critical alerts. ββ Recommended alert notifications actions could include: ββ Email. ββ Ticketing platform. ββ Microsoft Teams. ββ Incident management tooling. Recommended alert coverage: ββ Cluster health. ββ Node availability. ββ Disk usage and storage thresholds. ββ JVM memory pressure. ββ CPU utilisation. ββ Shard allocation issues. ββ Snapshot failures. ββ Ingestion failures. ββ Kibana Task Manager issues. 17 | elastic.co | Β© 2026 Elasticsearch B.V. All Rights Reserved. Elastic Stack Monitoring includes preconfigured alerting rules based on Elastic recommendations, which can be tailored to the environment. Data Source Inventory & Integration Status The ingestion review covered the current and planned data sources across servers, databases, platforms, applications, storage, network, and security devices. Elastic out-of-the-box integrations are being used where available, including System, PostgreSQL, IBM MQ, GitHub, Tenable, Palo Alto, Fortinet, Juniper SRX, Aruba CX, F5 BIG-IP, Forcepoint, and Kubernetes/OpenShift where applicable. Elastic integrations provide pre-packaged assets to collect, store, and visualise data from supported sources. For data sources where no suitable out-of-the-box integration is available, custom ingestion is required using supported ingestion patterns such as syslog, Logstash, API polling, HTTP Endpoint,, or custom Elastic Agent integrations. A quick overview on the data onboarding progress: Data Source Type OOTB Current Status / Notes Integration RHEL Servers Server System Logs & Onboarded using Elastic System integration Metrics for logs and metrics. ~210 host onboarded at this stage Physical Server System Logs & Servers Metrics Kubernetes / Platform Kubernetes / Onboarded, but some metrics are not being OpenShift OpenShift collected due to OpenShift SCC/context integration permissions. Requires follow-up. eMudhra Application N/A eMudhra Host is being monitored at this SecurePass / stage. Application logs are not onboarded iDAM yet. PostgreSQL Database PostgreSQL Onboarded using OOTB integration. IBM MQ Application IBM MQ OOTB integration used. Running through some challenges, support case follow up Neo4J Application N/A Not yet onboarded 18 | elastic.co | Β© 2026 Elasticsearch B.V. All Rights Reserved. NGINX + API Application N/A Onboarded. Need to verify sub_status Gateway parsing/visibility. JBoss Web Server / N/A Not yet onboarded Server Application GitHub Application GitHub Not yet onboarded ManageEngine Application N/A Not yet onboarded ITSM ManageEngine Application N/A Not yet onboarded Patch Tenable Application Tenable Not yet onboarded Imperva Application Imperva Not yet onboarded / Security Cisco SAN Storage Cisco Nexus, Logs : Cisco Nexus integration deployed. Switch / subject to Might need to validate if devices are Nexus Storage validation OS or IOS and use the appropriate integration (Cisco IOS, Cisco Nexus) Metrics: Can use SNMP polling t o get devices health, and metrics. NetApp Storage N/A SNMP polling is currently used to collect Storage device metrics. This may need to be fine-tuned to ensure all required metrics are covered. Alternative options include using NetApp REST APIs for metrics collection or reviewing NetApp Harvest as a potential metrics collection approach. Veritas Backup Backup N/A Not yet onboarded Palo Alto Security Palo Alto Onboarded Logs. Metrics are not yet Firewall β Next-Gen configured. Internal Firewall Fortinet Security Fortinet Logs are working. Metrics are not yet Firewall β FortiGate configured. External Firewall Logs Fortinet Security Fortinet Not yet onboarded Sandbox FortiGate Firewall Logs, subject to validation Juniper Router Network Juniper SRX OOTB integration is deployed. Review the incoming log patterns to confirm alignment with the default integration parsing 19 | elastic.co | Β© 2026 Elasticsearch B.V. All Rights Reserved. patterns. If the logs do not match the default patterns, create a custom pattern and add it to the ingest pipeline. Aruba Network HPE Aruba CX Metrics onboarded using SNMP poling Switches WAF Security F5 BIG-IP Not yet onboarded DDoS Security F5 BIG-IP Not yet onboarded Proxy Security Forcepoint Web Not working. Requires follow-up. Security DLP Security Forcepoint Web Not working. Requires follow-up. Security, subject to validation F5 Load Security / F5 BIG-IP Data is not coming in. Current issue Balancer Network appears to be that data is being sent as syslog rather than to the expected HTTP endpoint. Based on the current setup, Elastic is receiving syslog/log data from the devices through the available integrations, but metrics collection is not yet enabled or available for some device types through the same integrations. This is common, as many network, security, and storage integrations are primarily log-focused, while infrastructure and performance metrics may require additional collection methods. Elastic can provide recommendations around possible ingestion patterns, Elastic-supported integrations, data modelling, dashboarding, alerting, and best-practice alignment. However, the implementation, support, and maintenance of any custom integrations, vendor-specific scripts, third-party tools, exporters, SNMP configurations, API collectors, or non-Elastic-supported components would remain the responsibility of the client or the relevant third-party vendor. At a high level, the recommended approach would be a hybrid model: ββ Continue using Elastic integrations and syslog for logs and security events. ββ Use SNMP polling for standard device health and infrastructure metrics where suitable. ββ Use vendor APIs where richer platform-specific metrics are required. ββ Use Logstash or Elastic Agent/custom integrations where transformation, enrichment, or custom collection is required. 20 | elastic.co | Β© 2026 Elasticsearch B.V. All Rights Reserved. For the listed devices, the possible direction would be: Storage Devices For Cisco SAN switches/storage, SNMP polling may be considered for visibility into interface status, throughput, errors, discards, power supply, fan status, temperature, and general hardware health. For NetApp Storage, metrics can be collected using SNMP polling, NetApp REST APIs, or by exploring NetApp Harvest. SNMP polling may be used for device-level metrics, but it may need to be fine-tuned to ensure all required metrics are covered. NetApp REST API-based collection may provide richer visibility into areas such as volume capacity, aggregate usage, IOPS, latency, controller health, disk/shelf health, and replication status. For Veritas Backup, API-based collection or scheduled export through Logstash may be considered to monitor backup job status, failed jobs, success rate, backup duration, storage pool usage, and SLA compliance, depending on the available Veritas interfaces. Security Devices For Palo Alto Firewalls, the available Elastic Palo Alto integration capabilities should be reviewed to confirm whether metrics collection can be enabled for the PAN-OS environment. Elastic provides a Palo Alto Networks metrics integration that periodically fetches metrics from Palo Alto Networks firewalls and management systems. For Fortinet Firewalls, SNMP polling may typically be used for metrics such as CPU, memory, interface status, bandwidth, session count, HA status, and hardware health, while syslog continues to provide security and traffic events. For Sandbox, WAF, DDoS, Proxy, DLP, and F5 Load Balancer platforms, the preferred collection method depends on the vendor, model, supported interfaces, and required metrics. In many cases, SNMP can provide baseline infrastructure health, while vendor APIs may provide richer operational metrics such as policy events, blocked requests, queue depth, backend health, pool member status, throughput, latency, and service availability. Network Devices For Aruba Switches and Juniper Routers, SNMP polling is commonly used for metrics collection. This may cover device availability, CPU, memory, interface utilisation, interface errors/discards, port 21 | elastic.co | Β© 2026 Elasticsearch B.V. All Rights Reserved. status, temperature, fan/power status, and routing or protocol health where exposed by the device MIBs. The initial baseline metrics to consider may include: ββ Device availability ββ CPU utilisation ββ Memory utilisation ββ Interface status and utilisation ββ Interface errors and discards ββ Hardware health ββ Storage capacity ββ Backup success/failure status ββ HA/failover status ββ Last successful metric collection timestamp From an Elastic perspective, the metrics should ideally be normalised into dedicated metrics data streams such as network, security, storage, and backup metrics. For Logstash-based onboarding, it is recommended to use Elastic data streams rather than standalone indices where possible. Data streams are better suited for time-series data such as logs, metrics, and observability events. They also integrate well with index lifecycle management, rollover, and long-term retention strategies. Recommended approach: ββ Use data streams for metrics and logs. ββ Apply consistent dataset and namespace naming. ββ Align the data stream naming convention with the design documentation. ββ Apply ILM policies based on data type, value, and retention requirements. Example pattern: metrics-network.snmp-prod metrics-storage.netapp-prod logs-network.device-prod 22 | elastic.co | Β© 2026 Elasticsearch B.V. All Rights Reserved. For SNMP and custom metrics, it is recommended to map important fields to Elastic Common Schema (ECS) where practical. For example: Current / Recommended ECS-Aligned Field Notes Custom Field tag.device host.hostname or host.name Use keyword mapping for filtering and grouping. Device IP field host.ip Use IP mapping where possible. Device type host.type or custom device.type Helps classify routers, switches, storage, etc. Interface observer.ingress.interface.name, Use consistent naming for name observer.egress.interface.name, or network interface custom interface field dashboards. Interface event.status or custom mapped keyword Enables consistent filtering status field and visualisation. SNMP source data_stream.dataset Helps organise and route data. The exact mapping should be reviewed based on the source data structure and the dashboards required. To support dashboard filtering and aggregations, key fields should be mapped correctly. Recommended examples: ββ Hostnames and device names: keyword ββ IP addresses: ip ββ Interface names: keyword ββ Status fields: keyword ββ CPU, memory, disk, and bandwidth metrics: numeric fields, for example long, double. ββ Timestamps: date 23 | elastic.co | Β© 2026 Elasticsearch B.V. All Rights Reserved. For reference, the following documentation pages may be useful: ββ Elastic Agent and Fleet overview:β https://www.elastic.co/docs/reference/fleet ββ Manage Elastic Agent integrations:β https://www.elastic.co/docs/reference/fleet/manage-integrations ββ Elastic integrations catalogue:β https://www.elastic.co/integrations ββ Logstash SNMP integration plugin, including SNMP polling and SNMP trap support:β https://www.elastic.co/docs/reference/logstash/plugins/plugins-integrations-snmp ββ Logstash SNMP input plugin:β https://www.elastic.co/docs/reference/logstash/plugins/plugins-inputs-snmp ββ Palo Alto Networks logs integration for Elastic:β https://www.elastic.co/docs/reference/integrations/panw ββ Palo Alto Networks metrics integration for Elastic:β https://www.elastic.co/docs/reference/integrations/panw_metrics Out-of-Scope Dashboard Assistance Although dashboard creation was outside the original review and validation scope, the team raised an urgent request for assistance with creating an overall monitoring dashboard. In response, we supported the team remotely via screen sharing and helped structure the dashboard across server, network, and storage monitoring sections. The server monitoring section was completed where data is available, including CPU, memory, disk utilisation, and top utilisation views. The network monitoring section was structured, but current data availability is limited to Aruba device interface status. Network CPU and memory metrics, other network device types, and additional models are not yet onboarded into Elastic. The storage section was also structured and populated using the available NetApp SNMP data, currently limited to device status and global health. The overall monitoring dashboard structure was created to cover the main requested areas: ββ Server monitoring. ββ Network monitoring. 24 | elastic.co | Β© 2026 Elasticsearch B.V. All Rights Reserved. ββ Storage monitoring. The dashboard structure now provides a foundation for the required monitoring use cases, with visualisations populated where the relevant data is currently available. Where data is not available, the required sections were structured so that they can be completed once the missing data sources and metrics are onboarded into Elastic. Screenshots of the created dashboard: 25 | elastic.co | Β© 2026 Elasticsearch B.V. All Rights Reserved. 26 | elastic.co | Β© 2026 Elasticsearch B.V. All Rights Reserved. 27 | elastic.co | Β© 2026 Elasticsearch B.V. All Rights Reserved. Action Plan - Next Steps Priority Action Owner Target Critical Apply correct production license before 8 May DGARM Immediate 2026. Critical Review and fix snapshot repository issue. DGARM Immediate Critical Validate snapshot policies and snapshot success DGARM Immediate history. Critical Perform controlled restore test. DGARM Immediate High Review monitoring architecture against design. DGARM Short term High Configure alert notification actions. DGARM Short term High Enable log collection for monitoring. DGARM Short term High Review ILM, shard sizing, and retention. DGARM Short term Medium Review and enable Kibana audit logging if required. DGARM Medium term Medium Confirm whether Elasticsearch audit logs should be DGARM Medium ingested into Elastic. term Medium Back up configuration artefacts outside DGARM Medium Elasticsearch snapshots. term 28 | elastic.co | Β© 2026 Elasticsearch B.V. All Rights Reserved. </nowiki> </pre>
Summary:
Please note that all contributions to TetraWiki may be edited, altered, or removed by other contributors. If you do not want your writing to be edited mercilessly, then do not submit it here.
You are also promising us that you wrote this yourself, or copied it from a public domain or similar free resource (see
TetraWiki:Copyrights
for details).
Do not submit copyrighted work without permission!
Cancel
Editing help
(opens in new window)
