Admin: Auto-created from uploaded PDF text extraction

2026-07-26T16:27:10Z

Auto-created from uploaded PDF text extraction

New page

''Auto-generated from the uploaded PDF [[:File:Observibility_Cluster_Setup.pdf|Observibility_Cluster_Setup.pdf]]. This is an extracted-text rendering for searchability; see the original PDF for exact formatting, diagrams, tables, and images.''

<pre>
<nowiki>
Infosys - DGARM

Observability Cluster Setup Document

Last Updated:- 5th Feb, 2026

By

Prashant Kashyap

Document Control

Document No. Doc_1

Document Version 1

Prepared By Prashant Kashyap

Release Date 5th Feb 2026

Table of Contents

Infosys - DGARM ......................................................................................................................... 1

Document Control .......................................................................................................................2

Installation of Elasticsearch ....................................................................................................... 4

Kibana Installation ....................................................................................................................12

APM Installation on ELK setup .................................................................................................. 15

Fleet Installation in Elastic Cluster ........................................................................................... 18

Installation of Elasticsearch

1. Enable subscription-manager register
subscription-manager register --username actual_user --password
actual_pass

 Note :- After installation please subscription-manager unregister
Command :- subscription-manager unregister

2 . Install java on node (server), check the java version and Installation it
yum install java*
java -version

3. make Elasticsearch as user
useradd elasticsearch
groupadd elasticsearch

4. create data directory as per elasticsearch node functionality
mkdir /data // for normal node master and data
mkdir /ml-data // for ml node

5. create ELASTIC Directory for installation work
mkdir ELASTIC
cd ELASTIC

# Download Elasticsearch rpm package
wget
https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-
9.1.5-x86_64.rpm

# Install Elasticsearch
sudo rpm -ivh elasticsearch-9.1.5-x86_64.rpm

6. take backup of original elasticsearch.yml file before make any changes
cp /etc/elasticsearch/elasticsearch.yml
/etc/elasticsearch/elasticsearch_yml.backup

7. Change ownership of elk directories and owner
chown -R root:elasticsearch /etc/elasticsearch
chown -R root:elasticsearch /data
chown -R root:elasticsearch /var/log/elasticsearch

8. Put necessary node certificates in elasticsearch certs folder

cd /etc/elasticsearch/certs

Note :- .cnf, .dgarm.gov.in.crt, fullchain.crt and key will different as per node

9. Enter host entry in hosts file
Note : host entry wil be different for DR and Prod as per cluster design or setup

nano /etc/hosts
All ELK Cluster node entry

10. go to elasticserch folder and make changes in yml file
nano elasticsearch.yml

11. start 9200 and 9300 port
systemctl start firewalld
firewall-cmd --permanent --add-port=9200/tcp
firewall-cmd --permanent --add-port=9300/tcp
systemctl reload firewalld

12. start the elasticsearch service

sudo systemctl daemon-reload
sudo systemctl enable elasticsearch
sudo systemctl start elasticsearch
sudo systemctl restart elasticsearch
sudo systemctl status elasticsearch
sudo systemctl stop elasticsearch

13. Commands for check the error of Elasticsearch
sudo journalctl -u elasticsearch.service
sudo journalctl -u elasticsearch.service -f

14. Elasticsearch Instance
ps aux | grep elasticsearch
sudo netstat -plnt | grep 9200

15. command to change password for Elasticsearch user i custom form
/usr/share/bin/elasticsearch-reset-password -u elastic -i // set
custom password

Note:- The same steps are followed for all other Elasticsearch nodes.

Overview for Elasticsearch nodes:- 3 master nodes, 2 hot data nodes, 4
warm data nodes, 2 ml nodes.

Logstash Installation
1. Enable subscription-manager register
subscription-manager register --username ABC --password
abc@123
Note :- After installation please subscription-manager unregister
Command :- subscription-manager unregister

2. Install java on node (server), check the java version and Installation it
yum install java*
java -version

3. make logstash as user and group
useradd logstash
Groupadd logstash

4. create logstash Directory for installation work
mkdir /logstash
cd /logstash/
wget https://artifacts.elastic.co/downloads/logstash/logstash-
9.1.5-x86_64.rpm
rpm -ivh logstash-9.1.5-x86_64.rpm

5. make certs folder and copy all important certificate
/etc/logstash
mkdir certs

cd /etc/logstash/certs/

6. changer owner and directory permission

chown -R logstash:logstash /etc/logstash /usr/share/logstash
/var/log/logstash /var/lib/logstash

7. Go to config directory and create conf file as per requirement

/etc/logstash/conf.d
nano output.conf

8 . Start the port

firewall-cmd --permanent --add-port=5044/tcp
systemctl reload firewalld
sudo firewall-cmd --list-ports

9.Restart logstash service and check the status

sudo systemctl daemon-reload
sudo systemctl enable logstash

sudo systemctl start logstash
sudo systemctl restart logstash
sudo systemctl status logstash

Note :- Please put Certifcate and other details as per host and server.
Same process will be followed for the other logstash node also.

Kibana Installation

1. Enable subscription-manager register
subscription-manager register --username ABC --password abc@123

Note :- After installation please subscription-manager
unregister
Command :- subscription-manager unregister

2. Install java on node (server), check the java version and Install it
yum install java*
java -version

3. Make Kibana as user and group
useradd elasticsearch
groupadd elasticsearch

4. Create Kibana Directory for installation work
mkdir /kibana
cd /kibana/
wget https://artifacts.elastic.co/downloads/kibana/kibana-9.1.5-
x86_64.rpm
yum install net-tools
rpm -ivh kibana-9.1.5-x86_64.rpm

5. put all certificates in certs directory

cd /etc/kibana/certs/

6. make changes in kibana.yml file

Note : As per node, cluster and server host name will be different

7. open port and reload
firewall-cmd --permanent --add-port=5601/tcp
systemctl reload firewalld
netstat -an | grep 5601

8. Give permission to kibana respect folders
chown -R kibana:kibana /etc/kibana/certs
chown -R kibana:kibana /opt/kibana
chown -R kibana:kibana /var/log/kibana

9. restart kibana service
sudo systemctl daemon-reload
sudo systemctl enable kibana
sudo systemctl start kibana
sudo systemctl restart kibana
sudo systemctl status kibana

10. Command to reset kibana_system user password from elasticsearch master
node.
Cd /usr/share/elasticsearch
Sudo ./bin/elasticsearch-reset-password -u kibana_system

11. Now go to search engine and access kiban dashboard or url

https://kibanalocalhost:5601

Enter elastic and its password
Username :- elastic
Password :- custom or auto generated.

Note: The same process will be followed for the other Kibana node.

APM Installation on ELK setup
1. Enable subscription-manager register
subscription-manager register --username actual_user --password
actual_pass
Note :- After installation please subscription-manager unregister
Command :- subscription-manager unregister

2. Install java on node (server), check the java version and Installation it
yum install java*
java -version

3. make apm-server as user
useradd apm-server
groupadd apm-server

4. Create APM directory and install apm-server
mkdir /apm
cd /apm/
wget https://artifacts.elastic.co/downloads/apm-server/apm-server-9.1.5-
x86_64.rpm
rpm -ivh apm-server-9.1.5-x86_64.rpm

5. Take backup of yml file and make changes
cd /etc/apm-server/
cp apm-server.yml apm-server_yml.backup
nano apm-server.yml

6. Put certificate on certs location under apm directory
cd /etc/apm-server/certs/

7. Set permission for apm folders
chown -R apm-server:apm-server /etc/apm-server/
chown -R apm-server:apm-server /var/log/apm-server/

8. Make changes in apm conf yml file
cd /etc/apm-server/

nano apm-server.yml

Save yml file

9. Open port 8200 for apm
sudo firewall-cmd --zone=public --add-port=8200/tcp –permanent
sudo firewall-cmd –reload
sudo firewall-cmd --list-ports
firewall-cmd --list-all

Note:- The same process will be followed for other apm node.

10. Put ca certificate
cp ca.crt /etc/pki/ca-trust/source/anchors/
Note :- copy ca certificate from elastic master location and copy it on apm server given location

11. Start apm service and check the status
systemctl daemon-reload
systemctl status apm-server.service
systemctl enable apm-server.service --now
journalctl -xe -f -u apm-server.service --no-pager
journalctl -xe -f apm-server.service --no-pager
journalctl -xe -f -u apm-server.service --no-pager

Fleet Installation in Elastic Cluster
1. Go to Fleet Server and Enter All ELK Host entry in host file
nano /etc/hosts

2. Copy ca.crt certificate in certificate directory

Cd /etc/pki/ca-trust/source/anchors/
Copy certificate here and run update-ca-trust command.

3. Make fleet directory and go inside fleet and install elastic-agent rpm here

3. Go to Kibana and go to fleet section via 3 line in left side and add fleet.

5.select or click to add fleet server and under advanced section give Policy name as per our
requirements can give custom any name

NOTE :- Follow the step which shows after creating policy to install fleet. To install fleet we need to install
elastic-agent on server it automatically installs fleet on that particular server.

1. Select rpm or tar option for Installation of elastic-agent on our server and copy that
installation command and run in our /fleet directory of server

2. After installing elastic-agent bring or copy all respect certificate on fleet server under
elastic-agent directory

cd /etc/elastic-agent
mkdir certs
cd /etc/elastic-agent/certs
Give proper permission to certificate and certs directory to.

3. Open port 8220 for elastic-agent and fleet

4. Run installation or enroll command which is given in kibana UI for elastic-agent

5. Check status on both UI and CLI. It should be successfully enroll elastic-agent

Note :- same steps we need to follow for another 2nd node of fleet instead of default fleet we need to give
respect fleet url.

Add new fleet server host and Rest of the steps remaining same.

Note:- The same process will be followed for the other fleet node also.
</nowiki>
</pre>

Observibility Cluster Setup - Revision history

Admin: Auto-created from uploaded PDF text extraction