Structure of ELK

Auto-generated from the uploaded PDF Structure_of_ELK.pdf. This is an extracted-text rendering for searchability; see the original PDF for exact formatting, diagrams, tables, and images.

Structure of ELK

1. Elasticsearch Hierarchy – Simple Structure
   Cluster
   ├── Node A (Master)
   ├── Node B (Master)
   ├── Node C (Master)
   ├── Node D (Data)
   ├── Node E (Data)
   └── Node F (Data)


2. Index – Like a Database Table
   Index: logs-2025-06
    ├── Shard 0 (Primary) → Stored on Node D
    │ └── Documents: log1, log2, log3...
    ├── Shard 0 (Replica) → Stored on Node E
    ├── Shard 1 (Primary) → Stored on Node E
    │ └── Documents: log4, log5, log6...
    ├── Shard 1 (Replica) → Stored on Node F

       Index = A collection of documents (like a table in SQL)
       Shards = Pieces of the index for horizontal scaling
       Primary shard = Main copy
       Replica shard = Backup copy for redundancy


3. Document – Like a Row in SQL
   Each document is a JSON object:
   {
     "timestamp": "2025-06-13T12:34:56",
     "level": "ERROR",
     "message": "Failed to connect to DB"
   }
   Documents are stored inside shards, which are stored inside data nodes.


                                         Elastic design in DGARM
                                        Cluster name:- elastic-nonprod


Master Node :-          10.150.20.180              10.150.20.181         10.150.20.182

Data Node :-
                        10.150.20.183              10.150.20.184         10.150.20.185


                                        Data Node of Cluster
  10.150.20.183                              10.150.20.184                    10.150.20.185
                                              Index and Shards
  Index and Shard                            Index: Flight, Crew
                                             Primary shard 1,2                   Node + Shards
  Index: Passenger’s,
                                             flight,
  Primary Shards                             Primary Shards 1,2
  1,2, for Passenger                         crew                               Replica shard of
                                                                                  Flight Index.
  Replica Shards of
  Sh                                         Replica shard of
  crew
                                             Passenger


  Master Node :- 10.150.20.180, 10.150.20.181, 10.150.20.182
  Managing cluster state
  Creating or deleting indices
  Tracking which nodes are part of the cluster
  Allocating shards to nodes
  Monitoring node health and initiating recovery if needed


      1. Node 10.150.20.180 (Active Master)
       One of the nodes (likely A) is elected as the active master.
       It handles all cluster coordination tasks.
       The election is based on a quorum (majority) of master-eligible nodes.

 2. Node 10.150.20.181 and 10.150.20.182 (Standby Masters)
  These nodes are master-eligible but not currently active.
  They monitor the active master and are ready to take over if Node A fails.
  This ensures high availability and fault tolerance.