TetraWiki - User contributions [en]

Cluster of energoindia drbd and heartbeat

2015-07-11T19:47:59Z

Mithilesh:

[[category:Tetra Clients]]
[[category:Energo Group]]

We had challenge to create cluster with drbd and heartbeat of qmail because alreay there was running server of qmail. We have decided to create cluster by following below steps:

STEP 1. create a server with same configuration as running server.( Os, harddisk, drbd size)

STEP 2. create DRBD and HEARTBEAT on this server.

STEP 3. Migrate the users, password, and groups from running server to new server.

STEP 4. make link of required directory and file of qmail, ldap, clamd, spamassassin, squid, fetchmail in /home partition and copied data from running server.

STEP 5. Live this new server as primary node of cluster.

STEP 6. Create DRBD and HEARTBEAT on old server as secondary node.

STEP 7. Make link of required directory and file as created in primary node.

STEP 8. Test the cluster.

== '''############## ON NODE1 configuration ##############''' ==

OS- Red Hat Enterprise Linux Server release 5.3 (Tikanga)

ARCH - 64 bit

'''Note :'''

1. Before start DRBD and Heartbeat implementation , make sure NTP synchronization between node1 and node2 , otherwise it will not work.

2. Make both node or machine keyless, so it will not promt password at the time of login from one node to other node.

[root@node1 ~]# cat /etc/hosts
# Do not remove the following line, or various programs
# that require network functionality will fail.
127.0.0.1 mail.energoindia.com mail localhost.localdomain localhost
#::1 localhost6.localdomain6 localhost6
192.168.10.149 node1.energoindia.com node1 node.energoindia.com mail
192.168.1.150 node.energoindia.com node mail.energoindia.com mail
10.0.0.1 node1.energoindia.com node1 node.energoindia.com
10.0.0.2 node2.energoindia.com node2 node.energoindia.com
192.168.10.155 node2.energoindia.com node2 node.energoindia.com mail

NOTE: The virtual ip is 192.168.1.150.

''':::: Installing DRBD ::::'''

[root@node1 ~]# yum install drbd83-utils kmod-drbd83

[root@node1 ~]# mv /etc/drbd.d/global_common.conf /etc/drbd.d/global_common.sample

[root@node1 ~]# vim /etc/drbd.d/global_common.conf
global { usage-count no; }
common {
syncer { rate 100M; }
}

Create a new resource file that I called “main”,

[root@node1 ~]# vim /etc/drbd.d/main.res
resource main {
protocol C;
startup { wfc-timeout 0; degr-wfc-timeout 120; }
disk { on-io-error detach; }
on node1.energoindia.com {
device /dev/drbd0;
disk /dev/VolGroup00/LogVol02;
meta-disk internal;
address 10.0.0.1:7788;
}
on node2.energoindia.com {
device /dev/drbd0;
disk /dev/mapper/VolGroup00-LogVol02;
meta-disk internal;
address 10.0.0.2:7788;
}
}

Now we have to create the metadata on node1:

[root@node1 ~]#dd if=/dev/zero of= /dev/VolGroup00/LogVol02 bs=1M count=50

[root@node1 ~]# drbdadm create-md main

and we can now start DRBD

[root@node1 ~]# service drbd start

Starting DRBD resources: [ d(main) s(main) n(main) ].

We will tell DRBD that server1 is the Primary server:

[root@node1 ~]# drbdsetup /dev/drbd0 primary –o

[root@node1 ~]# cat /proc/drbd
version: 8.3.16 (api:88/proto:86-97)
GIT-hash: a798fa7e274428a357657fb52f0ecf40192c1985 build by phil@Build64R5, 2013-09-27 15:48:18
0: cs:Connected ro:Primary/Secondary ds:UpToDate/unknown C r-----
ns:182767328 nr:116152 dw:182883616 dr:829873130 al:1019560 bm:344 lo:0 pe:0 ua:0 ap:0 ep:1 wo:f oos:0

[root@node1 ~]# mkfs.ext3 /dev/drbd0

Now DRBD is configured , now you ready to mount your /dev/drbd0 to /home

[root@node1 ~]# mount –t ext3 /dev/drbd0 /home

[root@node1 ~]# df -h
Filesystem Size Used Avail Use% Mounted on
/dev/mapper/VolGroup00-LogVol00 96G 41G 50G 46% /
/dev/sda1 935M 27M 860M 4% /boot
tmpfs 3.8G 0 3.8G 0% /dev/shm
/dev/drbd0 446G 346G 78G 82% /home

'''::::::::::::Installing HEARBEAT:::::::::::'''

[root@node1 ~]# yum install heartbeat

Create the following file on node1 servers:

[root@node1 ~]# vim /etc/ha.d/ha.cf
keepalive 2
deadtime 30
bcast eth0
node node1.energoindia.com node2.energoindia.com
logfile /var/log/ha-log
debugfile /var/log/ha-debug

The next step is to create the resource file for heartbeat on node1 servers:

[root@node1 ~]# vim /etc/ha.d/haresources
node1.energoindia.com IPaddr::192.168.10.150/24/eth0 IPaddr::192.168.20.150/24/eth0
IPaddr::192.168.1.150/22/eth0 IPaddr::192.168.30.150/24/eth0 drbddisk::main Filesystem::/dev/drbd0::
/home::ext3 ldap httpd spamassassin clamd authdaemond imap imaps squid qmailctl

The last thing is to create the authentication file on node1 servers:

[root@node1 ~]# vim /etc/ha.d/authkeys
auth 3
3 md5 asd1234f

This password file should only be readable by the root user:

[root@node1 ~]# chmod 600 /etc/ha.d/authkeys

start heartbeat on node 1:

[root@node1 ~]# service heartbeat start

[root@node1 ~]# ifconfig

eth0 Link encap:Ethernet HWaddr E4:1F:13:C0:E9:CC
inet addr:192.168.10.149 Bcast:192.168.10.255 Mask:255.255.255.0
inet6 addr: fe80::e61f:13ff:fec0:e9cc/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:171872956 errors:0 dropped:0 overruns:0 frame:0
TX packets:229211317 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:80916792381 (75.3 GiB) TX bytes:190588412808 (177.4 GiB)
Interrupt:169 Memory:92000000-92012100

eth0:0 Link encap:Ethernet HWaddr E4:1F:13:C0:E9:CC
inet addr:192.168.10.150 Bcast:192.168.10.255 Mask:255.255.255.0
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
Interrupt:169 Memory:92000000-92012100

eth0:1 Link encap:Ethernet HWaddr E4:1F:13:C0:E9:CC
inet addr:192.168.20.150 Bcast:192.168.20.255 Mask:255.255.255.0
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
Interrupt:169 Memory:92000000-92012100

eth0:2 Link encap:Ethernet HWaddr E4:1F:13:C0:E9:CC
inet addr:192.168.1.150 Bcast:192.168.3.255 Mask:255.255.252.0
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
Interrupt:169 Memory:92000000-92012100

eth0:3 Link encap:Ethernet HWaddr E4:1F:13:C0:E9:CC
inet addr:192.168.30.150 Bcast:192.168.30.255 Mask:255.255.255.0
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
Interrupt:169 Memory:92000000-92012100

eth1 Link encap:Ethernet HWaddr E4:1F:13:C0:E9:CD
inet addr:10.0.0.1 Bcast:10.255.255.255 Mask:255.0.0.0
inet6 addr: fe80::e61f:13ff:fec0:e9cd/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:32136747 errors:0 dropped:0 overruns:0 frame:0
TX packets:92544365 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:3368179779 (3.1 GiB) TX bytes:128171113009 (119.3 GiB)
Interrupt:138 Memory:94000000-94012100

lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:16436 Metric:1
RX packets:7571848 errors:0 dropped:0 overruns:0 frame:0
TX packets:7571848 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
X bytes:5164760682 (4.8 GiB) TX bytes:5164760682 (4.8 GiB)

''':MIGRATION OF QMAIL USERS, SYSTEM USERS, NORMAL USERS FROM A RUNNING SERVER TO NODE1:'''

Commands to type on old Linux system:

1. First create a tar ball of old uses (old Linux system). Create a directory:
# mkdir /root/move/

2. Setup UID filter limit:
# export UGIDLIMIT=500

3. Now copy /etc/passwd accounts to /root/move/passwd.mig using awk to filter out system account (i.e. only copy user accounts)
# awk -v LIMIT=$UGIDLIMIT -F: '($3>=LIMIT) && ($3!=65534)' /etc/passwd > /root/move/passwd.mig

4. Copy /etc/group file:
# awk -v LIMIT=$UGIDLIMIT -F: '($3>=LIMIT) && ($3!=65534)' /etc/group > /root/move/group.mig

5. Copy /etc/shadow file:
# awk -v LIMIT=$UGIDLIMIT -F: '($3>=LIMIT) && ($3!=65534) {print $1}' /etc/passwd | tee - |egrep -f - /etc/shadow > /root/move/shadow.mig

6. Copy /etc/gshadow (rarely used):
# cp /etc/gshadow /root/move/gshadow.mig

7. Use scp or usb pen or tape to copy /root/move to a new Linux system.
# scp -r /root/move/* user@new.linuxserver.com:/path/to/location

Commands to type on new Linux system

1. First, make a backup of current users and passwords:

[root@node1 ~]# mkdir /root/newsusers.bak

[root@node1 ~]# cp /etc/passwd /etc/shadow /etc/group /etc/gshadow /root/newsusers.bak

2. Now restore passwd and other files in /etc/

[root@node1 ~]# cd /path/to/location

[root@node1 ~]# cat passwd.mig >> /etc/passwd

[root@node1 ~]# cat group.mig >> /etc/group

[root@node1 ~]# cat shadow.mig >> /etc/shadow

[root@node1 ~]# /bin/cp gshadow.mig /etc/gshadow

Now reboot system; when the Linux comes back, your user accounts will work as they did before on old system:

[root@node1 ~]# reboot

''':COPY ALL FILES OF QMAIL, LDAP ,SPAMASSASSIN , CLAMAV FROM A SERVER TO NODE1 USING LINK:'''

create below directory, make link of this and copy the data from running server.

[root@node1 ~]# mkdir -p /home/command

[root@node1 ~]# ln -s /home/command /command ( copy data from running server )

[root@node1 ~]# mkdir -p /home/downloads

[root@node1 ~]# ln -s /home/downloads /downloads

[root@node1 ~]# mkdir -p /home/packages

[root@node1 ~]# ln -s /home/packages /packages

[root@node1 ~]# mkdir -p /home/service

[root@node1 ~]# ln -s /home/service /service

[root@node1 ~]# mkdir -p /home/usr/local

[root@node1 ~]# ln -s /home/usr/local /usr/local

[root@node1 ~]# mkdir -p /home/var/qmail

[root@node1 ~]# ln -s /home/var/qmail /var/qmail

[root@node1 ~]# mkdir -p /home/usr/src

[root@node1 ~]# ln -s /home/usr/src /usr/src

[root@node1 ~]# mkdir -p /home/var/log/qmail

[root@node1 ~]# ln -s /home/var/log/qmail /var/log/qmail

[root@node1 ~]# touch /home/etc/tcp.smtp

[root@node1 ~]# ln -s /home/etc/tcp.smtp /etc/tcp.smtp

[root@node1 ~]# touch /home/etc/tcp.smtp.cdb

[root@node1 ~]# ln -s /home/etc/tcp.smtp.cdb /etc/tcp.smtp.cdb

[root@node1 ~]# mkdir -p /home/qmaildoc

[root@node1 ~]# ln -s /home/qmaildoc /qmaildoc

Install openldap:

[root@node1 ~]# yum install openldap

[root@node1 ~]# mkdir -p /home/etc/openldap

[root@node1 ~]# ln -s /home/etc/openldap /etc/openldap

[root@node1 ~]# mkdir -p /home/var/lib/ldap

[root@node1 ~]# ln -s /home/var/lib/ldap /var/lib/ldap

[root@node1 ~]# mkdir -p /home/var/www/html

[root@node1 ~]# ln -s /home/var/www/html /var/www/html

[root@node1 ~]# mkdir -p /home/var/www/sarg

[root@node1 ~]# ln -s /home/var/www/sarg /var/www/sarg

Install clamav:

[root@node1 ~]# yum install clamav clamd

[root@node1 ~]# mkdir -p /home/var/spool/qscan

[root@node1 ~]# ln -s /home/var/spool/qscan /var/spool/qscan

install spamassassin:

[root@node1 ~]# yum install spamassassin

[root@node1 ~]# mkdir -p /home/etc/mail/spamassassin/

[root@node1 ~]# touch /home/etc/mail/spamassassin/local.cf

[root@node1 ~]# ln -s /home/etc/mail/spamassassin/local.cf /etc/mail/spamassassin/local.cf

install squid:

[root@node1 ~]# yum install squid

[root@node1 ~]# mkdir -p /home/etc/squid

[root@node1 ~]# ln -s /home/etc/squid /etc/squid

configuration of fetchmail:

[root@node1 ~]# crontab -lu tetra

*/5 * * * * /usr/bin/fetchmail -v -U -F >> /home/tetra/fetchmail.log

copy the script of fetchmail...

[root@node1 ~]# rsync -aP 192.168.10.155:/usr/bin/fetchmail /usr/bin/

copy the rc.local file........

[root@node1 ~]# rsync -aP 192.168.10.155:/etc/rc.local /etc/

Make link of below file.......

[root@node1 ~]# ln -s /usr/local/sbin/authdaemond /etc/init.d/authdaemond

[root@node1 ~]# ln -s /var/qmail/bin/qmailctl /usr/bin/qmailctl

[root@node1 ~]# ln -s /usr/bin/qmailctl /etc/init.d/qmailctl

Now reboot the node1 to check all required files are copied properly and server is starting normally. Afterthen try below to check qmail, imap, ldap is working or not.

[root@node1 ~]# telnet localhost 25
Trying 127.0.0.1...
Connected to localhost.localdomain (127.0.0.1).
Escape character is '^]'.
220 mail.energoindia.com ESMTP

[root@node1 ~]# telnet localhost 110
Trying 127.0.0.1...
Connected to localhost.localdomain (127.0.0.1).
Escape character is '^]'.
+OK <2081.1436400429@mail.energoindia.com>

[root@node1 ~]# telnet localhost 143
Trying 127.0.0.1...
Connected to localhost.localdomain (127.0.0.1).
Escape character is '^]'.
* OK [CAPABILITY IMAP4rev1 UIDPLUS CHILDREN NAMESPACE THREAD=ORDEREDSUBJECT THREAD=REFERENCES SORT QUOTA
IDLE ACL ACL2=UNION STARTTLS] Courier-IMAP ready. Copyright 1998-2005 Double Precision, Inc. See COPYING
for distribution information.

[root@node1 ~]# telnet localhost 389
Trying 127.0.0.1...
Connected to localhost.localdomain (127.0.0.1).
Escape character is '^]'.

copy the /home from running server to node1......

[root@node1 ~]# rsync -alkvHP 192.168.10.155:/home/* /home/

After proper sync live this node1.

== '''############## ON NODE2 configuration ##############''' ==

[root@node2 ~]# cat /etc/hosts
# Do not remove the following line, or various programs
# that require network functionality will fail.
127.0.0.1 mail.energoindia.com mail localhost.localdomain localhost
#::1 localhost6.localdomain6 localhost6
192.168.10.149 node1.energoindia.com node1 node.energoindia.com mail
192.168.1.150 node.energoindia.com node mail.energoindia.com mail
10.0.0.1 node1.energoindia.com node1 node.energoindia.com
10.0.0.2 node2.energoindia.com node2 node.energoindia.com
192.168.10.155 node2.energoindia.com node2 node.energoindia.com mail

''':::::::::Installing DRBD:::::::'''

[root@node2 ~]# yum install drbd83-utils kmod-drbd83

Configuration should be same as in node1:

[root@node2 ~]# mv /etc/drbd.d/global_common.conf /etc/drbd.d/global_common.sample

[root@node2 ~]# vim /etc/drbd.d/global_common.conf
global { usage-count no; }
common {
syncer { rate 100M; }
}

Create a new resource file that I called “main”,

[root@node2 ~]# vim /etc/drbd.d/main.res
resource main {
protocol C;
startup { wfc-timeout 0; degr-wfc-timeout 120; }
disk { on-io-error detach; }
on node1.energoindia.com {
device /dev/drbd0;
disk /dev/VolGroup00/LogVol02;
meta-disk internal;
address 10.0.0.1:7788;
}
on node2.energoindia.com {
device /dev/drbd0;
disk /dev/mapper/VolGroup00-LogVol02;
meta-disk internal;
address 10.0.0.2:7788;
}
}

Now we have to create the metadata on node2:

[root@node2 ~]# dd if=/dev/zero of= /dev/VolGroup00/LogVol02 bs=1M count=50

[root@node2 ~]# drbdadm create-md main

Now start DRBD on this (node2) server:

[root@node2 ~]# service drbd start

Starting DRBD resources: [ d(main) s(main) n(main) ].

After starting drbd on node2 server, it start to synchronization between both nodes. It will take a little while to be completed. Please wait until it is done and move to the next step.

[root@node2 ~]# cat /proc/drbd

version: 8.3.16 (api:88/proto:86-97)
GIT-hash: a798fa7e274428a357657fb52f0ecf40192c1985 build by phil@Build64R5, 2013-09-27 15:48:18
0: cs:SyncSource ro:Primary/Secondary ds:UpToDate/Inconsistent C r-----
ns:503808 nr:0 dw:0 dr:504472 al:0 bm:30 lo:0 pe:0 ua:0 ap:0 ep:1 wo:b oos:41437916
[>....................] sync'ed: 1.3% (40464/40956)M
finish: 1:06:40 speed: 10,340 (10,280) K/sec

'''::::::Installing heartbeat::::::'''

[root@node2 ~]# yum install heartbeat

Create the following file on node2 servers with the exact same content as on node1:

[root@node2 ~]# vim /etc/ha.d/ha.cf
keepalive 2
deadtime 30
bcast eth0
node node1.energoindia.com node2.energoindia.com
logfile /var/log/ha-log
debugfile /var/log/ha-debug

The next setp is to create the resource file for heartbeat on node2 servers with exact same content as on node1 again:

[root@node2 ~]# vim /etc/ha.d/haresources
node1.energoindia.com IPaddr::192.168.10.150/24/eth0 IPaddr::192.168.20.150/24/eth0 IPaddr::192.168.1.150/22/eth0 IPaddr::192.168.30.150/24/eth0 drbddisk::main Filesystem::/dev/drbd0::/home::ext3 ldap httpd spamassassin clamd authdaemond imap imaps squid qmailctl

The last thing is to create the authentication file on node2 servers again with the same content as on node1:

[root@node2 ~]# vim /etc/ha.d/authkeys
auth 3
3 md5 asd1234f

This password file should only be readable by the root user:

[root@node2 ~]# chmod 600 /etc/ha.d/authkeys

[root@node2 ~]# service heartbeat start

Again create same directory on node2 as in node1 and make links but before take backup of these directory:

[root@node2 ~]# mkdir -p /home/command

[root@node2 ~]# ln -s /home/command /command

[root@node2 ~]# mkdir -p /home/downloads

[root@node2 ~]# ln -s /home/downloads /downloads

[root@node2 ~]# mkdir -p /home/packages

[root@node2 ~]# ln -s /home/packages /packages

[root@node2 ~]# mkdir -p /home/service

[root@node2 ~]# ln -s /home/service /service

[root@node2 ~]# mkdir -p /home/usr/local

[root@node2 ~]# ln -s /home/usr/local /usr/local

[root@node2 ~]# mkdir -p /home/var/qmail

[root@node2 ~]# ln -s /home/var/qmail /var/qmail

[root@node2 ~]# mkdir -p /home/usr/src

[root@node2 ~]# ln -s /home/usr/src /usr/src

[root@node2 ~]# mkdir -p /home/var/log/qmail

[root@node2 ~]# ln -s /home/var/log/qmail /var/log/qmail

[root@node2 ~]# touch /home/etc/tcp.smtp

[root@node2 ~]# ln -s /home/etc/tcp.smtp /etc/tcp.smtp

[root@node2 ~]# touch /home/etc/tcp.smtp.cdb

[root@node2 ~]# ln -s /home/etc/tcp.smtp.cdb /etc/tcp.smtp.cdb

[root@node2 ~]# mkdir -p /home/qmaildoc

[root@node2 ~]# ln -s /home/qmaildoc /qmaildoc

[root@node2 ~]# mkdir -p /home/etc/openldap

[root@node2 ~]# ln -s /home/etc/openldap /etc/openldap

[root@node2 ~]# mkdir -p /home/var/lib/ldap

[root@node2 ~]# ln -s /home/var/lib/ldap /var/lib/ldap

[root@node2 ~]# mkdir -p /home/var/www/html

[root@node2 ~]# ln -s /home/var/www/html /var/www/html

[root@node2 ~]# mkdir -p /home/var/www/sarg

[root@node2 ~]# ln -s /home/var/www/sarg /var/www/sarg

[root@node2 ~]# mkdir -p /home/var/spool/qscan

[root@node2 ~]# ln -s /home/var/spool/qscan /var/spool/qscan

[root@node2 ~]# mkdir -p /home/etc/mail/spamassassin/

[root@node2 ~]# touch /home/etc/mail/spamassassin/local.cf

[root@node2 ~]# ln -s /home/etc/mail/spamassassin/local.cf /etc/mail/spamassassin/local.cf

[root@node2 ~]# mkdir -p /home/etc/squid

[root@node2 ~]# ln -s /home/etc/squid /etc/squid

Make link of below file.......

[root@node2 ~]# ln -s /usr/local/sbin/authdaemond /etc/init.d/authdaemond

[root@node2 ~]# ln -s /var/qmail/bin/qmailctl /usr/bin/qmailctl

[root@node2 ~]# ln -s /usr/bin/qmailctl /etc/init.d/qmailctl

== '''### Test the cluster ###''' ==

1. Stop heartbeat on node1:

[root@node1 ~]# /etc/init.d/heartbeat stop

2. Watch log of heartbeat on node2, cluster will shift on node2.

[root@node2 ~]# tailf /var/log/ha-log

3. On node2, check the mails are working and /home is mounted.

[root@node2 ~]# telnet localhost 25
Trying 127.0.0.1...
Connected to localhost.localdomain (127.0.0.1).
Escape character is '^]'.
220 mail.energoindia.com ESMTP

[root@node2 ~]# telnet localhost 110
Trying 127.0.0.1...
Connected to localhost.localdomain (127.0.0.1).
Escape character is '^]'.
+OK <14050.1436639325@mail.energoindia.com>

[root@node2 ~]# telnet localhost 143
Trying 127.0.0.1...
Connected to localhost.localdomain (127.0.0.1).
Escape character is '^]'.
* OK [CAPABILITY IMAP4rev1 UIDPLUS CHILDREN NAMESPACE THREAD=ORDEREDSUBJECT THREAD=REFERENCES SORT QUOTA IDLE ACL ACL2=UNION STARTTLS] Courier-IMAP ready. Copyright 1998-2005 Double Precision, Inc. See COPYING for distribution information.

[root@node2 ~]# telnet localhost 389
Trying 127.0.0.1...
Connected to localhost.localdomain (127.0.0.1).
Escape character is '^]'.

[root@node2 ~]# df -h
Filesystem Size Used Avail Use% Mounted on
/dev/mapper/VolGroup00-LogVol00
95G 74G 17G 83% /
/dev/sda1 965M 24M 892M 3% /boot
tmpfs 2.0G 0 2.0G 0% /dev/shm
/dev/drbd0 446G 359G 65G 85% /home

4. now, to shift cluster again on node1.
stop heartbeat on node2 and start heartbeat on node1.

[root@node2 ~]# /etc/init.d/heartbeat stop

[root@node1 ~]# /etc/init.d/heartbeat start

5. Now test the cluster is shifting to node2, if node1 goes down. It will shift.

6. Now start node1, the cluster will shift automatically on node1 from node2.

2015-04-16T15:54:01Z

Mithilesh: Blanked the page

Architecture of sysnetglobal

2015-04-16T15:51:56Z

Mithilesh: Created page with "category:sysnetglobal '''Architecture''' image:sysnet-architechture"

[[category:sysnetglobal]]

'''Architecture'''

[[image:sysnet-architechture]]

Category:Sysnetglobal

2015-04-16T15:49:45Z

Mithilesh:

[[category:sysnetglobal]]

'''Architecture of sysnetglobal'''

[[image:sysnet-architechture.jpg]]

2015-02-06T13:34:58Z

Mithilesh: Created page with "'''STEPS TO CONFIGURE NRPE AND NRPE PLUGIN ON CLIENT MACHINE:-''' ON CLIENT MACHINE: Install libraries for dependencies as it would help in compiling the NRPE packages. ..."

'''STEPS TO CONFIGURE NRPE AND NRPE PLUGIN ON CLIENT MACHINE:-'''

ON CLIENT MACHINE:

Install libraries for dependencies as it would help in compiling the NRPE packages.

Step 1- install libraries

# yum install -y gcc glibc glibc-common gd gd-devel make net-snmp openssl-devel

step 2- user and password

# useradd nagios
# passwd nagios

step 3-extract plugins

# tar –xvf nagios-plugins-1.5.tar.gz

step 4- compile and install
# cd nagios-plugins-1.5
# ./configure
# make
# make install
step 5-permissions on plugins

# chown nagios.nagios /usr/local/nagios
# chown -R nagios.nagios /usr/local/nagios/libexec
step 6-install xinetd

#yum install xinetd

step 7-install nrpe

tar xzf nrpe-2.15.tar.gz
# cd nrpe-2.15
# ./configure
# make all
# make install-plugin
# make install-daemon
# make install-daemon-config
#make install-xinetd

step 8-add nagios monitoring server ip

#vim /etc/xinetd.d/nrpe
add only_from = 127.0.0.1 <nagios_ip_address>
#vim /etc/service
add nrpe 5666/tcp #NRPE

step 9-restart xinetd

#/etc/init.d/xinetd restart

step 10-verify nrpe daemon

# netstat -at | grep nrpe

tcp 0 0 *:nrpe *:* LISTEN

step 11-To check nrpe daemon is workig properly

# /usr/local/nagios/libexec/check_nrpe -H localhost
NRPE v2.15

step 12- customize nrpe commands and add commands

# vim /usr/local/nagios/etc/nrpe.cfg
command[check_users]=/usr/local/nagios/libexec/check_users -w 5 -c 10
command[check_load]=/usr/local/nagios/libexec/check_load -w 15,10,5 -c 30,25,20
command[check_hda1]=/usr/local/nagios/libexec/check_disk -w 20% -c 10% -p /dev/mapper/vg_mail1-LogVol01
command[check_zombie_procs]=/usr/local/nagios/libexec/check_procs -w 5 -c 10 -s Z
command[check_cpu]=/usr/local/nagios/libexec/check_cpu.sh -w 60 -c 70
command[check_mem]=/usr/local/nagios/libexec/check_mem_avail -w 10 -c 5
command[check_queue]=/usr/local/nagios/libexec/check_postfix_queue -w 200 -c 250
command[check_zimbra_lmtp]=/usr/local/nagios/libexec/check_smtp -H localhost -p 7025 -e '220 node.sandhar.in Zimbra LMTP server ready'
command[check_smtp]=/usr/local/nagios/libexec/check_smtp -H localhost

step 13 -run /etc/init.d/xinetd restart

step 14 – reload nagios on nagios server:
#/etc/init.d/nagios reload

step 15 -After then we should call all services from nagios server:
# /usr/local/nagios/libexec/check_nrpe -H 172.17.16.22 -c check_zimbra_lmtp
ERROR: could not complete SSL handshake
# /usr/local/nagios/libexec/check_nrpe -H 172.17.16.22 -c check_smtp
ERROR: could not complete SSL handshake

That means we are not able to call services from nagios server to client machine. This is happening because of openssl library missing. For this we need to run nrpe as daemon with non ssl. The command is:
# /usr/local/nagios/bin/nrpe -c /usr/local/nagios/etc/nrpe.cfg -d -n
Now in future whenever we will add services of mailserver in sandhar, we need to follow below steps:

step 16-kill the nrpe running and run nrpe as daemon.
# ps -elf | grep nrpe
0 S root 747 15083 0 80 0 - 25813 pipe_w 10:59 pts/0 00:00:00 grep nrpe
5 S nagios 22682 1 0 80 0 - 9810 poll_s Feb02 ? 00:00:00 /usr/local/nagios/bin/nrpe -c /usr/local/nagios/etc/nrpe.cfg -d -n

# kill -9 22682
# /usr/local/nagios/bin/nrpe -c /usr/local/nagios/etc/nrpe.cfg -d -n

step -17 Now test from nagios server.
# /usr/local/nagios/libexec/check_nrpe -H 172.17.16.22 -c check_zimbra_lmtp
SMTP OK - 0.001 sec. response time|time=0.000956s;;;0.000000

# /usr/local/nagios/libexec/check_nrpe -H 172.17.16.22 -c check_smtp
SMTP OK - 0.002 sec. response time|time=0.002258s;;;0.000000

# /usr/local/nagios/libexec/check_nrpe -H 172.17.16.22 -c check_cpu
OK - user: 23.22, nice: 1.11, sys: 2.95, iowait: 0.62, irq: 0.50, softirq: 0.74 idle: 74.33 | 'user'=23.22 'nice'=1.11 'sys'=2.95 'softirq'=0.74 'iowait'=0.62 'irq'=0.50 'idle'=74.33

# /usr/local/nagios/libexec/check_nrpe -H 172.17.16.22 -c check_queue
33 mail(s) in queue | mail_queue=33

step 18- Add host on nagios server of sandhar.
# vim /usr/local/nagios/etc/objects/linux-box.cfg
define host{
use linux-server
host_name CORP-MAILSERVER
alias CORP-MAILSERVER
address 172.17.16.22
hostgroups Windows Server
}

step 19- Add services on nagios server.

define service{
use generic-service
host_name CORP-MAILSERVER
service_description Current Users
notification_options w,u,c,r
check_command check_nrpe!check_users
}

define service{
use generic-service
host_name CORP-MAILSERVER
service_description Server Load
notification_options w,u,c,r
check_command check_nrpe!check_load
}
define service{
use generic-service
host_name CORP-MAILSERVER
service_description Disk Space
notification_options w,u,c,r
check_command check_nrpe!check_hda1
}
define service{
use generic-service
host_name CORP-MAILSERVER
service_description Zombie process
notification_options w,u,c,r
check_command check_nrpe!check_zombie_procs
}
define service{
use generic-service
host_name CORP-MAILSERVER
service_description CPU Load
notification_options w,u,c,r
check_command check_nrpe!check_cpu
}
define service{
use generic-service
host_name CORP-MAILSERVER
service_description Memory Utilization
notification_options w,u,c,r
check_command check_nrpe!check_mem
}
define service{
use generic-service
host_name CORP-MAILSERVER
service_description Mail Queue
notification_options w,u,c,r
check_command check_nrpe!check_queue
}
define service{ use generic-service
host_name CORP-MAILSERVER
service_description Mail Queue
notification_options w,u,c,r
check_command check_nrpe!check_queue
}
define service{
use generic-service
host_name CORP-MAILSERVER
service_description LMTP
notification_options w,u,c,r
check_command check_nrpe!check_zimbra_lmtp
}

define service{
use generic-service
host_name CORP-MAILSERVER
service_description SMTP
notification_options w,u,c,r
check_command check_smtp
}

step 20- verify the nagios.cfg file.

# /usr/local/nagios/bin/nagios -v /usr/local/nagios/etc/nagios.cfg
Checking services...
Checked 815 services.
Checking hosts...
Checked 108 hosts.
Checking host groups...
Checked 5 host groups.
Checking service groups...
Checked 0 service groups.
Checking contacts...
Checked 16 contacts.
Checking contact groups...
Checked 12 contact groups.
Checking service escalations...
Checked 2962 service escalations.
Checking service dependencies...
Checked 0 service dependencies.
Checking host escalations...
Checked 0 host escalations.
Checking host dependencies...
Checked 0 host dependencies.
Checking commands...
Checked 70 commands.
Checking time periods...
Checked 6 time periods.
Checking for circular paths between hosts...
Checking for circular host and service dependencies...
Checking global event handlers...
Checking obsessive compulsive processor commands...
Checking misc settings...

Total Warnings: 0
Total Errors: 0

If we get below error in the escalation.cfg file that means there is empty services defined for any host. For this we need to check config file.
ERROR: could not expand the services defined in ‘service escalation’ config ( /usr/local/nagios/etc/objects/escalation.cfg )

First disable the escalation.cfg in nagios.cfg file:
#cfg_file=/usr/local/nagios/etc/objects/escalation.cfg
Second verify the config file nagios.cfg file:
# /usr/local/nagios/bin/nagios -v /usr/local/nagios/etc/nagios.cfg
It will show you the warning like this:
Checking host escalations...
Checked 0 host escalations.
Checking host dependencies...
Warning: ‘node1.sandhar.in’
Checking commands...
Checked 70 commands.

To resolve this either you define the services for that host or disable that host. And verify the nagios.cfg file.
If total warnings is 0 and total errors is 0 then reload nagios.

# /etc/init.d/nagios reload

User:Mithilesh

2015-02-06T13:15:36Z

Mithilesh: Blanked the page

User:Mithilesh

2015-02-06T13:13:50Z

Mithilesh: NRPE installation in sandhar Mailserver

STEPS TO CONFIGURE NRPE AND NRPE PLUGIN ON CLIENT MACHINE:-

ON CLIENT MACHINE:

Install libraries for dependencies as it would help in compiling the NRPE packages.

Step 1- install libraries

# yum install -y gcc glibc glibc-common gd gd-devel make net-snmp openssl-devel

step 2- user and password

# useradd nagios
# passwd nagios

step 3-extract plugins

# tar –xvf nagios-plugins-1.5.tar.gz

step 4- compile and install
# cd nagios-plugins-1.5
# ./configure
# make
# make install
step 5-permissions on plugins

# chown nagios.nagios /usr/local/nagios
# chown -R nagios.nagios /usr/local/nagios/libexec
step 6-install xinetd

#yum install xinetd

step 7-install nrpe

tar xzf nrpe-2.15.tar.gz
# cd nrpe-2.15
# ./configure
# make all
# make install-plugin
# make install-daemon
# make install-daemon-config
#make install-xinetd

step 8-add nagios monitoring server ip

#vim /etc/xinetd.d/nrpe
add only_from = 127.0.0.1 <nagios_ip_address>
#vim /etc/service
add nrpe 5666/tcp #NRPE

step 9-restart xinetd

#/etc/init.d/xinetd restart

step 10-verify nrpe daemon

# netstat -at | grep nrpe

tcp 0 0 *:nrpe *:* LISTEN

step 11-To check nrpe daemon is workig properly

# /usr/local/nagios/libexec/check_nrpe -H localhost
NRPE v2.15

step 12- customize nrpe commands and add commands

# vim /usr/local/nagios/etc/nrpe.cfg
command[check_users]=/usr/local/nagios/libexec/check_users -w 5 -c 10
command[check_load]=/usr/local/nagios/libexec/check_load -w 15,10,5 -c 30,25,20
command[check_hda1]=/usr/local/nagios/libexec/check_disk -w 20% -c 10% -p /dev/mapper/vg_mail1-LogVol01
command[check_zombie_procs]=/usr/local/nagios/libexec/check_procs -w 5 -c 10 -s Z
command[check_cpu]=/usr/local/nagios/libexec/check_cpu.sh -w 60 -c 70
command[check_mem]=/usr/local/nagios/libexec/check_mem_avail -w 10 -c 5
command[check_queue]=/usr/local/nagios/libexec/check_postfix_queue -w 200 -c 250
command[check_zimbra_lmtp]=/usr/local/nagios/libexec/check_smtp -H localhost -p 7025 -e '220 node.sandhar.in Zimbra LMTP server ready'
command[check_smtp]=/usr/local/nagios/libexec/check_smtp -H localhost

step 13 -run /etc/init.d/xinetd restart

step 14 – reload nagios on nagios server:
#/etc/init.d/nagios reload

step 15 -After then we should call all services from nagios server:
# /usr/local/nagios/libexec/check_nrpe -H 172.17.16.22 -c check_zimbra_lmtp
ERROR: could not complete SSL handshake
# /usr/local/nagios/libexec/check_nrpe -H 172.17.16.22 -c check_smtp
ERROR: could not complete SSL handshake

That means we are not able to call services from nagios server to client machine. This is happening because of openssl library missing. For this we need to run nrpe as daemon with non ssl. The command is:
# /usr/local/nagios/bin/nrpe -c /usr/local/nagios/etc/nrpe.cfg -d -n
Now in future whenever we will add services of mailserver in sandhar, we need to follow below steps:

step 16-kill the nrpe running and run nrpe as daemon.
# ps -elf | grep nrpe
0 S root 747 15083 0 80 0 - 25813 pipe_w 10:59 pts/0 00:00:00 grep nrpe
5 S nagios 22682 1 0 80 0 - 9810 poll_s Feb02 ? 00:00:00 /usr/local/nagios/bin/nrpe -c /usr/local/nagios/etc/nrpe.cfg -d -n

# kill -9 22682
# /usr/local/nagios/bin/nrpe -c /usr/local/nagios/etc/nrpe.cfg -d -n

step -17 Now test from nagios server.
# /usr/local/nagios/libexec/check_nrpe -H 172.17.16.22 -c check_zimbra_lmtp
SMTP OK - 0.001 sec. response time|time=0.000956s;;;0.000000

# /usr/local/nagios/libexec/check_nrpe -H 172.17.16.22 -c check_smtp
SMTP OK - 0.002 sec. response time|time=0.002258s;;;0.000000

# /usr/local/nagios/libexec/check_nrpe -H 172.17.16.22 -c check_cpu
OK - user: 23.22, nice: 1.11, sys: 2.95, iowait: 0.62, irq: 0.50, softirq: 0.74 idle: 74.33 | 'user'=23.22 'nice'=1.11 'sys'=2.95 'softirq'=0.74 'iowait'=0.62 'irq'=0.50 'idle'=74.33

# /usr/local/nagios/libexec/check_nrpe -H 172.17.16.22 -c check_queue
33 mail(s) in queue | mail_queue=33

step 18- Add host on nagios server of sandhar.
# vim /usr/local/nagios/etc/objects/linux-box.cfg
define host{
use linux-server
host_name CORP-MAILSERVER
alias CORP-MAILSERVER
address 172.17.16.22
hostgroups Windows Server
}

step 19- Add services on nagios server.

define service{
use generic-service
host_name CORP-MAILSERVER
service_description Current Users
notification_options w,u,c,r
check_command check_nrpe!check_users
}

define service{
use generic-service
host_name CORP-MAILSERVER
service_description Server Load
notification_options w,u,c,r
check_command check_nrpe!check_load
}
define service{
use generic-service
host_name CORP-MAILSERVER
service_description Disk Space
notification_options w,u,c,r
check_command check_nrpe!check_hda1
}
define service{
use generic-service
host_name CORP-MAILSERVER
service_description Zombie process
notification_options w,u,c,r
check_command check_nrpe!check_zombie_procs
}
define service{
use generic-service
host_name CORP-MAILSERVER
service_description CPU Load
notification_options w,u,c,r
check_command check_nrpe!check_cpu
}
define service{
use generic-service
host_name CORP-MAILSERVER
service_description Memory Utilization
notification_options w,u,c,r
check_command check_nrpe!check_mem
}
define service{
use generic-service
host_name CORP-MAILSERVER
service_description Mail Queue
notification_options w,u,c,r
check_command check_nrpe!check_queue
}
define service{ use generic-service
host_name CORP-MAILSERVER
service_description Mail Queue
notification_options w,u,c,r
check_command check_nrpe!check_queue
}
define service{
use generic-service
host_name CORP-MAILSERVER
service_description LMTP
notification_options w,u,c,r
check_command check_nrpe!check_zimbra_lmtp
}

define service{
use generic-service
host_name CORP-MAILSERVER
service_description SMTP
notification_options w,u,c,r
check_command check_smtp
}

step 20- verify the nagios.cfg file.

# /usr/local/nagios/bin/nagios -v /usr/local/nagios/etc/nagios.cfg
Checking services...
Checked 815 services.
Checking hosts...
Checked 108 hosts.
Checking host groups...
Checked 5 host groups.
Checking service groups...
Checked 0 service groups.
Checking contacts...
Checked 16 contacts.
Checking contact groups...
Checked 12 contact groups.
Checking service escalations...
Checked 2962 service escalations.
Checking service dependencies...
Checked 0 service dependencies.
Checking host escalations...
Checked 0 host escalations.
Checking host dependencies...
Checked 0 host dependencies.
Checking commands...
Checked 70 commands.
Checking time periods...
Checked 6 time periods.
Checking for circular paths between hosts...
Checking for circular host and service dependencies...
Checking global event handlers...
Checking obsessive compulsive processor commands...
Checking misc settings...

Total Warnings: 0
Total Errors: 0

If we get below error in the escalation.cfg file that means there is empty services defined for any host. For this we need to check config file.
ERROR: could not expand the services defined in ‘service escalation’ config ( /usr/local/nagios/etc/objects/escalation.cfg )

First disable the escalation.cfg in nagios.cfg file:
#cfg_file=/usr/local/nagios/etc/objects/escalation.cfg
Second verify the config file nagios.cfg file:
# /usr/local/nagios/bin/nagios -v /usr/local/nagios/etc/nagios.cfg
It will show you the warning like this:
Checking host escalations...
Checked 0 host escalations.
Checking host dependencies...
Warning: ‘node1.sandhar.in’
Checking commands...
Checked 70 commands.

To resolve this either you define the services for that host or disable that host. And verify the nagios.cfg file.
If total warnings is 0 and total errors is 0 then reload nagios.

# /etc/init.d/nagios reload