Jboss and PostgresSQL Tuning for Muthoot ERP Environment

Objective : Tuning of Jboss (192.168.1.129) and PostgreSQl (Cluster- 192.168.1.140) for Muthoot Group

Issue : long Report from Application makes the JBOSS Server to hung and Finally we have restart the server . The Long report was not coming even after 1+ hours .

Approach :

1. Tune postgreSql to use high available memory
2. Tune JBOSS to use high available memory

PostgreSQL Tuning and Process

Install pgtune rpm package

yum install pgtune

Create a Sample configuration file based on current environment

pgtune -i /var/lib/pgsql/data/postgresql.conf -o post-pgtune.conf.sample

Check the difference between the created "post-pgtune.conf.sample" and Current "/var/lib/pgsql/data/postgresql.conf"

diff post-pgtune.conf.sample /var/lib/pgsql/data/postgresql.conf

The following lines were introduced in new configuration file

#custom_variable_classes = 		# list of custom variable class names
default_statistics_target = 50 # pgtune wizard 2013-07-25
maintenance_work_mem = 1GB # pgtune wizard 2013-07-25
constraint_exclusion = on # pgtune wizard 2013-07-25
checkpoint_completion_target = 0.9 # pgtune wizard 2013-07-25
effective_cache_size = 22GB # pgtune wizard 2013-07-25
work_mem = 192MB # pgtune wizard 2013-07-25
wal_buffers = 8MB # pgtune wizard 2013-07-25
checkpoint_segments = 16 # pgtune wizard 2013-07-25
shared_buffers = 7680MB # pgtune wizard 2013-07-25
max_connections = 80 # pgtune wizard 2013-07-25

As Muthoot Environment is on Redhat Cluster suite , check the cluster status via

clustat 

Stop the Cluster Services , which will bring down the postgresql too

clusvcadm -s Cluster-Service
cd /var/lib/pgsql/data/
chown postgres.postgres post-pgtune.conf.sample
cp -pvr postgresql.conf postgresql.conf.org_backedup_on_25_july_2013
cp -pvr post-pgtune.conf.sample postgresql.conf

Start the cluster so that PostgreSQL will start along

clusvcadm -R Cluster-Service

Test the environment

clustat 
ps -ef | grep postgre

JBOSS Tuning and Process

The Ram allocated to Jboss was less so needs to increase

Open the Jboss's run.conf file

vi /usr/bin/jboss-6.0.0.Final/bin/run.conf

Modify the old to new as shown below

#   JAVA_OPTS="-Xms128m -Xmx512m -XX:MaxPermSize=256m -Dorg.jboss.resolver.warning=true -Dsun.rmi.dgc.client.gcInterval=3600000 -Dsun.rmi.dgc.server.gcInterval=3600000"
JAVA_OPTS="-Xms12288m -Xmx12288m  -XX:MaxPermSize=256m -Dorg.jboss.resolver.warning=true -Dsun.rmi.dgc.client.gcInterval=3600000 -Dsun.rmi.dgc.server.gcInterval=3600000"

Updated On 22th Jan 2015 (By Amit Srivastava)

JAVA_OPTS="-Xms12288m -Xmx12288m -XX:MaxPermSize=512m -Dorg.jboss.resolver.warning=true -Dsun.rmi.dgc.client.gcInterval=3600000 -Dsun.rmi.dgc.server.gcInterval=3600000"

Test via

ps -ef | grep java

Page Compression & Connection timeout and pool (server.xml) in the connecter class

     <Connector protocol="HTTP/1.1" port="${jboss.web.http.port}" address="${jboss.bind.address}" 
        redirectPort="${jboss.web.https.port}"
        URIEncoding="UTF-8" connectionTimeout="20000"
        compression="on" compressableMimeType="text/html,text/xml,text/plain,text/css,application/x-javascript,application/xml,application/javascript,application/json,application/xhtml+xml"
       />

     <Connector protocol="AJP/1.3" port="${jboss.web.ajp.port}" address="${jboss.bind.address}"
        redirectPort="${jboss.web.https.port}" 
        URIEncoding="UTF-8" connectionTimeout="20000"
        compression="on" compressableMimeType="text/html,text/xml,text/plain,text/css,application/x-javascript,application/xml,application/javascript,application/json,application/xhtml+xml"
        max-connections="4800"

        />

Ref URL: http://imeji.org/install/server-setup/

Database Connection Tuning (postgres-ds.xml)

    <min-pool-size>0</min-pool-size>
   <max-pool-size>60</max-pool-size>
   <idle-timeout-minutes>10</idle-timeout-minutes>
   <new-connection-sql>SELECT 1</new-connection-sql>
   <check-valid-connection-sql>SELECT 1</check-valid-connection-sql>