From: bzaman on
Sending it once more by correcting the subject .

One of our applications is leaking memory on a Prod host . The
application hangs after sometime and we need to restart the
application .

====
Tasks: 449 total, 1 running, 448 sleeping, 0 stopped, 0 zombie
Cpu(s): 12.9% us, 0.1% sy, 0.0% ni, 87.0% id, 0.0% wa, 0.0% hi,
0.0% si
Mem: 8160920k total, 8088752k used, 72168k free, 75516k
buffers
Swap: 8388600k total, 3651908k used, 4736692k free, 807556k
cached

PID USER PR NI %CPU TIME+ %MEM VIRT RES SHR S
COMMAND
31712 nobody 16 0 100 286:20.91 14.4 1458m 1.1g 12m S
java_daemon
18427 nobody 16 0 2 391:17.48 27.7 4334m 2.2g 6200 S
java_daemon
====

Checking with sar , I can see around 40% of swapping taking place.

====
12:00:01 AM kbmemfree kbmemused %memused kbbuffers kbcached
kbswpfree kbswpused %swpused kbswpcad

09:10:02 AM 77104 8083816 99.06 79664 821680
4731308 3657292 43.60 1141444
09:20:01 AM 100904 8060016 98.76 73588 803964
4736692 3651908 43.53 1143748
Average: 294668 7866252 96.39 124533 614381
5268537 3120063 37.19 622012
====

But when checked with vmstat , I can see no swapping activity taking
place as 'so'' value is being displayed as 0 during my monitoring
interval of around 5 minutes.

====
procs -----------memory---------- ---swap-- ---io---- --
system-- ----cpu----
r b swpd free buff cache si so bi
bo in cs us sy id wa

1 0 3651908 73752 76652 812548 0 0 0 36 1058 655
13 0 87 0
1 0 3651908 69952 76688 813000 38 0 108 300 1181 36129
17 7 76 0
1 0 3651908 73408 76752 813072 0 0 6 143 1178 777
13 0 87 0
1 0 3651908 71696 76756 813036 6 0 6 33 1091
715 13 0 87 0
1 0 3651908 71440 76760 813148 45 0 45 86 1117 676
13 0 87 0
1 0 3651908 73240 76772 813136 0 0 0 17 1078 683
13 0 87 0
====

I am trying to understand this behavior. The following are some
questions for which
I am looking an answer .

1) Since vmstat is reporting no swapping activity , then why sar is
reporting around 40% of the swap space is used.
2) Since no swap activity is there as reported by vmstat , I am
getting
a feeling that the java applications are just using the memory and
they are not utilizing the memory . If that is the case , then it
looks like is because of the way Linux handles memory in Linux . Can
disabling "Linux memory overcommit " will help us in reducing the
memory usage for the java applications .

Please clarify .

Thanks in Advance
--Badiuzzaman