Tuning JVM settings correctly

Tuning JVM settings correctly to avoid full Garbage Collection. (Doc ID 1334842.1)

DISCONNECT - INFO  - conn=1317882 reason="other" msg="Exception caught while polling client connection LDAPS.X.Y.Z.W.64087 -- java.io.IOException: Connection reset by peer"

GC with old space cleanup

JVM space is separated in 3 different areas:

• Young Space
• Old Space
• Permanent Space

The way GC works is that there are partial GC's to recycle Young Space and Full GC to recycle Old Space.

There is also a promotion mechanism from young space to old space and old space to permanent, but the goal of this article is not to make a detailed presentation of the JVM but to be able to understand it's internal behaviour to diagnose it.

Full GC Old space is extremely costly, and is the cause of DPS being unreachable for limited amount of time.

Using Jmap and Jstat

You need to collect information about JVM  heap. 

Modifying JVM settings:

Old JVM settings (causing DPS outage)

• jvmArgs=-Xmx2000M  -Xms2000M -XX:NewRatio=2

 New JVM settings:

• jvmArgs: -Xms2g -Xmx2g -Xmn1g -XX:SurvivorRatio=4 -XX:+UseParNewGC  -XX:+UseConcMarkSweepGC

Explanation of parameters:

• Xms/Xmx: size of heap (2g, size at startup)
• Xmn: size of young space (1g to maximize young space, to avoid old space promotion)

• Garbage collation algorithm option choosen
• -XX:+UseConcMarkSweepGC (concurrent low pause Collector)
• -XX:+UseParNewGC To enable a parallel young generation GC with the concurrent GC

Results

Before applying JVM Setting Changes (system running since 6 days)

time: 06010018
 S0     S1     E      O      P     YGC     YGCT    FGC    FGCT     GCT
 0.00  41.27  81.54  74.68  49.73    849   39.436     1    8.526   47.962
 0.00  41.27  81.54  74.68  49.73    849   39.436     1    8.526   47.962
 0.00  41.27  81.56  74.68  49.73    849   39.436     1    8.526   47.962
 0.00  41.27  81.58  74.68  49.73    849   39.436     1    8.526   47.962
 0.00  41.27  81.66  74.68  49.73    849   39.436     1    8.526   47.962


time: 06160918
 S0     S1     E      O      P     YGC     YGCT    FGC    FGCT     GCT
56.63   0.00  86.07  73.87  49.96   2596  104.367     5   33.984  138.351
56.63   0.00  86.09  73.87  49.96   2596  104.367     5   33.984  138.351
56.63   0.00  86.11  73.87  49.96   2596  104.367     5   33.984  138.351
56.63   0.00  86.21  73.87  49.96   2596  104.367     5   33.984  138.351
56.63   0.00  86.43  73.87  49.96   2596  104.367     5   33.984  138.351

after applying JVM changes (system running after 11 days)

time: 06160948
 S0     S1     E      O      P     YGC     YGCT    FGC    FGCT     GCT
 0.00  29.60  87.95   0.00  61.88      3    0.197     0    0.000    0.197
 0.00  29.60  87.95   0.00  61.88      3    0.197     0    0.000    0.197
 0.00  29.60  87.98   0.00  61.88      3    0.197     0    0.000    0.197
 0.00  29.60  88.06   0.00  61.88      3    0.197     0    0.000    0.197
 0.00  29.60  88.06   0.00  61.88      3    0.197     0    0.000    0.197

time: 06271918
 S0     S1     E      O      P     YGC     YGCT    FGC    FGCT     GCT
 8.40   0.00  66.56  14.22  85.35   1134   38.757     0    0.000   38.757
 8.40   0.00  66.56  14.22  85.35   1134   38.757     0    0.000   38.757
 8.40   0.00  66.73  14.22  85.35   1134   38.757     0    0.000   38.757
 8.40   0.00  66.79  14.22  85.35   1134   38.757     0    0.000   38.757
 8.40   0.00  66.86  14.22  85.35   1134   38.757     0    0.000   38.757

Conclusion

The difference between them both is:

before: (6 days run time)

• YGC=2596 YGCT=104.367     FGC=5   GCT=33.984  FCGT=138.351

after: (11days run time)

• YGCT=1134   YGCT=38.757     FGC=0    GCT=0.000   FCGT=38.757

The major differences between before and after are:

• Young generation garbage collection time reduced of 2/3. (38.757 versus 104.367)
• No old space full garbage collection (0 compared to 5) (which is a killer for performance)

overall GC time was 38s in 11 days versus 140s in 6 days. Overall GC time reduced between 8 to 10.