Tuning WebSphere Application Server ND 7.0. Royal Cyber Inc.

Transcription

1 Tuning WebSphere Application Server ND 7.0 Royal Cyber Inc.

2 JVM related problems Application server stops responding Server crash Hung process Out of memory condition Performance degradation Check if the process ID is continually changing o Indicates the application server is probably crashing and being restarted Core files Javacore files VerboseGC logs Heap dump

3 Tuning Methodology The tuning process is iterative in nature. To find the optimal configuration several tests and evaluation should be performed

4 JVM Analysis Tools IBM Monitoring and Diagnostic Tools for Java - Garbage Collection and Memory Visualizer

5 JVM Analysis Tools IBM Pattern Modeling and Analysis Tool for Java Garbage Collector

6 Troubleshooting server hang issues WebSphere process hang detection steps a) Once a hang is suspected, obtain a thread dump or javacore b) For a typical hang, collect three dumps at a few minutes interval To see if anything is moving within the process (but slowly) a) Examine the thread dumps with Thread Analyzer or by hand Hang detection tools ThreadMonitor architecture was created to monitor thread pools within ThreadAnalyzer GUI-based tool Gathers and analyzes thread dumps from a WebSphere Application Server Provides recommendations based on analysis

7 ThreadAnalyzer

8 ThreadAnalyzer: Analysis

9 ThreadAnalyzer: Multiple dump analysis

10 ThreadAnalyzer: Overall thread analysis

11 How to troubleshoot crashes Symptoms Process terminated with Java exception or native signal Usual causes Out of memory exception Call stack overflow Unexpected exception (for example, out of disk space) Optimizer failure (for example, JiT) Bad JNI call or library problem Segmentation violations while executing native code

12 Dump Analyzer: Analysis completed

13 How to troubleshoot Out Of Memory error? Java virtual machine error Not enough memory to allocate an object can be caused by the following: The Java heap is too small Memory is available in the heap, but it is fragmented (for JDK and earlier) Memory leak in the Java code Not enough space in the native memory

14 Administrative Console TPV Graph

15 PMAT: Summary page

16 PMAT: Analysis and recommendations

17 PMAT: Chart view

18 Thread pool size Each task performed by the server runs on a thread obtained from one of WebSphere Application Server s many thread pools Three of the most commonly used (and tuned) thread pools within the application server are: Web container: Used when requests come in over HTTP Default: Used when requests come in for a message driven bean or if a particular transport chain has not been defined to a specific thread pool. ORB: Used when remote requests come in over RMI/IIOP for an enterprise bean from an EJB application client, remote EJB interface, or another application server

19 Default Thread Pool Setting A good practice is to use 5 threads per server CPU core for the default thread pool 10 threads per server CPU for the ORB and Web container thread pools. For a machine with up to 4 CPUs, the default settings are usually a good start for most applications There could be situations where the thread pool size might need to be increased to account for slow I/O or long running back-end connections

20 Monitoring and Tuning of Thread Pools The IBM Tivoli Performance Viewer is a tool embedded in the administrative console that lets you view the PMI (Performance Monitoring Infrastructure) data associated with almost any server component. The viewer provides advice to help tune systems for optimal performance and recommends alternatives to inefficient settings.

21 Monitoring and Tuning Connection Pool Each time an application attempts to access a back-end store (such as a database), it requires resources to create, maintain, and release a connection to that data store. PMI Metrics before tuning connection pool

22 Monitoring and Tuning Connection Pool PMI Metrics after tuning connection pool

23 Monitoring and Tuning Data source statement cache size Data source statement cache size specifies the number of prepared JDBC statements that can be cached per connection. Review the application code (or an SQL trace gathered from the database or database driver) for all unique prepared statements, and ensure the cache size is larger than that value. Iteratively increase the cache size and run the application under peak steady state load until the PMI metrics report no more cache discards.

24 Monitoring and Tuning Data source statement cache size PMI metrics after tuning the data source statement cache

25 Debug steps for Websphere Default Messaging Five steps for all messaging issues Check that the ME is in a started state Check SystemOut.log for the specific exception we are trying to debug. Check the FFDCs for further details on the specific exception we are trying to debug. Check the configuration. Trace and engage IBM support.

26