Oracle WebLogic Thread Pool Tuning AN ACTIVE ENDPOINTS TECHNICAL NOTE 2010 Active Endpoints Inc. ActiveVOS is a trademark of Active Endpoints, Inc. All other company and product names are the property of their respective owners. 2010 0
Content Oracle WebLogic Thread Pool Tuning... 2 Configure ActiveVOS to use its own Work Manager... 2 Create a global work manager for the ActiveVOS application... 2 Create a Request Class... 3 Create a Minimum Threads Constraint... 3 Create a Capacity Constraint... 3 Remember to adjust the sizing of other resources to handle concurrency needs... 4 Monitor Thread Pool & Resource Usage under Load... 4 Tuning stuck thread detection behavior... 5 Use Java s Built-in JVM monitoring facility... 5 About Active Endpoints... 6 Oracle WebLogic Thread Pool Tuning The description below explains how to configure Work Managers in Oracle WebLogic Server 10.3.x. Since the specific tuning parameters will vary from application to application and server to server, the specific values here are strictly examples and not specific recommendations. To determine the proper values for your application, you need to follow the steps outlined below, monitor your servers under load and make the appropriate adjustments until you are satisfied with the throughput. For detailed explanations of each of these components, please refer to the Oracle WebLogic Server documentation at http://www.oracle.com/technetwork/middleware/weblogic/documentat ion/index.html. Configure ActiveVOS to use its own Work Manager Using a separate Work Manager is highly recommended as it allows you to monitor and tune the thread pool used by your application. Create a global work manager for the ActiveVOS application 1. Open the Oracle Weblogic Admin Console 2. Navigate to Environment -> Work Managers 3. Choose New to create a Global Work Manager 4. Name your new work manager wm/activevos 5. Target the Work Manager to the same deployment target(s) as the ActiveVOS Server 6. Save Copyright 2010, Active Endpoints, Inc. Page 2 of 6
Create a Request Class The Request Class configuration allows you to choose how the self-tuning thread pool optimizes itself, based on your requirements. In this example, we would like to tune our thread pool to meet a target response time of 1 second per request. 1. Open the Work Manager you just created 2. Choose New next to the Request Class drop-down 3. Choose Response Time as the type 4. Enter a descriptive name for the new class 5. Enter the goal in milliseconds, in our example use 1000 for 1 second 6. Target to the same deployment target(s) as the ActiveVOS Server 7. Save Create a Minimum Threads Constraint This sets the minimum number of threads allocated to this Work Manager. Set this to a value that reflects the anticipated usage for your application to avoid having to deal with deadlocks. 1. Open the Work Manager you just created 2. Choose New next to the Minimum Threads Constraint dropdown 3. Enter a descriptive name for the new class 4. For an initial value, number of (threads per user) x (expected number of concurrent users) 5. Target to the same deployment target(s) as the ActiveVOS Server 6. Save Create a Capacity Constraint This sets the maximum number of queued (waiting) requests allowed before the server starts rejecting new requests. Setting a value here will prevent the server from getting into a bind where too many requests are backed up to the point where it can no longer do any work at all and needs to be bounced. 1. Open the Work Manager you just created 2. Choose New next to the Capacity Constraint drop-down 3. Enter a descriptive name for the new class 4. Enter the highest value of waiting threads that can be tolerated before the server gets completely flooded Copyright 2010, Active Endpoints, Inc. Page 3 of 6
5. Target to the same deployment target(s) as the ActiveVOS Server 6. Save Remember to adjust the sizing of other resources to handle concurrency needs Increasing the number of threads will increase the demand for other resources. Adjust the number of database connections supported in the connection pool to account for the increased concurrency. If there aren t enough database connections, threads will get backed up waiting to get to the database and you can exhaust the thread pool quickly. Increasing the thread count also increases demand for memory and CPU resources. Monitor usage of these resources and crank up the heap size to accommodate the load until your server starts paging to disk. When running with lots of threads and a large heap, you may also need to adjust the I/O and GC settings to avoid spending a lot of time managing the heap and dealing with network bottlenecks. Restart the servers. See the recommended settings for your platform in the Oracle WebLogic server tuning guide at http://download.oracle.com/docs/cd/e14571_01/web.1111/e13814/toc. htm. Monitor Thread Pool & Resource Usage under Load What you want to avoid is having a large pool of requests backing up in the server. This condition often leads to poor performance and deadlocks. In fact, it s much better to just reject new requests before it gets to the point of deadlock. The key server metrics here are the queue depths on the execute thread pools and database connections. Using the admin console, Navigate to Environment->Servers- >server to monitor->monitoring->threads Watch the value of the Queue Length column. On a healthy server, this number should be very small, preferably 0. If it s not, check the server s CPU and Memory utilization. If they re both below 100%, increase the values for the work manager in the ActiveVOS Copyright 2010, Active Endpoints, Inc. Page 4 of 6
console. If it s at 100%, your environment is maxed out and adding more threads is not going to help. Tuning stuck thread detection behavior Oracle WebLogic Server automatically detects when a thread in an execute queue becomes stuck. Because a stuck thread cannot complete its current work or accept new work, the server logs a message each time it diagnoses a stuck thread. Oracle WebLogic Server diagnoses a thread as stuck if it is continually working (not idle) for a set period of time. You can tune a server s thread detection behavior by changing the length of time before a thread is diagnosed as stuck, and by changing the frequency with which the server checks for stuck threads. To configure stuck thread detection behavior, in the admin console, click Lock & Edit and navigate to Environment > Servers > your server > Configuration > Tuning, update as necessary: 1. Stuck Thread Max Time -- Amount of time, in seconds, that a thread must be continually working before a server instance diagnoses a thread as being stuck. 2. Stuck Thread Timer Interval -- Amount of time, in seconds, after which a server instance periodically scans threads to see if they have been continually working for the configured Stuck Thread Max Time. 3. Click Save. Click Activate Changes. Use Java s Built-in JVM monitoring facility The Java 1.5 JDK provides a JVM monitoring feature that gives a nice graphical view of memory and thread usage. Add -Dcom.sun.management.jmxremote.port=<portnum> to the server s startup parameters. Monitor the server using JConsole found under $JAVA_HOME/bin. If you are using JDK 6, then you can instead use the excellent VisualVM monitoring tool, which includes the functionality of JConsole, and is included with the JDK. The JConsole utility is also available in a typical Oracle WebLogic installation, for example at C:\weblogic10_3_2\Middleware\jdk160\bin or C:\weblogic10_3_2\Middleware\jrockit160\bin Copyright 2010, Active Endpoints, Inc. Page 5 of 6
About Active Endpoints Active Endpoints (www.activevos.com) ActiveVOS is the leader in serviceoriented BPM software for process automation. ActiveVOS empowers project teams to create business process management (BPM) applications using services, making their businesses more agile and effective. ActiveVOS promotes mass adoption of SOA-enabled BPM applications by focusing on accelerating project delivery time with a complete, affordable and easy-to-use system. Active Endpoints is headquartered in Waltham, MA with development facilities in Shelton, CT. To find out how Active Endpoints can help your business, visit http://www.activevos.com, call +1 781 547 2900 and press 1 for Sales, or email us at info@activevos.com. Copyright 2010, Active Endpoints, Inc. Page 6 of 6