Configuring multiple Tomcat instances with a single Apache Load Balancer

Configuring multiple Tomcat instances with a single Apache Load Balancer How to set up Tomcat and Apache for load balancing HP Software Service Management Introduction... 2 Prerequisites... 2 Configuring the Tomcat instances... 2 Installing Tomcat... 2 Modifying the server.xml file... 2 Creating a Tomcat Service... 3 Installing and setting up Apache... 4 Modifying the http.conf file... 4 For more information... 8

Introduction For large implementation it is often required to set up multiple Web Application Servers with a single Web Server that functions as a load balancer. This document describes how to set up multiple Tomcat Web Application Servers with a single Apache Web Server as the load balancer. Prerequisites Both Tomcat version 5.0.xx (this document is based on 5.0.28) and Apache version 2.xxx (this document is based on 2.053) have to be available. Knowledge of how to set up Tomcat and Apache is required. Configuring the Tomcat instances The steps outlined in this section describe how to configure a single Tomcat instance and have to be performed on each instance. The following files within the Tomcat installation will need to be modified as outlined below: Server.xml Catalina.bat Service.bat The Tomcat5w.exe and Tomcat5.exe will have to be renamed as described below. Installing Tomcat Install Tomcat 5.0.28 according to Tomcat instructions. Install each Tomcat instance into its own directory, such as "C:\Tomcat1", "C:\Tomat2", "C:\Tomcat3", etc. If your installation involves multiple Tomcat Java Virtual Machines (JVMs) on multiple Servers, we still recommend naming the directories in sequential numbers. This will help analyzing log files in case of errors, since the JVM location is printed to the logs. After installation, verify that the Tomcat instance works correctly and that the JAVA_HOME and CATALINA_HOME environment variables are set up correctly by testing the http://<servername>:8080 URL. This URL should bring up the Tomcat screen. Next, duplicate this installation of Tomcat into the "C:\Tomcat2", etc. directories. Modifying the server.xml file The server.xml file of the additional instances needs to be edited to avoid port conflicts. Search for the following line: <Server port="8005" shutdown="shutdown" debug="0"> Then change it to: <Server port="<available port number>" shutdown="shutdown" debug="0">. Note: <available port number> can be any port number that is available on your system. To find out which port numbers are available, go to a DOS prompt on Windows and enter netstat a. This command lists all ports that are currently in use. Write down the port number used, for example this document uses 12005, and replace all references in this document that say 12005 with the port you chose. Next search all references to port 8009 and change that port to 12009 or a port that is available in your environment. 2

Now search for any references to port 8080 and change them to port 8082 or higher. A good naming convention is to give the first Tomcat instance the port number 8080, the second instance 8082, the third 8083 etc. Repeat these steps for all additional instances of Tomcat. Once the port conflicts are resolved, define the jvmroute parameter in the server.xml files to match the workers that will be defined in the Apache configuration file workers2.properties. For example you can name the worker for C:\Tomcat1 tomcat1, C:\Tomcat2 tomcat2, etc. In the first Tomcat instance, search the server.xml file for the Engine tag. Enter the jvmroute= tomcat1 parameter to the tag. The tomcat1 value needs to match the name you define as the worker for the directory of the first instance and match both the worker name you setup in the workers2.properties file. The port for this worker in the workers2.properties must match the 12009 port defined for that instance (or the port you defined for the tomcat2 instance). <Engine name="catalina" defaulthost="localhost" debug="0" jvmroute="tomcat2"> Change the base location of the web applications for all Tomcat instances after the initial install (tomcat2 and tomcat3 etc.). <!-- Define the default virtual host Note: XML Schema validation will not work with Xerces 2.2.--> <Host name="localhost" debug="0" appbase="c:\tomcat1\webapps" unpackwars="true" autodeploy="true" xmlvalidation="true" xmlnamespaceaware="true"> <Context path="/sm7" docbase="c:\tomcat1\webapps\sm7" crosscontext="false" debug="0" reloadable="false" /> Additionally, make these changes to the server.xml file: In the Connector tags the maxthreads, minsparethreads, and maxsparethreads parameters can be set to the following values, adjust them to your environment as needed. maxthreads="450" minsparethreads="75" maxsparethreads="225" The heap size needs to be adjusted according to your environment. In our environment we used between 1 and 1.5 GB. It is very important to calculate the heap size so that sufficient memory is left for the operating system. Creating a Tomcat Service In the \tomcat2 (3/4 )\bin\ directory, edit the catalina.bat file. set JAVA_HOME=C:\j2sdk1.4.2_12 set CATALINA_HOME=C:\Tomcat2 rem Guess CATALINA_HOME if not defined set CATALINA_OPTS=-server -Xms1024m -Xmx1024m -XX:+AggressiveHeap Next, edit the service.bat file in the same directory. In this file, add the port number for the Tomcat instance you are working on to the following lines. These ports must match the connector port in the server.xml file if exist "%CATALINA_HOME%\bin\tomcat5_8082.exe" goto okhome set EXECUTABLE=%CATALINA_HOME%\bin\tomcat5_8082.exe set SERVICE_NAME=Tomcat2 You will need to rename the Tomcat5w.exe to Tomcat<x>.exe, for example Tomcat2.exe according to which instance you are working on. Additionally, rename the Tomcat5.exe to Tomcat5_8082.exe according to the port of the Tomcat instance you are using. From a command prompt navigate to the \Tomcat2\bin 3

Enter the command: service install Tomcat2 Note: Adjust the command and the directory for each instance, eg. Tomcat3, Tomcat4, etc. If the command is run multiple times from the same directory, it will create multiple services for that directory. Installing and setting up Apache Follow these steps to install Apache: Download and install the Apache HTTP binaries. In our environment we used version 2.053. Download the latest mod_jk.dll file from the Apache website: http://archive.apache.org/dist/jakarta/tomcat-connectors/jk/binaries/win32/ Place the mod_jk.dll file in the <Apache Install Directory>\modules directory In the Apache installation, two files, httpd.conf and workers.properties, will have to be modified. Modifying the http.conf file Open the httpd.conf file in the \Apache2\conf directory. The following statements set up the alias for Tomcat to find the default content. Also, it defines the workers file, log file, and debug level. We can also choose to mount certain web context roots to specific workers defined in the workers2.properties. Additionally, if in your mod_jk log file you see messages pertaining to "max thread count reached", you may need to modify the "ThreadsPerChild" parameter in the httpd.conf file. Change the ServerName to <machine name>:80 Add the following to the end of the httpd.conf file. Alias /sm7 "C:/Tomcat1/webapps/sm7" Alias /sc6 "C:/Tomcat1/webapps/sc6" LoadModule jk_module modules/mod_jk.dll JkWorkersFile conf/workers2.properties JkLogFile logs/mod_jk.log JkLogLevel info JkMount /* loadbalancer JkMount /*.jsp loadbalancer JkMount /servlet/* loadbalancer JkMount /sm7/servlet/* loadbalancer JkMount /sm7/*.jsp loadbalancer JkMount /sm7/*.do loadbalancer JkMount /sm7/attachments/* loadbalancer JkMount /sm7/cwc/nav.menu loadbalancer JkMount /sm7/* loadbalancer JkMount /sc6/servlet/* loadbalancer JkMount /sc6/*.jsp loadbalancer JkMount /sc6/*.do loadbalancer JkMount /sc6/attachments/* loadbalancer JkMount /sc6/cwc/nav.menu loadbalancer 4

Start each of your Tomcat instances to verify that they start up successfully. Verify that you can see the Tomcat page when entering http://localhost:8080 or the respective HTTP port for each Tomcat. If any of the sites encounter errors, verify that there are no port conflicts, and research the errors that were issued when you tried to start Tomcat. Next we need to create a workers2.properties file as listed below and place it in the \Apache2\conf directory. Change the worker.tomcat_home parameter to point to your initial Tomcat instance and change your worker.java_home to point to the correct JAVA_HOME. Modify the ports, worker names, and URI settings at the bottom of the file to match your settings as well. ************ Begin worker.properties ************** the following is the connector timeout value, in seconds worker.maintain=1000 Specifies the load balance factor when used with a load balancing worker. Note: ----> lbfactor must be > 0 ----> Low lbfactor means less work done by the worker. Specify the size of the open connection cache. worker.ajp13.cachesize Comment out the worker.tomcat_home should point to the location where you installed tomcat. This is where you have your conf, webapps and lib directories. worker.tomcat_home=c:\tomcat1\ worker.java_home should point to your Java installation. Normally you should have a bin and lib directories beneath it. worker.java_home=c:\jdk1.5.0_10 You should configure your environment slash... ps=\ on NT and / on UNIX and maybe something different elsewhere. ps=\ ------ DEFAULT worker list ------------------------------------------ --------------------------------------------------------------------- The worker that your plugins should create and work with worker.list=ajp13, tomcat2, loadbalancer worker.list=loadbalancer worker.list=ajp13, tomcat1, tomcat2, tomcat3, tomcat4, tomcat5, loadbalancer Defining a worker named ajp13 and of type ajp13 Note that the name and the type do not have to match. worker.tomcat1.port=8009 worker.tomcat1.host=localhost worker.tomcat1.type=ajp13 worker.tomcat1.lbfactor=1 5

worker.tomcat1.cachesize=100 Second tomcat server on server 1 worker.tomcat2.port=12009 worker.tomcat2.host=localhost worker.tomcat2.type=ajp13 worker.tomcat2.lbfactor=1 worker.tomcat2.cachesize=100 Third tomcat on server 1 worker.tomcat3.port=13009 worker.tomcat3.host=localhost worker.tomcat3.type=ajp13 worker.tomcat3.lbfactor=1 worker.tomcat3.cachesize=100 Fourth tomcat on separate Server worker.tomcat4.port=14009 worker.tomcat4.host=new server worker.tomcat4.type=ajp13 worker.tomcat4.lbfactor=1 worker.tomcat4.cachesize=100 Fifth tomcat on separate Server worker.tomcat5.port=15009 worker.tomcat5.host=new server worker.tomcat5.type=ajp13 worker.tomcat5.lbfactor=1 worker.tomcat5.cachesize=100 Load Balancer worker worker.loadbalancer.type=lb worker.loadbalancer.balanced_workers=tomcat1, tomcat2, tomcat3, tomcat4, tomcat5 another new param from the doco to be more conservative worker.loadbalancer.lock=p Add to file worker.inprocess.type=jni worker.inprocess.class_path=$(workers.tomcat_home)$(ps)lib$(ps)tomcat.jar worker.inprocess.cmd_line=start worker.inprocess.jvm_lib=$(workers.java_home)$(ps)jre$(ps)bin$(ps)classic $(ps)jvm.dll worker.inprocess.stdout=$workers.log.home$(ps)logs$(ps)inprocess.stdout worker.inprocess.stderr=$workers.log.home$(ps)logs$(ps)inprocess.stderr *********** Begin uriworkermappings *** Simple worker configuration file 6

Mount the Servlet context to the loadbalance worker /servlet/*=loadbalancer Mount the examples context to the loadbalance worker /ROOT=loadbalancer /ROOT/*=loadbalancer Mount the examples context to the ajp13 worker /examples=ajp13 /examples/*=ajp13 Mount the sm7 context to the loadbalance worker /sm7=loadbalancer /sm7/*=loadbalancer [uri:/jkstatus/*] info=display status information and checks the config file for changes. group=status: [uri:/sm7/servlet/*] info=prefix mapping [uri:/sm7/*.do] info=extension mapping [uri:/sm7/*.jsp] info=extension mapping [uri:/sm7/answers/attachments/*] info=prefix mapping [uri:/sm7/attachments/*] info=attachment mapping [uri:/sm7/cwc/nav.menu] info=extension mapping Mount all jsp pages in the ROOT folder to the loadbalance worker /*.jsp=loadbalancer ************ End worker.properties ************** Once the Apache setup is completed, you can start the Tomcat services and Apache and the load will be distributed between the JVM's defined in your worker list. 7

For more information Please visit the HP Software support Web site at: http://www.hp.com/managementsoftware/support This Web site provides contact information and details about the products, services, and support that HP Software offers. HP Software online software support provides customer self-solve capabilities. It provides a fast and efficient way to access interactive technical support tools needed to manage your business. As a valued customer, you can benefit by being able to: Search for knowledge documents of interest Submit and track progress on support cases Submit enhancement requests online Download software patches Manage a support contract Look up HP support contacts Review information about available services Enter discussions with other software customers Research and register for software training Note: Most of the support areas require that you register as an HP Passport user and sign in. Many also require an active support contract. To find more information about support access levels, go to the following URL: http://www.hp.com/managementsoftware/access_level To register for an HP Passport ID, go to the following URL: http://www.managementsoftware.hp.com/passport-registration.html 2008 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. The only warranties for HP products and services are set forth in the express warranty statements accompanying such products and services. Nothing herein should be construed as constituting an additional warranty. HP shall not be liable for technical or editorial errors or omissions contained herein. HP, AssetCenter, and Service Manager are registered trademarks of Hewlett- Packard Development Company, L.P. 10/2008