Performance Comparison of Java Application Servers

Transcription

1 Buletinul Stiintific al Universitatii Politehnica din Timisoara, ROMANIA Seria AUTOMATICA si CALCULATOARE Transactions on AUTOMATIC CONTROL and COMPUTER SCIENCE Performance Comparison of Java Application Servers Bogdan Pop, Radu Medeşan and Ioan Jurca Department of Computer Science, Politehnica University of Timişoara, Faculty of Automatics and Computers, 1900 Timişoara, Romania Abstract This paper provides a comparative analysis of the performances of some of the most representative Java Application Servers. Every Web application, whether a shopping site, a portal, an online customer service system, a wireless site, or anything in between, has a key piece of software in common: an application server, which manages important aspects of an application, such as transactions, concurrency, scalability, persistence, security. The application servers are performing the work behind a Web site. Any Internet-based application that generates personalized or dynamic content relies on them [1]. In the following sections after a short description of the J2EE servers features and components, are presented the results obtained after doing two kinds of tests: for a business method of a session bean and work under stress conditions. After testing these four application servers our results suggest the most likely to be considered when dealing with e-commerce application: BEA WebLogic Application Server. Keywords: J2EE, Application servers, performance, comparison, EJB I. INTRODUCTION The application servers use a component-based development model, which separates business logic from presentation logic. This compartmentalization lets programmers retool large Web sites quickly: New code modules can be dropped in as needed, which means one can avoid rebuilding an entire system or searching through the spaghetti-code of scripts to find what has to be updated. Because of that, these products are the right choices for complex, often team-written Web applications such as high-volume e-commerce sites. By far the most common component object framework supported is Enterprise JavaBeans (EJBs). These come in three flavors: session beans, which perform business logic for an individual client during its browsing session; entity beans, which represent persistent data stored in one row of a database table, and message-driven beans, which use JMS (Java Message Service). The Java 2 Enterprise Edition (J2EE) specification (encompassing EJBs, servlets, JDBC and several other technologies) has been astonishingly successful at dominating the application server market. The application servers are reliable and scalable. One way to obtain reliability is through support for application server clusters. If one machine goes down, processing is automatically redirected to other systems without end-user interruption or data loss. Clustering can also help a site scale to handle increasing requests, but some products achieve scalability with multiple CPUs on large symmetric multiprocessing systems. Application servers are quickly conquering two other market niches: transaction monitors and enterprise application integration (EAI) products. Transaction monitors such as BEA's Tuxedo or IBM's CICS have traditionally provided connection pooling and state management. These features now appear in application servers. Application servers also provide connectivity to back-end systems such as databases, enterprise resource planning systems, mainframes, and message queuing systems. [1] Application servers are taking over other tangential markets, too, such as portals, wireless development servers, storefronts, and B2B integration products based on the foundation of their application server technology. In order to offer selection criteria based on the nature and the purpose of the application we have reviewed application servers from BEA, Borland, JBoss (open source) and the Reference Implementation from Sun. The performance tests were done using Java programs (EJBs applications) built in order to test the servers at heavy loads. II. FEATURES OF THE TESTED APPLICATION SERVERS Although every application server supports the J2EE specification, there are several major differences between them, depending on the EJB version, servlet version, supplied databases, supported operating systems, user interface, tools, etc. These basic features are illustrated in Table 1.

2 TABLE 1 Basic features of the Application Servers Features / Inprise Application Application Servers Server 4.1 BEA WebLogic 6.0 J2EE RI 1.3beta JBoss 2.0 Windows 9x, Windows NT, 2000 Windows NT 4.0 Windows 2000 Windows NT 4.0 Windows 2000 Supported OS Solaris Solaris Solaris 7,8 Any OS with HP-UX HP-UX 11.0 JVM Redhat Linux 6.2 AIX 4.3 Redhat Linux 6.2 Redhat Linux 6.2 EJB Version EJB 1.1 EJB 2.0 EJB 2.0 EJB 1.1 JDK Version Jdk Jdk Jdk Jdk J2EE licensed/certified Yes/Yes Yes/Yes Yes No/No Servlet version/jsp version Servlet 2.2/ JSP 1.1 Servlet 2.2/ JSP 1.1 Servlet 2.3/ JSP 1.2 Servlet 2.2/ JSP 1.1 (with Tomcat integration) Wizards Yes No Yes No Supplied database access JDataStore Cloudscape v3.6 Cloudscape v3.6 Hypersonic Remote console/full command line Yes/Yes Yes/Yes No/Yes No/Yes Monitoring tools Yes Yes No No Automatic deployment No Yes No No Hot deployment of EJBs No No No Yes Other relevant details about the four application servers are as follows: A. Inprise Application Server 4.1 The Web container of the Inprise Application Server [1] provides beside the basic features also the following: 1) A graphical interface for server configuration and administration. 2) Support for both standard (http) and secure (https) web engines. The secure web engine requires that the Inprise Security Service be installed. The Enterprise Java Bean (EJB) container offers beside the basic features the following: 1) Command line deployment utilities are provided such as java2iiop, Verify, and dd2xml. 2) It is based on VisiBroker for Java 4.1 with RMI/IIOP, Objects By Value (OBV), Portable Object Adapter (POA), and other VisiBroker features. 3) JNDI over CosNaming. 4) It offers a suite of components that works with JDBC 2.0 and uses XA protocol to enable full Two Phase Commit (2pc). 5) Support for transactions using JTS/JTA APIs. The EJB Container bundles an embeddable One Phase Commit (1PC) transaction engine. 6) Fault Tolerance and Load Balancing based on Smart Agent. 7) Pluggable Container Managed Persistence (CMP) provider mechanism. The EJB Container bundles a default CMP implementation based on "raw" JDBC. 8) JDataStore is bundled as a session state storage repository as well as a built-in RDBMS for CMP. B. BEA WebLogic 6.0 WebLogic Server [2] include beside the basics the following: 1) Rich client options - WebLogic Server supports Web browsers and other clients that use HTTP; Java clients that use RMI (Remote Method Invocation) or IIOP (Internet Inter-ORB Protocol); and mobile devices that use (WAP) Wireless Access Protocol. Connectors from BEA and other companies enable virtually any client or legacy application to work with a WebLogic Server application. 2) Enterprise e-business scalability - Critical resources are used efficiently and high availability is ensured through the use of Enterprise JavaBean business components and mechanisms such as WebLogic Server clustering for dynamic Web pages, backend resource pooling, and connection sharing. 3) Robust administration - WebLogic Server offers a Web-based Administration Console for configuring and monitoring WebLogic Server services. A command-line interface for configuration makes it convenient to administer WebLogic Servers with scripts. 4) E-commerce-ready security - WebLogic Server provides Secure Sockets Layer (SSL) support for encrypting data transmitted across WebLogic Server, clients, and other servers. WebLogic security realms feature user authentication and authorization for all WebLogic Server services. External security stores, such as Lightweight Directory Access Protocol (LDAP) servers, can be adapted to WebLogic realms, enabling single sign-on for the enterprise.

3 C. Java2 Enterprise Edition Reference Implementation 1.3 beta This release of the Reference Implementation from SUN [3] implements the following extended features: 1) Enhancements to the Servlet and JavaServer Pages (JSP) technologies with the addition of the Tomcat 4.0 container. 2) Servlets support application level filters and application event listeners and JSP pages support XML page syntax. 3) The J2EE Connector Architecture provides additional connectivity to existing enterprise information systems 4) Java API for XML Parsing (JAXP) is also included. 5) Enhancements to the Enterprise JavaBeans component model include message-driven beans, interoperability between EJB containers, and container-managed persistence version 2.0. D. JBoss (open source) JBoss [4] is an implementation of the EJB 1.1 specification, that is, it is a server and container for Enterprise JavaBeans. In this it is similar to Sun's `J2SDK Enterprise Edition' (J2EE), but JBoss is much more singleminded than J2EE. JBoss provides only an EJB server; it does not include support for JSP, SSL, and all the other protocols that the Sun product can handle. This means that it is smaller in memory and in disk space requirements. JBoss will run very effectively on a machine with 64 megabytes of RAM, and requires only some megabytes of disk (including source code). Because of its small memory footprint, JBoss starts faster than other servers. There is a built-in SQL database server for handling persistent beans, and this starts up automatically with the server. One of the nicest features of JBoss is its support for 'hot' deployment. What this means is that deploying a Bean is as simple as copying its JAR file into the deployment directory. If this is done while the Bean is already loaded, JBoss automatically unloads it, and then loads the new version. JBoss provides a GUI XML editor used to configure the deployment descriptor. JBoss is distributed under the GNU public license, which means that it is free, even for commercial work, and is likely to remain that way. III. PERFORMANCE TESTS We pushed the Java application servers to their limits and found significant differences in how they handle heavy loads. Our tests used Java programs that allowed us to measure the desired performances. The above-mentioned servers have been tested on Dell computers equipped with an Intel Pentium III processor (800MHz), 128 MB RAM, 20GB hard disk and Redhat Linux (6.2 and 7.0) as operating systems. The Oracle server, running on an Intel Pentium III processor at 667MHz with 256 MB RAM and 2x 16GB SCSI hard disks with RAID technology, plays an important role. The Oracle software version is We ran two tests: 1) The for a business method of a session bean, which measures the average time required by the applications server to execute a simple business method, and 2) Working under stress conditions which measures the servers capabilities to handle multiple connections and the capacity to work under stress. A. Average for a business method of a session bean The purpose of this benchmark is to evidentiate how much time is required for a server to execute a simple business method. The has three major components: 1) the time required to pass to server a call request for a business method 2) the on server side to process the business method 3) the time to transfer the results to the client. The test program is a client for the HelloWorld session bean, which is a simple stateless session bean with only one business method: sayhello(). This method returns the string Hello world! when called. After creating a bean, the program calls the bean s sayhello() method for 5, 10, 50, 100, 500, 1000, 5000, 10000, or times, specified as a run-time parameter. After running the program the following results were obtained: TABLE 2 Average for a method, expressed in milliseconds Number of calls Server Inprise Application Server WebLogic Application Server J2EE RI JBoss Figure 1 shows that under these conditions, the best performance is obtained with WebLogic server.

4 Test results After running the program the following results were obtained for each application server. The values represent the time spent by the client to obtain a reference of the home interface, the time for creating an entity bean and the of the test sequence. All values are measured in milliseconds. 1) Inprise Application Server Figure 1. The graphical representation of the test results. B. Testing servers under stress conditions This benchmark illustrates the way that the server handles multiple connections and its capacity to work under stress. For this, the Stress client creates a number of clients (5, 10, 50, 100, 500, 1000 or more specified at run-time as the first parameter), every client being represented by a thread. Every client thread creates an AccountBean an entity bean with container managed persistence- and the bean s id is unique and generated by the thread number. If Stress is running more than once at a time, then the beans created by threads with the same number would have the same id, so it is necessary to introduce a second parameter: idoffset and its value will be added to the number of the thread in order to obtain the new id. After creation, the client starts a sequence of business methods calls, as follows: for(long i=0;i<loop;i++){ account.getname(); account.getbalance(); account.getid(); account.deposit( ); account.getid(); account.getbalance(); account.withdraw( ); account.getbalance(); } The sequence is repeated loop times in order to ensure that every bean is working, because it can happen for a large number of beans for the first one to finish the sequence before the last bean is created. The value of loop represents the third parameter of the program. So, the Stress client ensures that for a convenient amount of time the server contains the desired number of beans and all beans perform data access. During this period, another client the Tester programcreates another bean and executes the same sequence of calls as above, measuring the elapsed time. Although the Inprise Application Server has the best for method calls under stress, it has failed for 1000 or more concurrent clients. It was tested using the default datasouce JDataStore- because it could not handle multiple concurrent connections using Oracle as datasource. IAS presented a strange behavior for beans loaded into server, throwing a NameNotFoundException every second time when running the test client. That s why it was tested loading the beans in a stand-alone container. TABLE 3 Test results for Inprise Application Server using JDataStore Lookup time Creation time ) BEA WebLogic Application Server The most reliable server is no doubt BEA s WebLogic Application Server. It is the only one which can resist to over 3000 concurrent clients without throwing exceptions. This server was tested in three configurations: standard configuration (using the default database: Cloudscape) and two configurations with Oracle. As the Cloudscape version shipped with WebLogic does not support more than one connection at a time, the connection pool had the capacity set to 1. The Oracle driver supports multiple connections (maximum 36), so the server was tested using a connection pool with 1 connection and then with 30 connections. The results obtained for the lookup time (the time elapsed until a reference to the EJB s Home is returned after a lookup in the JNDI tree) are presented in the following tables: TABLE 4 The lookup time for a bean deployed on BEA WebLogic Application Server Number of clients Configuration With Cloudscape With Oracle - 1 connection With Oracle - 30 conn

5 TABLE 5 The creation time for a bean deployed on BEA WebLogic Application Server Configuration With Cloudscape connection 30 connections TABLE 6 The for a business method of a bean deployed on BEA WebLogic Application Server Configuration With Cloudscape connection 30 connections ) Java 2 Enterprise Edition Reference Implementation The results obtained with Sun s J2EE RI are often the worst, but it is the easiest to use. The server was tested using the default datasource -Cloudscape- and Oracle. Server s best behavior with a large number of beans was obtained using Cloudscape, because with Oracle, it assigns a database connection to each bean created. While the number of simultaneous connections to Oracle is limited by the driver, the maximum number of beans loaded into server and working concurrently was around 39. Even with Cloudscape, the server fails loading more than 300 beans at a time. 4) JBoss (open source) JBoss has a quite good response, but fails when more than 100 beans are loaded and are working concurrently. It is the less reliable of all severs tested, and it could not manage transactions with Oracle: an exception is thrown indicating that the two-phase transactions are not supported. TABLE 9 Test results for JBoss using Hypersonic Lookup time Creation time C. Comparison of results The shortest time for obtaining a remote reference for the home interface is obtained with the WebLogic Application Server, as shown in fig. 2: The results obtained with Cloudscape are presented in the following tables: TABLE 7 Test results for J2EE RI using Cloudscape Number of clients Lookup time Creation time Figure 2. The lookup time comparison With 500 concurrent beans loaded into server, Inprise Application Server holds the best creation time; the only server that supports at least 1000 beans without throwing exceptions is WebLogic. TABLE 8 Test results for J2EE RI using Oracle Lookup time Creation time Figure 3. The creation time comparison

6 The best performance for the of a sequence of business calls belongs to Inprise Application Server, but this is only until 500 beans are loaded into server. time of Inprise Application Server is lower than the one obtained with Weblogic. IV. CONCLUSIONS While timing has always been a critical factor to adopting new technologies, the accelerated speed inherent in a virtual, information-driven business model has put even greater emphasis on response times. The companies that will efficiently develop and deploy distributed applications for use on both corporate intranets and over the Internet will gain strategic advantage in the information economy. Figure 4. Execution time comparison An overall view of the time elapsed for the execution of the test client is presented in the following charts: By providing the ability to deploy component-oriented enterprise applications across multiple computing tiers in a platform-neutral manner, J2EE can give fast-moving enterprises a significant and measurable competitive edge [6]. Having the best application server which implements the J2EE specifications leads to the top. Choosing the right application server depends on the application one has to deploy. For a beginner in the world of J2EE who doesn t need a complex server to hold simple applications, the best choice is the Reference Implementation from Sun. This is free and easy to use. For better performance and for applications that are not extremely complex, JBoss is the optimal choice. It is also free but is needed more knowledge about J2EE and have to manually create deployment packages. The WebLogic Server is the most reliable server and complex application server and offers the best support for the real-world applications. Although it needs a higher level of understanding of the J2EE concepts, has a complex configuration and is very expensive, this server is the best choice for a secure and fault-tolerant application. Figure 5. Performance for 10 beans REFERENCES [1] Sahil Gambhir Performance Tests: Java Application Servers, PC Magazine May 3, 2001 [2] BEA WebLogic Application Server Documentation [3] Inprise Application Server Documentation [4] J2EE RI Application Server Documentation [5] JBoss Application Server Documentation [6] Developing Enterprise Applications with the Java 2 Platform, Enterprise Edition, Sun Microsystems, 1999 Figure 6. Performance for 100 beans The above charts illustrates that the WebLogic server offers the best performance. An interesting situation appears for a higher overload (like 500 beans), when the total execution