Measurement-based Performance Analysis of E-commerce Applications with Web Services Components Venu Datla Katerina Goseva Popstojanova Presented at the IEEE Conference on e-business Engineering (ICEBE 5), Beijing, China, October 5 This work was funded by NSF CAREER grant number CNS-7715 and NASA OSMA SARP grant managed through NASA IV&V Facility Copyright K.Goseva 9 CS 73 Software Performance Engineering E-commerce applications & Web services Modern e-commerce applications are large-scale, distributed and depend on various inter-enterprise and intra-enterprise services for execution Integration with the application becomes complex task since these services are often developed on different platforms, using different programming languages and technologies The promise of Web services Interoperability Dynamic and flexible application integration Loose coupling Service oriented architecture Slide What are Web services? W3C Definition: A Web service is a software system designed to support interoperable machine-to-machine interaction over a network. It has an interface described in a machine-processable format. Web Service Description Language (WSDL) is an XML grammar for specifying the properties of a Web service (messages exchanged by the service, operations supported by the service, protocol bindingds and endpoints of the service, etc) Universal Discovery, Description, and Integration (UDDI) provides a standard way for businesses to publish and discover Web services Simple Object Access Protocol (SOAP) is a standard for sending messages and making Remote Procedure Calls over Internet. It uses XML for data encoding and HTTP as the transport protocol Other transport protocols (FTP, SMPT, or even TCP/IP sockets) may be used Slide 3 1
Web services communication stack Overheads: Marshalling and unmarshalling between objects and XML elements Time required to encode and parse SOAP messages Time consumed in encrypting and decrypting XML messages for Web services using security Slide Motivation Performance is an important quality attribute of Web services Only a few papers have focused on performance evaluation of Web services in the past throughput and overall system response time Slide 5 Contributions Measurement-based performance study of a threetier e-commerce application which includes Web services components at software architectural level we instrumented the application to log components response times in Application server logs at hardware resource level we used Windows performance monitor to capture the hardware resource level performance Implemented a session oriented workload generator based on TPC-W specification Measured application performance under different workload profiles and different workload intensities Slide
Benefits Our empirical results contribute toward quantifying the overhead introduced by Web services Help identifying software components and hardware resources which are bottlenecks in the system For example, we show that Web services components have significantly higher response time under the Ordering profile This information is important for system designers since customers in Ordering profile tend to have more ordering activity and generate revenue Slide 7 Prototype description We developed a prototype of an e-commerce application A travel agency that offers flight booking services to its customers and uses secure transfer for confidential data The prototype has a three-tier architecture user interface layer (JSP Web pages) business logic layer (EJB components, Web services components) data layer (Oracle database) The application is implemented using JEE technologies which facilitate development of scalable, robust, multi-tiered enterprise systems Slide 8 Software architecture The user interface layer consists of a set of Web pages Home, Search, Search results, Shopping cart, Customer login, Check credit, Process orders The business logic layer contains components that implement the core functionality of the travel agency application Flight-WS: Web service that searches for flight schedule information Currency-WS: Web service to compute exchange rate between currencies Credit-WS: Web service to check customer credit information Customer-EJB: EJB component which contains customer information Login-EJB: EJB component to authenticate customers Order-EJB: EJB component to store order information The data layer consists of a backend relational database management system which stores the persistent information in the form of database tables Slide 9 3
Development & Deployment We have developed all Web services components and hosted them locally The first version of our prototype integrated publicly available Web service for the Flight-WS this service had low availability when it was available, the server responded with server error whenever more than five simultaneous search request were generated This does not seem to be an isolated incident A study conducted in 1 showed that 8% of the production UDDI registry had links that were unusable A more recent study reported similar findings - over the period of six months (Aug 3 to Jan ) 7% of Web services registered in the UDDI registry were invalid (their WSDL files were either inaccessible or not registered) Slide 1 Development & Deployment By hosting all software components locally we avoid accounting for network latency which is beyond our control This does not limit the scope of our research since our goal is to study the contribution of software components to the response time of e-commerce interactions at server side rather than to study end-to-end response time as perceived by the user Even more, hosting all components locally supports experiments with higher workload which may not be possible with publicly hosted Web services components Slide 11 Deployment diagram Slide 1
Implementation details We use JEE, a widely used standard which facilitates development of scalable, robust, multi-tiered enterprise systems User interface layer Java Server Pages (JSP) which is a JEE technology for creating dynamic Web content Tomcat v5. as a Web server Business logic layer Web services and Enterprise Java Beans (EJB) JEE 1. Application Server Platform Edition 8. Data layer consists Oracle 9i Release Slide 13 Deployment details Workload generator runs on a laptop: 1. GHz Pentium M processor with 51 MB RAM Web server and EJB components run on the Application server 1: 3 GHz Pentium processor and 1 GB RAM Web services components run on the Application server : 3GHz Pentium processor and 1GB RAM Database server runs on a different machine with the same configuration and 1 GB disk drive All machines run Windows operating system and are connected through Ethernet LAN with 1 Mbps speed Slide 1 Why experiments with synthetic workload? The most realistic way to study performance is under the real workload. However, the real Web workload is complex, cannot be controlled and is not repeatable Alternative synthetic workload based on performance benchmarks Measurable and repeatable Enables comparative studies of products Vendors, developers, and users run benchmarks to accurately test new systems, pinpoint performance problems, or assess the impact of modifications to a system Slide 15 5
Workload generation: TPC-W TPC-W is a transactional Web benchmark which specifies an online bookstore e-commerce application simulates activities of a business oriented transactional Web server follows a closed loop model: a client sends new request only after receiving response to the previous request session oriented workload three workload profiles with different percentages of browse and buy operations Browsing Shopping Ordering Slide 1 Workload characterization in TPC-W We have implemented the workload generator based on TPC-W specification for our prototype The workload is modeled with a Discrete Time Markov Chain (DTMC) which characterizes users request pattern within a session In TPC-W it is called Customer Behavior Model Graph This represents the user view on the system Slide 17 TPC-W profiles We conducted measurements and analyzed results for two workload profiles: Browsing and Ordering Browsing profile (79-1) Ordering profile (5-5) Home 1 17 Search 3 17.5 Search Results 8 15.5 Shopping Cart 1 1 Customer login 3. 13 Check credit.9 11.5 Process order.9 11.5 Slide 18
Other TPC-W specifications Three phases: rump-up, steady-state, and ramp-down The data is collected during the steady-state interval with duration of 3 minutes The size of the database must be scaled accordingly to the number of clients We ran experiments with 5, 1, 15, and clients Following the TPC-W specification we populated the database with a customer table with 57, rows The average think time is 7 seconds and the average user session duration is 11 minutes For each Web interaction, the TPC-W measures at the client side (i.e., Emulated Browser) the Web Interaction Response time (WIRT) Slide 19 Measurement methodology We instrumented the application to capture the performance data for components running on different machines Service layer software architectural view System layer deployment view Each request involves processing by one or more components Search request: Search page Shopping Cart request: Shopping Cart page Customer Login request: Customer Login page Home request: Home page and Customer-EJB Search Results request: Search Results page and Flights-WS Check Credit request: Check Credit page, Login-EJB, and Currency-WS Process Order request: Process Order page, Credit-WS, and Order-EJB Slide Measurement methodology Component execution times are logged in the Application server logs using java.util.logging API Application server log record format We used scripts written in AWK scripting language to parse the Application server logs and extract response times of components participating in each request Slide 1 7
9 th percentile response time measurements Critical requests (user layer): Search Results, Process Order & Check Credit Critical components (software architectural layer): Flight-WS, Currency-WS & Credit-WS Slide Analysis of the results Response time increases with workload (number of customers) for both profiles Response times for 5,1 and 15 clients are approximately the same for the Browsing and Ordering profiles In case of clients response times for the Ordering profile are approximately % higher than for the Browsing profile Critical requests (user layer): Search results, Process order, and Check Credit Critical components (software architectural layer): Flight-WS, Currency- WS, and Credit-WS They have higher response time than other components (-8% of the overall response times of Search results, Process order, and Check Credit is spent in executing Web services) and are more sensitive to the workload This is due to the time used for parsing the XML requests and responses and the higher database activity Slide 3 Measurement methodology For hardware resource monitoring we used Windows performance monitoring tool On each machine we recorded The percentage of non-idle processor time spent in user mode (%User Time) The rate of read and write operations on the disk (Disk Transfers/sec) Slide 8
Results: Hardware resource measurements %Processor Utilization (Ordering profile) 5 3 1 13 39 5 5 78 91 1 Time (sec) %Processor Utilization (Browsing profile) 5 3 1 13 39 5 5 78 91 1 Time (sec) CPU utilization increases significantly for clients One of the reasons for increased response time of Web services components under higher workload is the overhead due to parsing and encoding the XML messages which leads to increased CPU utilization Disk Transfers / Sec (Ordering profile) 1 1 8 1 1 3 8 7 8 9 Time (sec) Disk Transfers / Sec (Browsing Profile) 1 1 8 1 1 3 8 7 8 9 Time (sec) EJB and Web services components perform operations on the backend database Disk transfers per second for Ordering profile are higher than for Browsing profile. The difference increases with the workload intensity, which explains the increase in response times of EJB and Web services components. Slide 5 New version of the application server We deployed the same implementation of the prototype on a newer version of the application server, JEE 1. Application Server - Platform Edition 8.1 This server was the latest available production version in December 5 The Web server, database server, operating systems, and the hardware configuration remain the same The results show that the new version of the application server improves the response times of Web interactions significantly Since Web services components experience the most dramatic improvements, we present more details for the Search Results, Credit Check, and Process Order interactions which include Web services components to fulfill the user requests Slide 9 th percentile response time measurements under the new application server Response Time (ms) Search Results request 5 5 35 3 5 15 1 5 5 1 15 5 1 15 Ordering Profile Browsing Profile Response Time (ms) Credit Check request 5 35 3 5 15 1 5 5 1 15 5 1 15 Ordering Profile Browsing Profile Flights-WS Search Results Page Currency-WS Login-EJB Check Credit Page Process Order request Response time of the Web services components 5 is significantly more sensitive to the workload 35 3 than the response time of other components 5 under the new application server as well, that is, 15 the Web services components still scale worse 1 than the other components 5 5 1 15 5 1 15 Also, the response time of the Web services Ordering Profile Browsing Profile Components is higher in the Ordering profile than in the Browsing profile, especially for higher Copyright K.Goseva Currency-WS 9Login-EJB Check CS Credit 73 Page Software Performance workload Engineering (i.e., 15 and customers). Slide 7 Response Time (ms) 9
Response times for Flight-WS normalized with respect to response time for 5 customers Normalized response time 5 3 1 Normalized response time 5 3 1 5 1 15 5 1 15 Ordering Profile Brow sing Profile JEE 1. Platform Edition 8. JEE 1. Platform Edition 8.1 JEE 1. Platform Edition 8. JEE 1. Platform Edition 8.1 Platform Edition 8.1 application server improves the response times of Web services components, as well as their scalability Slide 8 Response time improvement for Flight-WS component % of response time improvement 5 3 1 5 1 15 % of response time improvement 5 3 1 5 1 15 Ordering Profile Browsing Profile JEE 1. Application Server Platform Edition 8.1 improves the response time of Flight-WS component from 11% to % The improvement increases with the workload intensity which leads to better scalability of the Web services components The amount of improvement is not very sensitive to the workload profile The results for Currency-WS and Credit-WS components are consistent with these observations: response times of Currency-WS and Credit-WS components improve 1% - 3% and % - 35%, respectively Slide 9 Conclusion In contrast to the related work which evaluated the overall application response time, our study includes measurements and analysis of the server-side performance at different levels Web request level allows us to study the user point of view Software architecture level allows us to study the distribution of the Web request response time among different components used to process the request Hardware resource level provides additional insights and helps explaining the observed phenomena Enables system designers and application developers to improve the performance in a cost effective manner Slide 3 1
Conclusion Web services tend to become bottlenecks of the system, particularly under heavier load The phenomenon is attributed to the overhead introduced by additional processing of XML messages and basically, is the price paid for interoperability Possible solutions develop more efficient XML parsers Incorporate better mechanisms for encoding and decoding of SOAP messages The response time of the Ordering profile under higher workload becomes significantly worse than the response time of the Browsing profile Main reasons: higher database activity and contention for database resources which affect the performance of EJB components and even more the performance of Web services components Slide 31 Conclusion Software architectural level and hardware resource level measurements provide essential information, such as identification of critical software components and hardware resources, that can be used by system designers and application developers to improve the product performance Our latest results emphasize the importance of the serverside infrastructure selection, in addition to the software architecture selection, because both can profoundly affect the resulting performance and scalability of the application For our prototype, using the newer version of the JEE 1. application server led to improvement of Web services response times up to % under higher workload It follows that a simple upgrade of the server-side infrastructure, without additional development and implementation efforts, is a cost effective way to provide significantly improved performance and scalability levels Slide 3 11