Performance Testing and Optimization in Web-Service Based Applications Mesfin Mulugeta mesfin.mulugeta@blackboard.com Sr. Software Performance Engineer
Goals of the Presentation Brief introduction to software performance engineering and performance testing Web service performance metrics Web Service Performance testing and optimization tools Load Generation & Profiling Tools Non Goals Compare the performance of web services to other distribution technologies
Presentation Outcomes At the end of the session, participants will be able to do the following: Get acquainted with the important steps in a software performance engineering methodology Apply load generation and profiling tools in the performance testing and optimization of web service interfaces
Software Performance Engineering (SPE) A systematic, quantitative approach to constructing software systems that meet performance objectives (PO s). As part of SPE, we focus on: Identifying Critical Use Cases (e.g. opening a grade center) Assessing Performance Risk Establish PO s PO s are usually specified in: response time, Throughput, Resource Utilization, workload Integrate performance engineering into software development process
Performance Engineering in Blackboard TM SDLC Requirement Development - Assess Performance Risk - Mitigate Performance Risk -Identify critical se cases for Analysis - Establish performance objectives - Review Technical Design Document - Warn unacceptable anti-patterns - Model/Prototype Design Development -Profile the critical scenarios to identify inefficient executions - Code Review - Identify scalability issues in time - Develop Performance Data Model - Performance Verification Testing - Regression Testing - Focused Testing - Benchmarking & Certification - Platform and Advanced Configurations Testing Deployment (General Availability) Maintenance - Sizing & Capacity Guidance - Get Performance Problems - Diagnose Problems
Software Performance Testing (SPT) SPT is performed to determine how fast some aspect of a system performs under a particular workload Purpose Used to demonstrate that the system meets performance criteria Helps identify how each scenario performs How is SPT related to: Load Testing, Stress Testing, and Performance Tuning?
Performance Testing Tools Load Generators, Monitors & Frameworks LoadRunner JMeter Rational Performance Studio soapui Profilers JProbe Eclipse TPTP jmap, jhat, jstat, jconsole SAP Memory Analyzer
Performance Testing & Optimization Process Identify Performance Scenario Define Performance Objectives Specify & Implement Data Model Refactor or Performance Tune Used to mimic the real world scenario Specify Workload Problem could be at application, database, operating system, or network level. Use Profiler for diagnosing the problem Ensure some randomness Run the test by using appropriate tools No Report Result! Gather data & analyze Is Performance OK? Yes
Web Services a software system designed to support interoperable machine to machine interaction over a network. Uses open standards like XML, SOAP, WSDL Web-service based applications are applications built by using web services provided by third-parties.
Blackboard Web Services
A Web-Service Based Application Scenario Library University Portal loadcontent savecontent ContentWS (Bb Web Service) Web Service loadcontent
Why we need to measure web service performance? Clients need to know the response time, throughput of the APIs Multiple service providers offering same or similar service take advantage SPs need to know the resource demands of the APIs at different workloads
Web Service Performance Metrics Server Side: can be gathered by stress testing on the server end. Important parameters are: Server throughput (measured as the number of requests per second) Latency (time taken between the service request arriving and the request being serviced.) Client Side: client side monitoring or distributed load testing can be used to gather data. Important parameters are: Latency (WS processing time + network latency). This is the time taken for a service call to return the earliest response bytes Throughput (the average byte flows per unit of time, including latency) Error rate (identifies the dependability of the service)
JMeter (1) Apache Jakarta Project Used as a unit test for database connections, web services, FTP, LDAP, JMS, etc. Configuring JMeter for a web service test plan Thread Group No. of users to simulate Ramp-up period Samplers: define input from thread groups Web Service (SOAP) Request Listeners visualize test runs Graph Results
JMeter (2)
JMeter (3)
soapui A java-based web service testing tool Used for inspecting, invoking, developing, simulating, mocking, and load-testing web services
JMeter and soapui JMeter A general purpose load-testing tool. Used to load and performance test HTTP, JDBC, JMS, Web Service (SOAP), etc. Supports distributed load testing (important when simulating very high number of users) soapui Specific to web services Has a more user friendly IDE Doesn t support distributed load testing Plug-in support for most IDEs including Eclipse Automation support (Groovy scripting)
Profilers There are a number of open source and commercial profilers Profiling helps in finding: - Performance bottlenecks - Excessive memory consumption - Memory leaks Profiling Web Services Web services might be available in different ways. Developed from scratch Java classes, business components (e.g. EJB) are turned into web services
Eclipse TPTP TPTP - Test & Performance Tools Platform Can be used to profile local Java applications, complex applications running on multiple hosts, and Eclipse plug-ins Installations & Configuration Easier with the Eclipse Europa Version Pros: a free plug-in for Eclipse Cons: tightly integrated with the Eclipse Project
Using Eclipse TPTP
JProbe Offers memory analysis, performance analysis, and code coverage Can profile J2EE and J2SE based applications Can profile JUnitPerf tests Can be integrated with Eclipse
JProbe Method List View Snapshot (an example from Blackboard data generator tool) Method Cumulative Time Method Time Package No. of Calls Cumulative Objects Method Objects
Summary SPE integrates performance engineering into the software development process Clients and Service providers need to know the performance of web services for a number of reasons. Web service performance metrics are classified as client-side and server-side. Important parameters are: latency, throughput, and error rate. Load generators are used together with profiling tools to measure and optimize the performance of web service based applications.
References http://jakarta.apache.org/jmeter/ http://www.soapui.org/