Automated performance testing using Maven & JMeter George Barnett, Atlassian Software Systems @georgebarnett
Create controllable JMeter tests Configure Maven to create a repeatable cycle Run this build in your CI server every 8 hours Find performance regressions faster
WHY? Catch regressions quickly Stop hunting for bugs Code with confidence Know the performance cost of features Fearless external dependency upgrades Go home on time
ATLASSIAN 15,000+ customers 100+ developers 8 software streams 1 performance engineer
ATLASSIAN JIRA - Issue management Confluence - Enterprise Wiki Bamboo - Continuous Integration FishEye, Crucible & Clover - Dev Tools Crowd - Single Sign-On.
BEFORE WE BEGIN Continuous Integration is critical Keep code working with Unit and Functional tests Broken or irregular builds means no code to performance test Performance testing is NOT a replacement for other testing (but it helps)
PROCESS Write code (1-6 months) Compile code Send it to QA (or Production?!) Performance Regression PANIC! No performance testing!
A BETTER WAY Write code CI compiles code and runs tests Send it to QA (or Production?!) Performance Regression PANIC! No performance testing!
A BETTER WAY 2.0 Performance test artifacts as soon as they pass Unit and Functional testing Fast feedback for developers No Surprises
TOOLS JMeter Maven & plugins Bamboo & plugins A Profiler
JMETER Apache Project Java Supports HTTP, JDBC, SOAP, XML-RPC, FTP, SMTP, LDAP, JUnit & more.
MAVEN Apache Project Java Build Tool Dependency management Documentation
MAVEN CONCEPTS Project Object Model (POM) Standardised directory structure Build Lifecycle Reporting Lifecycle Plugins
MAVEN POM <project> <groupid>com.atlassian.com.performance.jira</groupid> <artifactid>automated-performance</artifactid> <properties> <test.dataset>/opt/dataset/dataset1.zip</test.dataset> </properties> <build> <!--Main Lifecycle plugins configured here--> </build> <reporting> <!--Reporting Lifecycle here--> </reporting> </project>
MAVEN PLUGINS Chronos Ant-run Cargo Maven Assembly plugin Maven Dependency plugin
BAMBOO Continuous Integration Can run builds on Amazon s EC2 JMeter Aggregator Plugin
A PROFILER Java Profiler CPU Snapshots GC & Memory information Thread usage and contention Triggers & offline sampling
STEP 1 Package the JMeter test Maven Assembly plugin Create a.zip from project directories `mvn deploy` to repository as a SNAPSHOT
STEP 2 Configure the Maven automated build Package application state into zip files Ant-run - application pre-setup Cargo - deploy container Chronos - configure the JMeter tests
MAVEN PROFILES <project>.. <properties> <test.dataset>/opt/dataset/dataset1.zip</test.dataset> </properties> <profile> <id>dataset1</id> <properties> <test.dataset>/opt/dataset/dataset1.zip</test.dataset>.. <profile> <id>dataset2</id> <properties> <test.dataset>/opt/dataset/dataset2.zip</test.dataset>..
MAVEN PROFILES Invoked on command line eg: mvn verify -Pmysql, tomcat5, dataset1 Configuration in profile is added to build Use for a wider variation in test environments Use profiles for individual work units, eg JMeter
ADDING A PROFILER $ export LD_LIBRARY_PATH= $JPHOME/bin/linux-x64/ <global.jvm.args>-xms2000m -Xloggc:${project.build.directory}/gc.log</global.jvm.args> <jvm.args>${global.jvm.args}</jvm.args> <profile> <id>jprofiler</id> <properties> <jvm.args>-agentlib:jprofilerti=offline,id=${jprofiler.profileid},config=${jprofiler.configfile} - Xbootclasspath/a:${jprofiler.home}/bin/agent.jar ${global.jvm.args}</jvm.args> </properties> </profile>
REPORTING CHRONOS Available in maven eg: mvn site / mvn chronos:report Generates graphs and a numerical report Can compare to historical data Static :(
REPORTING CHRONOS
REPORTING CHRONOS
REPORTING CHRONOS
REPORTING CHRONOS
REPORTING CHRONOS - HISTORY
REPORTING CHRONOS - HISTORY
REPORTING BAMBOO JMETER AGGREGATOR Open source Bamboo plugin (supported!) James Roper (Atlassian) 20% time project Can graph JMeter.JTL and CSV files Dynamic
REPORTING BAMBOO JMETER AGGREGATOR
TIPS Allocate time for fixing tests Your tests WILL break Work out a process for fixing bugs Gather as many artifacts from each test run as possible React sooner rather than later
Q&A gbarnett @ atlassian. com @georgebarnett Code samples online soon! blogs.atlassian.com/developer Meet Atlassian @ The Summit: May 31 - June 2 in SF www.atlassian.com/summit