Automated performance testing using Maven & JMeter. George Barnett, Atlassian Software

Transcription

1 Automated performance testing using Maven & JMeter George Barnett, Atlassian Software

2 Create controllable JMeter tests Configure Maven to create a repeatable cycle Run this build in your CI server every 8 hours Find performance regressions faster

3 WHY? Catch regressions quickly Stop hunting for bugs Code with confidence Know the performance cost of features Fearless external dependency upgrades Go home on time

4 ATLASSIAN 15,000+ customers 100+ developers 8 software streams 1 performance engineer

5 ATLASSIAN JIRA - Issue management Confluence - Enterprise Wiki Bamboo - Continuous Integration FishEye, Crucible & Clover - Dev Tools Crowd - Single Sign-On.

6 BEFORE WE BEGIN Continuous Integration is critical Keep code working with Unit and Functional tests Broken or irregular builds means no code to performance test Performance testing is NOT a replacement for other testing (but it helps)

7 PROCESS Write code (1-6 months) Compile code Send it to QA (or Production?!) Performance Regression PANIC! No performance testing!

8 A BETTER WAY Write code CI compiles code and runs tests Send it to QA (or Production?!) Performance Regression PANIC! No performance testing!

9 A BETTER WAY 2.0 Performance test artifacts as soon as they pass Unit and Functional testing Fast feedback for developers No Surprises

10 TOOLS JMeter Maven & plugins Bamboo & plugins A Profiler

11 JMETER Apache Project Java Supports HTTP, JDBC, SOAP, XML-RPC, FTP, SMTP, LDAP, JUnit & more.

12

13 MAVEN Apache Project Java Build Tool Dependency management Documentation

14 MAVEN CONCEPTS Project Object Model (POM) Standardised directory structure Build Lifecycle Reporting Lifecycle Plugins

15 MAVEN POM <project> <groupid>com.atlassian.com.performance.jira</groupid> <artifactid>automated-performance</artifactid> <properties> <test.dataset>/opt/dataset/dataset1.zip</test.dataset> </properties> <build> <!--Main Lifecycle plugins configured here--> </build> <reporting> <!--Reporting Lifecycle here--> </reporting> </project>

16 MAVEN PLUGINS Chronos Ant-run Cargo Maven Assembly plugin Maven Dependency plugin

17 BAMBOO Continuous Integration Can run builds on Amazon s EC2 JMeter Aggregator Plugin

18 A PROFILER Java Profiler CPU Snapshots GC & Memory information Thread usage and contention Triggers & offline sampling

19 STEP 1 Package the JMeter test Maven Assembly plugin Create a.zip from project directories `mvn deploy` to repository as a SNAPSHOT

20 STEP 2 Configure the Maven automated build Package application state into zip files Ant-run - application pre-setup Cargo - deploy container Chronos - configure the JMeter tests

21 MAVEN PROFILES <project>.. <properties> <test.dataset>/opt/dataset/dataset1.zip</test.dataset> </properties> <profile> <id>dataset1</id> <properties> <test.dataset>/opt/dataset/dataset1.zip</test.dataset>.. <profile> <id>dataset2</id> <properties> <test.dataset>/opt/dataset/dataset2.zip</test.dataset>..

22 MAVEN PROFILES Invoked on command line eg: mvn verify -Pmysql, tomcat5, dataset1 Configuration in profile is added to build Use for a wider variation in test environments Use profiles for individual work units, eg JMeter

23 ADDING A PROFILER $ export LD_LIBRARY_PATH= $JPHOME/bin/linux-x64/ <global.jvm.args>-xms2000m -Xloggc:${project.build.directory}/gc.log</global.jvm.args> <jvm.args>${global.jvm.args}</jvm.args> <profile> <id>jprofiler</id> <properties> <jvm.args>-agentlib:jprofilerti=offline,id=${jprofiler.profileid},config=${jprofiler.configfile} - Xbootclasspath/a:${jprofiler.home}/bin/agent.jar ${global.jvm.args}</jvm.args> </properties> </profile>

24 REPORTING CHRONOS Available in maven eg: mvn site / mvn chronos:report Generates graphs and a numerical report Can compare to historical data Static :(

25 REPORTING CHRONOS

26 REPORTING CHRONOS

27 REPORTING CHRONOS

28 REPORTING CHRONOS

29 REPORTING CHRONOS - HISTORY

30 REPORTING CHRONOS - HISTORY

31 REPORTING BAMBOO JMETER AGGREGATOR Open source Bamboo plugin (supported!) James Roper (Atlassian) 20% time project Can graph JMeter.JTL and CSV files Dynamic

32 REPORTING BAMBOO JMETER AGGREGATOR

33 TIPS Allocate time for fixing tests Your tests WILL break Work out a process for fixing bugs Gather as many artifacts from each test run as possible React sooner rather than later

34 Q&A atlassian. Code samples online soon! blogs.atlassian.com/developer Meet The Summit: May 31 - June 2 in SF