Develop Software with Confidence Continuous Integration, Continuous Agitation March 2006 Jeffrey Fredrick jtf@agitar.com 1
Topics What is Continuous Integration Continuous Integration at Agitar Toolset Questions 2
What is Continuous Integration Manual CI The practice of frequently integrating my work with the work of the team Automated CI the practice of giving automated feedback as rapidly as possible following a change Common goal lower the cost of failure 3
Feedback Code rules pass Code compiles Agitation tests pass Progress on test targets System tests pass Installer created Deployment successful People are generally interested in being good citizens. Alistair Cockburn 4
Developer Testing as Feedback Allows developer to confirm some part of the code works as expected not the same as the product working Provides feedback while doing faster feedback lower cost of failure downstream efficiency Developers write tests to change the code 5
Importance of Automating Tests Tests are code: software rots if not used Good tests fail Imperfect tests, run frequently, are much better than perfect tests that are never written at all. Martin Fowler 6
Overview of Agitator & Dashboard Agitator :: developer testing tool attempts to exercise all code and report on observed behavior observations/assertions evaluated for a range of input track expected and unexpected outcomes Management Dashboard managers: set targets, track trends developers: report status, progress on targets 7
Overview of A&D Code Bases Agitator (core) 1886 classes, 74126 lines 1164 test & harness classes, 33659 lines 5857 JUnit test points, 32668 total Dashboard 126 classes, 5052 lines 183 test & harness classes, 7028 lines 3360 JUnit test points, 5316 total 8
Our Builds quick build compiles & JUnit tests feedback < 10 minutes agitation build system tests & agitate changed classes feedback in under 1 hour nightly agitation agitate everything execution time ~ 10 hours (official build) 9
Feedback Mechanisms Lava lamps extreme Feedback Monitor Email CI tool & dashboard emails JSP reporting application 10
Lava Lamps 11
quick XFM 12
Failure Email 13
JSP Metrics Tab quick Results 14
Developer Testing Maturity Continuous Agitation Managed Developer Testing Machine independent tests Continuous Integration Automated nightly test execution Machine dependent tests No tests 15
Managed Developer Testing Why Because developer testing effort can be a non-trivial amount of development time How Set visible targets Show developers their progress Track trends position is nothing, momentum is everything 16
Agitator Executive Email 17
Agitator Developer Email 18
Agitation XFM 19
JSP Metrics Tab Agitation Results 20
Continuous Agitation Math Dashboard project agitation build 5316 test points x 1241 builds = 6.6 million opportunities for feedback Agitator project agitation build 32668 test points x 5407 = 176 million opportunities nightly build 32668 test points x 557 = 18 million opportunities 21
Impact of Continuous Agitation More features and higher quality Engineering staffing 40% lower than projected Latest release added new functionality 2 weeks before ship shipped 1 week early Oozing Confidence 22
Toolset CruiseControl open source CI framework Agitator ant tasks quicklist regression mode create.zsums console.xml Dashboard ant tasks generate & email reports 23
Resources CruiseControl Continuous Integration: An Unlikely Hero Continuous Integration by Martin Fowler DeveloperTesting.com Continuous Integration, Continuous Agitation A Bad Day With Continuous Integration extreme Feedback for Software Development 24
Questions 25