Continuous Integration bjorn.boisschot@ctg.com
1. It works on my machine.
Risk 1 Lack of Deployable Software
Risk 2 Lack of project visibility
2011 CTG, Inc. 9
2011 CTG, Inc. 10
Risk 3 Low quality software
Risk 4 Late discovery of defects
All these risks are related to Old School Development Practices
Old School Development Write Code Developer Write Code Write Code Builds sources Developer Lead Developer Commit Code Get latest sources Version Control System
Manual Effort Slow Feedback Debugging Dependencies
Continuous Integration 2011 CTG, Inc. 19
Build software better, faster, cheaper
Development practice where team members integrate their work daily verified by automated builds using different testing types
Development practice where team members integrate their work daily verified by automated builds using different testing types
3 building blocks of continuous integration
1. Version Control System
</CODE> </CODE> </CODE>
2. Automated Builds
</CODE> </CODE> </CODE>
3. Team Agreement
</CODE> </CODE> </CODE>
7 Principles of Continuous Integration
1. Commit code frequently
Check in at least once a day Avoid the 5PM check in mess Small Steps are beter than one big leap Check in after each task
But, we can check in as often as we want
Continuous Integration without testing = Continuous Compilation
1. Commit code frequently 2. Write automated unit tests
</CODE> </CODE> </CODE>
</CODE> </CODE> </CODE>
</CODE> </CODE> </CODE>
1. Commit code frequently 2. Write automated unit tests 3. Run private builds
</CODE> </CODE> </CODE>
</CODE> </CODE> </CODE>
</CODE> </CODE> </CODE>
1. Commit code frequently 2. Write automated unit tests 3. Run private builds 4. Don t commit broken code
</CODE> </CODE> </CODE>
</CODE> </CODE> </CODE>
</CODE> </CODE> </CODE>
1. Commit code frequently 2. Write automated unit tests 3. Run private builds 4. Don t commit broken code 5. Fix broken build immediately
You Break It (The Build) You Buy It (Round of Beer) You Fix It (Quickly)
1. Commit code frequently 2. Write automated unit tests 3. Run private builds 4. Don t commit broken code 5. Fix broken build immediately 6. Avoid getting broken code
1. Commit code frequently 2. Write automated unit tests 3. Run private builds 4. Don t commit broken code 5. Fix broken build immediately 6. Avoid getting broken code 7. All tests and inspections should pass
</CODE> </CODE> </CODE>
</CODE> </CODE> </CODE> <CODE ANALYSIS>
Development practice where team members integrate their work daily verified by automated builds using different testing types
one team
one team Developers
one team Testers Developers
Development practice where team members integrate their work daily verified by automated builds using different testing types
in CI speed matters
UI Test Execution Time Service/API Unit
Code Quality
Unit Testing
1 Unit 99,5% reliable 1 system = 50 Units 99,5% 50 = 78% reliable
Let s take a look at a Unit Test
A (very) simple class
Some Unit Tests
About Unit Testing Verify behavior of small elements in a software system, most often a single class Occasionally more than 1 class in case of tightly coupled classes No connections to DB, filesystem, Code Coverage per test is generally limited Should be very fast Crucial Component of CI, unit tests run in every build CI works well with TDD
Where in CI? Developer Feedback Mechanism Commit Code + Unit Tests Polls for changes CI Server VCS Build Server Builds sources Unit Tests
Build the code RIGHT Build the RIGHT code 2013 CTG, Inc. 78
About Unit Integration/Acceptance Testing Also a white box test May cross architectural boundaries More code coverage per test Longer running duration than Unit Tests Doesn t run in every build
Where in CI? Developer Send Feedback Feedback Mechanism Polls for changes CI Server Commit Build (Lightweight) Builds sources Unit Tests Commit Code + Unit Tests VCS If Commit Build Succesful Secondary Build (Heavyweight) Unit Integration Tests
Some Namedropping JUnit NUnit MS Tests TestNG DBUnit
Unit Performance Testing
JUnitPerf example
What to unit performance test?
2013 CTG, Inc. 85
2013 CTG, Inc. 86
Where in CI? Developer Send Feedback Feedback Mechanism Commit Code + Unit Tests VCS Polls for changes CI Server If Commit Build Succesful Commit Build (Lightweight) Builds sources Unit Tests Secondary Build (Heavyweight) Unit Integration Tests Unit Performance Tests
Some Namedropping JUnitPerf JUnitBench ContiPerf NUnitPerf
API/Service Testing
FitNesse
Simple Example
Run Test Case Red
Fixture Red
The actual Code Red
Run Test Case Red
Fix Code Red
Run Test Case Green
Refactor Code Refactor
More advanced example
SoapUI
Jmeter
Some Namedropping FitNesse Cucumber Concordion SOAPUI JMeter HP ServiceTest
Where in CI? Developer Send Feedback Feedback Mechanism Commit Code + Unit Tests VCS Polls for changes CI Server Commit Build (Lightweight) Builds sources Unit Tests Secondary Build (Heavyweight) Unit Integration Tests Unit Performance Tests Service/API Tests
GUI Testing
We re not done yet! GUI Automated Testing Selenium HP Unified Functional Testing - FASTBoX MS Coded UI Testing E2E Performance Testing JMeter HP LoadRunner MS Visual Studio RadView WebLoad Dynamic Security Testing HP WebInspect HP QAInspect
Where in CI? Developer Send Feedback Feedback Mechanism Commit Code + Unit Tests VCS Polls for changes CI Server Nightly Build GUI Automated E2E Performance Dynamic Security Commit Build (Lightweight) Builds sources Unit Tests Secondary Build (Heavyweight) Unit Integration Tests Unit Performance Tests Service/API Tests
Mapping Tests on Build Types
Builds Strategy Lightweight Commit Build Unit Tests Secondary Build (Heavyweight) Unit Integration Unit Performance API/Service Test Automated UI Smoke Tests Nightly Build Automated UI Tests Static Testing Static Application Security Testing Dynamic Application Security Testing Release Build Automated UI Tests Manual Tests
Manual Testing & CI
HP ALM: Application Lifecycle Intelligence
HP ALM: Build Reports
MS Test Manager: Recommended Tests
one team Testers Developers
one team Testers Operations Developers
120
Deployment pipeline
Reduce project Risks
Reduce repetitive manual processes
Generate deployable software
Remember our project risks?
Risk 1 Lack of Deployable Software
Risk 1 Lack of Deployable Software Risk 2 Lack of Project Visibility
Heatmap with LoC (Size), and Code Coverage (Color)
Risk 1 Lack of Deployable Software Risk 2 Lack of Project Visibility Risk 3 Low Structural Quality
Violations per category + trend
Level of Code Duplication
Risk 1 Lack of Deployable Software Risk 2 Lack of Project Visibility Risk 3 Low Structural Quality Risk 4 Late Discovery of Defects
Builds Strategy Lightweight Commit Build Unit Tests Secondary Build (Heavyweight) Unit Integration Unit Performance API/Service Test UI Smoke Tests Non-Functional Build (Nightly/Weekly) E2E Performance Testing Static Testing Static Application Security Testing Dynamic Security Testing Release Build (2 weekly) Manual Testing UI Tests
Risk 1 Lack of Deployable Software Risk 2 Lack of Project Visibility Risk 3 Low Structural Quality Risk 4 Late Discovery of Defects
2013 CTG, Inc. 139
CI is an attitude NOT a tool