CODING & DEVELOPMENT BORIS GORDON FEBRUARY 7 2013 Continuous Integration
Introduction About me boztek on d.o. (http://drupal.org/user/134410) @boztek boris@previousnext.com.au 2
Introduction About you 3
Introduction About you Probably a developer 3
Introduction About you Probably a developer Maybe don t know what CI is at all 3
Introduction About you Probably a developer Maybe don t know what CI is at all Probably have some idea but don t know where to start 3
Introduction About you Probably a developer Maybe don t know what CI is at all Probably have some idea but don t know where to start Wondering how/if it applies to Drupal projects 3
Introduction About you Probably a developer Maybe don t know what CI is at all Probably have some idea but don t know where to start Wondering how/if it applies to Drupal projects Have some scaling concerns 3
Introduction What we will cover 4
Introduction What we will cover Motivation 4
Introduction What we will cover Motivation Continuous integration basics 4
Introduction What we will cover Motivation Continuous integration basics Getting started 4
Introduction What we will cover Motivation Continuous integration basics Getting started Best practices 4
Introduction What we will cover Motivation Continuous integration basics Getting started Best practices Building, testing and deploying 4
Introduction What we will cover Motivation Continuous integration basics Getting started Best practices Building, testing and deploying Continuous integration servers 4
Introduction What we will not cover 5
Introduction What we will not cover Which software you need to install to integrate for you 5
Introduction What we will not cover Which software you need to install to integrate for you The perfect one size fits all project template 5
Introduction What we will not cover Which software you need to install to integrate for you The perfect one size fits all project template How to install Jenkins 5
Introduction What we will not cover Which software you need to install to integrate for you The perfect one size fits all project template How to install Jenkins How to write Drupal tests 5
Introduction What we will not cover Which software you need to install to integrate for you The perfect one size fits all project template How to install Jenkins How to write Drupal tests I won t really try to convince you that should be doing this 5
What s the problem? We are talking about construction 6
What s the problem? We are talking about construction 6
What s the problem? We are talking about construction 1. Pre-build (requirements, specs, etc.) 6
What s the problem? We are talking about construction 1. Pre-build (requirements, specs, etc.) 2. Design and develop and test locally 6
What s the problem? We are talking about construction 1. Pre-build (requirements, specs, etc.) 2. Design and develop and test locally 3. Deploy release to test server 6
What s the problem? We are talking about construction 1. Pre-build (requirements, specs, etc.) 2. Design and develop and test locally 3. Deploy release to test server 4. Testers review 6
What s the problem? We are talking about construction 1. Pre-build (requirements, specs, etc.) 2. Design and develop and test locally 3. Deploy release to test server 4. Testers review 5. Fix bugs 6
What s the problem? We are talking about construction 1. Pre-build (requirements, specs, etc.) 2. Design and develop and test locally 3. Deploy release to test server 4. Testers review 5. Fix bugs 6. Testers review 6
What s the problem? We are talking about construction 1. Pre-build (requirements, specs, etc.) 2. Design and develop and test locally 3. Deploy release to test server 4. Testers review 5. Fix bugs 6. Testers review 7. Fix bugs 6
What s the problem? We are talking about construction 1. Pre-build (requirements, specs, etc.) 2. Design and develop and test locally 3. Deploy release to test server 4. Testers review 5. Fix bugs 6. Testers review 7. Fix bugs 8. Deliver 6
What s the problem? We are talking about construction 1. Pre-build (requirements, specs, etc.) 2. Design and develop and test locally 3. Deploy release to test server 4. Testers review 5. Fix bugs 6. Testers review 7. Fix bugs 8. Deliver 6
What s the problem? We are talking about construction 1. Pre-build (requirements, specs, etc.) 2. Design and develop and test locally 3. Deploy release to test server 4. Testers review 5. Fix bugs 6. Testers review 7. Fix bugs 8. Deliver 7
Software Engineering 101 Not releasing defects is much cheaper than fixing them later. 8
How can we fix? Turns out it s cheaper to fail faster and more often. 9
How can we fix? Problems with divide and conquer 10
How can we fix? Integration is waste right? 11
How can we fix? Integration is waste right? 11
If it feels good, do it? 12
If it feels good, do it? Maybe if it s painful, you aren t doing it enough 13
If it feels good, do it? Maybe if it s painful, you aren t doing it enough 13
If it feels good, do it? Maybe if it s painful, you aren t doing it enough Integrating small changes reveals problems earlier 13
If it feels good, do it? Maybe if it s painful, you aren t doing it enough Integrating small changes reveals problems earlier Integrating regularly produces feedback quicker than integrating sporadically 13
If it feels good, do it? Maybe if it s painful, you aren t doing it enough Integrating small changes reveals problems earlier Integrating regularly produces feedback quicker than integrating sporadically 13
If it feels good, do it? Maybe if it s painful, you aren t doing it enough Integrating small changes reveals problems earlier Integrating regularly produces feedback quicker than integrating sporadically Maybe we should just integrate all the time? 13
Continuous integration Is that even possible? 14
Continuous integration Is that even possible? 14
Continuous integration Is that even possible? Yes 14
Continuous integration Is that even possible? Yes... sort of 14
Continuous integration Is that even possible? Yes... sort of... with discipline 14
Continuous integration Is that even possible? Yes... sort of... with discipline... and sufficient tools 14
Continuous integration How continuous is continuously? Continuously is more often than you think - Jez Humble 15
CI is a practice It s not really about tools It is about getting past our adolescent surety that I know better than everyone else and all I need is to be left alone to be the greatest. - Kent Beck 16
Goals and principles The CI mindset Without CI : consider your application s default state broken (aka under development ) With CI : the starting point for any development work is a stable, robust, tested application Mistakes are made early and quickly and addressed when it s cheapest to fix 17
CI is a practice In other words apt-get install jenkins!== CI Installing and maintaining a CI server may be an IT/Ops job Doing CI is something the developers practice 18
Getting started Defining some terms 19
Getting started Defining some terms build : some version-able, deployable, stand alone artefacts that can be tracked across different environments 19
Getting started Defining some terms build : some version-able, deployable, stand alone artefacts that can be tracked across different environments test : examine build artefacts and has own artefacts 19
Getting started Defining some terms build : some version-able, deployable, stand alone artefacts that can be tracked across different environments test : examine build artefacts and has own artefacts deploy : moving a build around 19
Implementing CI Start with the essentials check into mainline regularly test everything (that can break) building and testing should be fast 20
Implementing CI Regular mainline checkins 21
Implementing CI Comprehensive automated tests Confidence in a build is dependent on the breadth and quality of tests. Testing needs to be as frictionless as possible Tests need to run fast Drupal simpletest is pretty slow But there are other types of tests and you don t have to run all of them all the time 22
Implementing CI Comprehensive automated tests Some types of tests and recommended tools. 1. Unit tests - PHPUnit - DrupalUnittestCase 2. Functional tests - Simpletest 3. Smoke tests - Shell scripts, simpletest 4. Regression tests (use one of the above) 5. Automated acceptance tests - Behat with mink or selenium 23
Implementing CI Comprehensive automated tests That s a lot of tests! Different types of tests can be run at different stages of a CI pipeline 24
Implementing CI Productive development environments Consistent development environments are important. Writing tests needs to be as frictionless as possible too Tests need to run fast locally Ideally all tests need to be runnable locally 25
Implementing CI Productive development environments Configuration and data should be versioned Build scripts Test data Deploy configurations Dependency management 26
Implementing CI Productive development environments Building and testing should be fast 27
Implementing CI Productive development environments Why bother? Like TDD it can reveal bad smells you might otherwise ignore 28
Doing Continuous Integration Some best practices 29
Doing Continuous Integration Some best practices Don t check in on a broken build 29
Doing Continuous Integration Some best practices Don t check in on a broken build Run tests (locally) before merging into the main line 29
Doing Continuous Integration Some best practices Don t check in on a broken build Run tests (locally) before merging into the main line Keep the main line build stable 29
Doing Continuous Integration Some best practices Don t check in on a broken build Run tests (locally) before merging into the main line Keep the main line build stable Know how to manage instability 29
Doing Continuous Integration Some best practices Don t check in on a broken build Run tests (locally) before merging into the main line Keep the main line build stable Know how to manage instability Take responsibility 29
Doing Continuous Integration Some best practices Don t check in on a broken build Run tests (locally) before merging into the main line Keep the main line build stable Know how to manage instability Take responsibility Don t forget about the architecture 29
Doing Continuous Integration Some best practices Don t check in on a broken build Run tests (locally) before merging into the main line Keep the main line build stable Know how to manage instability Take responsibility Don t forget about the architecture Beware slow tests 29
Doing Continuous Integration Some best practices Don t check in on a broken build Run tests (locally) before merging into the main line Keep the main line build stable Know how to manage instability Take responsibility Don t forget about the architecture Beware slow tests Code style matters 29
Managing a Drupal Pipeline Build An automated build. This is the best place to start. 30
Managing a Drupal Pipeline Build 31
Managing a Drupal Pipeline Build.!"" README.md!"" build!"" build.properties!"" build.properties.dist!"" build.xml!"" db!"" src!"" test #"" tools 32
Managing a Drupal Pipeline Build - Demo $> phing 33
Managing a Drupal Pipeline Test Unit tests and integration tests are really part of the build. Static analysis is part of the build. 34
Managing a Drupal Pipeline Test So we can run tests with Phing. 35
Managing a Drupal Pipeline Deploy Capistrano Phing Environment differences 36
Continuous Integration Servers Automate all the builds! Once you have automated builds, automated tests and automated deployments it s time to wire them up. Jenkins Travis CI ThoughtWorks Go 37
Continuous Integration Servers Tie it all together with robots A CI server is the robot that sits between your version control system and your build system. Robots can do heavier lifting 38
Continuous Integration Servers Jenkins Jenkins is a popular CI server that provides the hub between your version control and build / test / deploy tools. VCS integrations Reporting Triggering Configurable UI / dashboard 39
Continuous Integration Servers Travis CI 40
Continuous Integration Servers References and further reading Continuous Delivery by Jez Humble http://www.amazon.com/continuous-delivery-deployment- Automation-Addison-Wesley/dp/0321601912 Continuous Integration by Duvall, Matyas and Glover http://www.amazon.com/continuous-integration-improving-software-reducing/dp/ 0321336380/ref=sr_1_1? s=books&ie=utf8&qid=1360206726&sr=1-1&keywords=continuous+integration Drupal phing build.xml http://reload.github.com/phing-drupal-template/ 41
Continuous Integration Servers References and further reading http://github.com/boztek/drupal-project-template 42
Integration complete Questions boztek on d.o. (http://drupal.org/user/134410) @boztek boris@previousnext.com.au 43