CONTINUOUS INTEGRATION REALISING ROI IN SOFTWARE DEVELOPMENT PROJECTS In the following pages we will discuss the policies and systems that together make up the process called Continuous Integration. This document is written for IT managment who are looking to improve the quality of the systems their teams build, along with increasing the Return on Investment of their IT department as a whole. Copyright 2012 Inviqa
Table of Contents Executive Summary 3 Introduction to Continuous Integration 4 The Systems Source Code Control Unit Testing Code Validation Documentation Integration Testing Regression Testing 6 6 6 6 7 7 8 The Tools Subversion PHPUnit CruiseControl/phpUnderControl PHP_CodeSniffer phpdocumentor Phing 9 9 9 10 10 10 10 Return on Investment 11 Conclusion 12 2
Executive Summary The main concept behind Continuous Integration as described by Martin Fowler is simple: Every developer checks in their code, every day. Adopting this policy reduces the time it takes to integrate a developer s code into the existing codebase to almost zero. Regular and frequent check-ins also allow testing systems to be developed that will regularly scan the entire code base for problems and raise red flags early in the process rather than later, when the bugs are more expensive to fix. These benefits allow teams to improve the quality of the software they deliver. Because developers are finding bugs earlier in the process rather than later, the cost of developing internal software is reduced, thus increasing the ROI of the team. When teams adopt this policy, systems can be put in place to help make sure that the code being written integrates with other developers code and the existing codebase. This methodology, along with the accompanying support systems, is known as a Continuous Integration system. From the system perspective, a stable and useful Continuous Integration system can be assembled using several best of breed, open source applications based on PHP. Subversion for Source Code Control phpundercontrol/cruise Control for Integration and regression testing 4 php Documentor documentation Code_Sniffer for code validation Phing for build tasks Combining these systems together, teams can integrate their code changes quickly, build on a scheduled basis and ensure that the system is not only working according to their understanding of what is to be built, but working according to the client s specifications. 3
Introduction to Continuous Integration Software development continues to grow increasingly more complex. Even when dealing with dynamic languages like PHP, the systems being built today dwarf the systems of even five years ago. Many development teams however have not upgraded their development process and systems to meet the new demands. Today s teams must be well versed in the best practices of software development like source code control, unit testing, integration testing and documentation. These practices, when coupled together, form the basis of a Continuous Integration system. Continuous Integration is a term coined by Martin Fowler in a paper by the same name. Continuous Integration is a software development practice where members of a team integrate their work frequently; usually each person integrates at least daily - leading to multiple integrations per day. Each integration is verified by an automated build (including test) to detect integration errors as quickly as possible. - Martin Fowler When programming in today s dynamic languages, the concept of a build does not mean the same thing as it does for traditional compiled languages. However the other concepts he discusses still have a place in software development. Continuously integrating changes into the main code base so that their effects can be quickly seen and fixed if necessary is an important step in controlling the cost of bugs and the I quality of your overall system. As discussed earlier, Continuous Integration is a series of inter-related services that, when used together, accomplish the goals behind Mr. Fowler s concept of Continuous Integration. Those goals are to eliminate the Integration phase of any software project and to identify bugs earlier rather than later. 4
The services used in Continuous Integration are: Source Code Control Unit Testing Code Validation Documentation Integration testing Regression testing (builds) All of these systems can be setup independently and used ad hoc. However, forward thinking teams are now integrating all of them to help identify and find fix small problems early rather than large problems late. 5
The Systems The underlying principle of Continuous Integration, frequent integration of code being written, is handled by the source code control system. The secondary goals, unit testing, code validation, documentation and integration testing, all revolve around this system. Source Code Control As stated, Source Code Control is the heart of any Continuous Integration system. Source Code Control is the main repository for the code and the first line of defense in many Continuous Integration systems. Aside from serving as the focal point for your source code and a backup in case a developer s machine breaks down, your source code control system provides you with important hooks that can be used to automate pieces like unit testing. Unit Testing Many teams use a two pronged approach for Unit testing. The first prong is the developer running the tests manually. This type of testing is helpful in creating the code but relies on the developer to remember to run the tests before check-in. The second prong addresses this shortcoming by automating unit testing and reporting. Many teams use hooks in programs like Subversion to automatically run Unit Test suites. There are two schools of thought when doing this. The first is to deny check-in if the code fails its unit testing. The other is that unit tests are run on check-in but the check-in can proceed but the error is reported. Either way, when bugs are introduced into the system, developers know about it immediately and can fix them quickly. Code Validation It is often said that the wonderful thing about standards is that there are so many to choose from. Nowhere is this more evident than with PHP coding standards; there is an embarrassment of riches when choosing a standard for your team. The problem, though, is not choosing one but enforcing it once chosen. Once you and your team have settled on a standard, a Continuous Integration system can help enforce it by routinely scanning the repository and identifying code snippets that do not match the agreed upon specification. 6
Documentation While nothing can replace well written end-user documentation, good API documentation is invaluable to developers. This is never truer than when new developers are moving on to a project or picking one up after having spent time away from it working on other projects. Many languages have adopted the JavaDoc style for API documentation. This method allows for the documentation to be read in-line by the developers while at the same time, can be parsed by appropriate systems to automatically generate a comprehensive set of docs for the project. Integration Testing The concept of software builds does not translate well into dynamic language development. Since all code is parsed and tokenized at runtime, the traditional processes of compiling and linking do not come into play. The routine of checkout, compile and linking is replaced by Integration Testing. Unit testing is an integral part of any software development project and is the best way to find bugs in your code early. Even the most rigorous unit testing cannot guarantee that all the pieces in the system will work together. For this you need to implement an Integration Testing regime. Integration testing is fitting all the individual pieces together and making sure that they work together. Integration testing can take three forms. Bottom Up By far the most popular methodology of Integration testing is bottom up testing. In Bottom Up testing the first step is to re-run all of your unit tests. Once they have been re- tested in full, you use a system like Selenium to test how the system as a whole works. Those reports are posted online for all to see. Top Up Using the Top Down methodology, all of the units are assembled into a system and then modules are tested. Below the modules, individual units are tested. Big Bang The Big Bang methodology of integration testing is where all pieces are assembled and tested; reports are generated and posted for all to see. No individual unit testing takes place as part of the Big Bang methodology. 7
Look for mindshare Test plans for integration testing are important Teams need to make sure that they are testing what the client will be testing in the Acceptance testing phase. Also as a consequence of the introduction of new bugs, program maintenance requires far more system testing per statement written than any other programming. Theoretically, after each fix one must run the entire batch of test cases previously run against the system, to ensure that it has not been damaged in an obscure way. In practice such regression testing must indeed approximate this theoretical idea, and it is very costly -Fred Brooks, The Mythical Man Month Regression Testing Once a milestone has been achieved, the test plans for the features in that milestone are integrated into the Regression Testing testing plan. Regression testing makes sure that once something is working, it stays working. It s not enough to simply test the features that you are currently working on, you have to continue to test completed features to make sure that new code does not have adverse affects. 8
The Tools Now that we ve discussed what a Continuous Integration System consists of, let s talk a bit about how to put one together. We will discuss the best of breed tools for PHPbased shops to use to implement Continuous Integration. While we will discuss the tools here, we will not go into the actual setup and integration of these tools. There is an excellent primer on that very subject on Inviqa s techportal site titled Getting Started with phpundercontrol i by Marc Veldman. Subversion ii The undisputed leader in Source Code Control for open source projects is Subversion. While the venerable CVS is still widely used, most teams have a migration plan already to move to Subversion. Since your Source Code Control system will be the heart of your Continuous Integration system it is recommended that that you get this important piece up and running before you move on to the next steps. PHPUnit iii Unit testing, whether done at check-in or only during scheduled builds is the cornerstone of the Continuous Integration system. PHP has several good Unit testing frameworks to choose from; however, the most complete and the most widely accepted is the PHPUnit testing framework. Used by itself, it is a powerful tool for ensuring the quality of a code base. When used as an integral part of a Continuous Integration system it helps improve the ROI of the project by finding problems earlier rather than later. 9
CruiseControl/ phpundercontrol iv The heart of any Continuous Integration system is the continuous build server. In the system being described CruiseControl and the PHP plugin for it, phpundercontrol, serve this role. CruiseControl is one of the most popular Continuous Build tools available and while it is written in Java there are builders and plugins available for many different languages, including Phing for PHP. phpundercontrol is the add-on application which allows for the integration of other PHP tools. PHP_CodeSniffer v By hooking PHP_CodeSniffer into the Continuous Integration system, you can automate the enforcement of your accepted coding standard. PHP_CodeSniffer will analyze the code in your project and report discrepancies. The report generated can be used to identify problems and resolve them quickly. Maintaining strict adherence to a coding standard will pay for itself in the maintenance phase of your project. phpdocumentor vi phpdocumentor is the accepted standard tool for building API documentation for PHP projects. Not only can it generate nicely formatted documentation, most modern IDEs support it, allowing for code completion on user code in addition to native language calls. Phing vii All build systems need a build utility. Unix/Linux has had make for many years now, Java has Ant and.net has Nant and more recently MSBuild. PHP s build tool is Phing. Phing is based on the concepts of Apache s Ant and allows you to complete any task you would be able to using Ant, make or any other build system. CruiseControl has plugins that support using Phing. 10
Return on Investment Any project undertaken by a company has to show that it is valuable to the company. When dealing with IT, this value, or the Return on Investment, is usually measured in either dollars contributed back to the bottom line or in time saved. Continuous Integration systems add value to the development process and the project as a whole and their value can be measured using both metrics. Overall, when properly installed, configured, and used, a Continuous Integration system will allow your teams to build projects faster and cheaper. The first, and arguably the most visible, benefit of Continuous integration is that it allows teams to identify and fix bugs early in the development cycle, as they appear. Conventional IT wisdom tells managers that the earlier you can find a bug, the cheaper it is to fix it. Since it s not yet possible to fix bugs as the programmer makes them, the next best thing is to find and report them when the code is checked in. This way the problem is fresh on the developer s mind and fixing it doesn t derail any other part of the project. Fixing bugs quickly, thus cheaply, means that systems are deployed with fewer bugs, and because expensive late- in-lifecycle debugging is not necessary, it allows the project to be delivered more quickly and within budget.... when properly installed, configured and used, a Continuous Integration system will allow your teams to build projects faster and cheaper The second and less visible improvement that Continuous Integration brings to projects is oversight. PHPUnit, CruiseControl, and phpundercontrol all bring with them published reports that can be viewed not only by the developers but by team leads, management and even clients. These reports give interested parties a snapshot into the health of the project. While the reports do not give a complete picture of the progress being made on the project, they 11
are useful for determining the health of the project. Project managers can use this information to make decisions on how to proceed with the project. Just like with the health of a human body, problems found early can be corrected more easily than those found later on. Continuous Integration systems require an upfront investment in time to install and configure. Thought needs to be given as to where the systems will physically reside. Once the initial investment has been made, however, all projects utilizing them can enjoy their benefits with only marginal costs involved in project setup and overall system maintenance. The latter can be distributed across all the projects using the system. References Conclusion This whitepaper has discussed the systems that make up a Continuous Integration system, made specific recommendations on which systems to use, and shown the ROI of using a Continuous Integration System. However, no system can make a difference if your team does not believe in them and use them. Implementing a Continuous Integration system is more about team attitude than the systems you deploy. The services we have described here can help teams identify bugs early but only if the system is used. The central tenant of Continuous Integration, Every developer checks in their code every day, cannot be enforced by a system, it has to be an attitude adopted by not only each developer but the team as a whole. As has been shown, however, the Return On Investment for deploying a Continuous Integration system can be measured in both the short term and long term. In the short term, bugs are cheaper to fix when they are found early in the development lifecycle. In the long term, as projects mature, the continuity of the code base will be maintained as different developers migrate in and out of the project. Continuous Integration and the systems that support it will keep your projects clean, tested and ready for deployment at a moment s notice. 12
i http://techportal.inviqa.com/2009/03/03/getting-started-with-phpundercontrol ii http://subversion.tigris.org/ iii http://phpunit.de iv http://phpundercontrol.org/about.html v http://pear.php.net/package/php_codesniffer vi http://www.phpdoc.org/ vii http://phing.info/ Inviqa focuses on bespoke IT project delivery, to large enterprises, based on best practice software engineering principles. Our team of 90 staff in the UK, including software engineers and support roles, provide solutions to enable our clients to get the most out of open source, cutting costs and reducing risks. We are passionate about open source technologies, having been successfully delivering complex IT solutions since 1999. We love web development and are inspired by helping enterprises and large organisations that rely on open source technologies for their business-critical applications. Inviqa offers web development, 24/7 support, PHP training and consulting services (audits, scalability challenges). 13