How To Implement Continuous Integration

Transcription

1 A Taming of the Tornado Using Ants on a Mobius Strip by M.C. Escher

2 Outline Integration Hell Software Development Nirvana What is Continuous Integration? Implementing Continuous Integration CI Toolbox Success Criteria Good Resources Questions, comments, stories, Contact info 2

3 Integration Hell 3

4 Software Development NIRVANA 4

5 Evolution of Continuous Integration 5

6 What is Continuous Integration? The macro process of object-oriented development is one of "continuous integration." ti... At regular intervals, the process of "continuous integration" yields executable releases that grow in functionality at every release.... It is through these milestones that management can measure progress and quality, and hence anticipate, identify, and then actively attach risks on an ongoing basis. -- Grady Booch, Object-OrientedOriented Analysis and Design with Applications, 2nd ed,

7 What is Continuous Integration? A common practice at Microsoft and some other shrink-wrap software companies is the daily build and smoke test process. Every file is compiled, linked, and combined into an executable program every day, and the program is then put through a smoke test, a relatively simple check to see whether the product smokes when it runs. -- Steve McConnell, Daily Build and Smoke Test, IEEE Software, July 1996 Microsoft called daily builds the Sync Pulse of a project -- Microsoft Secrets,

8 What is Continuous Integration? Continuous Integration is a software development practice where members of a team integrate their work frequently, usually each person integrates t at least daily - leading to multiple integrations per day. Each integration is verified by an automated build (including test) to detect integration errors as quickly as possible. Many teams find that this approach leads to significantly reduced integration problems and allows a team to develop cohesive software more rapidly. -- Martin Fowler, Continuous Integration,

9 What is Continuous Integration? Continuous Integration and the Möbius Strip Mobius Strip never-ending, one-sided shape continuous A surface with only one side and only one boundary component. mathematical property of being non-orientable a ruled surface discovered independently by the German mathematicians August Ferdinand Möbius and Johann Benedict Listing in

10 What is Continuous Integration? Provides a powerful method of finding defects early enough to fix them easily Continuous integration automatically: Integrating code frequently (2 hours recommended) Monitors source code Compiles after every change Unit tests compiled code Notifies developers of problems as encountered 10

11 Return on Investment Complexity = Cost Each connection represents potential for defect Less Complexity = smaller, manageable defects 11

12 What is Continuous Integration? Many statistics show projects fail more often than they succeed All studies agree: Projects very often fail. They are late. They are over-budget. They fail to deliver the product they were designed to produce. Some projects don't produce any product at all. 12

13 Why is it Important? -- History Lessons Toyota - Recalled 160,000 Prius (new hybrid) for software bug Mars Climate Orbiter - Destroyed due to navigation error Therac-25 - Delivered lethal l radiation doses 5 patients t died d National Cancer Institute - In Panama City, therapy planning software miscalculates radiation dosage - 28 overdoses, 8 died UK Air Traffic Control - Upgrade problems caused thousands of passenger delays and plane groundings across the country Denver International Airport - Baggage handling system problems. Cost estimate - $1.1 1 million lost per day FAA - Advanced Automation System overran planned budget by ~ $3 billion IRS - $8 billion software modernization program cost U.S. taxpayers $50 billion per year in lost revenue California motor vehicle - Registration system cancelled after topping $44 million in a long series of overruns B-2 bomber - Wouldn t fly on its maiden flight because of a software problem Ariane 5 rocket - Software error caused blow up on maiden launch Seattle, Computer-controlled ferries problems caused dozens of dozen dock crashes with damages > $7M 13

14 Why is it Important? -- The Future True Global Civilization - complete with a unified language, culture and planet-wide technological prowess. A global community capable of sustaining i and controlling its planet. Control of Global Warming - Human-caused global warming is visible in the air, water and melting ice, and is destined to get much worse in the future. Alternate Energies and Fuel - Managing the planet's resources effectively using advances in computing, engineering and other fields. Geothermal power showing promise. Cloak of Partial Invisibility - a cloaking device that reroutes certain wavelengths of light, forcing them around objects. the cloaked object would appear nearly invisible in microwave light. Molecular Memory - ultradense integrated memory circuit from nanowires and molecules developed. Problems are still being worked out of device; expect these as common by Microrobot - group of international scientists developing microrobot able to swim through arteries and digestive system. Exploding Robots - fleet of exploding probes could prepare p the way for warding off hazardous asteroids. Bomb/Drug Sniffing Robots - able to track dilute scents based on new smell-seeking algorithm called infotaxis. 14

15 Reality: Projects Fail - especially IT projects Battle cry is loud and clear Projects are failing more often than they are successful. Something must be done! But what??? One Good Answer: Continuous Integration 15

16 Not Just For Agile Popularized in the XP World Any Software Development Lifecycle benefits from Continuous Integration Requirements Analysis Find problems early Correct problems while fresh and less complex Promote communications/collaboration Preliminary Design Detailed Design Code & Unit Test Overlapping Waterfall Project Life Cycle Requirements Analysis System Integration & Test Preliminary Design Formal Test Detailed Design Rational Unified Process (RUP) Build 1. Requirements Analysis Preliminary Design Detailed Design Requirements Serial, Incremental, or Iterative Build Analysis Waterfall Project Life Cycle Project Code Life & Cycle Build n Unit Test Code & Unit Test Preliminary Design Build 1 Requirements Analysis System Integration Build 1... n & Test System Detailed Code & Integration Design Unit Test Formal Test & Test Note that the builds can be serial or overlapping Build n Preliminary Design Detailed Design Final System Integration & Test Scrum Spiral Project Life Cycle 1. Define Approach 2. Analyze & Avert Risk Management Review System Integration & Test Evolutionary Project Life Cycle Definition Review Develop/Update Estimations Define Cycle /Update Cycle Definition Update Plan Update Procedures Formal Test Review Progress Baseline Product. Identify Risks Analyze Risks Evaluate Risks Plan Risk Mitigation Monitor & Review Develop & Verify Product Risk Analysis Review Plan Commitment t Risk Walkthrough Development Plan Commitment Code & Unit Test Product Review 4. System Manage Integration & Plan 3. Develop Product 16 Formal NoteTest & that Test the figure only represents one cycle in the spiral. Formal Test

17 Getting from Here to There Are you in the tornado? How can you start the path to Nirvana? It is a matter of continuous incremental improvements! A CI platform becomes a place to bolt on repeatability 17

18 The CI Process 18

19 The CI Process (continued) 19

20 Getting There Backlog: Establish source repository and standards Identify and configure the Continuous Integration Server (the build machine ) Write a continuous integration script Automate the build process and install into CI tool Automate the deployment process and install into CI tool Automate the Unit Test Suite execution and install into CI tool Establish software development team standards to create (and increase coverage of) automated testing 20

21 Source Repository and Standards We must be able to revert to previous versions of any artifact. This is critical in being able to quickly recover from a CI build failure. Rule Version everything needed for the build. Don t version things that get built. Why Everything goes into the repository. Everything includes: test scripts, properties files, database schema, install scripts, SQL scripts, third party libraries, documentation Nothing that gets built through a tool should be stored in the repository, only those things that are needed to perform a build. 21

22 The CI Server This is the brains of CI Install and configure a CI tool Candidates: CruiseControl from ThoughtWorks ( Anthill Professional from Urbancode ( com) Repository Communications Portal ( , web interface, ) CI Server Deployment Servers 22

23 Clean Machine Deployment Clean Machine Automation This should include only those tools that are too difficult to script the installation. This would include the operating system, compilers, JVM,.Net run-time, database servers Deb s picture here Repository Communications Portal ( , web interface, ) CI Server Deployment Servers 23

24 Script the CI Process See Process Diagram from previous slide Timely Feedback Typically, less than 15 minutes. If more, consider a staged build approach Rule Maintain clean machine build scripts Launch the CI scripts from a single command. Why Scripts are written and maintained in such a way that someone should be able to walk up to a clean machine, execute the scripts, and fully build the system. Simplicity. 24

25 Automated Builds Automate the build from source to executable Common tools such as make and ant can be used Can identify other tools/best practices that should be included in automated build Candidates: Build documentation (e.g. JavaDocs) Automated code standards (e.g. FxCop, CheckStyle, and Jalopy) Automated test coverage checkers (Did the developer embed automated testing?) Rule The build process must be reusable across Dev, QA, and Production The output of the build should be placed in a location where anyone can retrieve it Why This creates repeatability as moving from one software development stage to the next. Builds need to be accessible to anyone on the team 25

26 Automated Deployments Automate the Deployment to the Clean Machine Typically done with the OS and DB scripting languages Rule The deployment script should allow different deployment targets. After deployment, system should execute a Smoke Test to ensure proper configuration Why This creates repeatability as moving from one software development stage to the next. This will allow the production support team, and others, to use the script for deployment and have some validation of proper configuration without running through the full Unit Test Suite 26

27 Automated Unit Tests Automate the execution of a Test Suite of Unit Test Cases Choose a testing tool for each language Candidates: Java - Junit,.Net - Nunit, The entire suite of tests should be able to be executed from a single command Establish automated testing standard Rule Why Always add tests to code when a code This will maintain and/or increase test module is created or modified coverage of code base. Synchronize with repository before checking in. Always synchronize your working copy with changes in the repository (mainline or branch) h)before checking code in. 27

28 Other Standards Rule Only check it in if it works. Run the scripts before you check in. Everyone commits every day. Why The mainline and branch must be kept in working condition so that others can safely check out working copies. Always run the automated build/deploy and test script in your local working copy and development environment before checking code in. This approximates the CI process as closely as possible and will avoid those pesky failure s. By doing this frequently, developers quickly find out if there's a conflict between two developers. The key to fixing problems quickly is finding them quickly. If the CI build breaks, fix it right The whole point of working with CI is that you're away. always developing on a known stable base. 28

29 Success Criteria Be prepared to invest some time and effort in the beginning Saves time later Builds quality into product Ensure you are working with the latest source code Never commit broken code Always do a clean build Never use CI as a shaming event Distribute ib failure feedback as quickly as possible to team Build/Compile/Test cycle timing should be reasonable Break build up into logical chunks Add more machines Minimize testing if Make the CI process a team decision Look at third-party tools rather than build your own tool 29

30 Continuous Integration Toolbox Tool Considerations Functionality Compatibility Reliability Longevity Usability 30

31 Continuous Integration Toolbox Typical Continuous Integration ti Toolbox Continuous Integration System provides reports, statistics, ti ti notifications, etc. to furnish the framework of the CI build process Build Tools scripting or automating ti code compilation to convert software to run on a computer Repository where software components are stored Automated Testing Tools automating the unit or developer level testing 31

32 Continuous Integration Toolbox Continuous Integration System Build Tools Repository Automated Testing Tools CruiseControl Beetlejuice Shell/command SubVersion NUnit (.NET) Hudson LuntBuild Continuum Gump Sin Parabuild script Ant NAnt CVS Visual Sourcecode JUnit (Java) SUnit (Smalltalk) Draco.NET Pulse Maven PVCS PYUnit CI Factory TeamCity Groovy MKS (Python) Drumbeat CI Bamboo MSBuild Dimensions CPPUnit (C++) Tinderbox Make Vault UnitTest BuildBot Visual Studio Surround Dogtail (GUI) Anthill Mercurial HttpUnit (Web) CMSynergy Parasoft Many tools available are Open Source and Free 32

33 Good Resources Continuous Integration: Improving Software Quality and Reducing Risk by Paul Duvall, Steve Matyas, and Andrew Glover Martin Fowler's introduction to Continuous Integration: Extreme Programming Website: t / /i t t h tml Daily Build and Smoke Test by Steve McConnell: 33

34 Questions, comments, stories, 34

35 Contact Us President/CEO Tod Pryor Business Development/ Process Improvement Deb Jacobs M Circle Omaha, NE Phone: Toll free: 866.PTI.CORP Fax: