Continuous Delivery Benefits, Best Practices and Practical Advice

Transcription

1 Continuous Delivery Benefits, Best Practices and Practical Advice Jeffrey Hammond Forrester Research Ajit Zadgaonkar Edmunds.com Mark Warren Perforce Software

2 Continuous Delivery: A Key Enabler of Feedback and Flow Jeffrey Hammond, VP, Principal Analyst Sept. 24, 2013

3 How Would You Describe Your Release Management Process? Source: Flickr ( 2 Entire contents 2010 Forrester Research, Inc. All rights reserved.

4 Developers Employ Agile Practices Which of the following project management practices does your development team currently use? Formal code review Short development iterations Test-driven development Daily standup/scrum meetings Product owner involved in development Use visualization, visual prototyping, storyboarding, or wire- Run retrospectives at the end of each development phase Use a burn-down list to prioritize day-to-day activity Pair programming Code coverage threshold must be met for release Stage-gate project approvals Kanban board used to show progress A/B testing used to evaluate proposed changes Minimum viable product approach used to focus releases None of the above 8% 8% 6% 5% 3% 3% 24% 20% 15% 12% 13% 19% 29% 28% 35% Base: 698 Professional Developers, Game Developers, IT Developers, Consultants Source: Forrsights Developer Survey, Q Forrester Research, Inc. Reproduction Prohibited 3

5 But Spend More Time On Than Deploying Code or Writing Tests How much time do you spend in a typical work day doing the following activities? 65% % of Developer That Spend More Than an Hour A Day 52% 50% 49% 38% 35% 26% 23% 19% 14% 12% Base: 651 Professional Developers, Game Developers, IT Developers, Consultants Source: Forrsights Developer Survey, Q Forrester Research, Inc. Reproduction Prohibited 4

6 This type of thinking is becoming increasingly obsolete Source: 2013 Forrester Flickr Research, ( Inc. Reproduction Prohibited 5

7 9/ M+ Android activations per day, 500 million+ total devices 9/ M+ ios devices sold to date Over 700M smart phones sold worldwide in 2012 Samsung Galaxy SIII: 96 million, iphone 5: 54 million The Mobile Shift is upon us Source: Flickr ( 6 Entire contents 2010 Forrester Research, Inc. All rights reserved.

8 IT Shops Struggle To Keep Up With Mobile How often does your development team release mobile apps? Other (please describe) 2% 4% We release patches for defects as needed and then release major updates when we are ready 10% 13% Releases to coincide with new versions of device OS 8% 12% Every month or more (as needed) 5+ regular releases/yr/device 2% 11% 13% 11% IT Dev Pro Dev 2-4 regular releases/yr/device 28% 39% <=1 release/yr/device 25% 21% 0% 20% 40% 60% Base: 172 Professional software developers and 87 IT developers building mobile applications Source: Forrsights Developer Survey, Q Forrester Research, Inc. Reproduction Prohibited 7

9 Mobile apps are just part of modern applications 2013 Forrester Research, Inc. Reproduction Prohibited 8

10 The way we develop is changing 7 Traits of Modern Applications 1. Omni-channel clients 2. Deployed on elastic infrastructure 3. Aggregate discrete services 4. Use managed APIs 5. Integrate open source software 6. Employ dev-ops techniques 7. Focus on measurable feedback

11 Delivering modern applications is hard Lifecycle Focus Time to Feedback Systems of Engagement Time to Certainty Systems of Record Time to Safety Systems of Operation

12 Getting real customer feedback is hard without frequent production releases!

13 Continuous Delivery Enables Fast Feedback Flow

14 What s your Null Release cycle time? What is the Null Release? If we changed one line of code in our application (or system), how long would it take us to deploy it into production using our regular release process?

15 5 ways to streamline delivery 1. Improve pre-build processes 2. Expand release management throughput 3. Optimize your release pipeline 4. Architect software for rapid change 5. Create a common release portal

16 Improving pre-build processes 1. Outlaw private builds and release streams 2. Employ gated checkins and pre-flight builds 3. Make your continuous integration process change aware Garbage in, garbage out

17 Expanding release throughput 1. Apply the concept of a release pipeline to your work. 2. Use "smoke tests" to evaluate builds quickly Sprint 1 Sprint 5 3. Decompose each process into the smallest possible logically independent steps Sprint 2 Sprint 3 X Sprint 6 Sprint 7 X 4. Automate repetitive steps wherever possible Sprint 4 Sprint 8 5. Use parallelism to speed up builds

18 Adopt A Simple Release Pipeline

19 Optimize your delivery pipeline Use virtual images to enforce consistency Use parallelism to compress the testing phase Use masked production data to improve testing fidelity Implement a developer self-service environment Use configuration management to automate deployment Use service virtualization to speed testing

20 An Optimized Delivery Pipeline

21 Architect Software For Rapid Change Modularize your architecture Manage deployment by configuration Design systems to support "blue-green" deployment Use runtime configurations of features to avoid rollbacks Web services, message passing to decouple layers

22 Create A Common Release Portal What you need 1. Status of the release pipeline 2. Drilldown metrics on individual releases 3. Consistent locations and links for "latest releases. 4. Links to all configuration recipes and instances 5. Alerting

23 Tools help increase your rate of delivery Portfolio Management Release Planning SCM Work Item Management and Tracking Continuous Integration and Build Management Test Management And Execution Release Automation and Orchestration Configuration Management Service Management Monitoring and APM 2012 Forrester Research, Inc. Reproduction Prohibited 22

24 The need for speed may vary Hypothesis driven Systems of Engagement Release on demand Systems of Record Milestone builds Systems of Operation

25 When release management works Amazon a release every 11.6 seconds, leading public cloud infrastructure Netflix testing in production w/ the Simian army, ~30% of NA internet traffic Instagram 0 14 million customers in 1 year, massive wealth creation for the dev team Eclipse foundation 50M LOC shipped like clockwork, every year for 6+ years Forrester Research, Inc. Reproduction Prohibited 24

26 Next steps 1. Make sure you are planning for faster releases as part of your modern application strategy 2. Assess your current level of maturity, and what it takes to get to the next level 3. Understand the cultural changes required to get to a state of continuous delivery 4. Tools and practices will help reinforce cultural change 5. Segment opportunities by complexity and experience and act accordingly 2013 Forrester Research, Inc. Reproduction Prohibited 25

27 Thank you Jeffrey Hammond

28 Continuous Delivery The Role of Version Management Mark Warren Product Strategy Director Perforce Software

29 Perforce Software Headquartered in Alameda, California, international operations in the United Kingdom, Canada and Australia. Trusted with storing and versioning the most valuable IP for the world's most innovative companies Over 10,000 customers worldwide From start-ups to Fortune 100 The leading cloud companies

30 Development Build & Test Release Operations & Monitoring CD Enabler Version Management SCM

31 Version Everything Java C#.doc.jpg Build Script Libraries.java.exe/.dll.jsp Ruby Not Just Software Developers Documenters, authors, DB artists,.mp4.svg Compilers project managers, Script release managers, Natural tools/ui/workflows.war.db COBOL PHP All.xls Teams.docx and Projects Installer Wherever they are Env. Config CSS Python.proj

32 Version Everything CCP Games Version absolutely *everything* Branching where it makes sense Nobody is born a versioning hero Versioning everything requires an investment. It s worth it. { } Versioning is the nerve center of the organization

33 Mainline Release Branching Strategy R R R Release 1.0.x merge bugfix R R Release 1.1.x merge bugfix Mainline Visibility Always deployable Continuous Integration, only create branches for releases Manage what gets into mainline

34 Mainline Salesforce.com 150 teams, 5,000 testing VMs, 500,000 files. Up to 10 million Perforce transactions per day, heading towards million Mainline code model Fast feedback is key Security, audit, compliance. Easy audit process Perforce is boring For ops team, that s a very good thing! { } Mainline is critical to collaboration

35 Best Practices Version *everything* Automate everything Love the Operations team Invest in infrastructure Feedback early & often Used by everyone

36 For more information For more use cases and information on the role of version management perforce.com

37 Continuous Delivery Ajit Zadgaonkar Sr Director, Software Engineering Sept 24, 2013 Copyright Edmunds.com, Inc. (the Company ). Edmunds and the Edmunds.com logo are registered trademarks of the Company. This document contains proprietary and/or confidential information of the Company. No part of this document or the information it contains may be used, or disclosed to any person or entity, for any purpose other than advancing the best interests of the Company, and any such disclosure requires the express approval of the Company.

38 Edmunds.com 170M monthly page views 13M monthly unique visitors 500 Employees Externally Exposed APIs: 175+ apps /month active developers

39 Software Delivery Pitfalls Longer shelf life of features waiting for releases Lower code readiness confidence Recurring code integration issues Release qualifications & promotions were ~ 90+man days Business was dependent on release schedules

40 Need for CD We wanted to Be more agile, lean and nimble towards delivering features to our consumers Operate in a proactive and faster manner Not compromise quality, stability and reliability.

41 Transformation Path Only mainline code branch Ensure backwards and forwards compatibility Keep code in good state Avoid code merge Reliable, stable automated tests and monitoring Automated deployments

42 The Difficult

43 Challenges Doing it right Automate for scalability and flexibility Employ tools to gain efficiency Transition to adoption Having automated tests result into higher confidence Code compatibility Accepting feedback

44 Critical Success Factors Everything is code Code need to be production ready all the time Remove conflicting or stale dependencies Clean up redundant code (feature, dependencies, code, monitors) Let test results define code/feature readiness Time is of the essence Faster running code, sooner feedback Build time, test execution time Immediate feedback is key - delay is build break! Fix-it-now (drop everything) culture Fixing issues in our work is part of the same day, same iteration

45 Critical Success Factors Feedback matters, not the mechanism Must be followed up with immediate actions i.e. Code corrections Tests added / fixed / removed Deployments scripts fixed, Application configuration updated Feedback Accountability Alerts / Monitors are for taking actions Feedback improves us, You will get feedback when you are open You can t improve what you don t measure No tests, no benchmark, no monitors = no feedback

46 Workflow Checkin > static code scanning > unit Tests > code coverage > build (parallel) > repository > CD invoke > deploy (parallel)> post deployment checks >smoke tests > regression tests > compatibility tests > Load Performance tests > metric monitors > promote > smoke test > monitor

47 Check-in Workflow CD invoke deploy(parallel) Static code scanning Load Performance tests repository unit tests Post deployment checks build(parallel) Compatibility tests code coverage smoke tests regression tests Metric monitors promote Smoke test monitor

48 Overview: Environment Continuous Delivery CI Build Watch DEV QA PROD Deploy Deploy Deploy Test/Monitoring Outside view: it s automation Inside view: it s confidence

49 We did it, you can too

50 Jeffrey Ajit Zadgaonkar: Mark