Continuous Delivery Workshop deployment pipelines Workshop materials created by Jez Humble, Martin Fowler, Tom Sulston, & Neal Ford
deployment pipelines tests, synergistic practices, incremental deployment data & infrastructure
Who are You? Manager DBA BA Operations Developer UX QA
Agile Transformation Manager DBA BA Operations Developer UX QA
Continuous Delivery Manager DBA BA Operations Developer UX QA
It s a people problem.
The Dangers of Silos The Tragedy of the Commons
Release Cadence
Release Cadence Mary Poppendieck Tom Poppendieck How long would it take your organization to deploy a change that involves just one single line of code? devopsy.com/blog/2013/08/16/devops-kata-single-line-of-code/ Can you do this on a repeatable, reliable basis?
Continuous Delivery Metrics lead time the time between the initiation and completion of a production process. cycle time the total elapsed time to move a unit of work from the beginning to the end of a physical process
Continuous Integration Integration early and often. Everyone checks into trunk at least once a day.
1200 Bring the pain forward. 900 eager vs. late pain 600 300 0 time
Continuous Integration Integration early and often. Everyone checks into trunk at least once a day.
Integration Integration early and often. Everyone checks into trunk at least once a day. Continuous Deployment Deploy as the final stage of continuous integration.
Integration Integration early and often. Everyone checks into trunk at least once a day. Deployment Deploy as the final stage of continuous integration. Continuous Delivery Software is always in a deployable state.
User Interface (Web, Swing) Business Analysts QA Domain Logic (Model) Application Server / Spring Persistance (OR Mapper) Firewall External Stuff Storage Application External Endpoints Mainframe Computer Modern software is complex!
Agile 101 "Agile" team Analysis + Design Development Centralized QA Integration + QA IT Operations Release and operation Customer Testing + Showcase Iteration 0 1 2 3 4 The "last mile"
Continuous Delivery "Agile" team Analysis + Design Development Centralized QA Integration + QA IT Operations Release and operation Customer Testing + Showcase Iteration 0 1 2 3 4 The "last mile" Customer Delivery team Constant flow of new features into production always business needs > operational constraints production ready
Potential Hindrances Lead time is too long Last mile is too painful Poor collaboration
Identify & remove friction
Continuous Integration Fast, automated feedback on the correctness of your application every time there is a change to code
Deployment Pipeline Fast, automated feedback on the production readiness of your application every time there is a change to code, infrastructure, or configuration production readiness 22
Deployment Pipelines
Prerequisites continuous integration comprehensive configuration management excellent automated testing at all levels
commit Stage Commit stage Version control source code commit tests build scripts Compile Unit test Assemble Code analysis deployable binaries test reports metadata Artifact repository Run against each check-in (continual integration) Starts building a release candidate If it fails, fix it immediately
Pipeline Construction commit functional test user acceptance test staging artifact respository artifact respository artifact respository artifact respository increasing confidence in production readiness Pipeline stages = feedback opportunities
UAT Stage Version control Artifact repository acceptance tests deployment scripts configuration data binaries Acceptance test stage Configure environment Deploy and smoke test Acceptance test Tear down test reports metadata Artifact repository End-to-end tests in production-like environment Triggered when upstream stage passes First DevOps-centric build 27
Manual Stage Version control Artifact repository deployment scripts configuration data binaries Later stages Configure environment Deploy and smoke test Tear down on request test reports metadata Artifact repository UAT, staging, integration, production, Push versus Pull model Deployments self-serviced through pushbutton process 28
Source code Env & app config Version control Env & app config Developers See code metrics and test failures Testers Self-service deployments UAT Configure environment Deploy binaries Smoke test Commit stage Compile Commit tests Assemble Code analysis Acceptance stage Configure environment Deploy binaries Smoke test Acceptance tests Capacity stage Configure environment Deploy binaries Smoke test Run capacity tests reports binaries metadata binaries reports metadata Operations perform push-button releases binaries Production Configure environment Deploy binaries Smoke test reports metadata Artifact repository 29
Machinery continuous integration ++ www.thoughtworks.com/products/go-continuous-delivery
www.go.cd
Increasing confidence in build's production readiness Environments become more production-like Commit stage Compile Unit test Analysis Build installers Acceptance test stage User acceptance testing Performance testing Production Faster feedback
chicken counting servers 0 1
Increasing confidence in build's production readiness Environments become more production-like Commit stage Compile Unit test Analysis Build installers Acceptance test stage User acceptance testing Performance testing Production Faster feedback
pipeline anti-patterns Increasing confidence in build's production readiness insufficient parallelization Environments become more production-like Commit stage Compile Unit test Analysis Build installers Acceptance test stage User acceptance testing Performance testing Production Faster feedback ideal time: < 10 minutes
pipeline anti-patterns Increasing confidence in build's production readiness insufficient parallelization Environments become more production-like Commit stage Compile Unit test Analysis Build installers Acceptance test stage User acceptance testing Performance testing Production Mingle: Faster feedback 3,282 test / 53 computers = ~1 hour
insufficient parallelization heuristic: make your pipeline wide, not long reduce the number of stages as much as possible parallelize each stage as much as you can create more stages if necessary to optimize feedback
pipeline anti-patterns inflexible workflow pipeline fans out as soon as it makes sense to do so
The Deming Cycle
The Scientific Method hypothesize experiment evaluate
principles of software delivery
principles automate almost everything build, deploy, test, release manual testing, approvals automatable high-value, humans only
principles keep everything you need to build, deploy, test, & release in version control requirements documents test scripts automated test cases network configuration scripts technical documentation database creation, upgrade, downgrade, and initialization scripts application stack configuration scripts libraries deployment scripts tool chains
When You Hire a New Developer
Infrastructure Consistency boxen.github.com
If it hurts... pain... do it more often bring the pain forward time principles
principles Cease dependence on mass inspection to achieve quality. Improve the process and build quality into the product in the first place build quality W. Edwards Deming in
continuous improvement Deming cycle principles
practices only build your binaries once
practices deploy the same way to every environment
practices smoke test your deployments
practices keep your environments similar
if anything fails, stop the line practices
Metrics & Monitoring
the driver for agile in the enterprise 21st century agility (business + IT) selling continuous delivery visibility risk mitigation responsiveness (cycle time)
nealford.com @neal4d