DO NOT DISTRIBUTE. Automating Applications with Continuous Delivery on AWS. Student Guide. Version 1.0

Transcription

1 Automating Applications with Continuous Delivery on AWS Student Guide Version 1.0

2 Copyright 2014 Amazon Web Services, Inc. or its affiliates. All rights reserved. This work may not be reproduced or redistributed, in whole or in part, without prior written permission from Amazon Web Services, Inc. Commercial copying, lending, or selling is prohibited. Corrections or feedback on the course, please us at: For all other questions, contact us at: All trademarks are the property of their owners Amazon Web Services, Inc. and its affiliates. All rights reserved. ii

3 Table of Contents Welcome to Automating Applications with Continuous Delivery on AWS... 1 Module 1: Your Application Deployments with AWS Management Services... 3 Module 2: Advanced Use of CloudFormation Module 3: Source Code Management & Automated Testing Module 4: Continuous Integration Concepts Module 5: Continuous Deployment Concepts Amazon Web Services, Inc. and its affiliates. All rights reserved. iii

4 Version 1.0 Welcome to: Automating Applications with Continuous Delivery on AWS Dean Samuels AWS Solution Architect What we will be covering today Check In Build & Test Git Atlassian Bamboo Tests Pass Deploy Adrian White AWS Solution Architect Elastic Beanstalk/AWS OpsWorks/AWS CloudFormation Repo CI Tools Orchestration Tools Code Test Build Provision Monitor Teardown 1 Continuous Integration and Continuous Deployment Pipeline Automating Applications with Continuous Delivery on AWS Application Deployment with AWS Management Services Amazon Web Services, Inc. and its affiliates. All rights reserved. 1

5 What you will be deploying today Developers Amazon Amazon Route 53 Amazon Route 53 Amazon Route 53 Route 53 Elastic Load Elastic Balancing Load Elastic Balancing Load Elastic Balancing Load Balancing DynamoDB Amazon S3 Auto Scaling group Auto Scaling group Auto App Scaling Instance group App Instance Auto App Scaling Instance group App Instance App Instance App Instance AWS Elastic Beanstalk App Instance App Instance AWS CloudFormation or AWS Elastic CloudFormation Shared Beanstalk or stacks AWS CloudFormation AWS Elastic CloudFormation Beanstalk Amazon RDSstacks or AWS Elastic CloudFormation Beanstalk or stacks Elastic Beanstalk stacks Automating Applications with Continuous Delivery on AWS Application Deployment with AWS Management Services Amazon Web Services, Inc. and its affiliates. All rights reserved. 2

6 Version 1.0 Welcome to: Automating Applications with Continuous Delivery on AWS Module 1: Your Application Deployments with AWS Management Services 1 Automating Applications with Continuous Delivery on AWS Application Deployment with AWS Management Services Amazon Web Services, Inc. and its affiliates. All rights reserved. 3

7 Infrastructure Environments Be prepared to be running multiple environments Development / Integration Testing/QA Staging/Pre-prod Production They should be running as close to the same stack as possible Use configuration management and infrastructure orchestration tools No one off hosts A goal: Go from nothing to fully running instances without human intervention Automating Applications with Continuous Delivery on AWS Application Deployment with AWS Management Services IF YOU CAN PROGRAM IT YOU CAN AUTOMATE IT 3 Automating Applications with Continuous Delivery on AWS Application Deployment with AWS Management Services Amazon Web Services, Inc. and its affiliates. All rights reserved. 4

8 What s the difference between CI and CD? Continuous Delivery is a method that advocates the creation of an automated deployment pipeline to release software rapidly and reliably into production Continuous Deployment is the next step of continuous delivery: Every change that passes the automated tests is deployed to production automatically. Continuous Integration Frequent integration of mainline code allows developers to rapidly discover integration problems and reduce development costs. Dev Deployment Pipeline Code Config Tests Commit to Git/master Version Control Config Get / Pull Code Repo Package Builder Push CI Server Generate Artifacts Distributed Builds Run Tests in parallel Deploy Server Test Env Staging Env Prod Env 6 Send Build Report to Dev Stop everything if build failed Automating Applications with Continuous Delivery on AWS Application Deployment with AWS Management Services Amazon Web Services, Inc. and its affiliates. All rights reserved. 5

9 Continuous Integration Pipeline DNS SOURCE CODE REPOSITORY CONTINUOUS INTEGRATION SERVER Automating Applications with Continuous Delivery on AWS Application Deployment with AWS Management Services AWS CloudFormation BUILDS PROJECT MANAGEMENT SERVER Elastic Beanstalk AMIs or App Bundles AWS OpsWorks AWS API 9 Automating Applications with Continuous Delivery on AWS Application Deployment with AWS Management Services Amazon Web Services, Inc. and its affiliates. All rights reserved. 6

10 Software Code Packaging and Delivery Options Software Repository vs Inventory of AMIs Your Code Your Code Fetch on Boot Your Code Fetch on Boot Web Container Framework Platform OS Web Container Framework Platform OS Container Framework Platform Automating Applications with Continuous Delivery on AWS Application Deployment with AWS Management Services 11 Web OS Install on Boot Eg with AWS CloudFormation Eg with Elastic Beanstalk Eg with AWS OpsWorks or AWS CloudFormation AWS Application Management Services Application Container Application & Resource Management Provision and manage AWS Resources You want to quickly get code running scalably and reliably without worrying about the underlying resources A shrink wrapped approach You want to run an application with the ability to customize and control software or resources DevOps Automation You want a simple, yet finely controllable way to model and version collections of AWS resources Fine grained control Elastic Beanstalk OpsWorks CloudFormation Amazon Web Services, Inc. and its affiliates. All rights reserved. 7

11 AWS OpsWorks A stack represents your application. One stack might be used for staging and another for production. AWS OpsWorks A layer defines how to setup and configure a set of instances and related resources such as volumes and software. Simply tell OpsWorks where it can find your code and define any additional deployment tasks. Then OpsWorks will take care of deploying your app in your desired configuration. Scale your stack based on time or load. Clone your production stack to a different region. Set up user permissions and access. Automate workflows for common tasks. setup configure deploy undeploy shutdown Amazon Web Services, Inc. and its affiliates. All rights reserved. 8

12 Demo Time Deploying Your Apps with AWS OpsWorks Automating Applications with Continuous Delivery on AWS Application Deployment with AWS Management Services CloudFormation Procedural definition Create it programmatically Parameter driven Dynamic and user-driven templates Template Known configuration Store stack configuration in source control Collaboration Share templates with ease as just files 16 Automating Applications with Continuous Delivery on AWS Application Deployment with AWS Management Services Amazon Web Services, Inc. and its affiliates. All rights reserved. 9

13 CloudFormation CloudFormation Template Stack JSON formatted file Parameter definition Resource creation Configuration actions Framework Stack creation Stack updates Error detection and rollback Automating Applications with Continuous Delivery on AWS Application Deployment with AWS Management Services Configured AWS services Comprehensive service support Service event aware Customisable Demo Time Deploying Your Apps with AWS CloudFormation 18 Automating Applications with Continuous Delivery on AWS Application Deployment with AWS Management Services Amazon Web Services, Inc. and its affiliates. All rights reserved. 10

14 Elastic Beanstalk IDE plugins Java.war file PHP.zip file Microsoft Web Deploy package Elastic Beanstalk User Application Application Service HTTP Service Language Interpreter Operating System Python.zip file Git integration 20 Host Amazon Web Services, Inc. and its affiliates. All rights reserved. 11

15 Elastic Beanstalk Demo Time Deploying Your Apps with Elastic Beanstalk 22 Automating Applications with Continuous Delivery on AWS Application Deployment with AWS Management Services Amazon Web Services, Inc. and its affiliates. All rights reserved. 12

16 Blue/Green Deployments Avoiding Downtime Avoiding Downtime Feature Flags A/B testing Managing costs by using A/B deployments Automating Applications with Continuous Delivery on AWS Application Deployment with AWS Management Services Blue/Green Deployments 100% Amazon Route 53 ELB EC2 Instances 24 DynamoDB MySQL RDS Instance ElastiCache Cache Node Amazon Web Services, Inc. and its affiliates. All rights reserved. 13

17 ELB EC2 Instances Amazon Route 53 90% 10% DynamoDB Amazon Route 53 MySQL RDS Instance 50% 50% ELB EC2 Instances ELB EC2 Instances ElastiCache Cache Node ELB EC2 Instances Auto scaling Max instances Min instances Scaling Trigger Custom Metrics Upper Threshold Lower Threshold Increment by, Update Policy As we shift more traffic over, let auto-scaling grow/shrink our instances of the new or old application 26 DynamoDB MySQL RDS Instance ElastiCache Cache Node Amazon Web Services, Inc. and its affiliates. All rights reserved. 14

18 ELB EC2 Instances DynamoDB EL B EC2 Instances Amazon Route 53 0% 100% MySQL RDS Instance Amazon Route 53 0% 100 % ELB EC2 Instances ElastiCache Cache Node ELB EC2 Instances As we shift more traffic over, let autoscaling grow/shrink our instances of the new or old application Shut down the old when no traffic there 28 DynamoDB MySQL RDS Instance ElastiCache Cache Node Amazon Web Services, Inc. and its affiliates. All rights reserved. 15

19 DynamoDB Amazon Route 53 MySQL RDS Instance 100 % ELB EC2 Instances ElastiCache Cache Node Blue/Green Deployments: Dealing with databases example Hot Deployment techniques Simple backup & restore (RDS user-initiated snapshots) Break large jobs into a series of backward compatible tasks & decouple Application deployment & DB migration 30 Automating Applications with Continuous Delivery on AWS Application Deployment with AWS Management Services Amazon Web Services, Inc. and its affiliates. All rights reserved. 16

20 Blue/Green Deployments: Dealing with databases example DB V1.1 V2.1 ID NAME ADDRESS ORDERID (Char) Joe Doe xxx Rob Smith xxxx Jane Smith xxxx Anne Lee xxxx V2.1 Deployment Timeline Blue/Green Deployments: Dealing with databases example Add a new column Deployment Timeline ID NAME ADDRESS ORDERID ORDERID_INT Joe Doe xxx Rob Smith xxxx Jane Smith xxxx Anne Lee xxxx Amazon Web Services, Inc. and its affiliates. All rights reserved. 17

21 Dealing with databases example DB V1.1 DB V1.1 Add a New Column V2.1 Change Apps to write to both new & old columns ID NAME ADDRESS ORDERID ORDERID_INT Joe Doe xxx Rob Smith xxxx Jane Smith xxxx Anne Lee xxxx Mark Fox cxxxx Dealing with databases example V2.1 DB V1.2 Add a New Column Deployment Timeline Change Apps to write to both new & old columns Migrate old column data to new column ( for older rows) Deployment Timeline ID NAME ADDRESS ORDERID ORDERID_INT Joe Doe xxx Rob Smith xxxx Jane Smith xxxx Anne Lee xxxx Mark Fox cxxxx Amazon Web Services, Inc. and its affiliates. All rights reserved. 18

22 Dealing with databases example DB V1.1 V2.1 DB V1.2 V2.2 Add a New Column Change Apps to write to both new & old columns Migrate old column data to new column ( for older rows) Update all Apps to read from new column ID NAME ADDRESS ORDERID ORDERID_INT Joe Doe xxx Rob Smith xxxx Jane Smith xxxx Anne Lee xxxx Mark Fox cxxxx Dealing with databases example DB V1.1 V2.1 DB V1.2 V2.2 V2.3 Add a New Column ID NAME ADDRESS ORDERID ORDERID_INT Joe Doe xxx Rob Smith xxxx Jane Smith xxxx Anne Lee xxxx Mark Fox cxxxx Mark Lee cxxxxx Deployment Timeline Change Apps to write to both new & old columns Migrate old column data to new column ( for older rows) Update all Apps to read from new column Update all Apps to only write to new column Deployment Timeline Amazon Web Services, Inc. and its affiliates. All rights reserved. 19

23 Dealing with databases example DB V1.1 V2.1 DB V1.2 V2.2 V2.3 ID NAME ADDRESS ORDERID_INT Joe Doe xxx Rob Smith xxxx 234 DB V Jane Smith xxxx Anne Lee xxxx Mark Fox cxxxx Mark Lee cxxxxx Add a New Column Change Apps to write to both new & old columns Migrate old column data to new column ( for older rows) Update all Apps to read from new column Update all Apps to only write to new column Drop the old column Deployment Timeline Deploying your Application Using AWS Management Services 38 Automating Applications with Continuous Delivery on AWS Application Deployment with AWS Management Services Amazon Web Services, Inc. and its affiliates. All rights reserved. 20

24 AWS Training & Certification Self-Paced Labs Training Certification Try products, gain new skills, and get hands-on practice working with AWS technologies aws.amazon.com/training/ self-paced-labs Skill up and gain confidence to design, develop, deploy and manage your applications on AWS aws.amazon.com/training Automating Applications with Continuous Delivery on AWS Application Deployment with AWS Management Services Demonstrate your skills, knowledge, and expertise with the AWS platform aws.amazon.com/certification Copyright 2014 Amazon Web Services, Inc. or its affiliates. All rights reserved. This work may not be reproduced or redistributed, in whole or in part, without prior written permission from Amazon Web Services, Inc. Commercial copying, lending, or selling is prohibited. Errors or corrections? us at For all other questions, contact us at: All trademarks are the property of their owners. 40 Automating Applications with Continuous Delivery on AWS Application Deployment with AWS Management Services Amazon Web Services, Inc. and its affiliates. All rights reserved. 21

25 Version 1.0 Welcome to: Automating Applications with Continuous Delivery on AWS Module 2: Advanced Use of CloudFormation 1 Automating Applications with Continuous Delivery on AWS Application Deployment with AWS Management Services Amazon Web Services, Inc. and its affiliates. All rights reserved. 22

26 Advanced use of CloudFormation Custom Resources Automating Applications with Continuous Delivery on AWS Application Deployment with AWS Management Services Amazon Web Services, Inc. and its affiliates. All rights reserved. 23

27 Parallel Stack Processing Conditions Dev 5 Prod Amazon Web Services, Inc. and its affiliates. All rights reserved. 24

28 Conditions "Parameters" : { "Environment" : { "Description" : "Specifies if this a Dev QA or Prod Environment", "Type" : "String", "Default" : "Dev", "AllowedValues" : [ "Dev", "QA", "Prod"] }, },... "Conditions" : { "ProdEnvironment" : { "Fn::Equals" : [ { "Ref" : "Environment" }, "Prod" ]} }, Conditions "DBInstance" : { "Type": "AWS::RDS::DBInstance", "Properties": { "DBName" : { "Ref" : "DBName" }, "Engine" : "MySQL", "MultiAZ" : { "Fn::If" : [ "ProdEnvironment", "true", "false" ] }, "DBSnapshotIdentifier" : { "Fn::If" : [ "ProdEnvironment", { "Ref" : "DBName" }, { "Ref" : "AWS::NoValue" } ] }, "DBInstanceClass" : { "Fn::If" : [ "ProdEnvironment", "db.m3.xlarge", "db.m3.medium" ] },... } }, Amazon Web Services, Inc. and its affiliates. All rights reserved. 25

29 Conditions "DBStorageAlarm" : { "Condition" : "ProdEnvironment", "Type" : "AWS::CloudWatch::Alarm", "Properties" : { "AlarmDescription" : "Alarm if db size grows beyond a threshold", "Namespace" : "AWS/RDS", "MetricName" : "FreeStorageSpace", }, }... Conditions Fn::If Fn::Equals Fn::Not Fn::And Fn::Or "Conditions" : {... "ProdOrLoadTestingEnv" : { "Fn::Or" : [ { "Condition" : "ProdEnvironment"}, { "Fn::Equals" : [... ]} ] } } "Fn::If": [{condition}, {value_if_true}, {value_if_false}] Amazon Web Services, Inc. and its affiliates. All rights reserved. 26

30 Bootstrapping user-data vs cfn-init user-data #!/bin/bash yum y install mysql mysql-server httpd php php-mysql wget unzip build-v4.zip -d /var/www/html wget -O \ /etc/httpd/conf/httpd.conf /sbin/chkconfig httpd on /sbin/service httpd start Only runs once at instance startup Bootstrapping user-data vs cfn-init cfn-init "AWS::CloudFormation::Init" : { "config" : { "packages" : { All that s required for instance UserData "yum" : { "mysql" : [], #!/bin/bash /opt/aws/bin/cfn-init s <STACK NAME> -r <LOGICAL_RESOURCE_ID> --region <REGION> "mysql-server" : [], "httpd" : [], "php" : [], "php-mysql" : [] } }, "sources" : { "/var/www/html" : }, "services" : { "sysinit" : { "httpd" : { Apply updates using cfn-hup service "enabled" : "true", "ensurerunning" : "true", "files" : ["/etc/nginx/nginx.conf"], "sources" : ["/var/www/html"] } } }, "files" : { "/etc/httpd/httpd.conf" : source : " } } Apply updates using cfn-hup service Amazon Web Services, Inc. and its affiliates. All rights reserved. 27

31 Bootstrapping user-data vs cfn-init cfn-init "AWS::CloudFormation::Init" : { "config" : { "packages" : { "yum" : { "mysql" : [], "mysql-server" : [], "httpd" : [], "php" : [], "php-mysql" : [] } }, "sources" : { "/var/www/html" : }, "services" : { "sysinit" : { "httpd" : { "enabled" : "true", "ensurerunning" : "true", "files" : ["/etc/nginx/nginx.conf"], "sources" : ["/var/www/html"] } } }, "files" : { "/etc/httpd/httpd.conf" : source : " } } Parameters and Mappings Apply updates using cfn-hup service Parameters are standard route into a stack Allow free-form user input Constrainable, but on a per-stack level Apply updates using cfn-hup service Mappings are traditionally used to map human-readable input to static values AMI IDs, instance type architectures, regional URLs However. Custom resources allow for centralized selection logic Lookups in: S3 DynamoDB/RDS APIs (EC2.DescribeImages, etc) Third Party datastore Amazon Web Services, Inc. and its affiliates. All rights reserved. 28

32 Custom Resources "mycustomresource" : { "Type": "Custom::MyCustomResource", } "Version" : "1.0", "Properties" : { } "ServiceToken": "arn:aws:sns:us-east-1:84969example:crtest", "CustomProperty" : "foo Automating Applications with Continuous Delivery on AWS Application Deployment with AWS Management Services Custom Resources "AMILookup": { }, "Type": "Custom::AmiLookup", "Properties": { } "ServiceToken": "arn:aws:sns:us-east-1:84969example:amilookup", "app": "WebApp", "version": "1.2", "os": "linux" "WebServer": { "Type": "AWS::EC2::Instance", } "Properties": { "ImageId" : { "Fn::GetAtt" : ["AMILookup", ami-id"] } } 15 Automating Applications with Continuous Delivery on AWS Application Deployment with AWS Management Services Amazon Web Services, Inc. and its affiliates. All rights reserved. 29

33 Custom Resources Create app:webapp version:1.2 os:linux 1 2 AWS CloudFormation 7 6 Output ami-id:ami-xxxxx Deploy EC2 Instance with AMI ami-xxxxx Custom Resource Topic Custom Resource Implementation Auto scaling Group DynamoDB Region SQS Queue Returns AMI id Query table, get AMI in ami-id key 5 4 ID where: app=webapp version=1.2 os is linux Third-Party Integration 3 Add New Resources Including AWS resources not currently supported by CFN Interact with the CloudFormation Workflow Inject dynamic data into a stack Extend the capabilities of existing resources Data management via CloudFormation It s really simple if you use aws-cfn-resource-bridge Install or fork from Major players: Chef Puppet Labs Puppet Ansible SaltStack Puppet and Chef are both enterprise-grade, ready for production Significant user bases within big organizations Battle-tested and proven Work across multiple operating systems Work in similar ways Have support options Ansible and SaltStack Becoming increasingly popular 17 Automating Applications with Continuous Delivery on AWS Application Deployment with AWS Management Services Amazon Web Services, Inc. and its affiliates. All rights reserved. 30

34 AWS Training & Certification Self-Paced Labs Training Certification Try products, gain new skills, and get hands-on practice working with AWS technologies aws.amazon.com/training/ self-paced-labs Skill up and gain confidence to design, develop, deploy and manage your applications on AWS aws.amazon.com/training Automating Applications with Continuous Delivery on AWS Application Deployment with AWS Management Services Demonstrate your skills, knowledge, and expertise with the AWS platform aws.amazon.com/certification Copyright 2014 Amazon Web Services, Inc. or its affiliates. All rights reserved. This work may not be reproduced or redistributed, in whole or in part, without prior written permission from Amazon Web Services, Inc. Commercial copying, lending, or selling is prohibited. Errors or corrections? us at For all other questions, contact us at: All trademarks are the property of their owners. 19 Automating Applications with Continuous Delivery on AWS Application Deployment with AWS Management Services Amazon Web Services, Inc. and its affiliates. All rights reserved. 31

35 Version 1.0 Welcome to: Automating Applications with Continuous Delivery on AWS Module 3: Continuous Integration and Deployment Source Code Management & Automated Testing Amazon Web Services, Inc. and its affiliates. All rights reserved. 32

36 What We Are Covering In This Module & Why Managing your source code Branching models supporting CI and CD Feature branching vs. long-lived branches Distributed vs. centralized source code management Source code management tools Automated testing Test driven development An example Python web application The CICD Pipeline Code Test Build Release Teardown Amazon Web Services, Inc. and its affiliates. All rights reserved. 33

37 The CICD Pipeline Code Code Test Build Release Teardown Distributed vs. Centralized Source Code Management Distributed Each developer gets their own source code repository, locally Supports parallel development workflows Local commit history Remove synchronizing around shared state, e.g. source code Central Commits are made against a centralized SCM Simplifies development workflow by forcing synchronization Reduces effectiveness of parallel streams of development on the same branch Amazon Web Services, Inc. and its affiliates. All rights reserved. 34

38 Branching Models Branching for features Branching for developers Prod Dev Feature Prod Alice Bob Branching for features can be much more effective when using CICD Amazon Web Services, Inc. and its affiliates. All rights reserved. 35

39 Source Code Management Tools Git SCM Behind the firewall or VPC Publicly hosted solutions Integrations with CI tool chains Others SVN Perforce Mercurial Managing Many Branches Feature Branching Prod UAT Feature Amazon Web Services, Inc. and its affiliates. All rights reserved. 36

40 Managing Many Branches Automated Merging UAT Feature 1 Feature 2 The CICD Pipeline Test Code Test Build Release Teardown Amazon Web Services, Inc. and its affiliates. All rights reserved. 37

41 Automated Testing Test driven and behavior driven development Reduce manual testing effort Improve consistency and accuracy through automation Inherit testing effort across future changes Automated Testing Unit Testing Test components of source code Helps find errors at a source code level early Supports code refactoring and simplifies integration Can provide detailed source level documentation Coupled with disciplines like TDD, ensures good test coverage Builds reusable test harnesses for future changes Amazon Web Services, Inc. and its affiliates. All rights reserved. 38

42 Automated Testing Code Coverage Measure how much code is covered by test suite Tests that all code is exercised Many approaches; function, statement, branch, exit Basic assumption: code not covered by tests is buggy Automated Testing What Else? Security and code auditing tests e.g. detecting SQL injection vulnerabilities Penetration testing Let AWS know before you do this on EC2 Automated load testing Distributed JMeter Bees with machine guns Web testing Selenium Amazon Web Services, Inc. and its affiliates. All rights reserved. 39

43 The Sample Web Application Self-Paced Labs Try products, gain new skills, and get hands-on practice working with AWS technologies aws.amazon.com/training/ self-paced-labs Setting up & Testing AWS Training & Certification Training Skill up and gain confidence to design, develop, deploy and manage your applications on AWS aws.amazon.com/training Certification Demonstrate your skills, knowledge, and expertise with the AWS platform aws.amazon.com/certification Amazon Web Services, Inc. and its affiliates. All rights reserved. 40

44 Copyright 2014 Amazon Web Services, Inc. or its affiliates. All rights reserved. This work may not be reproduced or redistributed, in whole or in part, without prior written permission from Amazon Web Services, Inc. Commercial copying, lending, or selling is prohibited. Errors or corrections? us at For all other questions, contact us at: All trademarks are the property of their owners Amazon Web Services, Inc. and its affiliates. All rights reserved. 41

45 Version 1.0 Welcome to: Automating Applications with Continuous Delivery on AWS Module 4: Continuous Integration and Deployment Continuous Integration Concepts Amazon Web Services, Inc. and its affiliates. All rights reserved. 42

46 What We Are Covering In This Module & Why The continuous integration pipeline SCM integration with feature branching Automated testing Some common CI tools Atlassian Bamboo Jenkins Demo: A CI pipeline for our example Python web application The CI Pipeline Code Test Build Release Teardown Amazon Web Services, Inc. and its affiliates. All rights reserved. 43

47 The CI Pipeline Code Test Deploy Release Teardown Code Test Deploy Release Teardown Prod UAT Feature 1 Code Test Deploy Release Teardown CI Tools Atlassian Bamboo & Jenkins Atlassian Bamboo Integration with other Atlassian toolchains Plugins on the Atlassian Marketplace, e.g. Tasks for AWS Vendor supported Jenkins Code Test Deploy Release Teardown Open source community Very widely deployed Broad plugin ecosystem Amazon Web Services, Inc. and its affiliates. All rights reserved. 44

48 Continuous Integration - Demo Staging a commit Bamboo has already been configured to run a build plan on commit Unit tests are automatically run Detailed feedback from build plan run (e.g. test log output) Green / red, or go / no-go The Sample Web Application Setting up & Testing Amazon Web Services, Inc. and its affiliates. All rights reserved. 45

49 Continuous Integration Self-Paced Labs Try products, gain new skills, and get hands-on practice working with AWS technologies aws.amazon.com/training/ self-paced-labs Atlassian Bamboo AWS Training & Certification Training Skill up and gain confidence to design, develop, deploy and manage your applications on AWS aws.amazon.com/training Certification Demonstrate your skills, knowledge, and expertise with the AWS platform aws.amazon.com/certification Amazon Web Services, Inc. and its affiliates. All rights reserved. 46

50 Copyright 2014 Amazon Web Services, Inc. or its affiliates. All rights reserved. This work may not be reproduced or redistributed, in whole or in part, without prior written permission from Amazon Web Services, Inc. Commercial copying, lending, or selling is prohibited. Errors or corrections? us at For all other questions, contact us at: All trademarks are the property of their owners Amazon Web Services, Inc. and its affiliates. All rights reserved. 47

51 Version 1.0 Welcome to: Automating Applications with Continuous Delivery on AWS Module 5: Continuous Integration and Deployment Continuous Deployment Concepts Amazon Web Services, Inc. and its affiliates. All rights reserved. 48

52 What We Are Covering In This Module & Why The continuous deployment pipeline Build, release, teardown Build approaches Baking AMIs Containers Demo: A CD pipeline for our example Python web application The CD Pipeline Code Test Build Release Teardown Amazon Web Services, Inc. and its affiliates. All rights reserved. 49

53 Build And Packaging Code Test Build Release Teardown Build And Packaging - Summary Baking an Amazon Machine Image (AMI) Immutable unit of deployment Ability to reference a point-in-time build artifact Robust roll forward and roll back Application packages Building a conventional application package Fast build and release cycle Containers Efficient build and release and immutable unit of deployment Amazon Web Services, Inc. and its affiliates. All rights reserved. 50

54 Build Baking An AMI Start a builder instance Bootstrap OS and application updates Bake your AMI Tag it Destroy/clean up the builder instance Build Bootstrapping Cfn-init, cfn-signal, cfn-hup Native AWS CloudFormation Chef & Puppet Re-use existing configuration management infrastructure Packer.io Can target different build outputs, e.g. AMIs and Docker containers Netflix Aminator (and other 3 rd party tools) Application update Bootstrap OS Builder instance Application update Bootstrap OS Builder instance Amazon Web Services, Inc. and its affiliates. All rights reserved. 51

55 Lightweight Linux Containers With Docker Docker provides Linux container infrastructure Lots of features: Very fast and cheap startup Portable across platforms Public/private registries Versioning support Re-use and container layers Automated bootstrapping Release Code Test Build Release Teardown Containers Docker AMI EC Amazon Web Services, Inc. and its affiliates. All rights reserved. 52

56 Release Approaches - Summary In-place to existing environment Managing interruption High risk Faster turn around time Rollback can be complex Rolling Release To Existing Environment Roll out change to each instance Don t release to all instances at once Think carefully about interruption Discrete environment Low risk A/B testing Testing in isolation Slower turn around time Rollback and roll-forward is easy Think carefully about state Amazon Web Services, Inc. and its affiliates. All rights reserved. 53

57 Rolling Release To Existing Environment Release carefully to each node and test Break connection between ELB and instance Use ELB connection draining Rolling Release To Existing Environment Release carefully to each node and test Break connection between ELB and instance Use ELB connection draining Amazon Web Services, Inc. and its affiliates. All rights reserved. 54

58 Rolling Release To Existing Environment Release carefully to each node and test Break connection between ELB and instance Use ELB connection draining Rolling Release To Existing Environment Release carefully to each node and test Break connection between ELB and instance Use ELB connection draining Amazon Web Services, Inc. and its affiliates. All rights reserved. 55

59 Rolling Release To Existing Environment What if applications versions can t co-exist? How do you do rollback with this approach? Is this complicated? Beanstalk also supports this method of deployment Discrete Environments AWS CloudFormation AWS CloudFormation and custom resources gives you all the flexibility you need Integration with CICD tool chains, e.g. Tasks for AWS plugin for Atlassian Bamboo Completely describes our environment and lifecycle Create Update Delete Amazon Web Services, Inc. and its affiliates. All rights reserved. 56

60 Discrete Environments - Release Lifecycle Release Releasing To Discrete Environment reinvent-1.cicd-bootcamp.com Test the new stack Match the traffic between the two stacks Update the floating DNS record Send Notification(s) ROLLBACK reinvent.cicd-bootcamp.com reinvent-2.cicd-bootcamp.com Amazon Web Services, Inc. and its affiliates. All rights reserved. 57

61 Traffic Matching For A Discrete Environment reinvent-1.cicd-bootcamp.com reinvent.cicd-bootcamp.com Releasing To Discrete Environment reinvent-1.cicd-bootcamp.com reinvent.cicd-bootcamp.com reinvent-2.cicd-bootcamp.com reinvent-2.cicd-bootcamp.com Amazon Web Services, Inc. and its affiliates. All rights reserved. 58

62 Rollback For A Discrete Environment reinvent-1.cicd-bootcamp.com reinvent.cicd-bootcamp.com Artifacts and Discrete Environments Build environments for artifacts, don t update environments with artifacts All environments are transient Standardisation, abstraction and portability reinvent-2.cicd-bootcamp.com Amazon Web Services, Inc. and its affiliates. All rights reserved. 59

63 Stateful Architectures But what about, Amazon Simple Queue Service Amazon DynamoDB Amazon Kinesis Amazon Relational Database Service? Application level awareness of state Application frameworks like Rails (Ruby), Django (Python) Application feature flags Teardown Code Test Build Release Teardown Amazon Web Services, Inc. and its affiliates. All rights reserved. 60

64 Tearing down discrete environments Teardown Continuous Delivery - Demo Staging a commit Ensure that no traffic is moving though ELB Teardown the CloudFormation Stack Deregister the AMI Bamboo is configured to run a build plan on commit for data stores and web application changes Data stores pipeline Web application pipeline Discrete stacks for each web application version Manual teardown (could be automated) Amazon Web Services, Inc. and its affiliates. All rights reserved. 61

65 Continuous Deployment Self-Paced Labs Try products, gain new skills, and get hands-on practice working with AWS technologies aws.amazon.com/training/ self-paced-labs Atlassian Bamboo AWS Training & Certification Training Skill up and gain confidence to design, develop, deploy and manage your applications on AWS aws.amazon.com/training Certification Demonstrate your skills, knowledge, and expertise with the AWS platform aws.amazon.com/certification Amazon Web Services, Inc. and its affiliates. All rights reserved. 62

66 Copyright 2014 Amazon Web Services, Inc. or its affiliates. All rights reserved. This work may not be reproduced or redistributed, in whole or in part, without prior written permission from Amazon Web Services, Inc. Commercial copying, lending, or selling is prohibited. Errors or corrections? us at For all other questions, contact us at: All trademarks are the property of their owners Amazon Web Services, Inc. and its affiliates. All rights reserved. 63