Modern CI/CD and Asset Serving Mike North 10/20/2015
About.me Job.new = CTO Job.old = UI Architect for Yahoo Ads & Data Organizer, Modern Web UI Ember.js, Ember-cli, Ember-data contributor OSS Enthusiast
Agenda Manual Integration Manual Releases Modern Asset Serving Continuous Integration Continuous Deployment
Integration (working together)
Integration MASTER FEATURE BRANCHES
Integration Hell Merge Conflicts Duplicate Work Difficult to automate Detect problems late Lack of visibility
Continuous Integration (CI) Technology + People Stay close to master Tests are automatically run on each small change Alert team when tests fail! Team keeps the pipeline healthy
Continuous Integration (CI) Staying close to master Hide (and merge) incomplete features It s ok to stub things out in the beginning Modular design practices Bottom up Top down and stub
Continuous Integration (CI) Test each change - Push vs PR build PULL REQUEST PUSH BUILD PR BUILD FEATURE BRANCH HEAD MASTER
Continuous Integration (CI) Test each change - Green Light?? Never Tested later
Continuous Integration (CI) Good team practices Pipeline breaks? Treat like a critical bug Don t pile on top of a break Assume things will fail Write tests that give you release confidence
Continuous Integration (CI) Keep the pipeline fast Trunk Staging Production 60m 10m 60m to fix
Continuous Integration (CI) Keep the pipeline fast Trunk Staging Production 10m 60m 10m to fix
Continuous Integration (CI) Keep the pipeline fast Selenium is slow JS tests (QUnit, Mocha) are fast Useless tests === tech debt Hitting a real API in your UI s CI pipeline should be kept at a minimum
Mocking an API Pretender.js Hijacks the XMLHttpRequest object Responds to requests with pre-defined fixtures Allows for passthrough on defined URLs Throws errors whenever unexpected requests are sent
Pretender github.com/ pretenderjs/ pretender Setup Server Define JSON Fixture XHR
Deployment (the shipping it part)
Releases Manual releases are painful, expensive demoralizing and risky Human testing scales terribly
Continuous Deployment (CD) Code goes from master to production, w/o human intervention required Heavy emphasis on automated testing Release early and often!
Continuous Deployment (CD) Good team practices Flow to production only when you want Report released changes back to the team Master === Production App versioning (SHA?)
Continuous Deployment (CD) Canary Environment 10% 90% Trunk Staging Production (Canary) Production (GA) Unit, Functional, Integration Tests Prod Verification Tests 24h Wait
Continuous Deployment (CD) Benefits Increased reliability, visibility, velocity, flexibility, focus, agility Reduced pressure to prematurely ship Deliver incremental value to users, early Consistency
Continuous Deployment (CD) for single-page apps index.html app.js vendor.js REDIS mystyle.css CDN
Asset Serving (the delivering it part)
Asset Serving Often overlooked, but important part of SPA development What do we want? Fast initial page load Maintenance page, API is down page, etc Notify users of new version available Canary environment
Asset Serving Fast initial page load S3 is not a CDN - Latency matters! CSS, JS, Images should be cached, index.html should not index.html app-1ab41cd781.js vendor-ab818d2175.js mystyle-b41cd1a832.css
Asset Serving for single-page apps *.{js, css, png, etc } index.html Server REDIS
Asset Serving Canary Environment - Multi tenancy Canary and GA environment are separate versions index.html index.html app-1ab41cd781.js vendor-ab818d2175.js app-cbab412.js vendor-abcd1d12.js mystyle-b41cd1a832.css mystyle-b41cd1a832.css
Asset Serving Canary Environment - Multi tenancy There s a version for each git SHA Via query param, you can ask for a version myapp:a1b231c myapp:d1241b myapp:abc11db http://myapp.com?key=bc147ba
Asset Serving Canary Environment - Multi tenancy There s a version for each git SHA Via query param, you can ask for a version There are also some named versions, that refer to other versions myapp:current myapp:a1b231c myapp:d1241b myapp:canary myapp:abc11db myapp:a1b231c myapp:d1241b
Asset Serving Canary Environment - Multi tenancy host & paths app myapp:current myapp:a1b231c localhost:3000/* myapp:current myapp:a1b231c lvh.me/* myapp:current myapp:d1241b canary.lvh.me/* myapp:canary myapp:canary myapp:d1241b myapp:abc11db Add a URL-to-App table to the mix
Asset Serving Maintenance Mode host & paths app myapp:current maintenance:current localhost:3000/* myapp:current myapp:a1b231c lvh.me myapp:current myapp:d1241b canary.lvh.me myapp:canary myapp:canary myapp:d1241b myapp:abc11db maintenance:1dbabc1 maintenance:current maintenance:1dbabc1
Asset Serving Making a PR Build available PULL REQUEST myapp:current maintenance:current PUSH BUILD PR BUILD myapp:a1b231c myapp:d1241b myapp:canary myapp:d1241b HEAD myapp:abc11db FEATURE BRANCH maintenance:1dbabc1 maintenance:current maintenance:1dbabc1 MASTER myapp:bc147ba http://myapp.com?key=bc147ba
A new version is here! Click below to start using it! http://myapp.com?key=bc147ba Asset Serving Notifying existing clients about deploys myapp:current maintenance:current myapp:a1b231c myapp:d1241b myapp:canary myapp:d1241b WebSocket myapp:abc11db maintenance:1dbabc1 maintenance:current maintenance:1dbabc1 myapp:bc147ba Pub/Sub Slack, Github, etc
It s coming https://github.com/mike-north/banker
TL;DR Manual testing and releasing can be awful at scale CI/CD is an investment worth making Don t treat your SPA like an API Your asset serving layer can boost productivity! Check out mike-north/banker soon for a turnkey implementation!