Achieving Continuous Delivery with AWS Lambda and Codeship
Timeline Ajay Nair from AWS Introduction to AWS Lambda Florian Motlik from Codeship Introduction to Codeship and Deploying to AWS Lambda Martin Tajur How Pipedrive uses Codeship and AWS Lambda Q&A
FAQ We re recording and will share the recording soon after Tweet with #awslambdawebinar Ask questions through GoToWebinar Question UI
Introduction to Lambda Ajay Nair
First, a little bit about Lambda
AWS Lambda COMPUTE SERVICE EVENT DRIVEN Run code without thinking about servers Code only runs when it needs to run
AWS Lambda: Benefits SERVERLESS EVENT-DRIVEN SCALE SUBSECOND BILLING
AWS Lambda: Capabilities BRING YOUR OWN CODE COMPUTE POWER LEVELS FLEXIBLE INVOCATION PATHS GRANULAR PERMISSIONS CONTROL
AWS Lambda: Putting it to work Mobile and Web backends File processing with Amazon S3 Amazon Dynamo DB triggers Microservices with API Gateway Amazon Echo skills AWS CloudFormation custom resources Amazon Kinesis processors Customized notifications with Amazon SNS Amazon Cognito triggers and the list will continue to grow!
AWS Lambda: How it works AUTHOR CONFIGURE DEPLOY LOG AND MONITOR
Codeship and Lambda Flo Motlik
Build - Release Cycle Build Validate Test Release
The top lesson that Cockcroft learned at Netflix is that speed wins in the marketplace. If you ask any developer whether a slower development process is better, no one ever says yes.
Codeship in a Nutshell
AWS Lambda on Codeship
{ "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Action": [ "lambda:updatefunctioncode", "lambda:updatefunctionconfiguration", "lambda:invokefunction", "lambda:getfunction" ], "Resource": [ "arn:aws:lambda:your_aws_region:your_aws_ac COUNT_ID:function:YOUR_FUNCTION_NAME" ] } ] }
Dockerized Codeship
Support any Software???????????
Flexible Workflows
Parity between local & CI & Production
Sign up at http://beta.codeship.com
How we use AWS Lambda to process incoming emails Martin Tajur CTO, Co-Founder
Web-based CRM for small teams with big ambitions.
Web-based CRM for small teams with big ambitions. Founded in 2010.
Web-based CRM for small teams with big ambitions. Founded in 2010. Used by well over 10,000 businesses worldwide.
Web-based CRM for small teams with big ambitions. Founded in 2010. Used by well over 10,000 businesses worldwide. 100+ employees, venture funded (BVP, Series A in 2015)
Web-based CRM for small teams with big ambitions. Founded in 2010. Used by well over 10,000 businesses worldwide. 100+ employees, venture funded (BVP, Series A in 2015) Very end user focused, helping the actual sales person do their job.
What we do with Lambda
What we do with Lambda TL;DR: We process incoming emails in Lambda.
What we do with Lambda
What we do with Lambda Gmail
What we do with Lambda Gmail Receive
What we do with Lambda Gmail Receive Process
What we do with Lambda Gmail Receive Process Store
What we do with Lambda Gmail Receive Process Store We process incoming emails in Lambda.
Pipedrive Gmail integration Work with your deals and contacts straight in the CRM
Pipedrive Gmail integration Work with your deals and contacts straight in the CRM Send and receive emails within the CRM
Pipedrive Gmail integration Work with your deals and contacts straight in the CRM Send and receive emails within the CRM Contact history stored
Pipedrive Gmail integration Work with your deals and contacts straight in the CRM Send and receive emails within the CRM Contact history stored No juggling between apps
Pipedrive Gmail integration Work with your deals and contacts straight in the CRM Send and receive emails within the CRM Contact history stored No juggling between apps Keep your focus
Pipedrive Gmail integration Work with your deals and contacts straight in the CRM Send and receive emails within the CRM Contact history stored No juggling between apps Keep your focus Close more deals
Pipedrive Gmail integration Work with your deals and contacts straight in the CRM Send and receive emails within the CRM Contact history stored No juggling between apps Keep your focus Close more deals $$$
Our Lambda function
Our Lambda function Called mail-message-parser
Our Lambda function Called mail-message-parser Node.js, Javascript based application
Our Lambda function Called mail-message-parser Node.js, Javascript based application Takes in regular HTML, returns nicer HTML
Our Lambda function Called mail-message-parser Node.js, Javascript based application Takes in regular HTML, returns nicer HTML Covered with 105 unit tests
Our Lambda function Called mail-message-parser Node.js, Javascript based application Takes in regular HTML, returns nicer HTML Covered with 105 unit tests Deployable zip file is 14MB (includes all npm modules)
What we do with Lambda Fix broken HTML emails
What we do with Lambda Fix broken HTML emails Filter out unwanted parts of HTML (<script>, attached event handlers, etc)
What we do with Lambda Fix broken HTML emails Filter out unwanted parts of HTML (<script>, attached event handlers, etc) Consolidate and localize stylesheets in emails
We process 300,000 emails per day on average
We process 300,000 emails per day on average of the 4,000 users enrolled to the Gmail sync feature on Pipedrive.
Processing incoming emails is...
Processing incoming emails is... Highly fluctuating in volume
Processing incoming emails is... Highly fluctuating in volume High peak hours (20k+ emails per hour), lower on weekends (2k per hour), bursts up to 200 emails/sec.
Processing incoming emails is... Highly fluctuating in volume High peak hours (20k+ emails per hour), lower on weekends (2k per hour), bursts up to 200 emails/sec. CPU intensive
Without Lambda, we d put it to a VM or a container.
Without Lambda, we would either...
Without Lambda, we would either......provision as much as instances as needed to serve the peak hours Bad. Waste of resources during less busy hours. Est. cost around $150/mo.
Without Lambda, we would either......provision as much as instances as needed to serve the peak hours Bad. Waste of resources during less busy hours. Est. cost around $150/mo....set up instances as autoscale group Bad. Complexity. Lots of moving parts to manage. Complex deployments. Est. cost up to $100/mo.
Without Lambda, it would cost us $100-$150/month. (At current scale.)
Our monthly spend on Lambda?
Our monthly spend on Lambda? $8
Without Lambda at scale?
Without Lambda at scale? 4K users, 300K emails/day = $150/month
Without Lambda at scale? 4K users, 300K emails/day = $150/month 100K users, 7.5M emails/day = $3750/month
With Lambda at scale?
With Lambda at scale? 4K users, 300K emails/day = $8/month
With Lambda at scale? 4K users, 300K emails/day = $8/month 100K users, 7.5M emails/day = $200/month
Without Lambda, it would take a day or two to set it all up.
Lambda production deployment setup through Codeship took us
Lambda production deployment setup through Codeship took us 20 minutes.
Time spent on scaling it down the road 0 hours.
How we deploy to Lambda
How we deploy to Lambda GitHub based workflow branch out write code pull request merge repeat
How we deploy to Lambda GitHub based workflow branch out write code pull request merge repeat Unit tests executed on each pull and commit on Codeship.
How we deploy to Lambda GitHub based workflow branch out write code pull request merge repeat Unit tests executed on each pull and commit on Codeship. Merge to master = production deployment, via Codeship.
Works perfectly for us
Works perfectly for us 0 hours spent on provisioning VMs, worrying about setting up autoscaling or other fine-tuning.
Works perfectly for us 0 hours spent on provisioning VMs, worrying about setting up autoscaling or other fine-tuning. 0 hours spent on configuration management.
Works perfectly for us 0 hours spent on provisioning VMs, worrying about setting up autoscaling or other fine-tuning. 0 hours spent on configuration management. $$$ saved on having the resources be provisioned 24/7.
Works perfectly for us 0 hours spent on provisioning VMs, worrying about setting up autoscaling or other fine-tuning. 0 hours spent on configuration management. $$$ saved on having the resources be provisioned 24/7. 20 minutes spent on setting up continuous delivery.
Works perfectly for us 0 hours spent on provisioning VMs, worrying about setting up autoscaling or other fine-tuning. 0 hours spent on configuration management. $$$ saved on having the resources be provisioned 24/7. 20 minutes spent on setting up continuous delivery. Developers can focus on the parser, not the deployment or supporting tools.
Thank you. Martin Tajur @tajur on Twitter www.pipedrive.com With this promo code CODESHIPROCKS 2 months free
Q&A Ajay Nair - @ajaynairthinks Florian Motlik - @flomotlik Martin Tajur - @tajur