Scalable Application Development on AWS Mikalai Alimenkou http://xpinjection.com 11.05.2012
Background Java Technical Lead/Scrum Master at Zoral Labs 7+ years in software development 5+ years of working by Agile methodologies Expert in Agile engineering practices Founder and coach at XP Injection Organizer of Selenium Camp, JEEConf, XP Days Ukraine, IT Brunch conferences
No Future!
Cloud computing Scale capacity on demand Turn fixed costs into variable costs Clean APIs and conceptual models Always available Cost-effective Reduced time to market Focus on product
Interesting startup Completely in Java Customer wants to use AWS New project arrived
AWS structure Elastic Compute Cloud (EC2) Elastic Map Reduce (EMR) Auto Scaling Elastic Load Balancing Simple Storage Service (S3) Elastic Block Store (EBS) Import/Export Storage Gateway SimpleDB DynamoDB Relational Database Service (RDS) ElastiCache Compute Storage Database Route 53 Virtual Private Cloud (VPC) Direct Connect CloudFront Networking Simple Queue Service (SQS) Simple Notification Service (SNS) Simple Email Service (SES) Simple Workflow Service (SWS) CloudSearch App Services CloudWatch Elastic Beanstalk CloudFormation Identity and Access Management (IAM) Deployment
Why Java? Runs everywhere Full technological stack Lots of tools for scalability Many frameworks for cloud development
EC2 quick introduction $.08-$2.4 per server hour $.05 - $.12 per GB data transfer
Simple boot model Instant-store AMI Stateless Cheaper Better IO Easier to scale EBS boot AMI Stateful Stop/start support Quick migration Faster loading AMI Instance Cloud
Different API access levels AWS management console AWS console tools REST API clients Plugins to IDEs and browsers
Elasticfox my superhero!
Everything on AWS Infrastructure Testing Staging Production Monitoring
Easy and reliable server configuration with AMI No work for system administrator Any number of servers available No waits for hardware More experimenting and testing Easy security model Happiness
Look ma, no hands!
What about money?
EC2 charging model
What I have for such money? Testing Server DB Server Production Server
Ways to save money Good and responsible team Reuse instances when possible rounding usage time Release instances when not more needed Monitor and control running instances Use reserved instances for continuously running jobs
EC2 physical distribution
Hourly payment is invisible Instances are virtual Same type of machine, but different performance Instance not immediately available Images creation require magical skills No multicast messaging
Amazon cares about you and other people
Everything is unreliable Retry policies everywhere Timeouts for all operations Automatic recovery
EBS quick introduction Raw data volumes from 1GB to 1TB Replicated automatically Snapshots to S3 May be used with different instances
EBS architecture
EBS fault tolerance
EBS cost model Not so cheap Additional costs for snapshots in S3
EBS!= hard drive Slow for time critical tasks Bottleneck for Lucene indexing and DB Use parallel EBS volumes Separate data storage between EC2 instances Don t use RDS Limited to 1TB Virtual RAID Data distribution
S3 quick introduction $.055 - $.125 per GB month storage $.05 - $.12 per GB data transfer
Load balancing with JMS or AMQP JMS and AMQP are very scalable and flexible Costs are less than SQS (Simple Queue Service) Configurable load balancing
Deployment options
Self-distributed system Host manager works via Java AWS API Centralized deployment Code distribution to EC2 hosts
Reduce coupling to AWS
Conclusions AWS is great for startups or research projects AWS increases development speed in times It is easy to create scalable and highly available Java application on AWS Development, testing and production may be done on AWS AWS save development costs AWS support Agile developmernt approaches
Useful books
@xpinjection http://xpinjection.com mikalai.alimenkou@xpinjection.com