Scalable Application. Mikalai Alimenkou

Transcription

1 Scalable Application Development on AWS Mikalai Alimenkou

2 Background Java Technical Lead/Scrum Master at Zoral Labs 7+ years in software development 5+ years of working by Agile methodologies Expert in Agile engineering practices Founder and coach at XP Injection Organizer of Selenium Camp, JEEConf, XP Days Ukraine, IT Brunch conferences

3 No Future!

4 Cloud computing Scale capacity on demand Turn fixed costs into variable costs Clean APIs and conceptual models Always available Cost-effective Reduced time to market Focus on product

5 Interesting startup Completely in Java Customer wants to use AWS New project arrived

6 AWS structure Elastic Compute Cloud (EC2) Elastic Map Reduce (EMR) Auto Scaling Elastic Load Balancing Simple Storage Service (S3) Elastic Block Store (EBS) Import/Export Storage Gateway SimpleDB DynamoDB Relational Database Service (RDS) ElastiCache Compute Storage Database Route 53 Virtual Private Cloud (VPC) Direct Connect CloudFront Networking Simple Queue Service (SQS) Simple Notification Service (SNS) Simple Service (SES) Simple Workflow Service (SWS) CloudSearch App Services CloudWatch Elastic Beanstalk CloudFormation Identity and Access Management (IAM) Deployment

7 Why Java? Runs everywhere Full technological stack Lots of tools for scalability Many frameworks for cloud development

8 EC2 quick introduction $.08-$2.4 per server hour $.05 - $.12 per GB data transfer

9 Simple boot model Instant-store AMI Stateless Cheaper Better IO Easier to scale EBS boot AMI Stateful Stop/start support Quick migration Faster loading AMI Instance Cloud

10 Different API access levels AWS management console AWS console tools REST API clients Plugins to IDEs and browsers

11 Elasticfox my superhero!

12 Everything on AWS Infrastructure Testing Staging Production Monitoring

13 Easy and reliable server configuration with AMI No work for system administrator Any number of servers available No waits for hardware More experimenting and testing Easy security model Happiness

14 Look ma, no hands!

15 What about money?

16 EC2 charging model

17

18 What I have for such money? Testing Server DB Server Production Server

19 Ways to save money Good and responsible team Reuse instances when possible rounding usage time Release instances when not more needed Monitor and control running instances Use reserved instances for continuously running jobs

20 EC2 physical distribution

21 Hourly payment is invisible Instances are virtual Same type of machine, but different performance Instance not immediately available Images creation require magical skills No multicast messaging

22 Amazon cares about you and other people

23 Everything is unreliable Retry policies everywhere Timeouts for all operations Automatic recovery

24 EBS quick introduction Raw data volumes from 1GB to 1TB Replicated automatically Snapshots to S3 May be used with different instances

25 EBS architecture

26 EBS fault tolerance

27 EBS cost model Not so cheap Additional costs for snapshots in S3

28 EBS!= hard drive Slow for time critical tasks Bottleneck for Lucene indexing and DB Use parallel EBS volumes Separate data storage between EC2 instances Don t use RDS Limited to 1TB Virtual RAID Data distribution

29 S3 quick introduction $ $.125 per GB month storage $.05 - $.12 per GB data transfer

30 Load balancing with JMS or AMQP JMS and AMQP are very scalable and flexible Costs are less than SQS (Simple Queue Service) Configurable load balancing

31 Deployment options

32 Self-distributed system Host manager works via Java AWS API Centralized deployment Code distribution to EC2 hosts

33 Reduce coupling to AWS

34 Conclusions AWS is great for startups or research projects AWS increases development speed in times It is easy to create scalable and highly available Java application on AWS Development, testing and production may be done on AWS AWS save development costs AWS support Agile developmernt approaches

35 Useful books

36 @xpinjection