AWS Performance Tuning Markus Albe @Percona Fernando Ipar @Percona Ryan Lowe @Square PLNY 2012
Amazon Web Services Cloud Formation CloudFront CloudSearch CloudWatch DirectConnect DynamoDB ec2 ElastiCache Elastic Beanstalk Elastic MapReduce Glacier IAM RDS Route 53 S3 SES SNS SQS Storage Gateway SWF VPC
Amazon Web Services Cloud Formation CloudFront CloudSearch CloudWatch DirectConnect DynamoDB ec2 ElastiCache Elastic Beanstalk Elastic MapReduce Glacier IAM RDS Route 53 S3 SES SNS SQS Storage Gateway SWF VPC
Amazon Web Services Cloud Formation CloudFront CloudSearch CloudWatch DirectConnect DynamoDB ec2 ElastiCache Elastic Beanstalk Elastic MapReduce Glacier IAM RDS Route 53 S3 SES SNS SQS Storage Gateway SWF VPC
Amazon Web Services Cloud Formation CloudFront CloudSearch CloudWatch DirectConnect DynamoDB ec2 ElastiCache Elastic Beanstalk Elastic MapReduce Glacier IAM RDS Route 53 S3 SES SNS SQS Storage Gateway SWF VPC
Amazon Web Services Cloud Formation CloudFront CloudSearch CloudWatch DirectConnect DynamoDB ec2 ElastiCache Elastic Beanstalk Elastic MapReduce Glacier IAM RDS Route 53 S3 SES SNS SQS Storage Gateway SWF VPC
Amazon Web Services Cloud Formation CloudFront CloudSearch CloudWatch DirectConnect DynamoDB ec2 ElastiCache Elastic Beanstalk Elastic MapReduce Glacier IAM RDS Route 53 S3 SES SNS SQS Storage Gateway SWF VPC
Amazon Web Services Cloud Formation CloudFront CloudSearch CloudWatch DirectConnect DynamoDB ec2 ElastiCache Elastic Beanstalk Elastic MapReduce Glacier IAM RDS Route 53 S3 SES SNS SQS Storage Gateway SWF VPC
Amazon Web Services Cloud Formation CloudFront CloudSearch CloudWatch DirectConnect DynamoDB ec2 ElastiCache Elastic Beanstalk Elastic MapReduce Glacier IAM RDS Route 53 S3 SES SNS SQS Storage Gateway SWF VPC
Amazon Web Services Cloud Formation CloudFront CloudSearch CloudWatch DirectConnect DynamoDB ec2 ElastiCache Elastic Beanstalk Elastic MapReduce Glacier IAM RDS Route 53 S3 SES SNS SQS Storage Gateway SWF VPC
Amazon Web Services Cloud Formation CloudFront CloudSearch CloudWatch DirectConnect DynamoDB ec2 ElastiCache Elastic Beanstalk Elastic MapReduce Glacier IAM RDS Route 53 S3 SES SNS SQS Storage Gateway SWF VPC
Regions & Availability Zones
Regions US (x3) EU (Ireland) APAC (x2) South America (Sao Paulo) GovCloud (US Gov t & Contractors)
AZ Differences Performance Characteristics Hardware Pricing Availability
AZ Strategy Multi-AZ Deployment Assume Entire AZs fail Smart Software Cost Optimization
ec2
ec2 (Elastic Compute Cloud) The Cloud On-Demand computers with minimal administration Best for organizations with light uptime requirements, highly variable workloads, horizontally distributable data and/or little time/skill to manage hosts
ec2 Instance Types Micro Small Medium Large X-XXXXL High-Memory Provisioned IOPS EBS SSD
ec2 Instance Pricing Micro $0.025 Small $0.105 Large $0.415 XL $0.830 XXL $1.170 XXXXL $2.340 ($1600/month)
EBS (Elastic Block Store) Persistent Data Storage Up to 1TB* Limited to single availability zone Automatic Replication Multiple EBS per ec2 Instance
EBS Pricing Standard EBS $0.10 per GB provisioned $0.10 per million IO Provisioned IOPs $0.125 per GB provisioned $0.10 per provisioned IOPS-month EBS Snapshots to S3 $0.125 per GB-month of data stored
ec2 First Write Penalty
ec2 Hardware Measured in Cores with Compute Units One EC2 Compute Unit provides the equivalent CPU capacity of a 1.0-1.2 GHz 2007 Opteron or 2007 Xeon processor Performance Highly Variable. Empirical evidence suggests > 1% unusable!
ec2 Gotchas Many organizations don t know what their monthly bill will be until it s too late. Overprovisioning Underprovisioning Forgetting mandatory maintenance Never testing restores
16G 16T 8F [rw]/sec (Large w/raid10) Random Reads Random R/W Random Writes Seq R Seq W 150000 112500 75000 37500 0
16G 16T 8F [rw]/sec (Large w/ebs) Random Reads Random R/W Random Writes Seq R Seq W 300000 225000 150000 75000 0
EBS vs. Hardware EBS RAID10 SSD SAN IOPS 0 375000 750000 1125000 1500000
Benchmark Scripts https://launchpad.net/~vadim-tk
RDS
RDS (Relational Database Service) SQL Server, MySQL, Oracle If you need A relational database service with minimal administration Best for organizations with light uptime requirements & little time/skill to manage databases
RDS Caveats Mandatory 30-minute maintenance window PER WEEK Magic Backups Auto-Upgrade Minor Versions!
RDS Performance Relatively Sane Defaults Somewhat limited access to MySQL configuration variables Very close to the corresponding ec2 size
Monitoring
Monitoring ec2 http://status.aws.amazon.com
Monitoring Instances Basic Monitoring: 5-minute granularity Detailed Monitoring: 1-minute granularity Custom Metrics & Alarms AutoScaling based on Metrics!!
AutoScaling To autoscale or not to autoscale? Monitoring/Alerting on cost Detailed monitoring on some % Regular Reviews
Architecture
Partition by Function ElastiCache for sessions CloudSearch for... wait for it... Search Per-Service data stores
Shard Active-Active AZs Shard by GeoIP?
Chaos Monkey https://github.com/netflix/simianarmy
Conclusions No magic scaling bullet Common sense (and $) will get you far Understand all AWS offerings Software Architecture & Design FTW
Questions?