PUBLIC CLOUD USAGE TRENDS

Transcription

1 PUBLIC CLOUD USAGE TRENDS 450 COMPANIES 165,000 INSTANCES 5.5 PB OF STORAGE FIRST QUARTER 2013 DAVID FEINLEIB UNDERWRITTEN BY thebigdatagroup.com Copyright 2013 The Big Data Group, LLC bigdatalandscape.com

2 Key Findings Public cloud buyers can achieve significant savings by optimizing usage Buyers continue to underutilize reserved and spot instances. Nearly a quarter of instance reservations are unused. Significant opportunity for cost-savings exists across reserved instances, underutilized instances, and shadow version storage consumption. Instances are underutilized with average utilization rates between 8-9% 52% of customers use reserved instances, but only 26% of all instances are reserved and 24% of instance reservations are unused Amazon Glacier, which was introduced in August of 2012, now accounts for 15% of storage usage Non-EC2 accounts for 38% of cost 2

3 Study Goals Benchmark industry public cloud costs Enable buyers to improve operational efficiency based on proven ROI model Expand on Public Cloud Cost Q Benchmark Study to provide additional market insights 3

4 Study Data 165,000 Amazon EC2 instances 6,700,000 instance hours 5.5 PB of storage Measured Mar 1, Mar 31,

5 Customer Distribution by Spend Just 4% of customers spend more than $1M annually, but account for more than half of total spend Customers who spend less than $50K per year make up the largest group of AWS users, yet account for only 4% of total spend. Just 4% of customers spend more than $1M per year, yet account for some 52% of total spend. Percent of Customers by Group First Quarter 2013 Percent of Spend by Group First Quarter % 52% 29% 22% 11% 2% 4% 4% 5% 10% < $50K $100K - $500K > $1M < $50K $100K - $500K > $1M 5

6 Total Spend by Service Type Public cloud spend isn t just on EC2-- non-compute services account for 38% of total spend While EC2 accounts for 62% of total public cloud spend, storage services such as EBS, RDS, and S3, along with other services, account for more than a third of total spend. EBS 8% RDS 7% S3 6% EC2 62% Other services 17% Other services: I/O, API calls, and bandwidth 6

7 Storage Usage by Type S3 remains the most popular and expensive storage type Simple Storage Service (S3) standard remains the most popular and expensive storage type. Glacier is gaining in popularity, likely due to data lifecycle management that can move old S3 objects to Glacier automatically. Storage Usage by Type First Quarter 2013 Cost per GB (First 1 TB per Month) April 2013 RR 12% Glacier 15% $0.10 $0.08 Standard S3 73% $0.01 RR: Reduced Redundancy Standard S3 Reduced Redundancy Glacier 7

8 Storage Visibility Public cloud users may be consuming nearly 10% extra storage without knowing it Shadow versions of files account for some 8.6% of versioned S3 storage. Shadow versions refer to multiple versions of objects stored in S3 that are difficult to see because the original named versions have been deleted, while the other versions still exist. Shadow Versions First Quarter 2013 Hidden 8.6% Visible 91.4% 8

9 Instance Types by Spend The m1.large instance type accounts for the biggest spend The m1.large and m2.4xlarge instance types account for the most spend, followed by c1.xlarge, m1.xlarge, and c1.medium. Top 5 Instance Types by Spend Q and Q % Q Q % 17.5% 18.0% 7.7% 11.6% 9.9% 10.0% 7.7% 7.6% m1.large m2.4xlarge c1.xlarge m1.xlarge c1.medum *According to Cloudyn benchmarking, two m1.medium instances deliver 30% more performance than a single m1.large instance. 9

10 Instance Usage by Type Small and Large are the most used instance types The m1.small and m1.large instances are the most popular instance sizes, followed by t1.micro, c1.medium, and m1. Medium. Top 5 Runtime Instance Types First Quarter % 19.3% 11.8% 11.4% 11.0% m1.small m1.large t1.micro c1.medium m1.medium 10

11 Usage Trends by Instance Type Customers are actively moving from Large to Medium instances Customers are actively moving from m1.large to m1.medium instances. Two m1.medium instances cost the same as one m1.large, but can produce 30% more output.* Top 5 Runtime Instance Types Q and Q % 24.8% 24.1% Q Q % 14.2% 11.8% 10.5% 11.4% 11.0% 6.9% m1.small m1.large t1.micro c1.medium m1.medium *According to Cloudyn benchmarking, two m1.medium instances deliver 30% more performance than a single m1.large instance. 11

12 Reserved Instance Usage Customers can achieve cost-savings by putting unused reservations to work More than half of all customers make some use of Reserved Instances (RIs). Customer Usage of Reserved Instances in Deployment Mix First Quarter 2013 Use RIs 52% Don t Use RIs 48% 12

13 Usage by Instance Pricing Model On-demand instances remain the choice of most customers Reserved Instances account for only 26% and spot instances for only 3% of total instance runtime usage. Despite their potential cost-saving benefits, customers remain hesitant to adopt these pricing models. Runtime Instances First Quarter 2013 Reserved 26% Spot 3% On-demand 71% 13

14 Reserved Instance Usage Customers can achieve cost-savings by putting unused reservations to work Some 24% of instance reservations are unused. Customers that are purchasing reserved instances aren t making use of nearly a quarter of them. Instance Reservations First Quarter 2013 Unused 24% Used 76% 14

15 Utilization of Most Common Instance Types Most instances are largely underutilized, presenting significant opportunity for costsavings The most common instance types are largely underutilized, with average utilization rates at just 8-9%. 91.0% 92.0% 91.0% Unutilized Utilized 9.0% 8.0% 9.0% m1.small m1.medium m1.large 15

16 Instance Types by Maximum Utilization Spiky usage may be one reason buyers of larger instances are reticent to move to smaller instances Less than half of all m1.medium and m1.large instances in the study exceeded 50% CPU usage, making them potential candidates for smaller instances. More than half of t1.micro instances had 75% utilization or greater and more than half of m1.small and c1.medium instances had 50% or greater CPU usage, making them potential candidates for larger instances. t1.micro 2% 10% 19% 16% 8% 45% m1.small 8% 11% 18% 20% 21% 22% m1.medium 9% 12% 35% 19% 10% 15% c1.medium 9% 12% 17% 23% 26% 13% m1.large 15% 18% 24% 25% 9% 9% < 10% 10%-24% 25%-49% 50%-74% 75%-89% > 90% 16

17 Emerging Trends Despite their potential cost-saving benefits, reserved instances remain underused As the range of AWS services continues to expand, hidden costs, such as those associated with storage in the form of shadow versions, can add up to big dollars Amazon Glacier has achieved significant traction in a short amount of time, likely due to its integrated data lifecycle management 17

18 Recommendations For Buyers Customers can achieve significant cost savings through proper public cloud management Evaluate Instance types relative to utilization Unused instance reservations Unreserved instances Consider Switching from Large to Medium instances Migrating infrequently-used data to Glacier Retiring unattached storage Ensure total response time stays within acceptable range ( seconds) 18

19 For More Information David Feinleib The Big Data Group consults with technology vendors and buyers on marketing strategy and market research. The Big Data Group produces The Big Data Landscape, Big Data Trends, Big Data TV, and The Big Data 100. Cloudyn s set of power tools helps companies optimize their cloud deployments. Organizations using Cloudyn have comprehensive insight and control over their cloud costs and usage. 19

20 Appendix 20

21 Customer Distribution by Spend Customers who spend less than $50K per year make up the largest group of AWS users, yet account for only 4% of total spend. Just 4% of customers spend more than $1M per year, yet account for some 52% of total spend. Customers by Spend Category First Quarter % 52% 29% 22% 4% 11% 10% 5% 2% 4% < $50K $50K - $100K $100K - $500K $500K - $1M > $1M % Customers % Spend 21

22 Utilization of Most Common Instance Types Most instances are largely underutilized, presenting significant opportunity for costsavings The most common instance types are largely underutilized, with average utilization rates at just 8-9%. 9.0% 8.0% 9.0% m1.small m1.medium m1.large 22

23 Instance Definitions For the purposes of this study, instances were grouped into Micro, Small, Medium, Large, and XLarge types. The instances included in each type are shown below. Type Micro Small Medium Large XLarge Instances t1.micro m1.small c1.medium, m1.medium m1.large *.*xlarge 23