Economic Cloud Computing What to keep in mind when using the Cloud...

Transcription

1 Economic Cloud Computing What to keep in mind when using the Cloud... Gary A. McGilvary edinburgh data.intensive research 1

2 OUTLINE 1. Introduction 2. Uncontrollable Cloud Costs 3. Human Factors Affecting Cost Motivation Cloud/Human Factors Cloud Load Instance Processors Cloud Correctness End-user Location Job Configuration Instance Security 4. Conclusions 2

3 INTRODUCTION Investigate factors influencing cost Cloud Computing Provider Underlying performance - Cloud load vs time of day - Instance processors Correctness of recorded cloud usage Human Factors End-user Location Configuration - Underutilization Security Lower costs for cloud users Optimal Execution Platform (etc...)

4 SPRINT Simple Parallel R INTerface - provides parallel functions of R Functions: - pcor: parallel correlation pcor(t (x, y = x) - pmaxt: parallel permutation test pmaxt(x, classlabel, B=150000)

5 OUTLINE 1. Introduction 2. Uncontrollable Cloud Costs Motivation Cloud/Human Factors Cloud Load Instance Processors Cloud Correctness 3. Human Factors Affecting Cost 4. Conclusions/Future Work 5

6 UNCONTROLLABLE COSTS Cloud Load vs Time of Day? Improving High-Performance Computations on Clouds Through Resource Underutilization 6

7 INSTANCE PROCESSORS Instances deployed on varying processors EC2 Compute Unit ( GHz, Xeon 2007) - Large instance = 2 EC2 CU s per core (2cores), GHz Processor Type Min Usage Max Usage Intel Xeon E GHz ( x GHz) Intel Xeon E GHz (x2-4.8 GHz) Intel Xeon E GHz (x GHz) 88.1% 100% 83.3% 100% 75.1% 90.22% Experiment Setup: - Availability Zone: US-East-1d Instance Type: Large

8 INSTANCE PROCESSORS 1 core: 2-GHz 2 cores: 4-GHz 8

9 CLOUD CORRECTNESS Variability of Data Transfer Usage Charges Network - Regions

10 CLOUD CORRECTNESS Variability of Data Transfer Usage Charges Network - Availability Zones IDT RDT AZDT

11 CLOUD CORRECTNESS Variability of Data Transfer Usage Charges Regional Data Transfer (RDT) Transferred 84.3 MB s from EC2 Ubuntu Repository to the instance Regional%Data%Transfer%In% MB's%Consumed% 90" 80" 70" 60" 50" 40" 30" 20" 10" 0" Usage"Report" tcpdump" 1" 2" 3" 4" 5" 6" 7" 8" 9" 10" 11" 12" Experiment%

12 OUTLINE 1. Introduction 2. Uncontrollable Cloud Costs 3. Human Factors Affecting Cost Motivation Cloud/Human Factors Cloud Load Instance Processors Cloud Correctness End-user Location Job Configuration Instance Security 4. Conclusions/Future Work 12

13 END-USER LOCATION Cost vs User Location: Submit in person, not VPN Typical application: - stage input data - execute job - return output Invoice and Usage report

14 Amazon Web Services Billing Statement: February 1 - February 28, 2011 Date Printed: February 23, 2011 Amazon Elastic Compute Cloud US East (Northern Virginia) Region Amazon EC2 running Linux/UNIX $0.085 per Small Instance (m1.small) about:blank Name: Gary McGilvary gary.mcgilvary@ed.ac.uk Account Number: Cost instance-hour vs (or User partial hour) Location: Amazon EC2 EBS COST VARIABILITY Totals 10 Hrs 0.85 $0.00 per GB-month of provisioned GB-Mo 0.00 storage under monthly free tier Amazon Web $0.00 Services per 1 million I/O requests under 5,422 IOs Name: Gary McGilvary 0.00 Billing Statement: monthly February free tier 1 - February 28, gary.mcgilvary@ed.ac.uk Date Printed: $0.00 February per 10,000 23, 2011 gets (when loading a 2,048 Requests Account Number: snapshot) under monthly free tier Amazon CloudWatch $0.015 per monitored instance-hour (or 5 Hrs Totals 0.08 Amazon Elastic partial Compute hour) Cloud US East (Northern Virginia) Region» 0.93 Amazon Amazon Simple EC2 Notification running Linux/UNIX Service US East (Northern $0.085 Virginia) per Small Region Instance (m1.small) First 10 Hrs 0.85 instance-hour 100,000 Amazon (or partial SNS hour) API 36 Requests 0.00 Amazon Requests EC2 EBSper month are free $0.00 per GB-month of provisioned» 0.00 Amazon Simple GB-Mo 0.00 storage Storage under Service monthly free tier US Standard Region $0.00 per 1 million I/O requests under $0.00 5,422 IOs 0.00 monthly per request free tier- PUT, COPY, POST, $0.00 or per LIST 10,000 requests gets (when under loading the a 17 Requests ,048 Requests 0.00 monthly snapshot) global under free monthly tier free tier Amazon CloudWatch» 0.00 Amazon Virtual $0.015 Private per monitored Cloud instance-hour (or 5 Hrs 0.08 partial hour)» 0.00 AWS Data Transfer (excluding Amazon CloudFront)» 0.93 Amazon Simple $0.000 Notification per GB - data Service transfer in under US East (Northern Virginia) Region GB 0.00 monthly global free tier $0.000 First 100,000 per GB Amazon - data transfer SNS API out under 36 Requests Requests per month are free GB 0.00 the monthly global free tier $0.010 per GB - regional data transfer -» 0.00 Amazon Simple in/out/between Storage Service EC2 Avail Zones or US Standard Region GB 0.01 when using public/elastic IP addresses or $0.00 ELBper request - PUT, COPY, POST, or LIST requests under the 17 Requests Taxes monthly global free tier Estimated Taxes VAT Registration» (Due Amazon March 1, Virtual 2011) Private Cloud» 0.00 Total Charges due on March 1, 2011 $1.13 AWS Data Transfer (excluding Amazon CloudFront) $0.000 per GB - data transfer in under GB 0.00 the monthly global free tier $0.000 per GB - data transfer out under GB 0.00 the monthly global free tier $0.010 per GB - regional data transfer - 23/02/ :33 <?xml version="1.0" encoding="utf-8"?> <ServiceUsage> <OperationUsage> <ServiceName>AmazonEC2</ServiceName> <OperationName>RunInstances</OperationName> <UsageType>DataTransfer-In-Bytes</UsageType> <StartTime>02/24/11 12:00:00</StartTime> <EndTime>02/24/11 13:00:00</EndTime> <UsageValue> </UsageValue> </OperationUsage> <OperationUsage> <ServiceName>AmazonEC2</ServiceName> <OperationName>GetMetricStatistics</OperationName> <UsageType>Calls</UsageType> <StartTime>02/24/11 12:00:00</StartTime> <EndTime>02/24/11 13:00:00</EndTime> <UsageValue>20</UsageValue> </OperationUsage> <OperationUsage> <ServiceName>AmazonEC2</ServiceName> <OperationName>RunInstances</OperationName> <UsageType>BoxUsage</UsageType> <StartTime>02/24/11 12:00:00</StartTime> <EndTime>02/24/11 13:00:00</EndTime> <UsageValue>1</UsageValue> </OperationUsage> <OperationUsage> <ServiceName>AmazonEC2</ServiceName> <OperationName>InterZone-Out</OperationName> <UsageType>DataTransfer-Regional-Bytes</UsageType> <StartTime>02/24/11 12:00:00</StartTime> <EndTime>02/24/11 13:00:00</EndTime> <UsageValue>42708</UsageValue> </OperationUsage> Page 1 of 1

15 COST VARIABILITY Cost vs User Location: Difference in taxation levels - Scale with use and expensive for prolonged time periods - Businesses/Individuals in a tax free zone will benefit e.g. $4000 per month equals $9600 of tax per year Difference in resource usages - dependent on location or cloud load?

16 JOB CONFIGURATION Underutilization Reserving more resources while using a small % of each e.g Reserving 4 instances but only using a total of 2 instances Two Cases: (Large: 2 cores at 2 EC2 CU s) 1. Each process per instance - e.g 4 processes = 4 instances (one core per instance) 2. Each process per instance core - e.g 4 processes = 2 instances (4 cores) 2 GHz R 2 GHz R 2 GHz 2 GHz 2 GHz 2 GHz VS R R R R 2 GHz R 2 GHz R

17 UNDERUTILISATION Case Study: SPRINT

18 UNDERUTILISATION Consequences of Underutilization: - Reserving more resources will increase costs - cost vs performance? - Paper: Improving High-Performance Computations on Clouds Through Resource Underutilization - Underutilization improves the expected execution time by two orders of magnitude -... it is more than 3 times cheaper to use 50% of the resources than 100% - Could potentially save businesses/individuals a substantial amount of time and money How do we determine the utilisation rate?

19 What does security have to do with cost? SECURITY Legal: If the intruder goes on to conduct illegal activity on or from your server, parties who suffer damage may hold you liable. Financial: Bandwidth consumed by your instances will be billed to your account. For example, if the intruder were to host a download site for (say) illegal software or music downloads and advertise the address on an underground mailing list, thousands of like minded downloaders may start downloading from your server, running up your usage charges.

20 CONCLUSIONS Dependent on user location, costs may differ as well as resources used EC2 s data transfer usage mechanism may be incorrect at times Application performance can vary significantly (execution times) Underutilization can increase performance - in every case? - utilization rate? The underlying instance processors affect performance and hence cost - correct EC2 Compute Units specified? Keep you instance secure! Its your job, not the provider s! 20

21 THANK YOU! Questions? 21