ECE6130 Grid and Cloud Computing

Transcription

1 ECE6130 Grid and Cloud Computing Howie Huang Department of Electrical and Computer Engineering School of Engineering and Applied Science

2 Cloud Computing Hardware Software Outline Research Challenges 2

3 Trends george washington un cloud computing 0.70 Source: Google Trends Source: Amazon 100+ billion dollar market in 2010 [Merrill Lynch, 2008] By 2012, customer spending on IT cloud services will grow to $42 billion. [IDC, 2008] 3

4 Scale Everyday Amazon AWS puts in servers that would support their entire business in 2000, when they were already a 2 billion dollar enterprise From James Hamilton s talk at UW, Internetscale storage, Nov mvis?id=1090 4

5 Google Data Center Tour h0p:// v=zrwpsfplx8i 5

6 Cloud Computing Provide service, compute and storage Utilize virtualization technology On-demand, pay as you go Reliability, availability, Client Client Client Internet Cloud Data Center Data Center Client Client Client 6

7 " A model of computation and data storage based on pay as you go access to unlimited remote data center capabilities. " A cloud infrastructure provides a framework to manage scalable, reliable, on-demand access to applications. " Examples: " Search, , social networks " File storage (Live Mesh, Mobile Me, Flicker, ) " Just about any large-scale web service is a cloud service. 7 Dennis Gannon, Microsoft

8 8 " Scale " Blue Waters = 40K 8-core servers " Road Runner = 13K cell + 6K AMD servers " MS Chicago Data Center = 50 containers = 100K 8-core servers. " Network Architecture " Supercomputers: CLOS Fat Tree infiniband " Low latency high bandwidth " protocols " Data Center: IP based " Optimized for Internet Access " Data Storage " Supers: separate data farm " GPFS or other parallel file system " DCs: use disk on node + memcache Fat tree network Standard Data Center Network

9 " Infrastructure as a Service (IaaS) " Provide App builders a way to configure a Virtual Machine and deploy one or more instances on the data center " The VM has an IP Address visible to the world " A Fabric controller manages VM instances " Examples: Eucalyptus.com, Amazon EC2 + S3, Flexiscale, Rackspace, GoGrid, SliceHost, Nimbus 9 Dennis Gannon, Microsoft

10 " An application development, deployment and management fabric. " User programs web service front end and computational & Data Services " Framework manages deployment and scale out " No need to manage VM images App Developer App User Web Access Layer Data & Compute Layer Examples: Microsoft Azure, Google App Engine, RightScale, SalesForce, Rollbase, Bungee, Cloudera 10 Dennis Gannon, Microsoft

11 11 " Online delivery of applications " Via Browser " Microsoft Office Live Workspace " Google Docs, etc. " File synchronization in the cloud Live Mesh, Mobile Me " Social Networks, Photo sharing, Facebook, wikipedia etc. " Via Rich Apps " Science tools with cloud back-ends " Matlab, Mathematica " Mapping " MS Virtual Earth, Google Earth " Much more to come. Dennis Gannon, Microsoft

12 " At one time the client was a PC + browser. " Now " The Phone " The laptop/tablet " The TV/Surface/Media wall " And the future " The instrumented room " Aware and active surfaces " Voice and gesture recognition " Knowledge of where we are " Knowledge of our health 12 Dennis Gannon, Microsoft

13 Experiments Simulations Archives Literature Instruments The Challenge: Enable Discovery. Deliver the capability to mine, search and analyze this data in near real time. Enhance our Lives Participate in our own heath care. Augment experience with deeper understanding. Petabytes Doubling every 2 years 13 Dennis Gannon, Microsoft

14 Amazon Elastic Compute Cloud a web service that provides resizable compute capacity in the cloud. It is designed to make web-scale computing easier for developers. Create Amazon Machine Image (AMI) Upload the AMI into Amazon S3 Use Amazon EC2 web service to manage Pay as you go 14

15 Amazon Simple Storage Service storage for the Internet. It is designed to make web-scale computing easier for developers. Write, read, and delete objects Unlimited objects Authorization mechanisms REST and SOAP interfaces HTTP/BitTorrent protocol 15

16 Amazon Pricing Compute $ Small Instance (Default) 1.7 GB of memory, 1 EC2 Compute Unit (1 virtual core with 1 EC2 Compute Unit), 160 GB of instance storage, 32-bit platform $ Large Instance $ Extra Large Instance Data Transfer $0.100 per GB - all data transfer in $0.170 per GB - first 10 TB / month data transfer out $0.130 per GB - next 40 TB / month data transfer out $0.110 per GB - next 100 TB / month data transfer out $0.100 per GB - data transfer out / month over 150 TB Looks inexpensive, but really? 16

17 EC2 Putting Numbers Together 1K instance hours, 1TB data in & out = $370 10K instance hours, 1TB data in & out = $1, K instance hours, 1TB data in & out = $100,270 S3 10TB storage, 100GB data in &out = 1, TB storage, 1 TB data in &out = 15, PB storage, 10 TB data in &out = 152,

18 Amazon Credit Please contact the instructor for $100 credit from Amazon EC2 Please use caution and terminate any idle instances when they are no longer used. You are solely responsible for any cost beyond the $100 Amazon credit. Thanks to the generous support from Amazon AWS 18

19 Downtime 7.1. Downtime and Service Suspensions. In addition to our rights to terminate or suspend Services to you as described in Section 3 above, you acknowledge that: (i) your access to and use of the Services may be suspended for the duration of any unanticipated or unscheduled downtime or unavailability of any portion or all of the Services for any reason, including as a result of power outages, system failures or other interruptions; and (ii) we shall also be entitled, without any liability to you, to suspend access to any portion or all of the Services at any time, on a Service-wide basis 19

20 Security 7.2. Security. We strive to keep Your Content secure, but cannot guarantee that we will be successful at doing so, given the nature of the Internet. Accordingly, without limitation to Section 4.3 above and Section 11.5 below, you acknowledge that you bear sole responsibility for adequate security, protection and backup of Your Content. We strongly encourage you, where available and appropriate, to use encryption technology to protect Your Content from unauthorized access and to routinely archive Your Content. We will have no liability to you for any unauthorized access or use, corruption, deletion, destruction or loss of any of Your Content. 20

21 Amazon S3 SLA 21

22 Research Challenges Scalable resource management Automatic Mechanism is of great interest Scheduling, maintaining, reporting, Power efficiency Availability Across multiple data centers and providers Performance Predictable performance Security Integration with mobile devices Many more 22

23 Facebook s Server Room v=nhoo1ztrh8c 23