CLOUD COMPUTING Y SU IMPACTO EN LA INFORMATICA

Transcription

1 CLOUD COMPUTING Y SU IMPACTO EN LA INFORMATICA Gustavo Alonso Systems Group Department of Computer Science ETH Zurich, Switzerland JISBD

2 Background ETH Zürich Systems Group Enterprise Computing Center CTO of Zimory GmbH 2

3 Overview Why cloud computing Cloud computing models A research agenda Computer science and the cloud 3

4 The motivation for cloud computing 4

5 Cloud computing for skeptics and everything on it I WANT YOUR DATA CENTER

6 Why cloud computing 6

7 Cloud computing for economists A natural step for the IT industry, moving from the secondary sector of the economy (manufacturing) to the tertiary/quaternary sector of the economy (services) and exploiting economies of scale 7

8 Service model is not what we know Closed systems and platforms Example: Appliances, automobiles Single provider Example: iphones / ipads Utility detached from technology Example: telephony, electricity, water 8

9 Widespread impact The cloud represents the true disappearing computer Mobile devices as the gateway to the cloud (= the computer) 9

10 Relevant aspects Costs Size Technology 10

11 Costs Oracle and Sun SPARC SOLARIS World Record TPC-C Performance Beats IBM's Best Results on DB2 with Power 595 Server Oracle and Sun Publish First World Record TPC-C Benchmark using Flash Technology Oracle OpenWorld, San Francisco, Calif. October 11, 2009 achieved a record-breaking 7.7 million tpmc at $2.34/tpmC 11

12 Cost of benchmarking set up 12

13 Cost of owning the system 13

14 Size matters Cost of bad weather Railway companies Airline reservations New services Postal services Financial services 14

15 Amadeus Workload Passenger-Booking Database ~ 600 GB of raw data (two years of bookings) single table, denormalized ~ 50 attributes: flight-no, name, date,..., many flags Query Workload up to 4000 queries / second latency guarantees: 2 seconds today: only pre-canned queries allowed Update Workload avg. 600 updates per second (1 update per GB per sec) peak of updates per second data freshness guarantee: 2 seconds 15

16 Amadeus Query Examples Simple Queries Print passenger list of Flight LH 4711 Give me LH hon circle from Frankfurt to Delhi Complex Queries Give me all Heathrow passengers that need special assistance (e.g., after terror warning) Problems with State-of-the Art Simple queries work only because of mat. views multi-month project to implement new query / process Complex queries do not work at all 16

17 Technology BEFORE Peak load provisioning AFTER Pay as you go One app per box In house facilities Virtualization on multicore Capacity on demand 17

18 Why cloud computing - III Business case Peak provisioning Costs of large installations Need for consolidation Computing becoming invisible Part 1 18

19 Cloud computing models 19

20 Types of cloud It is important to keep in mind that there are several types of clouds: Public clouds (services publicly available) Private clouds (services within one entity) And several types of services Infrastructure as a Service (Hardware) Platform as a Service (Hardware + Basic Software) Software as a Service (Full Applications) 20

21 Public Clouds Public clouds are a service offering by a provider that are open to the public and can be used following a contractual agreement Advantages: Economies of scale Robustness through numbers Disadvantages Legal and security aspects Load separation 21

22 Private Clouds Private clouds are virtualized resources made available as a service within one organization Advantages No legal and security aspects Your own infrastructure Fast provisioning of resources Disadvantages Limited sharing of resources Your own infrastructure 22

23 Private vs. public clouds The most common scenario in the short and medium term is likely to be private clouds connected to public clouds for quick provisioning Private cloud facilitates transition while maintaining control over IT infrastructure Public cloud (multiple providers) for expansion and outsourcing 23

24 IaaS: my hw, your sw Infrastructure as a Service Virtualized hardware forming a grid Machines, memory, CPUs, disk Networking plus load balancers Internet access Example IaaS: Amazon s EC2 Example of software: AppLogic from 3Tera SaaS PaaS IaaS 24

25 Iaas configuration Cloud user (company) Developers Users Users Users Cloud provider (IaaS) Company s software Company s software Provider s hardware 25

26 IaaS: pro and against Advantages Fast provisioning Reduced CapEx Large scale resources for small companies You control the software that runs Least vendor lock-in Disadvantages Hardware CapEx not the dominant factor Development and Maintenance costs remain Model with lowest margings for provider 26

27 PaaS: you do the tailoring Platform as a Service (Virtualized) Hardware resources Basic software databases or programming platforms Networking Internet access Example PaaS: Google Application Engine, Microsoft Azure SaaS PaaS IaaS 27

28 PaaS configuration Cloud user (company) Developers Users Users Users Cloud provider (PaaS) Company s software Provider s software Provider s software Provider s hardware 28

29 PaaS: pro and against Advantages Platform software already installed Simplified maintenance of platform software Control of the application and data Reduced CapEx for hardware and key software Disadvantages Some development and maintenance costs Often inflexible in what can be done and deployed Vendor lock-in at the platform level 29

30 SaaS: you use the service Software as a Service Full application Available through a web browser Internet access SaaS PaaS IaaS Example Saas: SalesForce, Google Apps 30

31 SaaS configuration Cloud user (company) Developers Users Users Users Cloud provider (SaaS) Provider s software Provider s hardware 31

32 Saas: pro and against Advantages No CapEx or OpEx Cost related to use No maintenance or development Model with highest margins for provider Larger software markets (no differentiation) Disadvantages Vendor lock-in at the application level Availability 32

33 A research agenda Systems Group ETH 33

34 Appliances for the cloud Access Application Platform Operating System Hardware Alfredo Crescando Multimed Barrelfish Data processing on FPGAs 34

35 CPU-based Systems [Streams on Wires, VLDB 09] packets processed 100% 80% 60% 40% 20% CPU-based FPGA 100% 100% 60% 36% CPU-based System drops UDP packets Networkmemory-CPU bottleneck 0% 300'000 pkts/s 1'000'000 pkts/s data input rate FPGA not a single UDP packet dropped 35

36 Programming FPGAs Comple x Event Query Queries Compilation VHDL Synthesis Map P&R Bitstream Network Data Stream FPGA Notification End System Complex Event Detection at Wire Speed with FPGAs 36 ETH Zurich

37 Optimal network utilization Maximum number of packets on a gigabit link Min. Ethernet frame = 64 bytes Interframe Gap = 12 bytes Preamble = 8 bytes 84 bytes (672 bits) per packet 1 Gbit/672 bits = packets Our measurements packets/second packets/second x 672 bits = Mbit/s 1 Gbit/s Complex Event Detection at Wire Speed with FPGAs 37 ETH Zurich

38 Barrelfish 38

39 Postgres on Multicore Architectures Nehalem EX 4 CPU Sockets 8 Cores per CPU 128 GB RAM TPC-W Browsing 24GB Database PostgreSQL can t take advantage of the extra resources 39

40 Multimed Query Q2 Q3 Q1 -R -W Query Q1 -R Query Q3 -R Query Q1 -R Query Q2 -W Query Q3 -R HPI - SOC Lab Symposium 40 40

41 Crescando on 1 Machine (N Cores) Scan Thread Scan Thread Input Queue (Operations) Split Scan Thread Scan Thread Merge Output Queue (Result Tuples)... Input Queue (Operations) Scan Thread Output Queue (Result Tuples) 41 Systems Group. D-INFK. ETH Zurich

42 Crescando in Data Center (N Machines) 42 Systems Group. D-INFK. ETH Zurich

43 Alfredo

44 Computer science and the cloud 44

45 New opportunities Cross layer optimizations System co-design New paradigms (system and languages) Algorithms in hardware Hybrid compilers (HW-SW) New data processing languages 45

46 New challenges Computer no longer visible Layers of abstractions blurring What is a program? What does a program do? Business models changing Engineering skills in high demand 46

47 Conclusions Three Chinese curses 1. May you live in interesting times 2. May the government be aware of you 3. May you find what you are looking for 47