CS 695 Topics in Virtualization and Cloud Computing and Storage Systems. Introduction

Transcription

1 CS 695 Topics in Virtualization and Cloud Computing and Storage Systems Introduction

2 Hot or not? source: Gartner Hype Cycle for Emerging Technologies,

3 Source: 3

4 Cloud Computing The in-vogue term Cloud is the network Network is the cloud Cloud storage, storage cloud Cloud-based e-learning Application in cloud, cloud application Smart cloud Cloud and phones... 4

5 What is cloud computing? Definition? Where is it? Why the hoopla? 5

6 scale your infrastructure on demand within minutes or even seconds, instead of days or weeks, thereby avoiding under-utilization (idle servers) and over-utilization 'internet centric software.'... shift from the traditional single tenant approach to software development to that of a scalable, multi-tenant, multi-platform, multi-network, and global a broad array of web-based services aimed at allowing users to obtain a wide range of functional capabilities on a 'pay-as-you-go' basis basically the broad concept of using the internet to allow people to access technologyenabled services... services must be 'massively scalable' to qualify as true 'cloud computing' The 'cloud' model initially has focused on making the hardware layer consumable as ondemand compute and storage capacity. Cloud computing really is accessing resources and services needed to perform functions with dynamically changing needs.... The cloud is a virtualization of resources that maintains and manages itself. outsourced, pay-as-you-go, on-demand, somewhere in the Internet, etc. Internet-based computing, whereby shared resources, software and information are provided to computers and other devices on-demand 6

7 Cloud computing Computing-as-a-service All forms of computing Analogies? 7

8 Cloud characteristics On-demand service User not worried about maintenance and setup issues etc. Shared Resources Large capacity distributed/multiplexed over several users Flexible Provisioning Dynamically scale resources Fine-grained metering pay-as-you-use model 8

9 Is the 'Cloud' brand new? NO! 60s-70s, large centralized servers carved up for individual access Cluster computing Grid computing Utility computing Peaking now Decreasing cost of powerful hardware Virtualization Economics 9

10 Cloud service models IaaS Infrastructure as a Service Amazon EC2, Ubuntu One, IBM Test Cloud PaaS Platform as a service Google Appengine, Windows Azure SaaS Software as a service Gmail, Google online services *aas 10

11 Cloud deployments 11

12 Cloud deployment options Private cloud Infrastructure for a single organization Public cloud Internal hosting, testing environment Infrastructure for general use Cloud provider owns service Hybrid cloud Composition of private+public Secure/non-secure or QoS/best-effort traffic, Cloud bursting 12

13 Cloud views User view Provider view 13

14 User-view of the Cloud On-demand provisioning Resource, software, services Quick deployment Security and isolation SLA requirements Service-level agreement Availability, isolation, performance metrics... Pay-on-use model Minimize cost for service 14

15 Providers-view of the Cloud Support on-demand provisioning Resource accounting and provisioning Efficient provisioning Reactive and quick High multiplexing desired Meet SLA requirements of users Allocate minimum to maximize profit/#users Power! 15

16 Scope of cloud computing source: Call for papers ACM SOCC 2015 Administration and Manageability Data Privacy Programming Models Data Services Architectures Provisioning and Metering Distributed and Parallel Query Processing Resource Management and Performance Energy Management Scientific Data Management Distributed and Cloud Networking Security of Services High Availability and Reliability Service Level Agreements Infrastructure Technologies Storage Architectures Large Scale Cloud Applications Transactional Models Multi-Tenancy Virtualization Technologies 16

17 Virtualization Didn't you mention virtualization in the course title? 17

18 Begin from the beginning Computer systems are incredibly complex silicon, transistors, disks, processors, network cards, buses... device drivers, operating systems, compilers, application libraries... But still, Systems evolving and flourishing Why? 18

19 Abstractions and Interfaces abstraction: concept or idea (w/o implementation specifics) interface: operations/process related to abstraction Complex systems organized as a hierarchy of abstractions Well-defined interfaces allow independent evolution of subsystems 19

20 Abstraction+Interface Examples 20

21 Abstraction+Interface Examples 1.Storage Physical disk: tracks, sectors, disk head... User view: file (abstraction) File operations (interface): OS + device driver 2.Network 3.ISA (Instruction Set Architecture) 4.WiFi x86 architecture (8086 based, 16-bit) Intel, AMD, VIA... manufacturers Software runs on all x86 hardware 21

22 Are we ready for world peace then? What is the issue with abstraction oriented development? 22

23 Are we ready for world peace then? What's the issue with abstraction oriented development? Interoperability Sub-systems designed for one interface will not work for other e.g., binary incompatibility 23

24 Virtualization virtual: not real, illusionary, perception of reality virtualize a component/resource Map visible interface/resources onto those of underlying physical system real system appears virtual 24

25 Types of virtual machines Process view of machine System view of machine memory, user-level instr., system calls for OS functions OS interface to hardware defines view of process Process VM environment to support multiple processes sharing resources hardware characteristics defines system view System VM 25

26 Process VM and System VM Application Guest Application Virtualization Software Operating System Runtime Operating System Virtualization Software Hardware Host Hardware 26

27 Process VM examples 27

28 Process VM examples Multi-processing systems Emulators Fetch, decode, interpret e.g., WINE, QEMU High-level Language VMs Compile for virtual architecture Translate/execute on native platform JAVA,.Net 28

29 System VM characteristics Provide machine-view (virtual machines) to multiple operating systems Resource allocations cpu, memory, network, disk Control/arbitrate VMs access to physical hardware Isolation and Security 29

30 System VMs Native VMs (type 1) a.k.a Bare-metal VMs Hypervisor/VMM on hardware, controls guests directly Guest unaware of VM translation Hosted VMs (type 2) Hypervisor/VMM hosted on OS Guests co-ordinated by VMM Para-virtualized or Fully-virtualized VMs Guest aware of underlying hypervisor? 30

31 Type of System VMs Native VMs (Bare-metal) VMware ESX Hosted VMs Xen kvm VMware GSX VirtualBox 31

32 Virtualization benefits Flexible resource provisioning Increased Inter-operability Standard virtualized interface Isolation Secure multiplexing, Local impact of crashes/bugs Resource usage isolation Machine State Capture VM is a file Snapshots, cloning, migration Content distribution Virtual appliances Vital building block of a Cloud! 32

33 Problem Areas Virtualization technolgies Physical to virtual (p2v) Capacity planning and profiling Resource Management Network I/O optimizations in virtual machines Migration and Consolidation management Placement and Provisioning Storage Issues with virtualization Energy, Energy, Energy Hot topics Security Measurements and Modelling 33