COM 444 Cloud Computing Lec 3: Virtual Machines and Virtualization of Clusters and Datacenters Prof. Dr. Halûk Gümüşkaya haluk.gumuskaya@gediz.edu.tr haluk@gumuskaya.com http://www.gumuskaya.com Virtual Machines and Virtualization of Clusters and Datacenters 1. What is Virtualization? 2. Implementation Levels of Virtualization 3. Virtualization Structures/Tools and Mechanisms 4. Virtualization of CPU, Memory, and I/O Devices 5. Virtual Clusters and Resource Management 6. Virtualization for Data-Center Automation Computing Engineering Department 1 2 What is Virtualization? Virtualization is a computer architecture technology by which multiple virtual machines (VMs) are multiplexed in the same hardware machine. The idea of VMs can be dated back to the 1960s. The purpose of a VM is to enhance resource sharing by many users improve computer performance in terms of resource utilization and application flexibility. A level of indirection between hardware and software VM abstraction: run all software written for physical machine 3 4
User s View of Virtualization Virtualization: Motivation Virtualization is a basic tenet of cloud computing. It simplifies the management of physical resources for the 3 abstractions: Multiplexing, Aggregation Emulation. The state of a Virtual Machine (VM) running under a Virtual Machine Monitor (VMM) can de saved and migrated to another server to balance the load. Virtualization allows users to operate in environments they are familiar with, rather than forcing them to idiosyncratic ones. 5 6 Virtualization: Motivation (cont.) Cloud resource virtualization is important for: System security, as it allows isolation of services running on the same hardware. Performance and reliability, as it allows applications to migrate from one platform to another. The development and management of services offered by a provider. Performance isolation. Virtualization: Motivation (cont.) Simulates the interface to a physical object by: Multiplexing: Creates multiple virtual objects from one instance of a physical object. Example - a processor is multiplexed among a number of processes or threads. Aggregation: Creates one virtual object from multiple physical objects. Example - a number of physical disks are aggregated into a RAID disk. Emulation: Constructs a virtual object from a different type of a physical object. Example - a physical disk emulates a Random Access Memory (RAM). Multiplexing and Emulation: Examples - virtual memory with paging multiplexes real memory and disk; a virtual address emulates a real address. 7 8
Virtualization: Top Strategic Technology to Change the Computer Industry According to a 2009 Gartner Report, virtualization was the top strategic technology poised to change the computer industry. With sufficient CPU power, memory and storage, any computer platform can be installed in another host computer, even if they use processors with different instruction sets and run with distinct operating systems on the same hardware. 9 Layering Layering a common approach to manage system complexity. Minimizes the interactions among the subsystems of a complex system. Simplifies the description of the subsystems; each subsystem is abstracted through its interfaces with the other subsystems. We are able to design, implement, and modify the individual subsystems independently. Layering in a computer system. Hardware, Software. Operating system. Libraries. Applications. 10 Interfaces Code Portability API - defines the set of instructions the hardware was designed to execute and gives the application access to the ISA; it includes HLL library calls which often invoke system calls. Application Binary Interface ISA: At the boundary between hardware and software. An application uses library functions (A1), makes system calls (A2), and executes machine instructions (A3). ABI - Allows the ensemble consisting of the application and the library modules to access the hardware; the ABI does not include privileged system instructions, instead it invokes system calls. 11 Binaries created by a compiler for a specific ISA and a specific operating systems are not portable. It is possible, though, to compile a HLL program for a virtual machine (VM) environment where portable code is produced and distributed and then converted by binary translators to the ISA of the host system. A dynamic binary translation converts blocks of guest instructions from the portable code to the host instruction and leads to a significant performance improvement, as such blocks are cached and reused 12
Traditional Computer and Virtual Machines Virtual Machine, Guest Operating System, and VMM (Virtual Machine Monitor) The Virtualization layer is the middleware between the underlying hardware and virtual machines represented in the system, also known as Virtual Machine Monitor (VMM) or hypervisor. 13 14 Hypervisor - Virtual Machine Monitor (VMM) A hypervisor is a hardware virtualization technique allowing multiple operating systems, called guests to run on a host machine. Type 1: bare metal hypervisor Sits on the bare metal computer hardware like the CPU, memory, etc. All guest operating systems are a layer above the hypervisor. The original CP/CMS hypervisor developed by IBM was of this kind. Type 2: hosted hypervisor Run over a host operating system. Hypervisor is the second layer over the hardware. Guest operating systems run a layer over the hypervisor. The OS is usually unaware of the virtualization 15 Hypervisor - Virtual Machine Monitor (VMM) Partitions the resources of computer system into one or more virtual machines (VMs). Allows several operating systems to run concurrently on a single hardware platform. A VMM allows Multiple services to share the same platform. Live migration - the movement of a server from one platform to another. System modification while maintaining backward compatibility with the original system. Enforces isolation among the systems, thus security. 16
Performance and Security Isolation Major VMM and Hypervisor Providers The run-time behavior of an application is affected by other applications running concurrently on the same platform and competing for CPU cycles, cache, main memory, disk and network access. Thus, it is difficult to predict the completion time! Performance isolation - a critical condition for QoS guarantees in shared computing environments. A VMM is a much simpler and better specified system than a traditional operating system. Example - Xen has approximately 60,000 lines of code; Denali has only about half, 30,000. The security vulnerability of VMMs is considerably reduced as the systems expose a much smaller number of privileged functions. 17 18 Major VMM and Hypervisor Providers Virtual Machines and Virtualization of Clusters and Datacenters 1. What is Virtualization? 2. Implementation Levels of Virtualization 3. Virtualization Structures/Tools and Mechanisms 4. Virtualization of CPU, Memory, and I/O Devices 5. Virtual Clusters and Resource Management 6. Virtualization for Data-Center Automation 19 20
Virtualization Ranging from Hardware to Applications in 5 Abstraction Levels 1. Virtualization at ISA Level Emulating a given ISA by the ISA of the host machine. e.g, MIPS binary code can run on an x-86-based host machine with the help of ISA emulation. Typical systems: Bochs, Crusoe, Quemu, BIRD, Dynamo Advantages: It can run a large amount of legacy binary codes written for various processors on any given new hardware host machines Best application flexibility Shortcomings & Limitations: One source instruction may require tens or hundreds of native target instructions to perform its function, which is relatively slow. V-ISA requires adding a processor-specific software translation layer in the complier. 21 22 2. Virtualization at Hardware Abstraction Level Low-Level VMM Operations Virtualization is performed right on top of the hardware. It generates virtual hardware environments for VMs, and manages the underlying hardware through virtualization. Typical systems: VMware, Virtual PC, Denali, Xen Advantage: Has higher performance and good application isolation Shortcoming & Limitation: Very expensive to implement (complexity) (Courtesy of Mendel Rosenblum, 2006) 23 24
3. Virtualization at Operating System (OS) Level The virtualization layer is inserted inside an OS to partition the hardware resources for multiple VMs to run their applications in virtual environments. 25 Virtualization at OSLevel It is an abstraction layer between traditional OS and user apps This virtualization creates isolated containers on a single physical server and the OS-instance to utilize the hardware and software in datacenters. Typical systems: Jail / Virtual Environment / Ensim's VPS / FVM Advantage: Has minimal starup/shutdown cost, low resource requirement, and high scalability; synchronize VM and host state changes. Shortcomings & Limitations: All VMs at the operating system level must have the same kind of guest OS Poor application flexibility and isolation. 26 Virtualization for Linux and Windows Platforms Virtualization Support for Linux and Windows Figure 3.3. The OpenVZ virtualization layer inside the host OS, which provides some OS images to create VMs quickly. 27 28
4. Virtualization at Library Support Level Virtualization at Library Support Level Library-level virtualization is also known as user-level Application Binary Interface (ABI) or API emulation. It creates execution environments for running alien programs on a platform rather than creating VM to run the entire operating system. It is done by API call interception and remapping. Examples: Wine, WAB, LxRun, VisualMainWin, vcuda Advantage: It has very low implementation effort Shortcoming & Limitation: Poor application flexibility and isolation 29 30 The vcube for Virtualization of General Purpose GPU CUDA is a programming model and library for general-purpose GPUs. It leverages the high performance of GPUs to run compute-intensive applications on host operating systems. However, it is difficult to run CUDA applications on hardware-level VMs directly. vcuda virtualizes the CUDA library and can be installed on guest OSes. The vcube for Virtualization of General Purpose GPU When CUDA applications run on a guest OS and issue a call to the CUDA API, vcuda intercepts the call and redirects it to the CUDA API running on the host OS. 31 32
5. Virtualization at User-Application Level It virtualizes an application as a virtual machine. This layer sits as an application program on top of an operating system and exports an abstraction of a VM that can run programs written and compiled to a particular abstract machine definition. Typical systems: JVM,.NET CLI, Panot Advantage: has the best application isolation Conditions for Efficient Virtualization A program running under the VMM should exhibit a behavior essentially identical to that demonstrated when running on an equivalent machine directly. The VMM should be in complete control of the virtualized resources. A statistically significant fraction of machine instructions must be executed without the intervention of the VMM. Shortcomings & Limitations: Low performance, low application flexibility and high implementation complexity. 33 34 Relative Merits of Virtualization Virtual Machines and Virtualization of Clusters and Datacenters 1. What is Virtualization? 2. Implementation Levels of Virtualization 3. Virtualization Structures/Tools and Mechanisms 4. Virtualization of CPU, Memory, and I/O Devices 5. Virtual Clusters and Resource Management 6. Virtualization for Data-Center Automation 35 36
Virtual Machines and Virtualization Middleware The main function of the software layer for virtualization is to virtualize the physical hardware of a host machine into virtual resources to be used by the VMs, exclusively. Virtualization Structures Depending on the position of the virtualization layer, there are several classes of VM architectures, namely Hypervisor (VMM - Virtual Machine Monitor) Para-virtualization Host-based virtualization They both perform the same virtualization operations. Three VM Architectures 37 38 Full Virtualization vs. Para-Virtualization Full virtualization a guest OS can run unchanged under the VMM as if it was running directly on the hardware platform. Does not need to modify guest OS, and critical instructions are emulated by software through the use of binary translation. VMware Workstation applies full virtualization, which uses binary translation to automatically modify x86 software on-the-fly to replace critical instructions. Advantage: no need to modify OS. Disadvantage: binary translation slows down the performance. 39 Full Virtualization vs. Para-Virtualization Para virtualization -a guest operating system is modified to use only instructions that can be virtualized. Reasons for paravirtualization: Some aspects of the hardware cannot be virtualized. Improved performance. Present a simpler interface. Para virtualization must modify guest OS, non-virtualizable instructions are replaced by hypercalls that communicate directly with the hypervisor or VMM. Reduces the overhead, but cost of maintaining a paravirtualized OS is high. The improvement depends on the workload. Para virtualization is supported by Xen, Denali and VMware ESX. 40
The XEN Architecture (1) The XEN Architecture (2) Xen is an open source hypervisor program developed by Cambridge University. Xen is a micro-kernel hypervisor, which separates the policy from the mechanism. The Xen hypervisor implements all the mechanisms, leaving the policy to be handled by Domain 0, as shown in Figure 3.5. Xen does not include any device drivers natively. It just provides a mechanism by which a guest OS can have direct access to the physical devices. As a result, the size of the Xen hypervisor is kept rather small. Xen provides a virtual environment located between the hardware and the OS. A number of vendors are in the process of developing commercial Xen hypervisors, among them are Citrix XenServer and Oracle VM. 41 42 The XEN Architecture (3) Full Virtualization 43 44
Binary Translation of Guest OS Requests using a VMM Para-Virtualized VM Architecture Para- Virtualization with Compiler Support. The KVM builds offers kernelbased VM on the Linux platform, based on paravirtualization 45 46 Para-Virtualized Guest OS Assisted by Compiler VMWare ESX Server for Para-Virtualization 47 48
Virtual Machines and Virtualization of Clusters and Datacenters 1. What is Virtualization? 2. Implementation Levels of Virtualization 3. Virtualization Structures/Tools and Mechanisms 4. Virtualization of CPU, Memory, and I/O Devices CPU Virtualization 5. Virtual Clusters and Resource Management 6. Virtualization for Data-Center Automation 49 50 Virtualization Support at Intel Intel Hardware-Assisted CPU Virtualization 51 52
Memory Mapping Memory Virtualization 53 54 Memory Virtualization Challenges Device Emulation for I/O Virtualization 55 56
Example 3.7 VMware Workstation for I/O Virtualization Conclusions on CPU, Memory and I/O Virtualization CPU virtualization demands hardware-assisted traps of sensitive instructions by the VMM Memory virtualization demands special hardware support (shadow page tables by VMWare or extended page table by Intel) to help translate virtual address into physical address and machine memory in two stages. I/O virtualization is the most difficult one to realize due to the complexity if I/O service routines and the emulation needed between the guest OS and host OS. 57 58 Multi-Core Virtualization Virtual Cores vs. Physical Processor Cores VCPU vs. traditional CPU Physical Cores The actual physical cores present in the processor. More burden on the software to write applications which can run directly on the cores. Hardware provides no assistance to the software and is hence simpler. Virtual cores There can be more virtual cores visible to a single OS than there are physical cores. Design of software becomes easier as the hardware assists the software in dynamic resource utilization. Hardware provides assistance to the software and is hence more complex. Figure 3.16 Four VCPUs are exposed to the software, only three cores are actually present. VCPUs V0, V1, and V3 have been transparently migrated, while VCPU V2 has been transparently suspended. (Courtesy of Wells, et al., Dynamic Heterogeneity and the Need for Multicore Virtualization, ACM SIGOPS Operating Systems Review, ACM Press, 2009 [68] ) Poor resource management. The lowest level of system software has to be modified. Better resource management. The lowest level of system software need not be modified. 59 60
Mapping of VMs into Adjacent Cores Virtual Clusters in Many Cores Space Sharing of VMs -- Virtual Hierarchy 61 62 Virtual Machines and Virtualization of Clusters and Datacenters 1. What is Virtualization? 2. Implementation Levels of Virtualization 3. Virtualization Structures/Tools and Mechanisms 4. Virtualization of CPU, Memory, and I/O Devices 5. Virtual Clusters and Resource Management 6. Virtualization for Data-Center Automation Virtual Cluster Characteristics The virtual cluster nodes can be either physical or virtual machines. Multiple VMs running with different OSs can be deployed on the same physical node. A VM runs with a guest OS, which is often different from the host OS, that manages the resources in the physical machine, where the VM is implemented. The purpose of using VMs is to consolidate multiple functionalities on the same server. This will greatly enhance the server utilization and application flexibility. VMs can be colonized (replicated) in multiple servers for the purpose of promoting distributed parallelism, fault tolerance, and disaster recovery. 63 64
Virtual Cluster Characteristics (cont.) Virtual Clusters vs. Physical Clusters The size (number of nodes) of a virtual cluster can grow or shrink dynamically, similarly to the way an overlay network varies in size in a P2P network. The failure of any physical nodes may disable some VMs installed on the failing nodes. But the failure of VMs will not pull down the host system. 65 66 Virtual Cluster based on Application Partitioning Live Migration of Virtual Machines 67 68
VM Migrated from one Failing Web Server to Another Live Migration of VM 69 70 Virtual Cluster Projects COD Partitioning a Physical Cluster into Multiple Virtual Clusters 71 72
Cluster-on-Demand (COD Project at Duke University Cluster Size Variations in COD over Eight Days 73 74 VIOLIN Project at Purdue University VIOLIN Project at Purdue University 75 76
Virtual Machines and Virtualization of Clusters and Datacenters 1. What is Virtualization? 2. Implementation Levels of Virtualization 3. Virtualization Structures/Tools and Mechanisms 4. Virtualization of CPU, Memory, and I/O Devices 5. Virtual Clusters and Resource Management 6. Virtualization for Data-Center Automation Virtualization for Datacenter Automation to serve millions of clients, simultaneously Server Consolidation in Virtualized Datacenter Virtual Storage Provisioning and Deprovisioning Cloud Operating Systems for Virtual Datacenters Trust Management in Virtualized Datacenters 77 78 Parallax for VM Storage Management Cloud OS for Building Private Clouds 79 80
Eucalyptus: An Open-Source OS for Setting Up and Managing Private Clouds Eucalyptus: An Open-Source OS for Setting Up and Managing Private Clouds 81 82 Eucalyptus for Building Private Clouds 83 84
Acknowledgements The slides have been based in-part upon original slides of a number of books and Profesors including: Distributed and Cloud Computing: From Parallel Processing to The Internet of Things, K. Hwang, G. Fox and J. Dongarra, Morgan Kaufmann Publishers, 2012. Cloud Computing: Theory and Practice, D. C. Marinescu, Morgan Kaufmann, 2013. 85 86