Virtualization Jukka K. Nurminen 23.9.2015
Virtualization Virtualization refers to the act of creating a virtual (rather than actual) version of something, including virtual computer hardware platforms, operating systems, storage devices, and computer network resources.
Hardware Virtualization Creation of a virtual machine that acts like a real computer with an operating system. Software executed on these virtual machines is separated from the underlying hardware resources. Guest Machine Hypervisor Host Machine
VM1 VM2 VM3 Hypervisor Host Machine Different operating system Multiple virtual machines on single hardware Isolation of problems to virtual machines Possibility to move virtual machines
Introduction What is virtualization? Virtualization is a broad term (virtual memory, storage, network, etc) Focus for this course: platform virtualization Virtualization basically allows one computer to do the job of multiple computers, by sharing the resources of a single hardware across multiple environments Virtual Container Virtual Container App. A App. B App. C App. D App. A App. B App. C App. D Operating System Virtualization Layer Hardware Nonvirtualized system A single OS controls all hardware platform resources Hardware Virtualized system It makes it possible to run multiple Virtual Containers on a single physical platform
Guest Hypervisor, VMM Host
The evolution of virtualization
Evolution of Virtualization How did it start? Server virtualization has existed for several decades IBM pioneered more than 30 years ago with the capability to multitask The inception was in specialized, proprietary, high-end server and mainframe systems By 1980/90 servers virtualization adoption initiated a revolution Inexpensive x86 hardware platforms Windows/Linux adopted as server Oss Today the ratio of virtual to physical servers estimated to be 8:1 growing strongly
Evolution of Virtualization Computing Infrastructure 2000 1 machine 1 OS several applications Applications can affect each other Big disadvantage: machine utilization is very low, most of the times it is below than 25% App App App App App App App App X86 Windows XP X86 Windows 2003 X86 Suse X86 Red Hat 12% Hardware Utilization 15% Hardware Utilization 18% Hardware Utilization 10% Hardware Utilization
Evolution of Virtualization Computing Infrastructure - Virtualization It matches the benefits of high hardware utilization with running several operating systems (applications) in separated virtualized environments Each application runs in its own operating system Each operating system does not know it is sharing the underlying hardware with others App. A App. B App. C App. D X86 Windows XP X86 Windows 2003 X86 Suse Linux X86 Red Hat Linux X86 Multi-Core, Multi Processor 70% Hardware Utilization
Other Benefits improved security hardware independence resource management Live migration Low server infrastructure utilization (10-18%) Increasing physical infrastructure costs (facilities, power, cooling, etc) Increasing IT management costs (configuration, deployment, updates, etc) Insufficient failover and disaster protection Server/workload consolidation (reduces server sprawl ) Compatible with evolving multi-core architectures Simplifies software distributions for complex environments Whole system (workload) migration Improved data-center management and efficiency Additional services (workload isolation) added underneath the OS security (intrusion detection, sandboxing, ) fault-tolerance (checkpointing, roll-back/recovery)
Virtualization on PC hardware VMware showed that it could be done Intel and AMD added hardware virtualization to their processors. Multi-core 64-bit machines, with many CPU cores and threads and lots of RAM Hard to use effectively in 32-bit OSs and apps but convenient for virtualization
Virtualization Virtualization Definitions A layer mapping its visible interface and resources onto the interface and resources of the underlying layer or system on which it is implemented Purposes Abstraction to simplify the use of the underlying resource (e.g., by removing details of the resource s structure) Replication to create multiple instances of the resource (e.g., to simplify management or allocation) Isolation to separate the uses which clients make of the underlying resources (e.g., to improve security) Virtual Machine Monitor (VMM) A virtualization system that partitions a single physical machine into multiple virtual machines. Terminology Host the machine and/or software on which the VMM is implemented Guest the OS which executes under the control of the VMM CS5204 Operating Systems 13
Virtualization Origins - Principles an efficient, isolated duplicate of the real machine Efficiency Innocuous instructions should execute directly on the hardware Resource control Equivalence Executed programs may not affect the system resources The behavior of a program executing under the VMM should be the same as if the program were executed directly on the hardware (except possibly for timing and resource availability) Communications of the ACM, vol 17, no 7, 1974, pp.412-421 CS5204 Operating Systems 15
Virtualization Origins - Principles Instruction types Privileged an instruction traps in unprivileged (user) mode but not in privileged (supervisor) mode. Sensitive Control sensitive attempts to change the memory allocation or privilege mode Behavior sensitive Location sensitive execution behavior depends on location in memory Mode sensitive execution behavior depends on the privilege mode Innocuous an instruction that is not sensitive Theorem For any conventional third generation computer, a virtual machine monitor may be constructed if the set of sensitive instructions for that computer is a subset of the set of privileged instructions. Signficance The IA-32/x86 architecture is not virtualizable. CS5204 Operating Systems 16
Virtualization Strategies GuestOS trap resource privileged instruction De-privileging VMM emulates the effect on system/hardware resources of privileged instructions whose execution traps into the VMM aka trap-and-emulate Typically achieved by running GuestOS at a lower hardware priority level than the VMM Problematic on some architectures where privileged instructions do not trap when executed at deprivileged priority change emulate change vmm resource Primary/shadow structures VMM maintains shadow copies of critical structures whose primary versions are manipulated by the GuestOS e.g., page tables Primary copies needed to insure correct environment visible to GuestOS Memory traces Controlling access to memory so that the shadow and primary structure remain coherent Common strategy: write-protect primary copies so that update operations cause page faults which can be caught, interpreted, and emulated. CS5204 Operating Systems 17
Virtualization System-level Design Approaches Full virtualization (direct execution) Exact hardware exposed to OS Efficient execution OS runs unchanged Requires a virtualizable architecture Example: VMWare Paravirtualization OS modified to execute under VMM Requires porting OS code Execution overhead Necessary for some (popular) architectures (e.g., x86) Examples: Xen, Denali CS5204 Operating Systems 19
Full vs Para Virtualization Full virtualization almost complete simulation of the actual hardware to allow software, which typically consists of a guest operating system, to run unmodified. Paravirtualization a hardware environment is not simulated; however, the guest programs are executed in their own isolated domains, as if they are running on a separate system. Guest programs need to be specifically modified to run in this environment.
Virtualization System VMMs Structure Type 1: runs directly on host hardware Type 2: runs on HostOS Primary goals Examples Type 1 Type 1: High performance Type 2: Ease of construction/installation/acceptability Type 1: VMWare ESX Server, Xen, OS/370 Type 2: User-mode Linux Type 2 CS5204 Operating Systems 21
Approaches to server virtualization
Evolution of Software solutions Server virtualization approaches 1 st Generation: Full virtualization (Binary rewriting) Software Based VMware and Microsoft Virtual Virtual Machine Machine Dynamic Translation Operating System 2 nd Generation: Paravirtualizatio n Cooperative virtualization Modified guest VMware, Xen VM Hypervisor VM 3 rd Generation: Silicon-based (Hardware-assisted) virtualization Unmodified guest Kvm, Vmware, Xen on virtualizationaware hardware platforms Virtual Machine Hypervisor Virtual Machine Hardware Hardware Time Hardware Virtualization Logic
App. C App. B App. A Full Virtualization 1 st Generation offering of x86/x64 server virtualization Dynamic binary translation The emulation layer talks to an operating system which talks to the computer hardware The guest OS doesn't see that it is used in an emulated environment All of the hardware is emulated including the CPU Two popular open source emulators are QEMU and Bochs Server virtualization approaches Virtual Machine Guest OS Device Drivers Emulated Hardware Device Drivers Host OS Hardware
Full Virtualization - Advantages The emulation layer Isolates VMs from the host OS and from each other Controls individual VM access to system resources, preventing an unstable VM from impacting system performance Total VM portability Server virtualization approaches By emulating a consistent set of system hardware, VMs have the ability to transparently move between hosts with dissimilar hardware without any problems It is possible to run an operating system that was developed for another architecture on your own architecture A VM running on a Dell server can be relocated to a Hewlett- Packard server
Full Virtualization - Drawbacks Hardware emulation comes with a performance price In traditional x86 architectures, OS kernels expect to run privileged code in Ring 0 However, because Ring 0 is controlled by the host OS, VMs are forced to execute at Ring 1/3, which requires the VMM to trap and emulate instructions Due to these performance limitations, paravirtualization and hardware-assisted virtualization were developed Application Ring 3 Server virtualization approaches Application Ring 3 Operating System Ring 0 Traditional x86 Architecture Guest OS Ring 1 / 3 Virtual Machine Monitor Full Virtualization Ring 0
App. C App. B App. A Para-Virtualization The Guest OS is modified and thus run kernel-level operations at Ring 1 (or 3) the guest is fully aware of how to process privileged instructions thus, privileged instruction translation by the VMM is no longer necessary The guest operating system uses a specialized API to talk to the VMM and, in this way, execute the privileged instructions The VMM is responsible for handling the virtualization requests and putting them to the hardware Virtual Machine Guest OS Server virtualization approaches Device Drivers Specialized API Virtual Machine Monitor Device Drivers Hypervisor Hardware
Para-Virtualization Today, VM guest operating systems are paravirtualized using two different approaches: Recompiling the OS kernel Paravirtualization drivers and APIs must reside in the guest operating system kernel You do need a modified operating system that includes this specific API, requiring a compiling operating systems to be virtualization aware Some vendors (such as Novell) have embraced paravirtualization and have provided paravirtualized OS builds, while other vendors (such as Microsoft) have not Installing paravirtualized drivers Server virtualization approaches In some operating systems it is not possible to use complete paravirtualization, as it requires a specialized version of the operating system To ensure good performance in such environments, paravirtualization can be applied for individual devices For example, the instructions generated by network boards or graphical interface cards can be modified before they leave the virtualized machine by using paravirtualized drivers
App. C App. B App. A Hardware-assisted virtualization The guest OS runs at ring 0 The VMM uses processor extensions (such as Intel -VT or AMD-V) to intercept and emulate privileged operations in the guest Hardware-assisted virtualization removes many of the problems that make writing a VMM a challenge The VMM runs in a more privileged ring than 0, a virtual -1 ring is created Server virtualization approaches Virtual Machine Guest OS Device Drivers Specialized API Virtual Machine Monitor Device Drivers Hypervisor Hardware
Hardware-assisted virtualization The hypervisor/vmm runs at Ring -1 super-privileged mode Server virtualization approaches VMX non-root VMX root
Hardware-assisted virtualization Pros It allows to run unmodified Oss (so legacy OS can be run without problems) Cons Speed and Flexibility Server virtualization approaches An unmodified OS does not know it is running in a virtualized environment and so, it can t take advantage of any of the virtualization features It can be resolved using paravirtualization partially
Additional Reading Early part of the paper is a good general intro Pearce, M., Zeadally, S., & Hunt, R. (2013). Virtualization: Issues, security threats, and solutions. ACM Computing Surveys (CSUR), 45(2), 17. WMWare story with details about virtualization challenges Bugnion, E., Devine, S., Rosenblum, M., Sugerman, J., & Wang, E. Y. (2012). Bringing virtualization to the x86 architecture with the original vmware workstation. ACM Transactions on Computer Systems (TOCS), 30(4), 12. Intel virtualization technology Uhlig, R., Neiger, G., Rodgers, D., Santoni, A. L., Martins, F., Anderson, A. V.,... & Smith, L. (2005). Intel virtualization technology. Computer, 38(5), 48-56.