Distributed System: Lecture 4 Virtualizations

Transcription

1 Distributed System: Lecture 4 Virtualizations Box Leangsuksun SWECO Endowed Professor, Computer Science Louisiana Tech University box@latech.edu CTO, PB Tech International Inc. naibox@gmail.com

2 Introduction to Virtualization System virtualization studied since the 70's (Goldberg, Popek) Fundamental Run multiple virtual machines (OSes) simultaneously Isolating between virtual machines. Controlling Resources sharing between VMs Increase resources utilization One of the hottest technologies since 2006

3 Virtualization: Key concepts Virtual Machine (VM), guest OS: complete operating system running in a virtual environment Host OS: operating system running on top the hardware, interface between the user and the VMM and VMs Virtual Machine Monitor (VMM):, Hypervisor: manage VMs (scheduling, hardware access)

4 Virtualization: Usage Ø Server consolidation (cloud) Ø Software testing Ø Security, Isolation (cloud) Ø Lower cost of ownership of server. (cloud) Ø Increase manageability (cloud) Ø Enhance server reliability

5 Major Fields of Virtualization Storage Virtualization Network Virtualization Server Virtualization

6 Architecture & Interfaces Architecture: formal specification of a system s interface and the logical behavior of its visible resources. API Applications ABI Libraries Operating System System Calls ISA System ISA User ISA Hardware n n n API application binary interface ABI application binary interface ISA instruction set architecture Credit: CS5204 Operating Systems from vtech u

7 Sample of API vs ABI 4/22/14 Towards survivable architecture 7

8 VMM Types System Provides ABI interface Efficient execution Can add OS-independent services (e.g., migration, intrusion detection) n Process Provdes API interface Easier installation Leverage OS services (e.g., device drivers) Execution overhead (possibly mitigated by justin-time compilation) CS5204 Operating Systems Credit: CS5204 Operating Systems from vtech u

9 System-level Design Approaches Full virtualization (direct execution) Exact hardware exposed to OS Efficient execution OS runs unchanged Requires a virtualizable architecture Example: VMWare n Paravirtualization OS modified to execute under VMM Requires porting OS code Execution overhead Necessary for some (popular) architectures (e.g., x86) Examples: Xen, Denali CS5204 Operating Systems Credit: CS5204 Operating Systems from vtech u

10 Design Space (level vs. ISA) API interface ABI interface Variety of techniques and approaches available Critical technology space highlighted CS5204 Operating Systems Credit: CS5204 Operating Systems from vtech u

11 System VMMs Type 1 Structure Type 1: runs directly on host hardware Type 2: runs on HostOS Primary goals Type 1: High performance Type 2: Ease of construction/installation/acceptability Examples Type 1: VMWare ESX Server, Xen, OS/370 Type 2: User-mode Linux Type 2 CS5204 Operating Systems Credit: CS5204 Operating Systems from vtech u

12 Hosted VMMs Structure Hybrid between Type1 and Type2 Core VMM executes directly on hardware I/O services provided by code running on HostOS Goals Improve performance overall leverages I/O device support on the HostOS Disadvantages Incurs overhead on I/O operations Lacks performance isolation and performance guarantees Example: VMWare (Workstation) CS5204 Operating Systems Credit: CS5204 Operating Systems from vtech u

13 Whole-system VMMs n n n Challenge: GuestOS ISA differs from HostOS ISA Requires full emulation of GuestOS and its applications Example: VirtualPC CS5204 Operating Systems Credit: CS5204 Operating Systems from vtech u

14 Strategies GuestOS trap change resource emulate change vmm resource privileged instruction De-privileging VMM emulates the effect on system/hardware resources of privileged instructions whose execution traps into the VMM aka trap-and-emulate Typically achieved by running GuestOS at a lower hardware priority level than the VMM Problematic on some architectures where privileged instructions do not trap when executed at deprivileged priority Primary/shadow structures VMM maintains shadow copies of critical structures whose primary versions are manipulated by the GuestOS e.g., page tables Primary copies needed to insure correct environment visible to GuestOS Memory traces Controlling access to memory so that the shadow and primary structure remain coherent Common strategy: write-protect primary copies so that update operations cause page faults which can be caught, interpreted, and emulated. CS5204 Operating Systems Credit: CS5204 Operating Systems from vtech u

15 Different Virtualization Concepts Full-virtualization: full virtual machine, from the boot sequence to the virtualized hardware Para-virtualization: the guest OS has to be modify for performance optimization Emulation: the guest OS architecture is different from the architecture of the host OS (translation on the fly). Ex: PPC VM on top of a x86 host OS.

16 Classification Two kinds of system virtualization Type-I: the virtual machine monitor and the virtual machine run directly on top of the hardware, Type-II: the virtual machine monitor and the virtual machine run on top of the host OS Host OS VM VM VMM Hardware Type I Virtualization (Bare-metal) VMware ESX, Microsoft Hyper-V, Xen VM VMM Host OS VM Hardware Type II Virtualization (hosted) VMware Workstation, Microsoft Virtual PC, Sun VirtualBox, QEMU, KVM

17 Bare-metal or hosted? Bare-metal Has complete control over hardware 17 Doesn t have to fight an OS Hosted Avoid code duplication: need not code a process scheduler, memory management system the OS already does that Can run native processes alongside VMs Familiar environment how much CPU and memory does a VM take? Use top! How big is the virtual disk? ls l Easy management stop a VM? Sure, just kill it! A combination Mostly hosted, but some parts are inside the OS kernel for performance reasons E.g., KVM

18 Available Solutions Example of Virtualization Projects Type I: Xen, L4, VMware ESX, Microsoft Hyper- V Type II: VMware Workstation, Microsoft Virtual PC, Sun VirtualBox, QEMU, KVM Different Benefits Type I: performances direct access to the hardware simple to implement para-virtualization possible Type II: development no limitation of para-virtualization emulation possible

19 How to run a VM? Emulate! Do whatever the CPU does but in software Fetch the next instruction Decode is it an ADD, a XOR, a MOV? Execute using the emulated registers and memory Example: addl %ebx, %eax is emulated as: enum {EAX=0, EBX=1, ECX=2, EDX=3, }; unsigned long regs[8]; regs[eax] += regs[ebx]; 19

20 How to run a VM? Emulate! Pro: Simple! Con: Slooooooooow Example hypervisor: BOCHS 20

21 How to run a VM? Trap and emulate! Run the VM directly on the CPU no emulation! Most of the code can execute just fine 21 E.g., addl %ebx, %eax Some code needs hypervisor intervention int $0x80 movl something, %cr3 I/O Trap and emulate it! E.g., if guest runs int $0x80, trap it and execute guest s interrupt 0x80 handler

22 How to run a VM? Trap and emulate! Pro: Performance! Cons: 22 Harder to implement Need hardware support Not all sensitive instructions cause a trap when executed in usermode E.g., POPF, that may be used to clear IF This instruction does not trap, but value of IF does not change!

23 How to run a VM? Dynamic (binary) translation! Take a block of binary VM code that is about to be executed Translate it on the fly to safe code (like JIT just in time compilation) Execute the new safe code directly on the CPU Translation rules? Most code translates identically (e.g., movl %eax, %ebx translates to itself) 23 Sensitive operations are translated into hypercalls Hypercall call into the hypervisor to ask for service Implemented as trapping instructions (unlike POPF) Similar to syscall call into the OS to request service

24 How to run a VM? Dynamic (binary) translation! Pros: No hardware support required Performance better than emulation Cons: Performance worse than trap and emulate Hard to implement hypervisor needs on-the-fly x86- to-x86 binary compiler Example hypervisors: VMware, QEMU 24

25 How to run a VM? Paravirtualization! Does not run unmodified guest OSes Requires guest OS to know it is running on top of a hypervisor E.g., instead of doing cli to turn off interrupts, guest OS should do hypercall(disable_interrupts) 25

26 How to run a VM? Paravirtualization! Pros: No hardware support required Performance better than emulation Con: Requires specifically modified guest Same guest OS cannot run in the VM and bare-metal Example hypervisor: Xen 26

27 Industry trends Trap and emulate With hardware support VMX, SVM 27

28 Linux-related virtualization projects Project Type License Bochs Emulation LGPL QEMU Emulation LGPL/GPL VMware Full virtualization Proprietary z/vm Full virtualization Proprietary Xen Paravirtualization GPL UML Paravirtualization GPL Linux-VServer OpenVZ Operating systemlevel virtualization Operating systemlevel virtualization GPL GPL

29 Hardware support for full virtualization and paravirtualization Recall that the IA-32 (x86) architecture creates some issues when it comes to virtualization. Certain privileged-mode instructions do not trap, and can return different results based upon the mode. For example, the x86 STR instruction retrieves the security state, but the value returned is based upon the particular requester's privilege level. This is problematic when attempting to virtualize different operating systems at different levels. For example, the x86 supports four rings of protection, where level 0 (the highest privilege) typically runs the operating system, levels 1 and 2 support operating system services, and level 3 (the lowest level) supports applications. Hardware vendors have recognized this shortcoming (and others), and have produced new designs that support and accelerate virtualization.

30 Hardware support for full virtualization and paravirtualization Intel is producing new virtualization technology that will support hypervisors for both the x86 (VT-x) and Itanium (VT-i) architectures. The VT-x supports two new forms of operation one for the VMM (root) one for guest operating systems (non-root). The root form is fully privileged, while the non-root form is deprivileged (even for ring 0). The architecture also supports flexibility in defining the instructions that cause a VM (guest operating system) to exit to the VMM and store off processor state. Other capabilities have been added

31 Hardware support for full virtualization and paravirtualization AMD is also producing hardware-assisted virtualization technology, under the name Pacifica. Among other things, Pacifica maintains a control block for guest operating systems that are saved on execution of special instructions. The VMRUN instruction allows a virtual machine (and its associated guest operating system) to run until the VMM regains control (which is also configurable). The configurability allows the VMM to customize the privileges for each of the guests. Pacifica also amends address translation with host and guest memory management unit (MMU) tables.

32 I/O Virtualization Typical methods to virtualize the CPU A computer is more than a CPU Also need I/O! Types of I/O: 32 Block (e.g., hard disk) Network Input (e.g., keyboard, mouse) Sound Video Most performance critical (for servers): Network Block

33 Xen Overview Para-virtualization possible full-virtualization is virtualization support at the hardware level (VT Intel technology, AMD-V/Pacifica technology) XenoLinux: port of the Linux kernel to the Xen Hypervisor Hypervisor based on a micro-kernel Ø Open Source, Linux based Ø Create and manage VMs via command line Ø Restricted hardware access though API Ø Host s kernel need to be patched.

34 VMware Overview Ø Commercial virtualization applications Ø Full Virtualization Ø Highly Portability Ø Simulate BIOS, PXE boot. Ø Simulate virtual Hardware for every VM Ø Support Bridge, NAT, and Host-Only Networks Ø Run wide range unmodified guest OSes such as Windows, Linux, Solaris, BSD, Netware, DOS,

35 VMware Overview Source :

36 VMware vs. Xen Relative performance on native Linux (L), Xen/Linux (X), VMware Workstation 3.2 (V), and User Mode Linux (U). Source : Xen and Art of Virtualization, Ian Pratt, University of Cambridge, Xensource Inc.

37 VMware vs. Xen (TCP results) L X V U Tx, MTU 1500 (Mbps) L X V U Rx, MTU 1500 (Mbps) L X V U Tx, MTU 500 (Mbps) L X V U Rx, MTU 500 (Mbps) TCP bandwidth on Linux (L), Xen (X), VMWare Workstation (V), and UML (U) Source : Xen and Art of Virtualization, Ian Pratt, University of Cambridge, Xensource Inc.

38 Qemu Emulation solution Direct access to the hardware possible if the host OS and the guest OS have the same architecture User Space User Space User Space User Space User Space Linux Windows Linux Mac OS X Solaris Drivers Drivers Drivers Drivers Drivers Qemu x86 Qemu x86 Qemu PPC Qemu PPC Qemu Sparc Host OS: Linux, Mac OS X, Windows Hardware: processor, memory, disk, network, etc. From

39 Xen Overview Para-virtualization possible full-virtualization is virtualization support at the hardware level (VT Intel technology, AMD-V/ Pacifica technology) XenoLinux: port of the Linux kernel to the Xen Hypervisor Hypervisor based on a micro-kernel Efficient virtualization: HPC possible

40 Xen Overview Ø Open Source, Linux based Ø High Performance Ø Support Bridge, and Routing Networks Ø Create and manage VMs via command line Ø Restricted hardware access though API Ø Host s kernel need to be patched.

41 Xen s Ring Model Ring 3 User Applications Ring 1 and 2 are not used Ring 0 Operating System Ring 3 User Applications Ring 2 is not used Ring 1 for VM s Ring 0 Xen s Hypervisor Standard x86 Architecture Xen on x86 Architecture

42

43 The architecture of Xen 43 Instructor s Guide for Coulouris, Dollimore, Kindberg and Blair, Distributed Systems: Concepts and Design Edn. 5 Pearson Education

44 Use of rings of privilege 44 Instructor s Guide for Coulouris, Dollimore, Kindberg and Blair, Distributed Systems: Concepts and Design Edn. 5 Pearson Education

45 Virtualization of memory management 45 Instructor s Guide for Coulouris, Dollimore, Kindberg and Blair, Distributed Systems: Concepts and Design Edn. 5 Pearson Education

46 Split device drivers 46 Instructor s Guide for Coulouris, Dollimore, Kindberg and Blair, Distributed Systems: Concepts and Design Edn. 5 Pearson Education

47 I/O rings 47 Instructor s Guide for Coulouris, Dollimore, Kindberg and Blair, Distributed Systems: Concepts and Design Edn. 5 Pearson Education

48 The XenoServer Open Platform Architecture 48 Instructor s Guide for Coulouris, Dollimore, Kindberg and Blair, Distributed Systems: Concepts and Design Edn. 5 Pearson Education

49 Virtualization Examples Server consolidation - Virtual machines are used to consolidate many physical servers into fewer servers, which in turn host virtual machines. Each physical server is reflected as a virtual machine "guest" residing on a virtual machine host system. This is also known as Physical-to-Virtual or 'P2V' transformation.

50 Virtualization Examples Disaster recovery - Virtual machines can be used as "hot standby" environments for physical production servers. This changes the classical "backup-and-restore" philosophy, by providing backup images that can "boot" into live virtual machines, capable of taking over workload for a production server experiencing an outage.

51 Virtualization Examples Testing and training - Hardware virtualization can give root access to a virtual machine. This can be very useful such as in kernel development and operating system courses.

52 Virtualization Examples Portable applications - The Microsoft Windows platform has a well-known issue involving the creation of portable applications, needed (for example) when running an application from a removable drive, without installing it on the system's main disk drive. This is a particular issue with USB drives. Virtualization can be used to encapsulate the application with a redirection layer that stores temporary files, Windows Registry entries, and other state information in the application's installation directory and not within the system's permanent file system. See portable applications for further details. It is unclear whether such implementations are currently available.

53 Virtualization Examples Portable workspaces - Recent technologies have used virtualization to create portable workspaces on devices like ipods and USB memory sticks. These products include: Application Level Thinstall which is a driver-less solution for running "Thinstalled" applications directly from removable storage without system changes or needing Admin rights OS-level MojoPac, Ceedo, and U3 which allows end users to install some applications onto a storage device for use on another PC. Machine-level moka5 and LivePC which delivers an operating system with a full software suite, including isolation and security protections.

54 Virtualization Tips In the VMware space, VirtualCenter is the management tool of choice for ESX Server. Other products, like Hewlett-Packard's Virtual Machine Management or IBM's Director modules, are adding functionality to deal with virtual machine [VM] environments. The problem is that most of these tools that are snap-ins lack much of the simple functionality you get in VirtualCenter. Most companies will end up buying both VirtualCenter and the vendor's tool and use both depending on what they are doing.

55 Virtualization Tips Shy away from large amounts of processing when doing consolidation. If you are doing virtualization for other reasons, like workload management, then you can get nearly anything to run virtualized if you are willing to change some of the things you do. However, if you are looking for maximum consolidation ratios and high ROIs, stay away from the quad boxes that are already running at 50%.

56 VM on Amazon 4/22/14 Towards survivable architecture 56

57 Security Tips Some standard minimum security at least: Disable remote root access use sudo when needed configure the AD PAM modules for Windows shops.

58 Security Tips Some organizations use too much surrounding security and end up making their environment slower, more difficult and expensive to manage. When dealing with the VMs, all of the standard procedures should be followed. The host systems themselves should often be considered appliances, and organizations should limit the amount of customized agents and security hacks performed on these systems.

59 Security Tips One should not go overboard with ESX hosts, since they are basically appliances serving up computing resources and should be treated as such. Nevertheless, taking a common sense approach to security on the servers is the best bet. The most common mistakes made with virtual security are based on ignorance, lack of knowledge of the Linux console, failure to understand how virtual switch architecture works, and what the host does not directly see in the data in the VM disk files.

60 Security Tips The same practices that are performed to secure a physical environment can, and should, be used in a virtual environment as well. Everything from proper VLAN/firewall organization to host-based intrusion detection should be leveraged to keep the environment secure.

61 Scalability Tips Simplicity. The more complicated the design and infrastructure, the less scalable it will be. For example, a common mistake in large organizations, is that they assume they cannot create a simple solution because they are big. One can argue that they should make the solution or design for VMware as simple as possible to make it scalable for the size of their organization and largest client base. Don't design the entire solution around the one-offs.

62 Scalability Tips When designing a virtual infrastructure, one should never look at the environment and try to plan one large infrastructure for the entire virtualization project. It won t work. Organize the overall environment into smaller groupings of servers and addressed individually. When approached this way, at the end of the project, a very scalable deployment methodology that uses the same principals with a manageable number of servers in various phases of the project will be in place