Evolution of Distributed Operating Systems and Virtualization. 21 November 2009 Jun Nakajima Principal Engineer

Transcription

1 Evolution of Distributed Operating Systems and ization 21 November 2009 Jun Nakajima Principal Engineer

2 Agenda History Microkernel based Distributed Operating Systems Current Generation and ization Next Generation (My View) 2

3 17 ½ Years Ago in Seattle, Washington, US 3

4 Some of them are still there 4

5 Distributed UNIX* SVR4 on Chorus Process Manger Stream Manger IPC Key Manger Object Manger (File) Workshop Proceedings Micro-kernels and Other Kernel Architectures, April 27 28, 1992, Seattle, Washington 5

6 Microkernel-Based Kernels Failed as foundation of distributed operating systems Required significant restructuring of the OS Maintenance /compatibility issues with original OS Did not match with the abstractions implemented by OS Design/logic changes Too simple or generic More complexity Distributed systems are more complex, hard to debug, APIs keep changing But provided innovations toward modular, optimized, wellstructured OS Hybrid kernel Nanokernel Used by various embedded/real-time OS today Distributing single OS in components was not a good idea 6

7 BTW: More Deterministic Latency on Micro-kernel Based OS Hypervisor-based Real-time systems can be good choice Workshop Proceedings Micro-kernels and Other Kernel Architectures, April 27 28, 1992, Seattle, Washington 7

8 Agenda History Microkernel based Distributed Operating Systems Current Generation and ization Next Generation (My View) 8

9 Machine Monitor (VMM) Machines (VMs) VM 0 Apps OS Apps OS Apps OS VM n Apps OS Physical Platform Resources Storage Network Processors Memory I/O VMM is similar to operating system; it monitors VMs instead of processes

10 ization-based Distributed OS ization on VMM ( Machine Monitor) maintains existing raw and machine interfaces No need to change or restructure the OS Well-established interface/behavior based on H/W Para-virtualization changes the interface, but the changes are typically mechanical. No design/logic changes ization-based approach is ideal for implementing distributed OS Scalability (next slide) Reliability (in Extended topics on virtualization Class) H/W virtualization features Hardware is fully ready for ization-based Distributed Operating System 10

11 Achieving Scalability Using VMM Scalability using single VM Scalability within VM Increase VCPUs (virtual CPU hot-plug), amount of memory (virtual memory hotplug), I/O capabilities (hot-plug, direct I/O, SR-IOV) Scalability using VMs Add more VMs Scalability using multiple VMs Use live migration VM 0 App #1 App #3 App #3 App #3 App #3 Guest OS 00 Guest OS 1 1 Guest OS OS 3 2 Guest OS 2 Guest OS 2 Guest OS 1 Guest OS 2 Guest OS 1 Guest OS 1 Guest OS 1 Guest OS 1 VM Monitor (VMM) VM Monitor (VMM) VM Monitor (VMM) VM Monitor (VMM) Physical Physical Host #1 Physical Physical Host #2 Physical Physical Host #m Physical Physical Host #x 11

12 Current Generation and Status VMM was not intended to be a distributed OS, but some features are emerging Resource balancing using live migration Live migration is not available across VMMs from different venders Probably it will not happen in the short term Management tools manage resources and VMMs react VM configuration VM creation/termination VM live migration Management tools are semi-automatic Better than by hand Not scalable (with e.g. >100) Current Generation is Incomplete as Distributed Operating System 12

13 Agenda History Microkernel based Distributed Operating Systems Current Generation and ization Next Generation (My View) 13

14 Issues with Current Generation Minimal H/W abstraction layer to VMs Advanced H/W virtualization features are not effectively advertized Direct I/O (e.g. VT-d), SR-IOV, HAP (Hardware Assisted Paging, e.g. EPT) or not Least common features advertized for live migration New H/W features are not utilized in VMs effectively VM configuration/placement is ad hoc (e.g. hand) Number of VMs and machines would increase in the future Multi-cores, Cloud Computing More frequent resource adjustment/management >10,000 VMs VM 0 App #1 App #3 App #3 App #3 App #3 Guest OS 00 Guest OS 1 1 Guest OS OS 3 2 Guest OS 2 Guest OS 1 Guest OS 2 Guest OS 1 Guest OS 2 Guest OS 1 H/W Abstraction Layer VM Monitor (VMM) VM Monitor (VMM) VM Monitor (VMM) VM Monitor (VMM) Physical We will need Physical real ization-based Physical Distributed Physical Physical Host #1 Physical Host #2 Operating System Physical Host #m Physical Host #x 14

15 Next Generation: Cloud Operating Systems More scalability using systematic and automatic resource management View the whole system Monitor each VMM and VMs Host level: CPU features, # of sockets, # of cores, CPU utilization, # of interrupts, amount memory available, power consumption, I/O throughput, devices available, H/W virtualization features, Per VM: VCPU features, # of sockets, # of cores, # of VM exits, # of interrupts, power consumption, I/O throughput and flows, devices available, VM creation/placement based on profile/requirements and policy/rules Priority Type of VM, e.g. large and I/O intensive, HPC, desktop, NUMA, Power management E.g. suspend if idle for a specific time Failover, etc. Prediction 15

16 Next Generation: Cloud Operating Systems (cont.) Utilize new/advanced H/W features in VMs as much as possible for cloud computing Based on VM profile/requirements Performance, power, workload characteristics, security, etc. Minimize virtualization overheads # of VMs can be very large More advance topics Grid computing, HPC cloud Parallel computing, memory intensive workload, high throughput 16

17 Conclusions ization technologies allow to implement real Distributed Operating System for next generation Scalable/reliable cloud operating systems High-performance/Grid computing Hardware virtualization features are ready for solving challenging problems Time for you to work in virtualization! 17