Is Virtualization Killing SSI Research? Jérôme Gallard Paris Project-Team Dinard November 2007 Supervisor : Christine Morin Co-supervisor: Adrien Lèbre
My subject! ;) Reliability and performance of execution in the context of grid environment Starting point: cluster and Kerrighed November 2007 2
Outline Context - Cluster / grid - SSI - Virtualization Combining Virtualization and SSI Conclusion - Lessons learnt - Perspectives November 2007 3
Introduction Cluster Exploitation: How to execute jobs on a cluster architecture? - Batch scheduler (BS) - Single System Image (SSI) N0 N1 N2 N3 N4 N5 November 2007 34
Batch Scheduler (BS) N1 N2 N3 N0 Wait queue of jobs N4 N5 N6 head node compute nodes Users interact only with the head node November 2007 65
Single System Image (SSI) 1/2 compute nodes N1 N2 N3 no head node N4 N5 N6 Users have illusion to manage a SMP machine with UNIX standard tools November 2007 6
Single System Image (SSI) 2/2 Functionalities: - Global management of processes - Migration of processes - Global management of ressources (such as the memory) November 2007 7
Virtualization Technologies 1/2 virtualized machines running on the compute node VM0 VM1 VM2 N0 compute node November 2007 8
Virtualization Technologies 2/2 Combining virtualization with BS N1 N2 N3 N0 Wait queue of VMs (JAWS [Grit2007]) N4 N5 N6 head node (BS) compute nodes November 2007 9
Combining BS + VM Simplify: - Deployment of applications - Management of applications - Management of application environment November 2007 10 10
VMs vs SSI Is virtualization killing system image research? What's about combining both approaches? November 2007 11 11
Introduction To Virtualization 1/5 Virtualization properties: Isolation VM1 simulation? VM2 Apache web server virtual machines virtual machines November 2007 12 12
Introduction To Virtualization 2/5 Virtualization properties: Isolation Server consolidation VM0 VM1 N0 N1 compute node compute node November 2007 13 12
Introduction To Virtualization 2/5 Virtualization properties: Isolation Server consolidation virtual machines VM0 VM1 N0 N1 compute node compute node November 2007 14 12
Introduction To Virtualization 2/5 Virtualization properties: Isolation Server consolidation virtual machines VM0 VM1 N0 N1 compute node compute node November 2007 15 12
Introduction To Virtualization 3/5 Virtualization properties: Isolation Server consolidation Virtual machine portability (architecture dependancy) virtual machines VM0 N0 compute node November 2007 16 12
Introduction To Virtualization 3/5 Virtualization properties: Isolation Server consolidation Virtual machine portability (architecture dependancy) virtual machines VM0 virtual machines VM0 N0 N1 compute node compute node November 2007 17 12
Introduction To Virtualization 4/5 Virtualization properties: Isolation Server consolidation Virtual machine portability (architecture dependancy) Application portability virtual machines VM0 OS0 capacity of running OS0 on VM0 without modifications of OS0 November 2007 18 12
Introduction To Virtualization 5/5 Virtualization properties: Isolation Server consolidation Virtual machine portability (architecture dependancy) Application portability Suspend/restart virtual machines VM0 November 2007 19 12
Virtualization Classification 1/3 Goldberg [Goldberg1973]: typei-virtualization Kernel Space User Space The Host OS manages devices. VMs are instances of virtual machines Host OS (Dom0) VM VM... The Hypervisor (like a superbios): * schedules the Host OS and VMs * forwards privilege instructions from VMs to Host OS Hypervisor Hardware Example of typei-virtualization: XEN [Barham2003] November 2007 20 13
Virtualization Classification 2/3 Goldberg: typeii-virtualization User space The Host OS manages the hypervisor as a process. VM VM... VMs are instances of virtual machines The hypervisor is a process running by the host OS. It's in charge of VMs management. Hypervisor Host OS Hardware Kernel space Example of typeii: Vmware [VMware2007], QEMU [Bellart2005] November 2007 21 14
Virtualization Classification 3/3 Container based virtualization NOT in Goldberg classification, but more and more used nowadays User Space cntr1 cntr2 Kernel Space Host Host OS OS + + Hypervisor Hypervisor Hardware Example of containers: OpenVZ [OpenVZ2007], chroot [Chroot2007] November 2007 22 15
Combining Container and SSI Container on the top of SSI 1/2 App1 App2 App3 App4 Advantages: + server consolidation + suspend/restart + Application portability Disadvantages: - Isolation - VM portability Ctnr1 Ctnr2 Ctnr3 Container + SSI N1 N2 N3 N4 November 2007 23 16
Combining container and SSI SSI on the top of Container 2/2 Architecture irrelevant: The same kernel is shared between all containers (they have not their own kernel) SSI Ctnr1 Ctnr2 Ctnr3 Ctnr4 Ctnr5 OS0 OS0 OS0 OS0 N1 N2 N3 N4 November 2007 24 17
Combining Virtualization and SSI TypeI-Virtualization upon SSI 1/2 App1 App2 App3 Advantages: + Isolation + server consolidation + Application portability + suspend/restart Disadvantage: - VM portability Host OS (Dom0) part2 of SSI VM1 VM2 Hypervisor + part1 of SSI N1 N2 N3 Example: ScaleMP is like a distributed superbios November 2007 25 18
Combining Virtualization and SSI SSI upon TypeI-Virtualization 2/2 App1 App2 App3 Advantages: + Isolation + server consolidation + Suspend/restart + Application portability SSI Disadvantage: - VM portability Host OS (Dom0) VM1 VM2 VM3 Host OS (Dom0) Hypervisor Hypervisor N1 N2 N3 November 2007 26 19
Combining Virtualization and SSI TypeII-Virtualization upon SSI 1/2 App1 App2 App3 Advantages: + Isolation + server consolidation + Suspend/restart + Application portability VM1 Hypervisor VM2 Disadvantages: - VM portability SSI N1 N2 N3 November 2007 27 20
Combining Virtualization and SSI SSI upon TypeII-Virtualization 2/2 Advantages: + Application portability + Isolation + server consolidation + Suspend/restart + VM portability App1 App2 App3 SSI VM1 VM2 VM3 VM4 Hyp1 Hyp2 Hyp3 OS1 OS2 OS3 N1 N2 N3 November 2007 28 21
Combining Virtualization and SSI Isolation of Two Distinct SSIs Possibility to run several SSIs on the same cluster. App1 App2 App3 SSI 1 SSI 2 VM1 VM2 VM3 VM4 VM5 Hyp1 Hyp2 Hyp3 OS1 OS2 OS3 N1 N2 N3 November 2007 29 22
Combining Virtualization and SSI Three layers of flexibilities Possibility to run a windows application on a linux SSI cluster running upon MACOS X systems App1 VM1 VM2 SSI VM1 VM2 N1 N2 N3 November 2007 30 23
Lessons Learnt 1/4 Containers on top of single system image clusters The SSI system combines all the advantages enabled by containers on a real SMP machine in a cluster environment November 2007 31 24
Lessons Learnt 2/4 Virtual Machines on top of SSI clusters Same isolation advantage as containers Virtualization solves application portability issues example : Possibility to run an application developed for process technology A and OS B on top of computers running SSI OS based on OS C and developped for achitecture processor D. November 2007 32 25
Lessons Learnt 3/4 SSI on top of VMs VM migration and suspend functionalities provide: Flexible, simple and ''On Demand'' resource allocation mechanisms for the applications Transparent adaptation mechanisms in case of hardware changes (for instance, node additions / removals). November 2007 33 26
Lessons Learnt 4/4 Virtualization and SSI complement each other SSI abstracts resources Virtualization adds another level of hardware flexibility November 2007 34 27
Conclusion Performance evaluation: VM usage does not significantly impact performances (4% with MrBayes application in MPI mode) Combining VM/SSI setup: - Very attractive - Installation and use are quite simple (evaluated with TypeII) Next step: migrate an SSI cluster from Rennes to Sophia November 2007 35 28
Future Works 1/2 Extend the Goldberg's classification to present in a uniform way: The hardware, The emulated hardware, The OS and SSI, The different kinds of virtualization techniques, The containers. November 2007 36 29
Future Works 2/2 Investigate SSI scheduler and hypervisor scheduler: does the SSI scheduler directly manage VMs? Extend to the Grid: interests of such hybrid approaches (VM/SSI) in XtreemOS November 2007 37 30
References [Grit2007] L. Grit, D. Irwin, V.Marupadi, P. Shivam, A. Yumerefendi, J. Chase, and J. Albrecht. Harnessing virtual machine resource control for job management. March 2007. [Goldberg1973] R. P. Goldberg. Architecture of virtual machines. AFIPS National Computer Conference, July 1973. [OpenVZ2007] OpenVZ. OpenVZ welcome page, 2007. Available as http://wiki.openvz.org/main_page. [Chroot2007] GNU. Chroot, 2007. Available as http://www.gnu.org/software/coreutils/manual/coreutils.html#chroot-invocation [Barham2003] P. Barham, B. Dragovic, K. Fraser, S. Hand, T. Harris, A. Ho, R. Neugebauer, I. Pratt, and Warfield. Xen and the art of virtualization. Bolton Landing, New-York, USA, October 2003. SOSP'03. [VMware2007] VMware. VMware welcome page, 2007. Available as http://www.vmware.com [Bellard2005] F. Bellard. QEMU, a fast and portable dynamic translator. Technical report, USENIX Association, 2005. [Huelsenbeck] J. P. Huelsenbeck and F. Ronquist. MrBayes: a program for the Bayesian inference of phylogeny. http://golab.unl.edu/teaching/sbseminar/manual.pdf November 2007 38 31