Is Virtualization Killing SSI Research?

Size: px

Start display at page:

Download "Is Virtualization Killing SSI Research?"

Edgar Quinn
8 years ago
Views:

1 Is Virtualization Killing SSI Research? Jérôme Gallard Paris Project-Team Dinard November 2007 Supervisor : Christine Morin Co-supervisor: Adrien Lèbre

2 My subject! ;) Reliability and performance of execution in the context of grid environment Starting point: cluster and Kerrighed November

3 Outline Context - Cluster / grid - SSI - Virtualization Combining Virtualization and SSI Conclusion - Lessons learnt - Perspectives November

4 Introduction Cluster Exploitation: How to execute jobs on a cluster architecture? - Batch scheduler (BS) - Single System Image (SSI) N0 N1 N2 N3 N4 N5 November

5 Batch Scheduler (BS) N1 N2 N3 N0 Wait queue of jobs N4 N5 N6 head node compute nodes Users interact only with the head node November

6 Single System Image (SSI) 1/2 compute nodes N1 N2 N3 no head node N4 N5 N6 Users have illusion to manage a SMP machine with UNIX standard tools November

7 Single System Image (SSI) 2/2 Functionalities: - Global management of processes - Migration of processes - Global management of ressources (such as the memory) November

8 Virtualization Technologies 1/2 virtualized machines running on the compute node VM0 VM1 VM2 N0 compute node November

9 Virtualization Technologies 2/2 Combining virtualization with BS N1 N2 N3 N0 Wait queue of VMs (JAWS [Grit2007]) N4 N5 N6 head node (BS) compute nodes November

10 Combining BS + VM Simplify: - Deployment of applications - Management of applications - Management of application environment November

11 VMs vs SSI Is virtualization killing system image research? What's about combining both approaches? November

12 Introduction To Virtualization 1/5 Virtualization properties: Isolation VM1 simulation? VM2 Apache web server virtual machines virtual machines November

13 Introduction To Virtualization 2/5 Virtualization properties: Isolation Server consolidation VM0 VM1 N0 N1 compute node compute node November

14 Introduction To Virtualization 2/5 Virtualization properties: Isolation Server consolidation virtual machines VM0 VM1 N0 N1 compute node compute node November

15 Introduction To Virtualization 2/5 Virtualization properties: Isolation Server consolidation virtual machines VM0 VM1 N0 N1 compute node compute node November

16 Introduction To Virtualization 3/5 Virtualization properties: Isolation Server consolidation Virtual machine portability (architecture dependancy) virtual machines VM0 N0 compute node November

machine portability (architecture dependancy)

17 Introduction To Virtualization 3/5 Virtualization properties: Isolation Server consolidation Virtual machine portability (architecture dependancy) virtual machines VM0 virtual machines VM0 N0 N1 compute node compute node November

portability (architecture dependancy) virtual machines VM0

18 Introduction To Virtualization 4/5 Virtualization properties: Isolation Server consolidation Virtual machine portability (architecture dependancy) Application portability virtual machines VM0 OS0 capacity of running OS0 on VM0 without modifications of OS0 November

(architecture dependancy) Application portability virtual machines

19 Introduction To Virtualization 5/5 Virtualization properties: Isolation Server consolidation Virtual machine portability (architecture dependancy) Application portability Suspend/restart virtual machines VM0 November

20 Virtualization Classification 1/3 Goldberg [Goldberg1973]: typei-virtualization Kernel Space User Space The Host OS manages devices. VMs are instances of virtual machines Host OS (Dom0) VM VM... The Hypervisor (like a superbios): * schedules the Host OS and VMs * forwards privilege instructions from VMs to Host OS Hypervisor Hardware Example of typei-virtualization: XEN [Barham2003] November

.. The Hypervisor (like a superbios): * schedules the Host OS and VMs * forwards privilege

21 Virtualization Classification 2/3 Goldberg: typeii-virtualization User space The Host OS manages the hypervisor as a process. VM VM... VMs are instances of virtual machines The hypervisor is a process running by the host OS. It's in charge of VMs management. Hypervisor Host OS Hardware Kernel space Example of typeii: Vmware [VMware2007], QEMU [Bellart2005] November

22 Virtualization Classification 3/3 Container based virtualization NOT in Goldberg classification, but more and more used nowadays User Space cntr1 cntr2 Kernel Space Host Host OS OS + + Hypervisor Hypervisor Hardware Example of containers: OpenVZ [OpenVZ2007], chroot [Chroot2007] November

23 Combining Container and SSI Container on the top of SSI 1/2 App1 App2 App3 App4 Advantages: + server consolidation + suspend/restart + Application portability Disadvantages: - Isolation - VM portability Ctnr1 Ctnr2 Ctnr3 Container + SSI N1 N2 N3 N4 November

24 Combining container and SSI SSI on the top of Container 2/2 Architecture irrelevant: The same kernel is shared between all containers (they have not their own kernel) SSI Ctnr1 Ctnr2 Ctnr3 Ctnr4 Ctnr5 OS0 OS0 OS0 OS0 N1 N2 N3 N4 November

25 Combining Virtualization and SSI TypeI-Virtualization upon SSI 1/2 App1 App2 App3 Advantages: + Isolation + server consolidation + Application portability + suspend/restart Disadvantage: - VM portability Host OS (Dom0) part2 of SSI VM1 VM2 Hypervisor + part1 of SSI N1 N2 N3 Example: ScaleMP is like a distributed superbios November

26 Combining Virtualization and SSI SSI upon TypeI-Virtualization 2/2 App1 App2 App3 Advantages: + Isolation + server consolidation + Suspend/restart + Application portability SSI Disadvantage: - VM portability Host OS (Dom0) VM1 VM2 VM3 Host OS (Dom0) Hypervisor Hypervisor N1 N2 N3 November

27 Combining Virtualization and SSI TypeII-Virtualization upon SSI 1/2 App1 App2 App3 Advantages: + Isolation + server consolidation + Suspend/restart + Application portability VM1 Hypervisor VM2 Disadvantages: - VM portability SSI N1 N2 N3 November

28 Combining Virtualization and SSI SSI upon TypeII-Virtualization 2/2 Advantages: + Application portability + Isolation + server consolidation + Suspend/restart + VM portability App1 App2 App3 SSI VM1 VM2 VM3 VM4 Hyp1 Hyp2 Hyp3 OS1 OS2 OS3 N1 N2 N3 November

29 Combining Virtualization and SSI Isolation of Two Distinct SSIs Possibility to run several SSIs on the same cluster. App1 App2 App3 SSI 1 SSI 2 VM1 VM2 VM3 VM4 VM5 Hyp1 Hyp2 Hyp3 OS1 OS2 OS3 N1 N2 N3 November

30 Combining Virtualization and SSI Three layers of flexibilities Possibility to run a windows application on a linux SSI cluster running upon MACOS X systems App1 VM1 VM2 SSI VM1 VM2 N1 N2 N3 November

31 Lessons Learnt 1/4 Containers on top of single system image clusters The SSI system combines all the advantages enabled by containers on a real SMP machine in a cluster environment November

32 Lessons Learnt 2/4 Virtual Machines on top of SSI clusters Same isolation advantage as containers Virtualization solves application portability issues example : Possibility to run an application developed for process technology A and OS B on top of computers running SSI OS based on OS C and developped for achitecture processor D. November

33 Lessons Learnt 3/4 SSI on top of VMs VM migration and suspend functionalities provide: Flexible, simple and ''On Demand'' resource allocation mechanisms for the applications Transparent adaptation mechanisms in case of hardware changes (for instance, node additions / removals). November

34 Lessons Learnt 4/4 Virtualization and SSI complement each other SSI abstracts resources Virtualization adds another level of hardware flexibility November

35 Conclusion Performance evaluation: VM usage does not significantly impact performances (4% with MrBayes application in MPI mode) Combining VM/SSI setup: - Very attractive - Installation and use are quite simple (evaluated with TypeII) Next step: migrate an SSI cluster from Rennes to Sophia November

36 Future Works 1/2 Extend the Goldberg's classification to present in a uniform way: The hardware, The emulated hardware, The OS and SSI, The different kinds of virtualization techniques, The containers. November

37 Future Works 2/2 Investigate SSI scheduler and hypervisor scheduler: does the SSI scheduler directly manage VMs? Extend to the Grid: interests of such hybrid approaches (VM/SSI) in XtreemOS November

38 References [Grit2007] L. Grit, D. Irwin, V.Marupadi, P. Shivam, A. Yumerefendi, J. Chase, and J. Albrecht. Harnessing virtual machine resource control for job management. March [Goldberg1973] R. P. Goldberg. Architecture of virtual machines. AFIPS National Computer Conference, July [OpenVZ2007] OpenVZ. OpenVZ welcome page, Available as [Chroot2007] GNU. Chroot, Available as [Barham2003] P. Barham, B. Dragovic, K. Fraser, S. Hand, T. Harris, A. Ho, R. Neugebauer, I. Pratt, and Warfield. Xen and the art of virtualization. Bolton Landing, New-York, USA, October SOSP'03. [VMware2007] VMware. VMware welcome page, Available as [Bellard2005] F. Bellard. QEMU, a fast and portable dynamic translator. Technical report, USENIX Association, [Huelsenbeck] J. P. Huelsenbeck and F. Ronquist. MrBayes: a program for the Bayesian inference of phylogeny. November

Is Virtualization Killing SSI Research?

Is Virtualization Killing SSI Research? Jérôme Gallard, Geoffroy Vallée, Adrien Lèbre, Christine Morin, Pascal Gallard and Stephen L. Scott Aug, 26th Outline Context Cluster BS/SSI Virtualization Combining