Is Virtualization Killing SSI Research? Jérôme Gallard, Geoffroy Vallée, Adrien Lèbre, Christine Morin, Pascal Gallard and Stephen L. Scott Aug, 26th
Outline Context Cluster BS/SSI Virtualization Combining Virtualization and SSI Conclusion Lessons learnt Perspectives August 2008 2
Introduction Cluster exploitation: How to execute jobs on a cluster architecture? - Batch scheduler (BS) - Single System Image (SSI) August 2008 3
Batch Scheduler (BS) August 2008 464
Batch Scheduler (BS) N1 N2 N3 N0 Wait queue of jobs N4 N5 N6 head node compute nodes Users interact only with the head node August 2008 56 4
Single System Image (SSI) August 2008 65
Single System Image (SSI) compute nodes N1 N2 N3 no head node N4 N5 N6 Users have the illusion to manage a SMP machine with UNIX standard tools August 2008 76 5
Single System Image (SSI) Interesting functionality: - Global management of ressources (such as the memory) August 2008 86
Virtualization Technologies August 2008 97
Virtualization Technologies Virtualized machines running on the compute node VM0 VM1 VM2 N0 compute node August 2008 107
Virtualization with BS N1 N2 N3 N0 Wait queue of VMs (JAWS, Haizea) N4 N5 N6 head node (BS) compute nodes August 2008 118
Combining BS + VM Simplify: - Management and deployement of applications - Management of resources August 2008 129
VMs vs SSI Is virtualization killing system image research? What's about combining both approaches? August 2008 13 10
VMs vs SSI bottom TypeI top TypeI TypeII Container SSI X X X O TypeII X X X O Container X X X O SSI O O O X August 2008 14
System Level Virtualization Goldberg: TypeII-virtualization August 2008 15 11
System Level Virtualization Goldberg: TypeII-virtualization App1 User space Guest OS Guest OS VM VM... Hypervisor Host OS Hardware Example of typeii: Vmware, QEMU Kernel space August 2008 16 11
TypeII Virtualization upon SSI N1 N2 N3 August 2008 17 12
TypeII Virtualization upon SSI Host OS: SSI N1 N2 N3 August 2008 18 12
TypeII Virtualization upon SSI Hypervisor Host OS: SSI N1 N2 N3 August 2008 19 12
TypeII Virtualization upon SSI VM1 VM2 Hypervisor Host OS: SSI N1 N2 N3 August 2008 20 12
TypeII Virtualization upon SSI App1 App2 Guest OS Guest OS VM1 VM2 Hypervisor Host OS: SSI N1 N2 N3 August 2008 212
TypeII Virtualization upon SSI Advantages: VM could use virtual resources composed with several physical nodes (for instance, memory) App1 Guest OS VM1 App2 Guest OS VM2 Hypervisor Host OS: SSI N1 N2 N3 August 2008 22 12
SSI upon TypeII Virtualization August 2008 23 13
SSI upon TypeII Virtualization Advantages: Possibility to have Virtual Cluster and to migrate it from nodes to others App1 App2 App3 SSI VM1 VM2 VM3 VM4 Hyp1 Hyp2 Hyp3 Host OS1 Host OS2 host OS3 N1 N2 N3 August 2008 24 13
SSI upon TypeII Virtualization Advantages: Possibility to have Virtual Cluster and to migrate it from nodes to others App1 App2 App3 SSI VM1 VM2 VM3 VM4 Hyp1 Hyp2 Hyp3 Host OS1 Host OS2 host OS3 N1 N2 N3 August 2008 25 13
SSI upon TypeII Virtualization Advantages: Possibility to have Virtual Cluster and to migrate it from nodes to others App1 App2 App3 SSI VM1 VM2 VM3 VM4 Hyp1 Hyp2 Hyp3 Host OS1 Host OS2 host OS3 N1 N2 N3 N4 August 2008 26 13
SSI upon TypeII Virtualization Advantages: Possibility to have Virtual Cluster and to migrate it from nodes to others App1 App2 App3 SSI VM1 VM2 VM3 VM4 Hyp1 Hyp2 Hyp3 Hyp4 Host OS1 Host OS2 host OS3 host OS4 N1 N2 N3 N4 August 2008 27 13
SSI upon TypeII Virtualization Advantages: Possibility to have Virtual Cluster and to migrate it from nodes to others App1 App2 App3 SSI VM1 VM2 VM3 VM4 Hyp1 Hyp2 Hyp3 Hyp4 Host OS1 Host OS2 host OS3 host OS4 N1 N2 N3 N4 August 2008 28 13
SSI upon TypeII Virtualization Advantages: Possibility to have Virtual Cluster and to migrate it from nodes to others App1 App2 App3 SSI VM1 VM2 VM3 VM4 VM4 Hyp1 Hyp2 Hyp3 Hyp4 Host OS1 Host OS2 host OS3 host OS4 N1 N2 N3 N4 August 2008 29 13
SSI upon TypeII Virtualization Advantages: Possibility to have Virtual Cluster and to migrate it from nodes to others App1 App2 App3 SSI VM1 VM2 VM3 VM4 VM4 Hyp1 Hyp2 Hyp3 Hyp4 Host OS1 Host OS2 host OS3 host OS4 N1 N2 N3 N4 August 2008 30 13
System Level Virtualization Goldberg: TypeI-virtualization August 2008 314
System Level Virtualization Goldberg: TypeI-virtualization Kernel Space App1 User Space Guest OS Guest OS Host OS (Dom0) VM VM... Hypervisor Hardware Example of typei-virtualization: XEN August 2008 32 14
TypeI Virtualization upon SSI App1 App2 App3 Guest OS Guest OS Host OS (Dom0) VM1 VM2 Hypervisor + part of SSI Example: ScaleMP is like a distributed superbios N1 N2 N3 August 2008 33 15
TypeI Virtualization upon SSI App1 App2 App3 Guest OS Guest OS Host OS (Dom0) VM1 VM2 Characteristics: Distributed hypervisor Hypervisor + part of SSI Example: ScaleMP is like a distributed superbios N1 N2 N3 August 2008 34 15
SSI upon TypeI Virtualization App1 App2 App3 Same as SSI upon TypeII-Virtualization SSI Host OS (Dom0) VM1 VM2 VM3 Host OS (Dom0) Hypervisor Hypervisor N1 N2 August 2008 35 16
Container Based Virtualization NOT in Goldberg classification, but more and more used nowadays August 2008 36 17
Container Based Virtualization NOT in Goldberg classification, but more and more used nowadays User Space cntr1 cntr2 Kernel Space Host Host OS OS + + Hypervisor Hypervisor Hardware Example of containers: OpenVZ, chroot August 2008 37 17
Container on the top of SSI App1 App2 App3 App4 Ctnr1 Ctnr2 Ctnr3 Container + SSI N1 N2 N3 N4 August 2008 38 18
Container on the top of SSI App1 App2 App3 App4 Ctnr1 Ctnr2 Ctnr3 Distributed Host OS including containers Container + SSI N1 N2 N3 N4 August 2008 39 18
Container on the top of SSI Advantages: To have containers taken advantage of distributed SMP App1 App2 App3 App4 Ctnr1 Ctnr2 Ctnr3 Distributed Host OS including containers Container + SSI N1 N2 N3 N4 August 2008 40 18
SSI on the top of Container Architecture irrelevant: The same kernel is shared between all containers (they have not their own kernel) SSI Ctnr1 Ctnr2 Ctnr3 Ctnr4 Ctnr5 OS0 OS0 OS0 OS0 N1 N2 N3 N4 August 2008 419
SSI on the top of Container Architecture irrelevant: The same kernel is shared between all containers (they have not their own kernel) SSI Ctnr1 Ctnr2 Ctnr3 Ctnr4 Ctnr5 OS0 OS0 OS0 OS0 N1 N2 N3 N4 August 2008 42 19
Lessons Learnt Containers on top of single system image clusters The SSI system combines all the advantages enabled by containers on a real SMP machine in a cluster environment August 2008 43 20
Lessons Learnt Virtual Machines on top of SSI clusters Same advantage as containers Virtualization solves application portability issues example : Possibility to run an application developed for process technology A and OS B on top of computers running SSI OS based on OS C and developped for achitecture processor D. August 2008 44 21
Lessons Learnt SSI on top of VMs VM migration and suspend functionalities provide: Flexible, simple and ''On Demand'' resource allocation mechanisms for the applications Transparent adaptation mechanisms in case of hardware changes (for instance, node additions / removals). August 2008 45 22
Lessons Learnt Virtualization and SSI complement each other SSI abstracts resources Virtualization adds another level of hardware flexibility August 2008 46 23
Combining Virtualization and SSI Isolation of Two Distinct SSIs App1 App2 App3 Possibility to run several isolated SSIs on the same cluster. SSI 1 SSI 2 VM1 VM2 VM3 VM4 VM5 Hyp1 Hyp2 Hyp3 OS1 OS2 OS3 N1 N2 N3 August 2008 47 24
Combining Virtualization and SSI Three layers of flexibilities Possibility to run a windows application on a linux SSI cluster running upon MACOS X systems App1 VM1 VM2 SSI VM1 VM2 N1 N2 August 2008 48 25
Conclusion Combining VM/SSI setup: - Very attractive - Installation and use are quite simple (evaluated with TypeII) Developpement of a framework allowing to - Deploy VM on G5K - Deploy SSI (Kerrighed) on physical/virtual nodes Facility of deploying experiments August 2008 49 26
Future Work Performance evaluation Extend the Goldberg's classification to present in a uniform way: The hardware, The emulated hardware, The OS and SSI, The different kinds of virtualization techniques, The containers. August 2008 50 27
Future Work Investigate SSI scheduler and hypervisor scheduler: does the SSI scheduler directly manage VMs? Extend to the Grid: interests of such hybrid approaches (VM/SSI) in XtreemOS August 2008 51 28
Thank You Questions? August 2008 529