Virtual Machine Lifecycle Management The example of CernVM
INTRODUCTION - What is a virtual machine lifecycle - Current Implementation - Ideal Solution
INTRODUCTION What is a Virtual Machine Lifecycle Collect Software Build Appliance Define Configuration Test Builds Deploy & Manage Publish
INTRODUCTION Current Implementation Collect Software Build Appliance Define Configuration Test Builds Manual configuration Deploy & Manage Publish
INTRODUCTION Current Implementation Collect Software Build Appliance Manual collection of software Define Configuration Test Builds Manual configuration Deploy & Manage Publish
INTRODUCTION Current Implementation Collect Software Build Appliance Manual collection of software Using a custom tool: ibuilder Define Configuration Test Builds Manual configuration Deploy & Manage Publish
INTRODUCTION Current Implementation Collect Software Build Appliance Manual collection of software Using a custom tool: ibuilder Define Configuration Manual configuration Test Builds Manually testing the images on every hypervisor Deploy & Manage Publish
INTRODUCTION Current Implementation Collect Software Build Appliance Manual collection of software Using a custom tool: ibuilder Define Configuration Manual configuration Test Builds Manually testing the images on every hypervisor Deploy & Manage Publish Manual publish
INTRODUCTION Current Implementation Collect Software Build Appliance Manual collection of software Using a custom tool: ibuilder Define Configuration Manual configuration Test Builds Manually testing the images on every hypervisor Deploy & Manage Publish VMWare vsphere & Amazon Cloud Manual publish
INTRODUCTION Problems As the project is getting bigger, it s not possible to do all those steps manually Each phase of the lifecycle requires a different software There is no unified solution Commercial licenses for the software involved on every step make the whole process very expensive
INTRODUCTION Commercial Software Building: rbuilder Not supporting SLC any more Managing: VMWare vsphere Ideal but too expensive Again, no unified solution
INTRODUCTION Ideal solution Each component is independent They connect with each other seamlessly into a single solution They can all be managed from a single, userfriendly interface They can communicate with each other to perform automated jobs Components can be added on demand It s open source
ARCHIPEL Closest open-source starting point
ARCHIPEL Features Virtual Machine and Hypervisor manager written in python and Objective-J Uses a distributed agent/client architecture Provides a powerful browser-based front-end Requires no server-side scripting Modular design Can be extended It uses open-source components It is based on the XMPP (Jabber) protocol Widely used, and tested (Google Chat, ichat, Facebook Chat) Can be clustered
ARCHIPEL Architecture Agents Agent Hypervisor A Agent Hypervisor B Virtual Machine 1 @ Hypervisor A Virtual Machine 2 @ Hypervisor B Virtual Machine 3 @ Hypervisor B XMPP XMPP XMPP Server ejabberd BOSH XMPP Chat Client Chat Client Archipel GUI Client
ARCHIPEL How can Archipel be extended Agents and clients can easily be created since they are using a wide-spread protocol with many implementations (XMPP) Custom front-ends can be added on the existing front-end, due to it's modular design Since all of the components are using the same transport, they can also talk with each other.
ARCHIPEL Problems Poor documentation regarding the module development (Official documentation It's under construction) Heavy front-end Cannot run on portable devices
PROGRESS Development and current status
PROGRESS Our additions Agent Agent Agents Hypervisor A Hypervisor B Agent Virtual Machine 1 @ Hypervisor A XMPP Repository Virtual Machine 2 @ Hypervisor B XMPP XMPP Server ejabberd XMPP Image Builder Virtual Machine 3 @ Hypervisor B BOSH Image Tester Archipel GUI New Agents Client
PROGRESS Our additions Agent Agent Agents Hypervisor A Hypervisor B Agent Virtual Machine 1 @ Hypervisor A XMPP Repository Virtual Machine 2 @ Hypervisor B XMPP XMPP Server ejabberd XMPP Image Builder Virtual Machine 3 @ Hypervisor B BOSH Image Tester Archipel GUI Client PicoClient GUI Client New client for portable devices
PROGRESS Our additions Agent Agent Agents Hypervisor A Hypervisor B Agent Virtual Machine 1 @ Hypervisor A XMPP Repository Virtual Machine 2 @ Hypervisor B XMPP XMPP Server ejabberd XMPP Image Builder Virtual Machine 3 @ Hypervisor B BOSH Image Tester New frontend modules for the new agents Archipel GUI Client PicoClient GUI Client
PROGRESS Our additions Agents Agent Hypervisor A Agent Hypervisor B Enable Inter-Agent communication Agent Virtual Machine 1 @ Hypervisor A XMPP Repository Virtual Machine 2 @ Hypervisor B XMPP XMPP Server ejabberd XMPP Image Builder Virtual Machine 3 @ Hypervisor B BOSH Image Tester Archipel GUI PicoClient GUI Client Client
PROGRESS Our additions New agents that cover the first stages of the VM Lifecycle Software Collection: Repository agent Image Building: Building agent (ibuilder) Image Testing: Testing agent (Taper) New front-end for portable devices New front-end modules for Archipel GUI Enable Inter-Agent communication
PROGRESS What is done so far We have submitted a patch to the Archipel team that allows us to create custom modules on the front-end. The patch was accepted and it's on the current release. We have created a minimal perl agent that can wrap existing components into Archipel-Compatible agents. Despite the poor documentation, we managed to create some front-end modules, and a skeleton that can be used as a starting point to create other modules. We have created a modular, lightweight client that can run on portable devices.
PROGRESS Perl Agent (iagent) Minimal starting point for new Archipel- Compatible Agents Modular design Based on PoE Single-threaded Multitasking New messaging additions on PoE Broadcasting with module hierectary LDAP Authentication
PROGRESS Message Broadcast Perl Agent (iagent) XMPP Module XMPP XMPP Server ejabberd Authentication Module User module(s) iagent
PROGRESS ibuilder Module Agent Uses iagent Command-line interface to ibuilder Connection to ibuilder database Client ArchipelClient front-end to define probjects, prepare and initiate builds
PROGRESS ibuilder Module
PROGRESS ibuilder Module
PROGRESS ibuilder Module
PROGRESS PicoClient Lightweight browser-based client for portable devices UI based on Sencha Touch framework Modular system similar to Archipel Client HTML5 offline-capable Compatible with the existing archipel agents Provides native application look-and-feel for iphone, ipad, Android and BlackBerry
PROGRESS PicoClient
PROGRESS PicoClient
DEMONSTRATION Archipel Client: http://cernvm16.cern.ch/project/admin/archipel Pico Client: http://cernvm16.cern.ch/project/admin/picoclient