GRIDSEED: A Virtual Training Grid Infrastructure Iztok Gregori CNR-IOM DEMOCRITOS Trieste, Italy iztok@democritos.it Stefano Cozzini CNR-IOM DEMOCRITOS Trieste, Italy cozzini@democritos.it Tyanko Aleksiev ICTP, elab/sissa Trieste, Italy aleksiev@sissa.it Abstract GRIDSEED provides a simple tool to setup a portable and interoperable Grid infrastructure based on virtual machines. The GRIDSEED tool was originally developed to easily deploy a training glite Grid infrastructure almost everywhere in the world with a set of machines (simple PC s) locally connected among them as the only requirement. It uses standard virtualization tools (VirtualBox and/or VMmare) easily and widely available. Recently the virtual environment is enriched with other different middleware (Arc and Globus Toolkit ) to make it the first virtual training laboratory for interoperability and interoperation among different middleware. GRIDSEED is therefore a complete training environment formed by a virtual infrastructure complemented by some demo applications and training materials ready to be used both in standard training events and advanced interoperation demo/events session. In this paper we present the motivations behind this tool, the way it could be installed and used for training events and some technical details. Keywords Interoperability, Grid Middleware, Virtualization, e-infrastructure. 1 Introduction Small training grid infrastructures play a fundamental role in promoting and wide-spreading Grid computing technologies and Grid infrastructures. Such tool allows beginners to play and experience how to use a Grid without the burden to be officially enrolled in a Grid production environment. In case of glite middleware [1] used in EGI and related projects a dedicated training infrastructure named GILDA [8] has been setup since a few years. Beside this permanent infrastructure several efforts have been done to setup temporary Grid infrastructures to be used during training and dissemination events. There are several methods to setup such temporary Grid infrastructures. Right now almost all proposed solutions are based on a virtualization approach. The availability of free virtual software like xen [2] and VMware server and player [3] has indeed recently boosted the use of virtualization for temporary infrastructures dedicated to training (see for instance [4]). The basic idea behind this approach is to have many different virtual machines each serving a specific Grid service. These virtual machines can be hosted in one or more physical servers and then appropriately configured in order to start all the needed Grid services. GRIDSEED is just a step further in this direction: based on virtual machines this tool was originally developed to hide the complex task to configure, by hand, all the services available on a glite-base Grid infrastructure. In GRIDSEED a set of pre-configured virtual machines can be installed on a set of physical machines connected in a LAN and once booted, they offer all the Grid services without any further configuration. Having shown robust enough in many different occasions and events this tools was then enlarged and complemented with grid services based on different grid middleware and becomes also a virtual lab for interoperability experiments. The GRIDSEED training infrastructure can be turned on with minimal effort and easily used and exploited to provide a full training environment where state of the art interoperability approached can be evaluated and tested. In the remainder of this paper, which is an updated and enhanced version of the lecture notes on GRIDSEED we presented at ICTP [5], we will discuss in details our tool and its evolution. In the next section we describe the context where GRID- SEED was deployed discussing the motivation and the evolution of the project. Section 3 introduces the general architecture and some details: we also discuss a few technical choices done during the development. Section 4 discusses the full GRIDSEED environment and the way it can be configured and used for both standard training events and as interoperable virtual laboratory. We also discuss advantages and limitations with respect to other experiences/tools. Finally in section 5 we set forth our future plans and draw some conclusions. 61
2 Context The research group behind this project is based in Trieste and involves EU-IndiaGRID [7] team at ICTP and a team at the joint SISSA/Democritos laboratory for e-science [13] (in short elab) which is actually maintaining the package. The original motivation behind GRIDSEED was to have a fully fledged training Grid infrastructure ready to be setup anywhere in the world, even if the internet connection is not available. This approach is actually followed right now in many training events at European level as well. The main objectives that GRIDSEED originally met were: Very small requirements GRIDSEED should not require sophisticated hardware and/or advanced software setup in a generic computer lab. Requirements should be kept to the minimum to avoid any difficulty in providing what is needed by local organizers. Ease of use GRIDSEED should make the setup and the management of the training infrastructure as simple as possible. The complete infrastructure should be setup and then managed by people that could be completely unaware about the complexity to configure global and local Grid services. Stability and reliability The GRIDSEED infrastructure should be stable and reliable: we are marginally interested here in performance. The system must be responsive under some demo loads to be useful but it is tolerable that under heavy real load some slow downs can occur in favour of stability. Expandable and portable GRIDSEED should be easy to expand in terms of new services, new Grid-sites, new kinds of middleware. Advanced users should easily be able to expand it depending on their needs. GRIDSEED was initially developed as a natural evolution of the EGRID Live CD [9] a pioneering solution in making glite middleware installation and configuration simple. The kick-off of GRIDSEED activities took place during the EU-IndiaGrid tutorial organized in June 2007 in Kolkata (India): in that occasion a fully fledge computational EGEE/gLite Grid was setup based on VMware virtual machines and this can be considered as an alpha version of the product. In all the subsequent training activities of the EU-India project enhanced versions of GRIDSEED were used and tested. From May 2008 the initiative was sponsored and supported by the elab team which became more and more actively involved in the development. Since then many different training event used successfully the package. Starting from release Release 1.4 (january 2010) we added ARC middleware support on user interface and new ARC middleware VMs. This first interoperable effort was consolidated in version 1.5 who was actually successfully presented in a demo at the Uppsala EGEE User forum [6]. Latest version (1.6) was released in December 2010 with Globus Toolkit services enabled as well. This version will be presented and discussed in the rest of the paper. 3 GRIDSEED architectures and technical implementation GRIDSEED consists of a set of VirtualBox virtual machines (VMs) each of them hosting one or more Grid services, working in a coordinated way within a dedicated local network. We decided to use VirtualBox virtualization tools due to the free availability of the software on both Linux and Windows architectures. This choice helped us to keep requirements to the minimum: for instance, a Windows-based laboratory can be easily used by installing VirtualBox software. VirtualBox is not difficult to install and configure and its graphical interface allows an easily control of the VMs. We are distributing the VMs in Open Virtualization Format (ovf) which allows to use different virtualization softwares (like VMware) to run GRIDSEED. Thus users of GRIDSEED can leverage industrial strength virtualization products, for free. Prerequisite for GRIDSEED, is therefore the presence of VirtualBox and/or VMware software in the target hosts and a LAN connecting them. 3.1 Architecture We can distinguish three kinds of VMs based on their logical roles within the architecture, as follow. 3.1.1 Central Services The core services of the GRIDSEED environment are: the glite central services (VOMS, BDII, WMS...), the Certification Authority, the monitoring service and the network services which GRIDSEED relies on (DNS, DHCP...). This environment is currently distributed over four VMs: master The core of the architecture is based on the master VM, which provides a set of services to keep all the VMs coordinated and easily manageable. We therefore installed on it the following services and tools, commonly used to manage clusters of workstations: 62
user in the two Virtual Organizations pre-configured on the infrastructure: GRIDSEED and elab. central-1 The Central VM is configured to host both information services (Top-bdii) and the logical file catalog services (LCG File Catalog LFC). Figure 1. GRIDSEED architecture: see text for discussion. c3 tools, for remote management of all the VMs. DNS, completely configured up to the maximum level of expandability. DHCP server, working on non standard tcp port, used to configure the Worker Nodes and the User Interfaces. Iptables service, to provide outbound connectivity from VMs to external network. NTP service, to keep VMs synchronized. Unfortunately, Virtual Machines have problems to keep time syncronization accuracy within the second, which is critical for Grids in general, and especially for the security component of glite called GSI. In some cases the installation of VirtualBox Guest Additions fixes the problem but for safety an ad hoc simple shell script, that keeps the machine synchronized, could be activated. The master VM runs also two glite core services: VOMS server MYproxy server The above services are complemented by a fake Certification Authority. It consists of an Apache web server and a set of CGIs that issues all the X509 certificates required by the Grid infrastructure. Any user can generate his/her personal certificate and/or a host certificate in order to add, for instance, a new VM host to the system. Indeed, a user is not actually requested to do these steps manually: a script (ask cert.sh) available on the standard UI asks for user certificate and registers it automatically on the VOMS Server to enroll each WMS The WMS VM hosted both glite Resource Broker and Logging and Bookkeeping services. Those two services are resident on a dedicated machine due to high load that can be reach during job submissions. Also WMS is patched to allow MPI support, through the match making procedure, to CREAM Computing Elements. nagios The monitoring Nagios service is running on this machine. The functionality of Grid sites is tested by the execution of Site Availability Monitor (SAM) tests that launches periodic test at various Grid Service instances (CE, SE, etc) to check their status. 3.1.2 Site Services These are VMs needed to operate a Grid site. The base GRIDSEED site is composed by a Storage Element, a Computing Element and a Worker Node VMs. The Worker Node is associated to a site and automatically configured once powered on, so if you want to increase their number, you have to just power on another virtual machine. Such configuration can be expanded to include up to 16 WNs. From version 1.6 of GRIDSEED there are two kind of Grid sites, both of them are enabled to support MPI jobs: GRIDSEED-1 This Grid Site is based on glite 3.2 64 bit middleware and includes a CREAM Computing Element (with site BDII and TORQUE batch system), a DPM Storage Element and up to 16 Worker Nodes. GRIDSEED-2 Differently from the previous site GRIDSEED-2 is focusing on interoperability. The core of the Grid Site is a glite 3.1 Computing Element, which shares the TORQUE queue system and some other resources with Globus Toolkit 4.0.8 and ARC 0.8.1 Virtual Machines. The number of Grid Sites can be easily increased up to a maximum of 20. 63
3.1.3 User Interfaces The latest version of Milu, the Miramare Lightweight User Interface [10] is serving here a standard User Interface for GRIDSEED. Both 32 and 64 bit versions of the virtual machine are available. Such UIs have a set of predefined accounts with predefined user certificates. The current version of GRIDSEED operates for glite version 3.1/3.2, Nordeugrid ARC 0.8.1 and Globus Toolkit 4.0.8 and is kept constantly updated. 3.2 Deployment All the VMs can be booted from the same physical host, or alternatively they may be booted from different ones: it depends on the hardware available. Clearly, the more VMs are booted from the same physical host, the bigger the hardware requirements are for that host. We tried to keep to the minimum memory and size requirements for all the VM machines. Machine Compressed Size RAM master.grid.seed 1.7 GB 256 MB central-1.grid.seed 1.4 GB 256 MB wms.grid.seed 1.8 GB 512 MB milu-64.grid.seed 957 MB 256 MB milu-32.grid.seed 849 MB 256 MB ce-1.grid.seed 630 MB 256 MB ce-1wnx 987 MB 256 MB se-1.grid.seed 473 MB 256 MB ce-2.grid.seed 1.3 GB 256 MB ce-2wnx 1.7 GB 256 MB se-2.grid.seed 1.3 GB 256 MB arc-ce.grid.seed 1.1 GB 256 MB globus.grid.seed 1.4 GB 256 MB Table 1. List of GRIDSEED VMs The compressed size for each machine is in order of 1 OR 2 GB and the minimal GRIDSEED set of machine compressed is of about 8 GB. The relative small size makes it very handy to download it on a couple of usb pen and carry it to remote training locations where there is either no internet connection or very slow internet. Once all the machines are decompressed a few tens of gigabyte are needed to host all of them. Memory requirements are quite easily satisfied with recent hardware as well. The basic GRIDSEED setup can be managed with 8GB of RAM: this is the minimum requirement in terms of RAM. Of course, this amount could be provided by more than one machine. To run 64 bit Virtual Machines you must have modern CPU with enabled hardware virtualization feature. A fairly comfortable setup, able to deal with 10/15 users at the same time requires a total of 32GB of RAM, easily available in any medium size modern computer lab. At the moment we are currently keeping the working version of GRIDSEED on 5 servers, each of them with 8GB each: this allows us to keep up and running, beside all the central services and 1 UI, 3 different Grid-sites in a comfortable way. As mentioned GRIDSEED is fairly flexible. It is designed to create and configure a Grid infrastructure made up of different middleware, up to 10 UI and up to 20 sites, each one consisting of 1 CE + 1 SE + up to 16 WN. This represents a formidable Grid to play with. So it serves well the needs of training towards a large number of users: this however requires adequate hardware to run on. The easiness of copying and rebuilding VMs makes it feasible also for advanced training dedicated to system administrators as well. Once the VM are all booted only a few network configuration steps (described in detail in the wiki project [12]) are needed in order to allow users reach, via ssh, the UI(s). This is because GRIDSEED VMs are on an isolated network and the only open gateway should be the one to reach the User Interfaces available. On Milu UI 20 predefined accounts are also available and ready to be used by users with a few scripts to facilitate the creation of the jobs and testing of the infrastructure. 3.3 Availability The GRIDSEED software project is now hosted on the gforge portal for scientific software development hosted at elab [11] and we welcome people from the Grid community who want to contribute to the project. On the gforge portal all configuration scripts and tools to deploy and develop the product will be made available. It is also possible to require other features and/or Grid services not yet developed. 4 GRIDSEED environment GRIDSEED virtual training infrastructure is associated with a set of tutorials on Grid computing. These tutorials are available online on the GRIDSEED website and represent the latest and updated effort to complement the infrastructure with materials ready to be used in training events and/or for self-training. The web site is actually hosted in a wiki: the materials can easily be improved, updated and completed by anybody interested in collaborating with the project. At the moment the GRIDSEED wiki reports some basic and advanced tutorials dedicated to the users; these are standard tutorials prepared following the glite User Guide. These tutorials can be used jointly with the GRIDSEED 64
infrastructure and offer some advantages with respect to a standard and generic guide like, for instance: Output of the command obtained from command line is actually the same reported on the wiki: this does not confuse beginners. There is a clear correspondence among what is written on the wiki and what is obtained on the system. Cut and paste of the command is almost always working: in case of quite complex commands this allows trainees to focus on the command itself and not on the typesetting of the command. There is no need to change important parameters in the commands like, for instance, the Virtual Organization; again this allows trainers (and tutors as well) to focus on the meaning of the commands and the proposed exercises without worrying about many details that at first iteration can cause some confusion. All the software needed is there and no specific configurations are required. The GRIDSEED environment is then enriched by some case studies of successfully ported scientific applications performed within the Grid projects our team is involved with. Thus, besides the standard tutorials there are also ad hoc tutorials intended to illustrate such porting activities. These tutorials are devoted to users eager to learn and understand tricks and tools developed to port scientific applications on the EGEE/gLite Grid infrastructure. We consider this an important added value because real porting experience can be analyzed in detail with all the software needed installed and under direct control of the trainees. We tried to keep our case studies to be of general interest as much as possible for larger communities. This GRIDSEED set of tutorials is also an important tool for self training. It has to be noted that people trained once on GRIDSEED are able to replicate their training infrastructure easily everywhere and are then ready to offer it to beginners. In this context GRIDSEED seems to be an efficient tool to allow trainees to become future trainers. To enhance and simplify the middleware interoperability the standard installation of Milu User Interface has native commands to interact with different middlewares and on milu-64 there is also Gridway metascheduler software that permits you to submit job directly to different middleware using a common language and commands. There are still of course some limitations: for absolute beginners installing and using GRIDSEED from scratch can be difficult, and in case of a large number of users and applications the virtual approach overhead can be significant. GRIDSEED environment has been successfully used several times in different contexts and for different purposes (see for instance [14] and the web site of the project). We note that in training events focused on specific user communities, temporary infrastructure like GRIDSEED can easily be adapted and customized in order to provide specific tools and services requested by the user community. 5 Conclusions We presented GRIDSEED, a tool to setup a temporary training infrastructure using virtual technologies. Such system makes it easy to setup a training testbed to start experiencing with Grid infrastructure based on glite middleware. GRIDSEED users and even trainers can avoid the burden of installing Grid middleware and properly configure it and can therfore fully dedicate to learn how to use the Grid and not how to install the middleware. GRIDSEED provides also its own Certification Authority so no administrative requests have to be issued to start playing within a Grid environment. Our tools are under development and there is room for further improvements; we are currently considering the following developments : Merge central-1 and master in only one VM. Tools for creating VMs from template. Automatic nagios configuration Scripts for cloning VMs GRIDSEED was recently used as a virtual laboratory for interoperability: we added some Arc and Globus services and we were able to perfectly mimic some interoperation case studies among production grid infrastructure like EGI and GARUDA. In future releases we plan to enhance and expand the interoperability features by adding new services and new middleware as well. In particular the challenge ahead of us is to fully integrate the recently release EMI-1 distribution which includes three different european middleware (glite/ Arc and Unicore). References [1] See http://glite.cern.ch/ [2] P. Barham, B. Dragovic, K. Fraser, S. Hand, T. Harris, A. Ho, R. Neugebauer, I. Pratt and A. Warfield, (2003). Xen and the Art of Virtualization. In Proceedings of the Nineteenth ACM Symposium on Operating Systems Principles. ACM. Energy and Nuclear Physics, March, 2003 [3] S. Devine, E. Bugnion and M. Rosenblum, (1998). Virtualization system including a virtual machine monitor for a comwith a segmented architecture. US Patent 65
[4] R. Berlich and M. Hardt, (2005). Grid in a box - virtualisation techniques in Grid training. Presented at EGEE conference, Athens. Available via: http://www.ep1.rub.de/ruediger/pandoraathens.pdf [5] ICTP Lecture Notes Series, Volume 24 (ISBN 92-95003-42-X) [6] See http://gridseed.esciencelab.org/uploads/main/gridseed-poster.pdf [7] See www.euindiagrid.eu [8] GILDA (Grid INFN Laboratory for Dissemination Activities) see https://gilda.ct.infn.it/ [9] See http://www.egrid.it/sw/livecd [10] See http://doc.escience-lab.org/index.php/main/milu [11] See http://eforge.escience-lab.org [12] See http://gridseed.escience-lab.org [13] See www.escience-lab.org [14] B.R. Amarnath, T. Selvi Somasundaram, M. Kashif, A. Masoni and S. Cozzini EU-IndiaGrid interoperability experiments using GridSeed tool Multiagent and Grid Systems An International Journal 6 (2010) 19 66