III Level Course, 2011 Free Software. Dott. Bertoldo Silvano Ing. Terzo Olivier

Transcription

1 III Level Course, 2011 Free Software Dott. Bertoldo Silvano Ing. Terzo Olivier 1

2 1. Introduction to Grid and Cloud Computing 2. Open Source Software in Grid and Cloud Computing 2.1 Hypervisor 2.2 Cloud Platform 2.3 Grid Platform 2.4 Open Source OS 2.5 Scheduler 3. Some VIP opinions in Open Source Grid and Cloud Computing 2

3 60 % of Informative System has planned cloud strategic projects for the next 5 years (source IBM) In 5 years time, specific cloud market will have sales proceeds for more than 12 billions of dollars only in US. There will be 196 millions of users with their own cloud. (source Forrester) 3

4 GRID COMPUTING Grid computing is a term referring to the combination of computer resources from multiple administrative domains to reach a common goal. The grid can be thought of as a distributed system with non-interactive workloads that involve a large number of files. What distinguishes grid computing from conventional high performance computing systems such as cluster computing is that grids tend to be more loosely coupled, heterogeneous, and geographically dispersed. 4

5 Although a grid can be dedicated to a specialized application, it is more common that a single grid will be used for a variety of different purposes. Grids are often constructed with the aid of general purpose grid software libraries known as middleware. 5

6 CLOUD COMPUTING Cloud computing refers to the use and access of multiple server-based computational resources via a digital network (WAN, Internet connection using the World Wide Web, etc.). Cloud users may access the server resources using a computer, netbook, pad computer, smart phone, or other device. In cloud computing, applications are provided and managed by the cloud server and data is also stored remotely in the cloud configuration. 6

7 Users do not download and install applications on their own device or computer; all processing and storage is maintained by the cloud server. The on-line services may be offered from a cloud provider or by a private organization. 7

8 The term is often used to describe the interface provided by the specific cloud computing functionality infrastructure as a service (IaaS). The term "hypervisor" was first used in 1965, referring to software that accompanied an IBM RPQ for the IBM 360/65. It allowed the model IBM 360/65 to share its memory: half acting like a IBM 360; half as an emulated IBM The software, labeled "hypervisor," did the switching between the 2 modes on split time basis. The term hypervisor was coined as an evolution of the term "supervisor, the software that provided control on earlier hardware. 8

9 On open source context the two major hypervisors are Xen and Kvm and consist in two aspects the full and paravirtualization. In computing, a hypervisor, also called virtual machine manager (VMM), is one of many hardware virtualization techniques that allow multiple operating systems, termed guests, to run concurrently on a host computer. The hypervisor presents to the guest operating systems a virtual operating platform and manages the execution of the guest operating systems. Multiple instances of a variety of operating systems may share the virtualized hardware resources. Hypervisors are installed on server hardware whose only task is to run guest operating systems. Non-hypervisor virtualization systems are used for similar tasks on dedicated server hardware, but also commonly on desktop, portable and even handheld computers. 9

10 KVM (for Kernel-based Virtual Machine) is an open source hypervisor software KVM's parts are licensed under various GNU licenses: KVM kernel module: GPL v2 KVM user module: LGPL v2 QEMU virtual CPU core library (libqemu.a) and QEMU PC system emulator: LGPL Linux user mode QEMU emulator: GPL BIOS files (bios.bin, vgabios.bin and vgabios-cirrus.bin): LGPL v2 or later 10

11 KVM It is a full virtualization solution for Linux on x86 hardware containing virtualization extensions (Intel VT or AMDV). It consists of a loadable kernel module, kvm.ko, that provides the core virtualization infrastructure and a processor specific module. Can run multiple virtual machines running unmodified Linux or Windows images. Each virtual machine has private virtualized hardware: a network card, disk, graphics adapter. 11

12 Xen open source virtualization software that is used to partition workstations and servers into separate virtual machines, each containing its own copy of an OS. Pronounced "zen," and developed at the University of Cambridge in the U.K., Xen is noted for its fast response and low overhead. The Xen community develops and maintains Xen as free software, licensed under the GNU General Public License (GPLv2). 12

13 Xen uses one or more privileged "guest" operating systems for handling the actual device drivers for the hardware. Known as "paravirtualization" and unlike VM environments where the OS runs as is, the OS that runs on top of Xen must be programmed to call Xen virtual drivers, which, in turn, call the real drivers. Linux and versions of Unix were the first operating systems ported to Xen. No Porting for Hardware Virtualization There is no requirement to port the operating system to Xen if the hardware platform offers support for virtualization, such as Intel's VT, AMD's AMD-V and IBM's POWER5 architecture. visit. 13

14 OpenNebula is the Industry standard for on-premise IaaS cloud computing, offering a comprehensive solution for the management of virtualized data centers to enable private, public and hybrid (cloudbursting) clouds. OpenNebula was first established as a research project back in 2005 by Ignacio M. Llorente and Rubén S. Montero. Since its first public release of software in March 2008, it has evolved through open-source releases and now operates as an open source project. OpenNebula is fully open-source software distributed and licensed for use under the terms of the Apache License, Version 2.0. Copyright , OpenNebula Project (OpenNebula.org). The OpenNebula open source project is being managed by C12G Labs. 14

15 OpenNebula is: Open: fully open source (not open core) software Adaptable: customizable to fit inyo your own data center Proven: many massive scale production development Powerful: advanced enterprise class functionality Inteoperable: most popular cloud APIs and standard-based No lock-in: platform independent on major hypervisors 15

16 Eucalyptus was born as a University project in the MAYHEM labs of the Computer Science Dept. at UC Santa Barbara. The MAYHEM team's experience in Grid Computing, HPC, and massively scalable systems (Rich Wolski's team of NWS and EveryWare fame) made it the natural place for the birth of Eucalyptus. The name EUCALYPTUS is an acronym and stands for Elastic Utility Computing Architecture for Linking Your Programs To Useful Systems. Eucalyptus is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, only version 3 of the License. 16

17 Eucalyptus is: Open: it is open source Modular: its components can be easily swapped out for custom components Distribuited: it allows its components to be installed strategically close to the needed/used resources Designed to perform: it is scalable and to achieve optimal performance in diverse environments Flexible: it is flexible and can be installed on a very minimal setup Compatible: it is compatible with the most popular and widely used Cloud API currently available: Amazon EC2 and S3 Hypervisor agnostic: it is designed to easily support most available and future hypervisors. Currently Eucalyptus fully supports KVM and Xen Hybrid cloud: its compatibility with Amazon's EC2 API allows for a natural hybrid cloud with the biggest public cloud available. 17

18 Grid Platform are composed of several components GSI: (Grid Security Infrastructure) CA Authentication and autorization: Certification Authority GRAM: Grid Resource Allocation Management Process cexecution and monitoring. Needed files staging GridFTP :(Data Management) Protected file transfer MDS: Monitoring and Discovery system (Information Services) Grid node informatrion management: GRIS: Grid Resource Information Service GIIS: Grid Index Information Service 18

19 GLOBUS: On opensource context for the creation of grid services. Globus is open source Grid software that addresses the most challenging problems indistributed resource sharing. The Globus middleware is a fundamental enabling technology for building grids that allow distributed computing power and storage resources, scientific instruments, and other tools to be shared securely across corporate, institutional, and geographic boundaries. Almost every large-scale HPC production grid, including EGEE, LCG, and NorduGrid in Europe, uses Globus Toolkit components as its base. The Globus Alliance is a community of organizations and individuals developing fundamental technologies behind the "Grid," which lets people share computing power, databases, instruments, and other on-line tools securely across corporate, institutional, and geographic boundaries without sacrificing local autonomy. 19

20 Globus Toolkit is an open source computational grid technology from Globus Alliance, an international association incorporating the University of Edinburgh, the Royal Institute of Technology in Sweden, the National Center for Supercomputing Applications and Univa Corporation, which are involved in building grid computing standards and technology. This community allows people to share computing power, databases, instruments, and other on-line tools securely across corporate, institutional, and geographic boundaries without sacrificing local autonomy. 20

21 Globus Toolkit is a portable software package which includes components such as information infrastructure and resource management, data management, colloborative communication, fault detection and security. A growing number of projects and companies are using Globus Toolkit to unlock the potential of grids to further their interests. For example Network for Earthquake Engineering and Simulation (NEES), FusionGrid, the Earth System Grid (ESG), the NSF Middleware Initiative, and the National Virtual Observatory). 21

22 The most common open source Operating System used in Grid and Cloud computing is obviously Linux. The development of Linux is one of the most prominent examples of free and open source software collaboration; typically all the underlying source code can be used, freely modified, and redistributed, both commercially and non-commercially, by anyone under licenses such as the GNU General Public License. AND SO ON 22

23 Portable Batch System (or simply PBS) is the name of computer software that performs job scheduling. Its primary task is to allocate computational tasks, i.e., batch jobs, among the available computing resources PBS was originally developed for NASA under a contract project that began on June 17, The main contractor who developed the original code was MRJ Technology Solutions. MRJ was acquired by Veridian in the late 1990s. Altair Engineering acquired the rights to all the PBS technology and intellectual property from Veridian in Altair Engineering currently owns and maintains the intellectual property associated with PBS, and also employs the original development team from NASA. The following open source versions of PBS are currently available: OpenPBS: original Open Source version released by MRJ in 1999 (not actively developed) TORQUE: a fork of OpenPBS that is maintained by Cluster Resources Inc (actually developed) 23

24 TORQUE Resource Manager provides control over batch jobs and distributed computing resources. It is an advanced open-source product based on the original PBS project and incorporates the best of both community and professional development. It incorporates significant advances in the areas of scalability, reliability, and functionality and is currently in use at tens of thousands of leading government, academic, and commercial sites throughout the world. TORQUE may be freely used, modified, and distributed under the constraints of the included in the proper licence. 24

25 R. Stallman founder of the Free Software Foundation and creator of the computer operating system GNU M. Shuttleworth Funder of the the development of Ubuntu, a Linux distribution based on Debian, and other free software projects "It's stupidity. It's worse than stupidity: it's a marketing hype campaign. somebody is saying this is inevitable and whenever you hear somebody saying that, it's very likely to be a set of businesses campaigning to make it true. On the Linux server side, the cloud is the big thing. It needs connectivity, mobility, and ability to scale up virtual instances. The desktop has key value here. By having a desktop that is network aware, you get the best of both worlds. Linux will be at the heart of the cloud and mobile devices. 25