CLEVER: a CLoud-Enabled Virtual EnviRonment

Transcription

1 CLEVER: a CLoud-Enabled Virtual EnviRonment Francesco Tusa Maurizio Paone Massimo Villari Antonio Puliafito {ftusa,mpaone,mvillari,apuliafito}@unime.it Università degli Studi di Messina, Dipartimento di Matematica. Facoltà di Ingegneria, Contrada di Dio, S. Agata, Messina, Italy. ISCC 10 IEEE Symposium on Computers and Communications, June 22-25, 2010, Riccione, Italy

2 Outline Introduction 1 Introduction Inside the Cloud State of the art Our contribution 2 3 UML description Implementation 4

3 Summary Introduction Inside the Cloud State of the art Our contribution 1 Introduction Inside the Cloud State of the art Our contribution 2 3 UML description Implementation 4

4 A cloud definition Introduction Inside the Cloud State of the art Our contribution Cloud Computing is a large-scale distributed computing paradigm that is driven by economies of scale, in which a pool of abstracted, virtualized, dynamically-scalable, managed computing power, storage, platforms, and services are delivered on demand to external customers over the Internet. [I. Foster et al. (2008)]

5 Three different cloud service levels Inside the Cloud State of the art Our contribution Virtualization technology for abstracting resources. Services to the customers mainly at three different levels: Iaas, PaaS and SaaS.

6 Inside the Cloud State of the art Our contribution A Cloud classification: Public, Private and Hybrid Clouds Public Clouds Management of Virtual Machine instances within a proprietary infrastructure. Many different customers can run and control their own applications. Access from a remote interface using a specific protocol. Private Clouds Infrastructure owned by a single organization offering its internal computing resources to local users: do not sell computing capacity. Open Source tools employment, dedicated operating environment offered to local users with high trust level. Hybrid Clouds A private cloud which adds to the local infrastructure more computing capacity with resources coming from an external public clouds. External resources access allowed over the Internet, using remote interfaces.

7 Inside the Cloud State of the art Our contribution Private/Hybrid Cloud middlewares: a reference stack Features of Private/Hybrid cloud middlewares: Middleware for Virtual Infrastructure Management: essentially dynamic orchestrator of Virtual Environments (VEs). Middleware for High-level Management: transforms existing infrastructures into an IaaS clouds with cloud-like interfaces; adds Security, Contextualization, Federation and other high-level mechanisms.

8 Inside the Cloud State of the art Our contribution Private/Hybrid Cloud middlewares: existing solutions Virtual Infrastructure Management Middlewares: OpenQRM and OpenNebula Deploy and manage VEs: individually or in groups needing parallel scheduling on local resources or external public clouds. Automate VE setup regardless of the underlying virtualization layer. Lack mechanisms for building hybrid IaaS clouds: public cloud-like interfaces, the ability to deploy VMs on external clouds and other High-level functionalities. High-level Middlewares: Globus Nimbus and Eucalyptus Transform existing infrastructure into an IaaS cloud with cloud-like interfaces. Compatible with the Amazon EC2 or Web Services Resource Framework (WSRF) interfaces and offers self-configuring virtual cluster support. Include Cloud-like interfaces and higher-level functionalities for security, contextualization. Limited VI management capabilities: lack the features of middlewares specialized in VI management.

9 Inside the Cloud State of the art Our contribution A new cloud computing middleware: CLEVER Acts as a middleware for the management of Private and Hybrid cloud computing infrastructures. Specifically integrates VI Management layer functionalities. Tries to minimize the design and implementation issues of the existing solutions: integrates fault-tolerance and scalability features. Abstract low-level technologies using a plug-ins design. Provides simple and easily accessible interfaces: Integration of security, contextualization and other high-level functionalities made available from higher level software components; Interconnection of different heterogeneous cloud computing infrastructures.

10 Summary Introduction 1 Introduction Inside the Cloud State of the art Our contribution 2 3 UML description Implementation 4

11 Reference Scenario Introduction Hierarchical reference stack Private/Hybrid cloud scenarios usually composed of a cluster of computers. Virtual Infrastructure Management layer functionalities arranged according to the physical infrastructure: Host Management and Cluster Management sub-layers.

12 General Architecture on the reference scenario N computing nodes containing one host level Management module: Host Manager. One node includes a cluster level Management module: Cluster Manager. External components: XMPP Server and Distributed Database. Middleware entities talks in an XMPP chat room exploiting the presence feature.

13 CLEVER Architecture Introduction Host Manager (HM) Communicates with the hosts OS, hypervisor and distributed file-system on which the VE disk-images are stored. Performs both physical resources and VEs monitoring. Runs VEs on the physical hosts even performing their migration. Cluster Manager (CM) Coordinates the HMs and performs operations on the Distributed Database. Acts as an interface between the clients and the HM. Performs the user VE disk-images management and the monitoring of the overall cluster state. At least one CM has to be deployed on each cluster: many of them should exist to enable fault-tolerance. A master CM will be in active state while the other ones will remain in a monitoring state: automatic active CM re-election.

14 CLEVER Architecture: XMPP and Fault Tolerance XMPP Offers a decentralized communication channel: more XMPP server could exists. Using more servers a central point of failure will not exist. Both CMs and HMs talks in a chat room exploiting the presence feature. Information about the cluster state (connected hosts) offered directly by the protocol. It is easy to add new nodes in the infrastructure: scalability. Distributed Database Database containing the overall set of information related to the middleware: the current state of the VEs, data related to the XMPP connection. Developed according to a well structured approach, for enabling fault tolerance features. Used by both the Active/Idle CMs and XMPP server(s).

15 Host Manager Components: Host Coordinator Each HM component mapped an a different OS process: high modularity and fault tolerance. The core of the Host Manager: it coordinates all the HM internal components using a specific Internal Communication protocol based on Java Message Service (JMS). Through the CM interface communicates with the CMs exchanging XMPP chat messages on the specific room (VEs allocation, Monitoring State, etc.).

16 Host Manager: Monitor and Low-level components Monitor: Resource usage monitoring for each host. Information organized and made available for the HM coordinator. Hypervisor Interface: middleware back-end to the host hypervisor. Different virtualization technologies could be employed using different plug-ins structure has to be developed. Image Manager: supply to the Hypervisor Interfaces the needed VE disk-image corresponding to a specific VE. Different plug-ins associated to different data access/transfer method. Network Manager: Gathers information about the host network state. Manages host network (OS level) according to the guidelines provided by the HM Coordinator: dynamic creation of network bridges, routing and firewalling rules.

17 Cluster Manager Components: Cluster Coordinator Each CM component mapped an a different OS process: high modularity and fault tolerance. The core of the Cluster Manager: like the HM Coordinator it coordinates all the CM internal components using a specific Internal Communication protocol based on Java Message Service (JMS). Performs Virtual Environment deploying based on the cluster workload and features of each host. HMs Interface, Client Interface and External Cluster Interface for communicating on different XMPP chat with the cluster HMs, the cloud clients (or High-level software) and other federated infrastructures.

18 Cluster Manager: Low-level components Database Manager: interacts with the database used to store information needed to the cluster handling. Database Manager must maintain the data strictly related to the cluster state. Performance Estimator: Analysis of the set of data collected from the Coordinator, in order to compute and provide a probable trend estimation of the collected measures. Image Manager: manages registration and upload within the Cluster Storage System of the VEs disk-images. The Storage Manager is used to perform the registration process of such files and manage the internal cluster distributed file system.

19 Summary Introduction UML description Implementation 1 Introduction Inside the Cloud State of the art Our contribution 2 3 UML description Implementation 4

20 Design state Introduction UML description Implementation Whole system behavior described through UML: Use Case Diagrams: Global system, Middleware Initialization, VEs execution and VEs registration. Activity Diagrams: XMPP communication, Component Interaction, VE Migration,... Class Diagram: one for each component and Internal Communication Protocol. Sequence Diagrams: VEs Migration, Inter-process Communication Protocol.

21 UML description Implementation Design state example: General Use Case

22 UML description Implementation Design state example: Host Coordinator Class Diagram

23 Implementation State Introduction UML description Implementation Fully Implemented Components HM-CM Communication Protocol over XMPP Host Manager: Host Coordinator and Internal Communication Protocol Hypervisor Interface with Libvirt plugin Monitor Network Manager Cluster Manager: Cluster Coordinator and Internal Communication Protocol Partially Implemented Components Host Manager: Image Manager Cluster Manager: Storage Manager, Database Manager

24 Summary Introduction 1 Introduction Inside the Cloud State of the art Our contribution 2 3 UML description Implementation 4

25 A CLEVER summary Introduction Mainly acts as CLEVER Virtual Infrastructure Management cloud middleware. Exposes suitable interfaces at the High-level Management layer. Enables the integration of Public Cloud Interfaces, Contextualization, Security, Federation and other High-level functionalities. Using Monitoring, is able to perform best VEs allocation and migration. Thanks to its pluggable design, CLEVER grants scalability, modularity, flexibility. Abstraction from the underlying technologies. Fault Tolerance using more CMs. Presence concept exploited to know the state of the cluster: available hosts, disconnected hosts.

26 Future Works Introduction Complete the middleware implementation with other remaining components already designed. Performance Analysis of the middleware, evaluating the overhead introduced by XMPP. Compare the middleware performance with the VM allocation rate of the Hypervisor (OS tools/api).

27 Questions and Answers Introduction