RESOURCE MONITORING SYSTEM BASED ON SFA FOR A HETEROGENEOUS CLOUD FEDERATION

Transcription

1 RESOURCE MONITORING SYSTEM BASED ON SFA FOR A HETEROGENEOUS CLOUD FEDERATION 1 SEUNGHYUN SEO, 2 MYOUNGJIN KIM, 3 YUN CUI, 4 SEUNGHO HAN, 5 SEUNGBUM SEO, 6 HANKU LEE 1,2,3,4,5,6 Department of Internet & Multimedia Engineering, Konkuk University dekiller2, tough105, ilycy, shhan87, gpsb, hlee}@konkuk.ac.kr Abstract- A large number of corporations are jointly developing various Cloud platforms. However, problems have occurred in resource management due to multiple number of platforms that were not integrated at the time of development of the Cloud federation service. If user uses heterogeneous platform, there arises difficulty in resources management. Therefore, there is a need to develop a federation monitoring system that can perform efficient management, exclude overlapping data and integrate the management of each platform when using heterogeneous platform. As such, this thesis proposes resource monitoring system for the Slice Federation Architecture (SFA)-based heterogeneous Cloud federation that can integrate the physical resources. This system is composed of 4 modules, namely, the Component Manager, Aggregator Manager, Registry Manager and Slice Manager. Keywords- Cloud Computing, Cloud Platform, Cloud Brokerage Service, Federation, SFA. I. INTRODUCTION Cloud computing provides the IT resource as service by utilizing the internet technology, which has the computing characteristic of borrowing and using the IT resources (SW, storage, server, network) as much as needed, receiving support for real time expandability in accordance with the service load, and paying the cost for the actual use. According to Gartner, a global research company in the IT area, Cloud computing is expected to grow enormously in the future as it had been considered as one of the 10 major strategic technology of the world in 2012 as well as in In addition, it was forecast that, 40% of the Cloud services will be provided through the Cloud brokerage service by 2015 and the Cloud brokerage service will become the fastest growing Cloud computing area. Cloud computing is divided into the SaaS (Software as a Service), PaaS (Platform as a Service) and IaaS (Infrastructure as a Service) models depending on the types of the service provided. Of these, the IaaS is a service in the format of managing the key infrastructural resources (CPU resources, memory, disc and network environment) for providing IT service in the form of shared resources and providing it by dividing the resources. The service user uses the services by installing the OS and middleware (data base and web server) by having the resource allocated on such infrastructure. In general, the hardware resources are virtualized for distribution by using the virtualization technology. The user may borrow and use only as much server, storage and network as needed through IaaS, thereby reducing the cost of manpower and resources for the management of computing infrastructure and IT infrastructure. Moreover, it has the advantage of being able to expediently coping with the rapidly changing business through flexible expansion and reduction of the computing resources. Representative overseas IaaS vendors include Amazon Web Services, AT&T and Rackspace, while the Ucloud of KT, which is built on the basis of Cloudstack, is undergoing development and offering commercially available services in Korea. Accordingly, most active researches are being carried out for the IaaS. Among these, the Openstack and the Cloudstack are the representatives of the open source based Cloud platforms. Among the open source Cloud platforms, the largest number researches are on the Openstack with as many as more than 200 companies participating in this project. The Cloudstack was as popular as the Openstack prior to the emergence of the Openstack and active development on the Cloudstack still continues. The Ucloud of KT that provides commercial services on the basis of the Cloudstack is known most widely in Korea. Although such development of numerous platforms is enormously helpful for the advancement of the Cloud computing platform, there arises the following problem that despite the multiple numbers of Cloud platforms providing IaaS service, they still don t provide integrated resources management function. From the user s perspective, additional cost will be incurred due to wasting of unnecessary resources since duplicate data will be generated if multiple platforms are used together. In order to achieve efficient management, duplicated data must be excluded when using 4

2 heterogeneous platforms that are different from each other. In addition, for the Cloud brokerage services in the future, there is a need for federation monitoring system integrated management of the platforms, and a diverse range of federation system architectures are being developed at the moment. Slice Federation Architecture (SFA) under GENI environment has a Slice-base that enables integration of the physical resources (CPU, memory and disc). In this thesis, resource monitoring system for SFA-based heterogeneous Cloud federation that can harmoniously manage and monitor the Openstack and Cloudstack, which are heterogeneous Cloud platform on the basis of the Slice Federation Architecture (SFA) that is able to integrate physical resources (CPU, memory and disc) on the basis of Slice is proposed. The services of the proposed system is composed of Component Manager (CM) that brings the resources of the Cloud service provider and managing it by defining it with Sliver, Aggregator Manager (AM) that brings the Sliver of CM in accordance with the particular user conditions and managing it with Slice, Registry Manager (RM) that manages the user log information and Slice information in the format of database and file, and Slice Manager (SM) that controls the Slice and provides interface to the user. The composition of this thesis is as follows. Chapter 2 explains representative open source platform Openstack and Cloudstack, Cloud brokerage service, and SFA under GENI environment, Chapter 3 proposes the SFA-based resources monitoring system architecture for heterogeneous Cloud federation, Chapter 4 deals with system realization and prototype, and Chapter 5 provides conclusion and presents future research directions. II. LITERATURE SURVEY Openstack is a Cloud computing open source project in IaaS format that enables general server to generate and execute the Cloud computing service. The service composition of the Openstack is largely made up of 5 service projects, namely, the NOVA (Compute) that controls and manages the virtual compute instance, Swift (Object Storage) that provides object storage service, Cleanse (Image Storage) that provides image services, Keystone (Identity) that is in charge of certification services and Horizon (Dashboard) that provides web interface service. In addition, a diverse range of other projects are being developed at the moment. Cloudstack is an IaaS platform software developed by cloud.com and is composed of 2 nodes, namely, the Cloudstack management server that manages the resources of the system and the Cloudstack server host with virtual instances. The management server determines which host will execute the virtual instances, allocates the IP and storage to the virtual instances, and manages the snapshot, template and ISO images. The server host is where the VM is directly executed and provides physical resources to VM. The servers of the host is grouped into Cluster composed of more than 1 host and main storage, Pod at Rack unit that includes more than 1 Cluster and Layer2 switch, and Zone, which is a data center unit with several Pods. B. Cloud Brokerage Services (CBS) A. Openstack / Cloudstack Fig. 1 Openstack & Cloudstack service configuration Fig. 2 Cloud Brokerage Service concept map With the proliferation of the use of Cloud by individuals and companies, Cloud brokerage service that provides support for the use of a diverse range of Cloud platforms as a single Cloud platform is receiving highlight. Cloud brokerage service plays triple role of accumulation, integration and customization. Accumulation refers to collection of various Cloud services and providing them for the end user while integration refers to the role of connecting the Cloud service with the internal system as an intermediary. Customization refers to adjustment of the Cloud service in accordance with the needs of the customer or development of application to be operated in the Cloud. With the triple role of the Cloud brokerage service, it is able to overcome the weaknesses of single Cloud service such as loss of data 5

3 due to reliance on single particular Cloud and system disability, and the weaknesses of multiple numbers of Cloud service for which data integrated management is difficult. As a representative example of adoption of the system, there is the Defense Information Systems Agency (DISA) newly inaugurated by the US Department of Defense to take charge of the Cloud service mediation related tasks for the corporation, and Infosys, Accenture, Capgemini and Hitachi, etc. which are currently providing the services. C. Global Environment for Network Innovation GENI has diversified research characteristics that include wide range of issues ranging from the network theories to network policies and researches from social and economic perspectives. GENI is defining the infrastructure on the basis of the results derived from various projects including PlanetLab, ProtoGENI and VINI, etc., and the architecture of the GENI infrastructure is referred to as the Slice Federation Architecture (SFA). That is, GENI provides the environment under which network experiments can be executed by sharing the resources being operated at various institutions. Moreover, GENI shares the resources from diverse platforms and operates Slice by defining the SFA in order to compose ecologic system of the heterogeneous platform. - Ticket: As the RSpect signed by AM, it is a certification that the resources requested by the user will be allocated during the given time period. The ticket issued will be retrieved upon allocation of the requested resources. - Credential: In order for the user to have the Slice issued and acquire the permission for use of the resources from the AM, Credential must be provided in order to prove that the user has legal authority. The specific format for the Credential is determined in accordance with the control framework. AM, which plays core role in GENI, provides API including the GetVersion() that returns the Cloud information, ListResources() that retrieves Cloud resources, CreateSliver() that generates instances in accordance with the requested information of RSpec, DeleteSliver() for deletion of instances and SliverStatus() that retrieves the information of Slice and Sliver, etc. In the system we are proposing, API for direct AM interface on the basis of the API provided by GENI was realized, and the same name of API was given. In addition, the system was realized to have executed the tasks that are the same as those of GENI AM AP. Thus, we are proposing the resources monitoring system of the Openstack and Cloudstack of the heterogeneous Cloud Platform on the foundation of GENI AM API and SFA. The role of using the GENI in SFA is divided into 3 categories: - Management Authority (MA): Authority for normal operation of Aggregator Manager (AM) and manages the allocation and operation of resources. - Slice Authority (SA): Authority for generation and use of Slice, and in charge of Slice registration and controlling of slice user access. - User: All the researchers who experiment new network model or execute the desired services through GENI, system administrator who manages the entire or particular AM of the GENI infrastructure, personnel in charge of the relevant project, and providers of the physical resources who participated in GENI federation belong to the category of users. In addition, although user at another level could exist when particular experiment and services are provided through Slice, it is deemed that it is solved independently at the corresponding Slice without separate regulation at the level of SFA. In addition, SFA presents 3 types, namely, RSpec, Ticket and Credential. - RSpec: Format used when expressing the current status of resources provided by the Aggregator Manager (AM) and requesting resources. It is used when delivering the specification table of the available resources or requesting the necessary resources. III. SFA-BASED RESOURCE MONITORING SYSTEM ARCHITECTURE FOR HETEROGENEOUS CLOUD FEDERATION This thesis proposes integrated monitoring system for heterogeneous Cloud platform service by utilizing SFA of GENI. SFA is a structure defined as a collection of interfaces for interlocking purpose and compatibility of the available Slice-based network, and Slice is composed of the collection of Sliver. Sliver signifies the virtual resources dispersed within the Component Manager (CM), which are the 6

4 physical resources of the Cloud service provider while the Slice refers to the collection of the dispersed virtual resources and dispersed collection of Slivers, that is, the collection of the virtual resources of the Cloud service provider currently operated in each of the different CM s. Fig. 3 illustrates the SFA-based Cloud federation monitoring system architecture proposed for the integration of heterogeneous Cloud platform service. In the overall system, physical and logical resources of service providers based on diverse heterogeneous Cloud platforms including the Openstack and Cloudstack are brought to CM through CM API and managed by being defined with each Sliver. Information of Sliver stored in such CM is delivered to AM through Aggregator Manager API (AM API) and AM composes Slice on the basis of the information of Slivers. Each of the Slices is managed by the Slice Manager and forwards data of each of the Slices to the administrator in order for the administrator to manage and monitor the resources on type of the platforms and the type of instances of each of the users. In addition, by maintaining the log information of the user and the information of Slice and Cloud service providers in the format of database and file in the Registry Manager, the administrator manages the information of the users and Slice and Cloud service providers. Such SFA-based Cloud platform resources federation monitoring system architecture was largely composed of 4 elements including Slice Manager, Registry Manager, Aggregator Manager and Component Manager. A. Component Manager (CM) CM plays the role of checking the resources of the Cloud service provider and returning the general circumstances related to the resources to the Aggregator Manager (AM) in the proposed system. Moreover, CM plays the role of collecting and segregating the resources information (VCPU, VMemory and VStorage) on the user instance, and provides Sliver-based control. CM is composed of the management interface and CM API. Management interface manages the physical resources such as the instant Cloud service providers such as the Openstack and Cloudstack by defining them as each of the dispersed Slivers. In other words, a Sliver includes one virtual resource among the VCPU, VMemory or VStorage, which are the infrastructure resources of the Cloud service provider. B. Aggregator Manager (AM) AM plays the role of binding and managing the slivers that corresponds to the same conditions with Slice and delivering the Slice to the Slice Manager by aligning the Sliver of Component Manager (CM) that defines the Cloud service provider resources as virtual resources to particular user conditions in the proposed system. Particular user conditions refer to searching and filtering between the desired resources among the VCPU, VMemory and VStorage. AM, similar to CM, is composed of management interface and AM API. The management interface is the Slice generated by the Slice Manager for integrated management of multiple number of CMs by binding the dispersed virtual resources of the Cloud service providers defined by Sliver within each CM in accordance with the particular user conditions. Moreover, it delivers the each of the bound Slices to the Slice Manager. AM API plays the role of bringing the infrastructure resources of Cloud service provider defined by the Sliver at each CM. In addition, it has the role of delivering the Slices bound in accordance with the particular user conditions to the Slice Manager. C. Registry Manager (RM) RM provides information on the type of the Cloud and the extent of usage for each time interval of the users by storing the log information of the users including the date, time and IP of the logging in of the user with ID. It is therefore possible to confirm the extent of user access for each time interval by using such user log-in information as well as prepare for the assault of viruses or DDos server. In addition, it maintains the Slice information and Cloud service provider including the number, generation and deletion time of Slice in the formats of database or file by continuously tracking them, and enables Slice control by delivering such information to the Slice Manager. D. Slice Manager (SM) SM brings and manages the Slice generated and integrated at the Aggregator Manager (AM), and enables the initialization of Slice, and supply and control of resource by bringing the information of Slice from the Registry Manager (RM). In addition, SM provides Slice-based control. In addition, it plays the role of the interface that can directly access the administrator and provides web interface, thereby offering the environment in which the user can monitor the Slice. CM API plays the role of brining the resources from the Cloud service provider. In addition, it plays the role of delivering the Sliver that defined the resources brought from the Cloud service provider to AM. SM is composed of Slice management interface and SM API. Slice management interface brings the Slice that user and administrator wants from AM or deletes the unnecessary Slice by receiving the Slice 7

5 information from RM. Slice that user wants provides the environment that enables monitoring of the infrastructure resources of Slice through web. SM API performs the role of bringing the Slice that user wants from the AM, and delivers and stores the information of Slice brought in RM. In addition, it plays the role of delivering the Slice to web when providing the web environment under which the user can monitor. IV. SYSTEM REALIZATION AND PROTOTYPE overview menu and the physical resources not being used together simultaneously, as illustrated in the Fig. 4. A. Realization Environment Openstack was installed on 2 units of cluster, with Control Node (Master) established in 1 of the units and Compute Node (Slave) in the other one. For the Control Node, eth0 connected to the external internet network and 2 units of eth1 LAN cards connected to the internal network were used. In the Compute Node, eth0 connected to the internal network was included. For installation, devstack was used after having set the internal network. Table. 1 Cloud platform version and cluster Specifications Categorie Contents s Cloud Platform Openstac k Openstac k Version Grizzly Grizzly CPU Intel Xeon 4 core 2.13 GHZ Intel Xeon 4core 2.13 GHZ Memory 4GB 4GB 4GB Cloudstack Intel Xeon 4core 2.13 GHZ HDD 500GB 500GB 500GB OS Ubuntu LTS 64Bit Ubuntu LTS 64Bit Ubuntu LTS 64Bit C. Cloudstack dashboard Similar to the Openstack, when the instances are generated through the Cloudstack dashboard by accessing the web through the local host after having completed the installation, it is possible to generate the infrastructure resources (CPU, Memory, Storage) of cluster as virtual resources(vcpu, VMemory, VStorage). It is possible to monitor the generated virtual resources and physical resources not being used together simultaneously, as illustrated in the Fig. 5. Cloudstack was installed in 1 unit of cluster, and management server and the host server were composed. For installation, Cloudstack Apache website manual was used. The Table 1 above illustrates the version of the heterogeneous Cloud platform and the system specifications of the server. B. Openstack Dashboard Following completion of installation, it is possible to generate the infrastructure resources (CPU, Memory, Storage) of the cluster as virtual resources (VCPU, VMemory, VStorage) by using the instance menu through the Openstack dashboard by accessing the web through the local host. The virtual resources generated can monitor the virtual resources used in the Fig. 5 Cloudstack Dashboard D. Integrated Dashboard for Heterogeneous Cloud Federation System Web server was established by suing Django for the integrated dashboard of the Openstack dashboard and the Cloudstack dashboard, and Restful API was used for data exchange. Moreover, Integration dashboard for heterogeneous Cloud platform that can confirm the resources of the Openstack and Cloudstack, which provides web interface, together and simultaneously was realized by using Aptana. 8

6 2 units of the Openstack clusters generated 670GB VStorage among the total of 1TB by suing 500GB HDD each, while the Cloudstack generated 165GB VStorage among the total of 500GB. Fig. 6 illustrates the integrated dashboard for the heterogeneous Cloud platform and is the window that appears when the Storage item is clicked. The storage menu in use displays 67% for the cluster with Openstack installed and 33% for the cluster with Cloudstack installed. In the storage menu not used, it is possible to confirm that 33% of the Openstack is not being used while 67% of the Cloudstack is not being used. Moreover, it is possible to confirm in the total menu that 62% of the storage among the storage of the 3 units of clusters for which Openstack and Cloudstack are installed is being used while the storage not being used is 38%. It is possible to confirm the resources being used as well as the non-used resources by means of the other items on the left (CPU and Memory) in addition to the Storage item, and Openstack and Cloudstack items were realized to enable generation and deletion by directly accessing each of the dashboards by connecting the link to each of the Openstack and Cloudstack dashboards. CONCLUSION From the viewpoint of the administrator, the difficulties in administration arise if the user uses multiple Cloud platforms together, and, from the viewpoint of the user, the problem of additional cost incurred due to wasting of unnecessary resources due to the heterogeneous Cloud generates duplicating data, thereby requesting monitoring information on the service environment of the heterogeneous Cloud platform. In order to resolve such issues, this thesis proposed SFA-based resources monitoring system for the heterogeneous Cloud federation. The system was composed largely of 4 services, namely, Component Manager (CM) that brings the resources of the Cloud service provider and manages it by defining it with Sliver, Aggregator Manager (AM) that brings the 9 Sliver of CM in accordance with the particular user conditions and managing it with Slice, Registry Manager (RM) that manages the user log information and Slice information in the format of database and file, and Slice Manager (SM) that controls the Slice and provides interface to the user. It is able to directly monitor resource distribution management by monitoring the status of heterogeneous Cloud computing resources in real time, and confirm the resource use rate by the heterogeneous Cloud service provider. We are planning to elevate the utilization of diverse resources and provide adoptive type Cloud service and Cloud brokerage service by linking the Cloud service resources of the same user between the heterogeneous Cloud computing through improvement and use of the heterogeneous Cloud platform resources federation monitoring system in the future. REFERENCES [1] Openstack, [2] Cloudstack, [3] KT Ucloud, [4] Johan Trodsson, Ruben S.Montero, Rafael Moreno-Vozmediano, Ignacio M.Llorente, Cloud brokering mechanisms for optimized placement of virtual machines across multiple providers, Future Generation Computer Systems, vol 28, pp , 2012 [5] Antonio Celesti, Francesco Tusa, Massimo Villari and Antonio Puliafito, How to Enhance Cloud Architectures to Enable Cross-Federation, 2010 IEEE Cloud, pp , 2010 [6] B. Rochwerger, D. Breitgand, E. Levy, A. Galis, K. Nagin, I.M. Llorente, R. Montero, Y. Wolfsthal, E. Elmroth, J. Caceres, M. Ben-Yehuda, W. Emmerich, F. Galan, The Reservoir model and architecture for open federated cloud computing, IBM Journal of Reasearch and Development, vol.53, no.4, pp.4:1, 4:11, 2009 [7] Muli Ben-Yehuda, Michael D.Day, Zvi Dubitzky, Michael Factor, Nadav Har El, Abel Gordon, Anthony Liguori, Orit Wasserman, Ben-Ami Yassour, The Turtles Project: Design and Implementation of Nested Virtualization, Proceedings of the 9th USENIX conference on Operating systems design and implementation, p.1-6, 04-06, 2010 [8] J.H. Ra, S.H. Han, B.Y. Sung, Y.G. Kim, Open Source Cloud Platforms : Openstack and CloudStack, KIISE, pp , 2012 [9] I.H. Jung, S.H. Lee, Y.I. Eom, Comparative Analysis of Open Source Cloud Computing Platforms, KIISE, pp , 2012 [10] GENI, [11] Larry Peterson, Soner Sevinc, Jay Lepreau, Robert Ricci, John Wroclawski, Ted Faber, Stephen Schwab and Scott Baker Slice-Based Federation Architecture, [12] PlanetLab Implementation of the Slice-based Facility Architecture, tart/sfa-impl.pdf [13] K.H. Nam, S.J. Jeong, M.K. Shin, H.J. Kim, Technology and Trends of GENI Control Framework,

7 [14] J.W. Kim, B.L. Cha, S.W. Han, N.K. Kim, Y.H. Kim, K.H. Nam, S.W. Lee, S.B. Mun, A Review on Experimental Facilities for Future Internet Testbed, &value=fi-tb-whitepaper-v04r1.pdf [15] Openstack Installation, guides/ multinode-lab.html [16] Cloudstack Installation, docs/en-us/apache_cloudstack/4.2.0/html/installation_guid e/installation.html 10