Function and Performance Test of pen Source CloudStack Platform for HPC Service Function and Performance Test of pen Source CloudStack Platform for HPC Service 1 Jaegyoon Hahm, 2 Sang Boem Lim, 3 Guohua Li, 4 Hyeyoung Cho, 4 Seung Chan Shin, 4 Jaekeun Yeom, 4 Jungha Lee 1, First Author Korea Institute of Science and Technology Information (KISTI) Deajoen Republic of Korea, jaehahm@kisti.re.kr *2,Corresponding Author Department of Advanced Technology Fusion, Konkuk University, Seoul, Republic of Korea, sblim@konkuk.ac.kr 3 Department of Advanced Technology Fusion, Konkuk University, Seoul, Republic of Korea, yust@konkuk.ac.kr 4 Korea Institute of Science and Technology Information (KISTI) Deajoen Republic of Korea {chohy, scshin, jaekeun, jungha07}@kisti.re.kr Abstract Cloud computing technology is receiving attention from many companies and institutes. In order to construct a cloud computing environment, different architectures are being selected and tested for different types of services. Many cloud platforms are contributed to the Apache foundation as part of the trend of open source. penstack, CloudStack, and pennebula are the representative cloud management platforms. In this paper, we select the CloudStack platform to test its function and performance. To check if the functions work well and to test the most important performance aspects of the virtual machine, we constructed a function test-bed and performance test-bed for the virtual machines. We explain about the process of construction and test scenario. After that, we analyze the results. Through this process, we test the CloudStack platform to check the function and performance. Keywords: Cloud, Cloud computing, CloudStack, pen Source, HPC, Virtual Machine 1. Introduction Cloud computing is a computing technology which involves a large number of distributed resources through a real-time communication network. It also provides on-demand self-service to the end users. Nowadays, the cloud market is becoming more and more popular, and many companies and organizations are starting to study different types of cloud platforms for their own suitable services. Furthermore, many companies are using software based on the Web to provide cloud services. Most companies or institutes are also working on open source cloud platforms. The four most popular open source cloud platforms are penstack [1], CloudStack [2], pennebula [3] and Eucalyptus [4]. Among them, CloudStack platforms provided stable services on commercial platforms. Until 2012, CloudStack was released as open source. We have tested the functions of the open source version of CloudStack and the performance of virtual machines. In order to take advantage of the accuracy of the test, we constructed a function test-bed and performance test-bed. In chapter 2, we will explain the process of construction and installation issues of the platforms. Chapter 3 shows the test scenario and results. Chapter 4 analyzes the results. In chapter 5, we will make arrangements for technical issues. Finally, we explain the conclusion and future work in chapter 6. 2. Test-Bed System and Experiments Setup We constructed two test-bed systems. We prepared 4 cluster nodes for the function test and 18 cluster nodes for the performance test. We use the resources in the KISTI supercomputing center [5]. Journal of Next Generation Information Technology(JNIT) Volume 5, Number 2, May 2014 56
Function and Performance Test of pen Source CloudStack Platform for HPC Service 2.1. Function Test-Bed System The following are the hardware, software, and network information for the function test-bed system. We use a cluster named Venus, which consists of the 4 nodes in Table 1. ne 1-TB node is used as an NFS server, and the other 3 nodes are used as hosts. A host uses an X2 - Intel Xeon E5420 2.5GHz Quad Core with VT-x, 16GB of RAM, and one 250GB hard disk. We installed the CentS 6.4 2.6.32 64bit S on every host and constructed a 1 Gigabit Dual Ethernet Intel Corporation 80003ES2LAN network. Table 1. Function Test-Bed Hardware Information Information Cluster Name NFS Server Capacity Host Nodes CPU RAM Disk S Network Venus 1TB 3 X2 - Intel Xeon E5420 2.5GHz Quad Core with VT-x 16Gytes 250GB, 1EA CentS 6.4 2.6.32 64bit 1Gigabit Dual Ethernet Intel Corporation 80003ES2LAN We installed CloudStack version 4.1.1, which was released on 6 Aug 2013, as shown in Table 2. We also installed KVM [7] libvirt 0.10.2 as a hypervisor. ne of three nodes is used as a management server, and we installed mysql version 5.1.6 for the database. The deployment language of the installation source code is java, and the patch technique is based on a web interface. Using this web interface, we installed the command line tool CloudMonkey 5.0.0 [8] made a script written in Python for the performance test. The CloudStack platform is open source and uses Apache License 2.0 as a software license. Both Linux (Ubuntu, CentS, Fedora) and Windows are supported. Table 2. Function Test-Bed Software Information Information Version Release Date Hypervisor Database Source Code Language Patch License Supported S CloudStack4.1.1 2013-08-06 KVM(libvirt 0.10.2) Mysql 5.1.6 Java Web Interface (Using Command Line Tool CloudMonkey5.0.0) Apache License 2.0 Linux Series (Ubuntu, CentS, Fedora), Windows The Default physical network is constructed as a 1-Gigabit Ethernet Public/Private network, which each host can access. We set the Public network router as a default gateway and implemented a router using NAT for the Private network to access the Internet. The Public network has a 1-Gigabit bandwidth and B class netmask. Every host has a public IP. We also constructed a firewall in this network. For the virtual machine network, each host has bridge br0 to the eth0 interface. For example, in Figure 1, there are 8 virtual machines, and virtual machines between different hosts can communicate through bridge br0. 57
Function and Performance Test of pen Source CloudStack Platform for HPC Service 2.2. Performance Test-Bed System Figure 1. Bridge for Virtual Machine (VM) network The following are hardware, software, and network information for the virtual machine performance test-bed system. We use a cluster named Solar Cygnus and test on 18 nodes. In Table 3, one 4.5TB node is used as an NFS server, and the others are used as host nodes. This host uses an X2 - Intel Xeon E5420 2.5GHz 4 Cores with VT-x with 16GBytes of RAM and one 1TB hard disk. We installed CentS 6.4 2.6.32 64bit (with penstack kernel 2.6.32-358.123.2.openstack.el6.x86_64) and constructed 1-Gigabit Dual Ethernet and 10-Gigabit Ethernet Mellanox connectx3 [9] for the physical network. Table 3. Performance Test-Bed Hardware Information Information Cluster Name NFS Server Capacity Host Nodes CPU RAM Disk S Network Solar Cygnus 4.5TB 17 X2 - Intel Xeon E5420 2.5GHz 4 Cores with VT-x 16Gytes 1TB, 1EA CentS 6.4 2.6.32 64bit (with penstack kernel 2.6.32-358.123.2.openstack.el6.x86_64) 1Gigabit Dual Ethernet/10 Gigabit Ethernet Mellanox connectx3 The software information is the same as the function test-bed in Table 2. We constructed 1-Gigabit Ethernet as a public network and a 10-Gigabit Ethernet private network. Each host can access both the Public and Private networks. We set the private network router as the default gateway. The public network has a 1-Gigabit bandwidth and B class netmask. Every host has a public IP. We also constructed a firewall in this network. The Private network has a 10-Gigabit bandwidth, B class netmask and NAT implemented router which can access the Internet. In order to manage the host and virtual machines more efficiently, we used 10-Gigabit high-speed private networks for data communication. For the virtual machine network, each host has bridge br0 to the eth0 interface. It is the same as the function test-bed in Figure 1. 58
Function and Performance Test of pen Source CloudStack Platform for HPC Service 3. Test Scenario and Experimental Results The scenarios and results for the function test and performance test are as follows. 3.1. Function Test Scenario and Results According to the open source cloud platform standard document, we consider 6 categories: VM management, user management, image and template management, network, storage, and host. VM management is divided into stop, start, deploy, destroy, reboot, cold-migration, live-migration, resize (CPU, memory), and hotplugging functions, as shown in Table 4. The cold-migration and resizing CPU and memory functions do not work well on CloudStack 4.1.1. A bare-metal provisioning function will be supported in CloudStack 4.2.0. Table 4. VM management function test Types Function Check VM management stop Start Deploy Destroy Reboot cold-migration X live-migration resize(cpu, memory) X Hotplugging (disk, not iscsi) bare-metal provisioning X Table 5 shows the results of the virtual machine, storage, network, and account. All detail functions of each type work well. Table 5. User management function test Types Small Types Function Check User management Virtual Machine Max Instance Storage Max Volumes Max Snapshot Network IP allocation Security Groups VPC Account Lock Account Disable Account Assign VM Table 6 shows the test results of the image and template management functions. The functions failed to download an image file, and an attempt at adding a disk to the running virtual machine failed. Table 6. Image and Template management function test Types Function Check Image and Template management Set template without image file Image download X Add disk when deploying X VM Image snapshot 59
Function and Performance Test of pen Source CloudStack Platform for HPC Service Table 7 shows the test results of the network management functions. Even if VLAN is working well, it just works on the advanced zone [10]. The SDN (Software defined network) is a commercial version. Table 7. Network management function test Types Function Check Network IP allocation VLAN (in advanced zone) SDN(Software Defined X (commercial version ) Networking) Image snapshot Table 8 shows test results of storage management functions. The results of testing the snapshot, quota, block, object functions show that all work well except for the snapshot. The snapshot function is not supported on CloudStack 4.2.0 in a KVM environment. CloudStack uses penstack s swift object storage. Table 8. Storage management function test Types Function Check Storage Snapshot Quota Block bject (Using swift) Table 9 shows the test results of host management functions. We can deploy virtual machines on a selected host by using the tag option. We can migrate running virtual machines by clicking maintenance mode. The add host and delete host functions both work well. Table 9. Host management function test Types Function Check host Select Host when creating Maintenance mode Registration, deletion 3.2. Performance Test Scenario and Results We tested the virtual machine performance in the following scenarios. In Figure 2, which shows the VM life-cycle, there are Created/Stop, Running, Destroyed, and Shutdown states. Deploy, Stop, Start, Destroy, Expunging, Recover, and Reboot commands can be used for control. 60
Function and Performance Test of pen Source CloudStack Platform for HPC Service Figure 2. VM Life-cycle [11] The command for creating a virtual machine is Deploy, and its state follows Creating->Starting- >Running flow. The command for stopping the virtual machine is Stop, and its state follows Running- >Stopped flow. This flow includes S shutdown time, resource return time, and deleting metadata of the virtual machine from the database. The command for restarting a stopped virtual machine is Start, and its state follows Stopped->Running flow. This flow includes time for re-scheduling and rebooting the S. The command for deleting a virtual machine is Destroy, and its state follows Running- >Stopping->Stopped->Destroyed flow. This flow is the same as the Stop flow. The only difference is that it is marked as a deleted virtual machine in the database. The command for cleaning up a virtual machine is Expunging, and it means that the virtual machine will be cleaned up in 24 hours by default. It can be modified in Global Setting. It follows Destroyed->Expunging flow. The command for recovery is Recover, and it follows Recovering->Stopped flow. It can recover Destroyed virtual machines. According to the virtual machine life-cycle, we select deploy, stop, start, snapshot, destroy, recover, and migrate for 10 test scenarios. For each test, we delete all cache files in the shared storage. The algorithms for allocating host resources are Random, First fit, User dispersing, User Concentrated Pod Random, and User Concentrated Pod First Fit. In order to control 8 virtual machines on 1 host, we select the User Dispersing option to work with virtual machines. User Dispersing Algorithms mean that virtual machines deploy on a resource which has the least number of virtual machines of the same account. The Default option Random means virtual machines deploy from a list of available resources across the Zone, and a random resource is picked and allocated to the virtual machine. First Fit means virtual machines are created from a list of available resources in a database sequence. The other 2 options are the same as Random and First Fit, except that they can only be created on a selected pod. Figures 3 through 9 show the create, stop, start, snapshot, destroy, recover, and migrate results tested with 1, 8, 16, 32, 64, and 128 virtual machines. We will discuss the details of the analysis of the results in the next section. 61
Function and Performance Test of pen Source CloudStack Platform for HPC Service Figure 3. Create VM time Figure 4. Stop VM time Figure 5. Restart VM time 62
Function and Performance Test of pen Source CloudStack Platform for HPC Service Figure 6. Snapshot time Figure 7. Destroy VM time Figure 8. Recovery VM time 63
Function and Performance Test of pen Source CloudStack Platform for HPC Service Figure 9. Migrate VM time Figure 10 shows all functions and standard deviation values. 4. Analysis Figure 10. All VM commands and standard deviations CloudStack has two storage types: NFS shared type and local type. We use the default type NFS shared environment. What follows is about the analysis of the results of the virtual machine performance tests. As shown in Figure 3, the virtual machine creation time includes time for obtaining an image file from the NFS server, time for scheduling resources, and time for creating and booting. The creation time for 1, 8, and 16 virtual machines ranges from 24 to 29 min. The creation time from 32 virtual machines to 128 virtual machines increases in a proportional way. As a result, in the NFS shared storage type, most of the creation time is for obtaining an image from the NFS server. In this test, we use a 3.5GB qcow2 image file to create a virtual machine, and construct 10G Ethernet. In the CloudStack platform, the SSVM (Secondary Storage Virtual Machine) [12] is created automatically by 64
Function and Performance Test of pen Source CloudStack Platform for HPC Service CloudStack, and its role is to manage all kinds of images. SSVM has a network speed limitation of 200Mbps. We have tested on 16 hosts and use User Dispersing algorithms. In Figure 4, the virtual machine stop time includes time for shutdown of the virtual machine S, time for returning all resources to the host of the virtual machine, and time for deleting all metadata about the virtual machine. In the graph, the stopping time between 8 to 32 virtual machines has 5 to 8- min changes. The stopping time between 32 to 128 virtual machines increases in a proportional way. As a result, the time from 32 virtual machines increases uniformly. In Figure 5, the virtual machine restart time includes time for re-scheduling host resources and time for rebooting the S. It takes 5 sec to create 1 virtual machine. As a result, with the increasing number of virtual machines, the time is increasing uniformly. In Figure 6, the time for the virtual machine taking a snapshot includes time for creation on the primary storage and time for backup on the secondary storage. After taking a snapshot, a 2.2GB snapshot backup file is created on the secondary storage. It takes 1 min to take a snapshot for 1 running virtual machine. The time change between 32 to 64 virtual machines is the largest. In Figure 7, the virtual machine destroy time is the same as the stop time. It takes 15 sec to destroy 1 virtual machine. From 1 to 64 virtual machines, it is increasing uniformly. After 64 virtual machines, the change is smaller than before. In Figure 8, the virtual machine recovery time is restoring the destroyed virtual machine. It takes 1sec to recover 1 destroyed virtual machine. It is increasing uniformly. In Figure 9, the virtual machine migration time is moving the running virtual machine from host 1 to host 2. It takes 7 sec to migrate 1 virtual machine to another host. In this test, we have 16 hosts that can only migrate 64 virtual machines. In Figure 10, the time for taking a snapshot is longer than for other commands. The stop, destroy, and restart time changes are similar to each other. For creation time, it is more beneficial to create a large number of virtual machines. We have done 10 tests and calculated the standard deviation to confirm the width of the time change. As a result, in Figure 10, on the top of the snapshot bar which is on 64 nodes (virtual machines) is longest. It means that taking a snapshot on 64 nodes is very instable. 5. Conclusion and Future Work Although many cloud platforms are released as open source, there are actually many problems in the process of installing and operating. In order to address this disadvantage of open source and confirm the possibility and stability, in this paper, we prepared hardware resources and selected CloudStack as a test-bed. In this scenario, we checked HPC service on 1, 8, 16, 32, 64, and 128 virtual machines and did 10 tests. In the future, improvement of the performance needs to be tested. The CloudStack web interface currently offers the CloudMonkey command line tool. It has no parallel command to work with virtual machines at the same time. For these commands, parallel functions need to be added to command line interfaces. It will be more efficient to work with virtual machines. 6. References [1] "penstack Launches as Independent Foundation, Begins Work Protecting, Empowering and Promoting penstack". BusinessWire. 19 September 2012. Retrieved 7 January 2013. [2] Sheng Liang, Architecting for the cloud:lessons learned from 100 CloudStack deployments, CT, Cloud Platforms, Citrix, 2012 [3] Peter Sempolinski and Douglas Thain, A Comparison and Critique of Eucalyptus, pennebula and Nimbus, IEEE International Conference on Cloud Computing Technology and Science, November, 2010. [4] "Amazon Web Services (AWS) and Eucalyptus Partner to Bring Additional Compatibility Between AWS and n-premises IT Environments". News release (Eucalyptus Systems). March 22, 2012. Retrieved June 1, 2013. [5] Korea Institute of Science and Technology Information, KISTI, http://www.kisti.re.kr/ 65
Function and Performance Test of pen Source CloudStack Platform for HPC Service [6] Apache CloudStack 4.1.1 Installation Guideline, https://cloudstack.apache.org/docs/en- US/Apache_CloudStack/4.1.1/html/Installation_Guide/index.html [7] G.Charles, A performance analysis of Xen and KVM hypervisors for hosting the Xen Worlds Project, Iowa State University, 2011. [8] CloudMonkey5.0.0, https://pypi.python.org/pypi/cloudmonkey/ [9] Mellanox 10/40/56GbE Converged Network Adapters, http://www.mellanox.com/page/ethernet_cards_overview [10] Ristov, S. Gusev, M., Security evaluation of open source clouds, Cyril & Methodius Univ. Rugjer, Skopje, Macedonia, 2013 [11] Apache CloudStack 4.1.1 Admin Guideline, https://cloudstack.apache.org/docs/en- US/Apache_CloudStack/4.1.1/html/Admin_Guide/vm-lifecycle.html [12] SSVM, templates, Secondary storage troubleshooting. https://cwiki.apache.org/confluence/display/cludstack/ssvm,+templates,+secondary+stora ge+troubleshooting [13] Qingling Wang, Carlos A. Varela, Impact of Cloud Computing Virtualization Strategies onworkloads Performance, Rensselaer Polytechnic Institute, Troy, NY, USA, 2011. 66