Comparative Analysis of Virtual Desktops in Cloud

Transcription

1 Thesis no: Comparative Analysis of Virtual Desktops in Cloud Performance comparison of OpenStack Private Cloud and AWS Public cloud Pruthvi Raj Kanmantha Reddy Faculty of Computing Blekinge Institute of Technology SE Karlskrona Sweden

2 This thesis is submitted to the Faculty of Computing at Blekinge Institute of Technology in partial fulfillment of the requirements for the degree of Masters in Electrical Engineering with emphasis on Telecommunication Systems. The thesis is equivalent to 20 weeks of full time studies. Contact Information: Author: Pruthviraj Kanmantha Reddy External advisor: Sujoy Chatterjee, General Manager, ITC Infotech, Bangalore, India University advisor: Prof. Adrian Popescu Department of Communication Systems (DIKO) BTH, Karlskrona Faculty of Computing Blekinge Institute of Technology SE Karlskrona, Sweden ii Internet : Phone : Fax :

3 ABSTRACT Context. DaaS is a cloud service that provides desktop environment of virtual machine running on the cloud. In DaaS, backend of VDI is managed by a cloud service provider. There are many enterprise based virtual desktop solutions such as AWS WorkSpaces, OpenStack VDI/DaaS, etc. One of the main challenge that service providers face is to deploy the desktops to user at a faster pace and the performance of the virtual desktops to be good. Objectives. The main objective of this thesis is to mark the key requirements for Desktop as a Service (DaaS) in various cloud offerings. This thesis has been divided into two scenarios. In the first scenario boot up times for both OpenStack nova instance and AWS EC2 were considered at instance level. In the second scenario AWS performance metrics such as CPU usage, memory usage, and disk performance metrics were considered to compare AWS workspace virtual desktop with OpenStack VDI/DaaS. Methods. This research has both qualitative and quantitative analysis. In qualitative analysis literature study and survey was conducted to analyze the cloud adoption in enterprises. The performance metrics such as CPU, memory usage, Disk average IO size, throughput, average latency etc. were calculated for virtual desktops running on top of AWS workspace and OpenStack VDI/DaaS. Results. Results are analyzed for two test cases. In the first case the VM boot up time was calculated. Here it was seen that overall average boot up time for OpenStack nova instance was less when it is compared to AWS EC2. In the other case performance metrics such as CPU usage, memory usage and disk performance parameters like IOPS, average latency, average IO and throughput were calculated when put under various test scenarios. It was seen that OpenStack VDI instance performed better in most of the test scenarios. Conclusions: Virtual desktop on OpenStack VDI/DaaS performance was much better compared to AWS workspace. While average latency was considerably higher in OpenStack VDI/DaaS than AWS workspace. Even the boot up times was less in OpenStack than AWS in both cases when the VM s were launched in serial or in parallel, all at a time. Keywords: VDI, Performance, VM Boot up time, OpenStack, AWS WorkSpaces iii

4 ACKNOWLEDGEMENTS First of all, I would like to take this opportunity to thank my parents who well supported me at every single stage of my career. I would not have gained enough strength in many difficult situations without their support and guidance. I am grateful for all the support I received from my supervisor Prof. Adrian Popescu for the help and guidance he has given me. Without his advices and support this thesis would have not been possible. I would also like to thank our examiner, Dr. Kurt Tutschku for his efforts throughout the course. I owe my gratitude to my external advisor Sujoy Chatterjee and the whole team at ITC Infotech, who gave us an opportunity to work in their assistance. I would like to thank my friend who is also my thesis partner Mr. Malkannanagari Akash Reddy for his relentless contribution in making our thesis successful. I cannot thank them him/her enough my friends, professors and well-wishers who gave me enough confidence and support in both professional as well as in personal life. Finally I would like to thank every single staff at BTH, Karlskrona and JNTU, Hyderabad for giving us an opportunity to develop in every particular aspect. iv

5 CONTENTS ABSTRACT... III ACKNOWLEDGEMENTS... IV CONTENTS... V LIST OF FIGURES... VII LIST OF TABLES... VIII ABBREVIATIONS... IX 1 INTRODUCTION PROBLEM STATEMENT AIM AND OBJECTIVES RESEARCH QUESTIONS CONTRIBUTION OUTLINE OF THE THESIS BACKGROUND VIRTUALIZATION CLOUD COMPUTING CLOUD COMPUTING MODELS CLOUD COMPUTING DEPLOYMENT MODELS DESKTOP AS A SERVICE AMAZON WEB SERVICES EC2-Elastic Computing Cloud WorkSpaces OPENSTACK VARIOUS COMPONENTS IN OPENSTACK VARIOUS COMPONENTS TO BUILD AN OPEN STACK VDI PERFORMANCE MONITORING PERFORMANCE METRICS EXPERIMENT METHOD SURVEY EXPERIMENTAL ENVIRONMENT OpenStack Server: For OpenStack VDI For AWS WorkSpaces EC2 - M4 large instance specifications PERFORMANCE METRICS Experiment Experiment EXPERIMENTAL SETUP OpenStack AWS WorkSpaces EXPERIMENT EXPERIMENT Test 1 - Full load on CPU Test 2 Full load on Memory Test 3 Write temp file Test 4 Simulation of Disc Accesses ASSUMPTIONS RESULTS SECTION v

6 4.1.1 Case 1: Serial Launch Case 2: Parallel Launch SECTION Scenario 1: Full CPU Load Scenario 2: Full Memory Load Scenario 3: Write Test File Scenario 4: Simulation of Disk access ANALYSIS AND DISCUSSION RELATED WORK ANSWERING RESEARCH QUESTIONS CONCLUSION AND FUTURE WORK FUTURE WORK REFERENCES APPENDIX SURVEY RESULTS AWS WORKSPACE INSTALLATION BASH SCRIPT FOR FULL CPU LOAD vi

7 LIST OF FIGURES Figure 1: VDI Architecture... 1 Figure 2: Cloud Computing Model... 5 Figure 3: OpenStack Architecture Model... 8 Figure 4: Various Components of a VDI... 9 Figure 5: Experimental Methodology Figure 6: Setup for OpenStack Environment Figure 7: AWS WorkSpace Environment Figure 8: Graphs for Full Load CPU test case Figure 9: Graphs for Full Load CPU test case Figure 10: Graphs for Full Load Memory test case Figure 11: Graphs for Full Load Memory test case Figure 12: Graphs for Write Temp File test case Figure 13: Graphs for Write Temp File test case Figure 14: Graphs for Simulate Disc Access test case Figure 15: Graphs for Simulate Disc Access test case vii

8 LIST OF TABLES Table 1: Experiment Environment Table 2: Experiment Environment Table 3: Experiment Environment Table 4: Experiment Environment Table 5: VM's Serial Launch - Boot up Time Table 6: VM's Parallel Launch - Boot up Times viii

9 ABBREVIATIONS VDI API OS IT SAAS PAAS IAAS DAAS CPU AWS VM vcpu EC2 S3 IAM UI IOPS RDP Virtual Desktop Infrastructure Application Program Interface Operating System Information Technology Software as a service Platform as a Service Infrastructure as a Service Desktop as a Service Central Processing Unit Amazon Web services Virtual Machine Virtual CPU Elastic Compute Cloud Simple Storage Service Identity and Access Management User Interface Input/Output Operations Per Second Remote display protocol ix

10 1 INTRODUCTION Virtual desktop infrastructure (VDI) is one of the emerging technologies that replaces as an alternative to traditional desktop computing. The theory behind Virtual desktop infrastructure is based on isolating a logical OS, Operating System instance from the client or end users who access them [1]. VDI comes with a feature of High Availability i.e. when a physical desktop crashes it might take several hours to rebuild it whereas creating a new virtual machine will take minutes to deploy it. As the processing takes place remotely in datacenter, users can access the VDI s from several devices which can include smartphones, tablets, desktops and etc. Connectivity between client device and OS executing in the data center require the users to view and interact with the desktops over a network using remote display protocols. Due to the paradigm shift from traditional desktop computing to VDI there are several advantages which includes decrease in management costs, both operational as well as the capital costs. The costs are significantly lower because OS images, Patches or updates and applications are no longer handled by the large distributed systems but by a well-controlled and managed datacenters[2]. Figure 1: VDI Architecture [3] Cloud Computing has transformed a large part of the IT industry, making software even more attractive as a service and shaping the way IT hardware is designed and purchased. Developers with innovative ideas for new internet services no longer require the large capital outlays in hardware to deploy their service or the human expense to operate it. The enterprises need not be concerned about overprovisioning for a service whose popularity does not meet their predictions, thus wasting costly resources, or under provisioning for one that becomes wildly popular, thus missing potential customers and revenue. This elasticity of resources, is one of the main features of cloud which has made it so popular. [4]. The main attributes of cloud are: on demand provisioning of resources, underlining infrastructure and software s are offered as a service, infrastructure is scalable and flexible, 1

11 provisioning and de-provisioning of resources are elastic and pay per use of computing resources and workloads. Cloud computing has categorized its services in to three different types which are Software as a service, SaaS, Platform as a service, PaaS and Infrastructure as a Service, IaaS. In SaaS the cloud offerings provides already created running applications on cloud infrastructure. PaaS provides a platform allowing customers to build and deploy applications and services using various tools and programming languages supported by the provider. Here the customer has no complexity of building and maintaining the infrastructure. In cloud computing, IaaS is a type of service where the virtualized computing resources are provided by a third party provider hosts over the internet. Cloud computing comes in three forms: private cloud, public cloud and hybrid cloud. In public cloud, the infrastructure and services are made available to the customers over a network, usually through internet. In a public cloud there is a greater level of efficiency in shared resources but is more vulnerable than private cloud in terms of security [5]. In Private cloud, infrastructures are maintained in companies own data centers which are managed internally or by a third party. Hybrid cloud is a combination of both private and public cloud. Basically, the infrastructure is on companies own data centers but relies on public cloud when it requires a higher level of computation. The main advantage of using a public cloud service provider is that, one need not worry about deployment and management of resources. All the hardware, applications and bandwidth costs are covered by the provider. There is no wastage of resources as one pay for what they use [6]. Some of the popular public cloud providers are Amazon web services[7] and Microsoft azure [8]. But these commercial clouds usually charge high price based on how many resources you use. In some cases, it becomes more suitable for enterprises to build their own cloud. They are also multi tenancy environments that are less secure and the customer has no control over the underlining technology. This is where open source cloud platforms fit in. It removes all the vender lock in s and is not proprietary. Customer has whole control over the underlining infrastructure. The main difficulty with open source platforms is that it requires high initial costs and skilled persons to do the job. Now with the emergence of open source it is important to choose the most suitable cloud platforms based on the specific characteristics of each platform and the one which meets the customer requirements. The performance study is important because understanding the application performance, boot up times will help the cloud users to choose the better solution from the list of various cloud DaaS offerings. 1.1 Problem Statement Desktop as a Service is a Software as a Service (SaaS) offering. While there could be several parameters which determines the performance and in turn the user experience, Boot up times are essential to improve the service offerings and there hasn t been enough work carried on it. Also, the impact of multiple scenarios which involve optimizing memory, CPU, throughput needs to be studied for efficient usage and improvement of this service offering. This is needed to resolve issues arising due to running CPU, memory etc. intensive applications. 1.2 Aim and Objectives 2

12 The aim of this research is to figure out the extensibility of public and private cloud offerings especially Desktop as a Service offerings. This was envisioned as a way of understanding the variations in performance and features provided by the service providers. The detailed objectives could be enlisted as below. To conduct a survey and investigate the problems associated with public cloud providers and shift towards open stack in cloud. To conduct a literature review on Amazon web service public cloud and have a detailed review on open stack. Testing the performance at both instance level and the virtual desktops infrastructure level. Testing boot up times and discussing various steps to launch an instance. Compare the results of open stack VDI and AWS Workspace in terms of CPU utilization, Average latency, Average IO size, throughput and IOPS. 1.3 Research Questions 1. How do the Amazon EC2 and Open stack nova compute Instance perform in terms of Boot up times? 2. How do the Desktop as a Service offerings based on AWS and in house Open stack perform in multiple scenarios in terms of metrics like CPU Utilization, Average Latency, and Throughput and etc.? 1.4 Contribution To understand the needs and extensibility of using OpenStack and public cloud like AWS, a survey was conducted to understand the requirements and needs of enterprises. A testbed was setup to have an on premise Desktop as a Solution model. Experiments were conducted to measure the boot up times at instance level on AWS and OpenStack Nova instances. Additionally, performance of these solutions was also measured in terms of various metrics in several cases like maximizing CPU usage, Memory usage etc. 1.5 Outline of the thesis Chapter 1 gives an introduction about public and private cloud and their offerings. It also presents the use cases and contains the problem statement, research questions and the contribution of this thesis work. Chapter 2 gives an account of the virtualization, cloud computing and the various models that are offered under it. A brief description is given regarding AWS, Open stack and the Desktop as a Service offerings under them. Chapter 3 gives details about the experiment environment, setup used and also the parameters used for the experiment. Chapter 4 presents the results of the experiment. They are followed by relevant explanations and observations. Chapter 5 has brief description about the research work carried out earlier in this field. Chapter 6 has the conclusions derived from this work. It also gives directions for future work. 3

13 2 BACKGROUND Virtual Desktop Infrastructure (VDI) is a service that hosts user desktop environments on a virtual machine running on a centralized server. VDI is a variation on the client/server computing model, sometimes referred to as server-based computing. Usually these types of services do not use cloud environments because few clouds support such a demanding workload for user-facing applications [9]. As cloud environments become more robust, vendors are starting to provide services that provide virtual desktops on cloud. Some of the popular VDI solutions are Amazon Workspaces [10] and VMWare Horizon[11]. VDI frees the user from Operating System stigma usually attached with a single machine. It provisions the centralized management of desktop environments and thereby rapidly propagating the changes and updates in a production environment. These are usually offered in the form of applications where one can have an experience of using multiple OS within a single hardware machine. Also it facilitates enhanced management from the operator s perspective giving them an overall view and ability to manage hundreds of users. This also has a significant benefit in terms of licensing costs and also gives a greater mobility for the users to use these services on devices of his choice [12]. 2.1 Virtualization In the field of IT, major transformation of technology has occurred after the introduction of virtualization technology. It has been the key enabler in the development and the usage of cloud computing. According to[13] Virtualization is the simulation of the software and hardware upon which other software runs. This simulated environment is called a virtual machine. There exists several type of virtualization in the domains of network, storage and server. Prior to Server Virtualization, there was a tight coupling between OS and the underlying hardware resource. It was not feasible to implement multiple OS and often it lead to inefficient utilization of the hardware resources. This was in fact a bottleneck and often led to cost escalation. With virtualization, an abstract layer of the underlying resources could be achieved and hence decoupling of OS from hardware. This enabled rapid provisioning of Virtual machines which could be started and stopped based on usage and requirements [14]. 2.2 Cloud Computing According to NIST Cloud computing is a model which provides on demand access of shared resources over a network, generally over the internet. These resources can be rapidly provisioned with minimum management overhead. [15]. Cloud services in general are offered in three different levels, namely: SaaS, Software as a Service, PaaS, Platform as a Service and IaaS, Infrastructure as a Service. The Cloud computing services popularity is because of elasticity and pay per usage policies. Resources can be scaled up or down based on usage and requirements. Thereby, reducing capital expenditure investment. 4

14 Figure 2: Cloud Computing Model [16] 2.3 Cloud Computing Models SaaS, Software as a Service. The capability provided to the consumer is to use the provider s applications running on a cloud infrastructure. The consumer does not manage or control the underlying cloud infrastructure including, network, servers, operating systems, storage, or even individual application capabilities, with the possible exception of limited user specific application configuration settings. This lets the consumer free from handling licensing costs, ensuring that the system is up to date with the latest software patches etc. The consumer can directly consume the services by paying for the level of services he consumes. This is gaining popularity for the sake of simplicity it involves [15]. Examples of SaaS include Salesforce [17], Google Apps etc. PaaS, Platform as a Service. In Platform as a Service model, the end user is provided with the platform required to build his applications. This is optimized for developers to build their applications giving them the required tools and libraries for standard development environments like.net, Ruby, Java etc. The customer still need not manage the underlying cloud infrastructure like network, storage, compute resources etc. and they are all handled by the platform itself. Yet he can exercise control over configuration and deployment settings [15]. Examples of this model include AWS Elastic Beanstalk[7], Google App Engine [18], and etc. IaaS, Infrastructure as a Service. This is the service where the customer has the greatest control over the infrastructure. He can provision and limit the resources which best suits his environment and usability. Ability to allow traffic, Access Control Lists, Security Groups etc. are within the control of the customer. It is up to choice of the customer to choose the operating system and making any updates or changes to the underlying configuration settings. This is typically used by consumers who are willing to deploy or shift their on premise infrastructure to the cloud [15]. Some of the popular IaaS services include AWS EC2 [19] and Microsoft Azure [8]. 5

15 2.4 Cloud Computing Deployment Models Private cloud In this deployment model the cloud platform is implemented in the organizations own datacenters. This cloud deployment is provisioned only for a single enterprise. In this model the cloud services are owned by a single organization which is managed by them or a third party [15]. Public cloud Public cloud is a cloud platform where the infrastructure is provisioned to use by general public. In this deployment model the infrastructure is owned and managed by cloud service provider [15]. Hybrid cloud Hybrid cloud model is a combination of both private and public cloud. Generally the organization use their own infrastructure for the cloud deployment but use public cloud services where there is a scarcity of resources. The different cloud models are bound together with standardized technology that provisions data and applications portability [15]. 2.5 Desktop as a Service DaaS, Desktop as a Service allows clients to pay for what they are actually using rather than the site license for the entire database [20]. DaaS is among one of the major services in cloud and it is similar to IaaS because a user utilizes infrastructure in both cases. However, IaaS provides CLI but DaaS provides GUI and can be directly used without the need for any installation process. DaaS is a service of remotely providing the desktop of an instance in the cloud. With DaaS, the users can be more productive they now have an easy single login access to any of the applications they need from anywhere, since they want with an uncompromised user experience that seamlessly moves with them and compatible with any device they prefer to use. It is more flexible because user tasks are executed in server-side, only PC is not available but thin-client, tablet and smart phone are also available. 2.6 Amazon Web Services EC2-Elastic Computing Cloud Amazon elastic compute cloud is defined as a web service that provides resizable compute capacity in the cloud. It provides easily accessible web services, API to manage and deploy virtual machine instances in the AWS virtual computing environment. In amazon, the time required to boot a new server instances up and running has reduced to minutes, allowing you to scale up and scale down the capacity easily with the change in the requirement. Considering the performance characteristics of an instance, Amazon has divided its environment in to various instance families (compute optimized, storage optimized and memory optimized). Amazon pricing model has changed the economies of computing by allowing the user to pay only for the resources that you actually use[21] WorkSpaces Amazon WorkSpaces is a Desktop as a Service utility and is hosted in a public cloud. Public cloud offers an easy way to provide a cloud-based desktop experience to the end-users. The user has a choice to explicitly select the amount of CPU, memory and storage required for his desktop client. The user can use these services through a thin client and connecting to 6

16 it. It is worth to note that this is available across several devices like PC, Mac, Desktop or even mobile platforms like ipad or Android thereby giving the end user wide options to utilize the services[22] Workspaces also provides features to integrate services like Active Directory for managing large number of users. The integration could also be done through a secured VPN connection. This is particularly useful for enterprises as they can securely enable their employees to use this services while giving them flexibility and mobility[22]. This also has an advantage of using bring your own device (BYOD) model as the medium enterprises which cannot afford much can reduce their expenditure on hardware provisioning. There is also a good benefit in the sense that data would remain within the corporate infra premises and can satisfy any regulatory requirements. Moreover as the data does not reside in one s machine locally, storing it in secure way and backing up is a hassle less process[22]. 2.7 OpenStack OpenStack is one of the most popularly used open-source software in computing world. OpenStack project has been created with an objective of being the pervasive software choice that can be used to convert compute and storage hardware into an IaaS Cloud [23]. OpenStack project was proposed by Rackspace and NASA and is released under Apache 2.0 License. It is an open source cloud computing software that allows IT to control large pools of compute, storage and networking resources throughout a datacenter. Focusing on every particular aspect of the datacenter, OpenStack software consists of several independently developed components with well-defined APIs. OpenStack design architecture typically uses the seven following components[24]: OpenStack Compute (nova) OpenStack Block Storage (cinder) OpenStack Object Storage (swift) OpenStack Image service (glance) OpenStack Identity (keystone) OpenStack Networking (neutron) or legacy networking (nova-network) OpenStack dashboard (horizon) Open stack Orchestration (heat) 2.8 Various components in OpenStack OpenStack Compute (also called Nova): Nova is the Computing Fabric controller for the OpenStack Cloud. It is similar to Amazon Web Services -EC2. All activities such as provisioning and life-cycle management of VMs within the OpenStack cloud are handled by Nova. It supports most available hypervisors such as VMware ESX and ESXi, Microsoft Hyper-V, Xen and KVM. But, Nova itself does not provide any virtualization capabilities but uses libvirt APIs to interact with the hypervisors. It is the project that ultimately spins up desktops or servers[25]. OpenStack Block Storage (Cinder): Cinder, is a Block Storage service for OpenStack and is an extraction of nova-volume. This has the ability to virtualize pools of block storage devices. It provides end user with a self-service API to request for the storage volume without the awareness of where it is actually deployed[26]. 7

17 However, cinder is the most suitable project for Desktop workloads. It provides a persistent volume which is an important criteria for desktops, it can be attached to a running instance and now the instance boots up from the volume attached. OpenStack Object Storage (swift): Swift provides a distributed, eventually consistent and scalable virtual object store for OpenStack. It is similar to amazon web services - simple storage service (S3). Swift is capable of storing billions of objects efficiently, safely, and cheaply that are distributed across nodes [27]. OpenStack Image service (glance): Glance is an imaging service which provides the users the provision to upload and discover data assets that are meant to be used with other services. Glance is a virtual Desktop image repository which Handles imaging. These image services include discovering, registering, and retrieving virtual machine images. The Images which are made available are from glance can be stored in anywhere from file systems to object storage (swift)[27]. OpenStack Identity (keystone): Keystone is an authentication system that provides Identity, Token, Catalog and Policy services for use specifically by services in the OpenStack family. It is similar to IAM service in AWS[28]. OpenStack Networking service (neutron): Neutron is an OpenStack project to provide "networking as a service" between interface devices like vnics managed by other Open stack services (e.g., nova)[29]. It provides tools that can build per-tenant private networks, which comes in handy for multi-tenant environments. Open stack Dash board service (Horizon): Horizon is the Open stack dashboard project. It provides a web based UI on top of your OpenStack cloud, where you can deploy instances, create snapshots, networks, and more[30]. Figure 3: OpenStack Architecture Model[31] 8

18 2.9 Various Components to build an Open stack VDI Hypervisor: OpenStack supports a wide range of hypervisors. But among them open source hypervisor for building a VDI solution can become a viable and cost effective approach. But, when it comes to the features needed to successfully manage DaaS, the feature sets provided by any of the hypervisors are adequate[32]. Display protocol: A display protocol is required for a VDI solution as it provides the end users with graphical interface to view their virtual desktops which run in the service providers servers or in cloud. Teradici PCoIP, Spice, HP RGS, Microsoft RDP are some of the popular protocols. Choosing an appropriate display protocol is very essential as it provides the end user with visual graphics for the virtual desktops. Also, Complex workloads often require complex visualization. Connection Broker: A Connection broker is a component which lets the user connected to the virtual desktop. It generally acts as an intermediate between end user and the OpenStack cloud. The connection broker which can handle every use case depends on various factors like Windows/Linux desktops, display protocol and the type of client devices. A connection broker is a management product which enable automated deployment and provision of remote desktops. [32]. Figure 4: Various Components of a VDI 2.10 Performance Monitoring Performance Monitor is a built in GUI based monitoring tool in Windows 7, Windows Server 2008 R2, Windows Server It provides a visual display of the in built Windows performance counters such as processing time, memory etc.., the representation is either in real time or as a way to review historical data. Performance counters are used to provide information as to how well the operating system or an application, driver is performing. The counter values can help determine system bottlenecks to tune up and down the application performance. As performance can affect the resource usage in a system, continues monitoring 9

19 of counter data such as operating system, network, and devices that an application can consume can give a clear picture of how well the system is performing.[33] The process of adding counters follows a single step which is dragging and dropping or also by creating Data Collector Sets. It has a feature of multiple graph view that lets user to visually review performance log data. The user can create custom views in Performance Monitor that can be exported as Data Collector Sets with performance as well as logging features Performance Metrics 1. CPU Usage: CPU utilization refers to a system usage of processing resources, or it is the amount of workload handled by the CPU. It is one of the main core component of performance. 2. Memory Usage: Memory usage refers to the temporary storage of a system for immediate use. 3. Average IO size: This means the size of a request. IO size is inversely proportional to IOPS because as the IO size increases the no of IOPS decreases. 4. Average Latency: Latency is referred as the delay occurred in processing a request or it is also known as how fast a single I/O-request is handled. 5. IOPS: This means the amount of read and write operations that could be handled per second. 6. Throughput: Throughput is the multiplication of average IO size with Inputs and outputs operations per second. It is usually used to measure device performance. 10

20 3 EXPERIMENT METHOD In this chapter, we discuss about the methodology we adopted during the course of this work. We present the framework and results of our survey. Also the setup for the experiment, metrics used for measurement are explained. A brief description about the tools and the environment is also presented so as to gain an understanding of this work. The methodology of this work includes a three stage procedure which is represented in fig 5. Figure 5: Experimental Methodology 3.1 Survey A survey was chosen as an adopted model to identify the adoption of Cloud in enterprises and various challenges faced while using them. It also gave a scope to understand their requirements, the reasons for choosing public and private cloud based services. This was real time scenario based survey and helped us to understand and mark the key differentiators, which finally helped in understanding their adoptability. Based on results obtained, AWS, Amazon Web Services and OpenStack were the cloud based services that were chosen because of their wide usage and extensive availability of documentation. Also, we had resources from which we could setup and test an experimental platform/testbed and understand the flexibility they provide both in terms of migration and usability. The results of the survey are included in the appendix section. 3.2 Experimental Environment Hardware specifications OpenStack Server: The platform that was used is based on Intel Xeon CPU with enterprise class hardware specifications. The hardware and software specifications are listed below. Processor 2.4 GHz Intel Xeon Cache 30 MB RAM allotted 16 GB OS Cent OS Hard Drive 1 TB Table 1: Experiment Environment The above hardware was used for building Open stack framework. Version used for Open stack is kilo. 11

21 3.2.2 For OpenStack VDI The virtual desktop instance running on top of OpenStack has the following specifications: Processor Cache RAM allotted OS Hard Drive 2.4 GHz Intel Xeon 30 MB 4 GB Windows 2008 Server R2 SP1 50 GB Table 2: Experiment Environment The above hardware was used for building Open stack framework. Version used for Open stack is kilo For AWS WorkSpaces Instance level specifications for Workspaces Processor 2.4 GHz Intel Xeon Cache 30 MB RAM allotted 4 GB OS Windows 2008 Server R2 SP1 Hard Drive 50 GB Table 3: Experiment Environment Amazon Workspaces is a Desktop as a Service model provided by AWS EC2 - M4 large instance specifications Processor 2.4 GHz Intel Xeon Model(Instance Type) m4.large vcpu 2 Mem (GiB) 8 SSD Storage (GB) EBS-only Table 4: Experiment Environment 3.3 Performance metrics Experiment 1 Boot up times: When an instance is initiated, boot up time is the amount of time the system takes to complete the entire boot cycle. It could be dependent on several factors such as Operating system, Image size, type of instance (large or small). Often, large instances with large image sizes take more amount of time to get started. Steps to calculate boot up times in EC2 and OpenStack Configuring the security groups in AWS and OpenStack in such a way that all inbound traffic for http port 80, SSH port 22 and ICMP ports from need to be kept open. Initiating the launch of a new instance and continuously trying to send http, SSH and ping requests to the instance. 12

22 The running state of an instance is confirmed by the first successful response to the request sent through http, SSH and ping requests Experiment 2 In order to assess the performance of the virtual desktops on top of AWS and OpenStack we have chosen few metrics such as CPU Usage, Memory Usage, Average IO size, and Average Latency, throughput and processor queue length. Perfmon, an SNMP based monitoring tool was used for measurements. It provides a simple interface to monitor and collect data in the form of counters. The measurements were taken for 5 iterations so as to improve the reliability and accuracy of the results obtained. Following is the brief descriptions for each of the metrics and how they are calculated. 1. CPU Usage: We measure the extent to which CPU was utilized while running applications. It is calculated from the counters values of % Processor (_Total) / % Processor Time and Processor (_Total) / % User Time. Also, CPU Usage is the average of the above two counter 2. Memory Usage: This metric gives the total consumption of system RAM. The counter value used for determining memory usage are \Memory\Available Mbytes. The percentage memory usage is calculated on the basis of this counter value. 3. Average IO size: We have considered only a single logical disk for measuring IO size. The counter name representing average IO size is Average Disk Bytes/Transfer. 4. Average Latency: Latency is measure of time taken to process our requests. The counter Average Disk sec / Transfer was used for representing average latency. This is the counter used to measure IO latency. 5. Throughput: This gives the total amount of data processed which include reading from disk (read operations) and writing to the disk (write operations). Disk Bytes / sec counter was used for measuring throughput. They were stored usually for a sample rate of 1 sec. 6. IOPS: It gives an insight into measuring the number IO requests made to the disk, Disk Transfers / sec was the counter which specifies IOPS. 3.4 Experimental Setup OpenStack One of our aims was to identify the feasibility of running Desktop as a Service based model by using popular open source tool like OpenStack. For this a server with Intel Xeon CPU was used. The guest operating system was CentOS 6. On top of that, we ran KVM hypervisor which is a type 2 hypervisor. As KVM is a para virtualization hypervisor it does not sit directly on top of hardware so a thin operating system such as CentOS is installed between KVM and the hardware layer. This was done to create a virtualized environment. OpenStack was deployed on this hypervisor. Microsoft Windows is a popular desktop operating system. We selected Windows 2008 R2 SP1 OS as the host operating system and ran this inside OpenStack environment. The VDI solution is built on top of OpenStack, Fig5 shows the user is requesting for provisioning a VDI. For the user to access the OpenStack cloud an enterprise connection broker was used. Now an instance is created based on the specifications mentioned by the user. The user tries to access the VDI instance through a Remote display protocol which is spice protocol in this case. There are two experiments conducted as a part of our work. The first experiment is related to measuring the boot up time of an operating system. Second experiment is to setting up Desktop as a Service on top of Open stack. 13

23 3.4.2 AWS WorkSpaces Figure 6: Setup for OpenStack Environment AWS Workspaces is a fully managed Desktop as a Service. Since this is a managed service and based on Software. As a Service model, we will not have access to the core hardware on which there services are built upon. The instance level maintenance and security are handled by AWS themselves and only the service is exposed to the outside world. Typically, a Workspace runs on top of customized Xen based hypervisor that runs most of the systems at AWS. The installation procedure is mentioned in the appendix section. Figure 7: AWS WorkSpace Environment[34] 14

24 3.5 Experiment 1 The first experiment is related to measuring the boot up time of an instance. This is critical because desktop as a service model is highly dependent on the boot up times of the instance, they play a crucial role in time critical applications and auto scaling. and is directly related in improving the user experience. This however, can depend on several factors such as the guest and host operating system, the core hardware utilized, level of abstraction needed for the user and etc. Parameters: The following are the parameters considered for this experiment. Boot up time: Boot up time is calculated for 10 iterations and average is taken into consideration Operating Systems: Windows 2008 R2 and Fedora were considered for both AWS and OpenStack based instances. Instance types: M4 large for AWS and a similar M4 large flavor for OpenStack with similar specifications. Number of instances: Boot up times are also measured by launching n instances parallely. 3.6 Experiment 2 This experiment was conducted to test how the virtual desktop, at the user end behaves when put under various load test scenarios. This metrics were considered because when you identify server performance degradation, the usual suspects are CPU, Memory, and the Disk. In application performance, there are many kinds of applications which optimize high CPU, memory and disk individually, so the following test cases were constructed to meet the requirements. The OS considered for the virtual desktop is Windows 2008 R2 Desktop with specifications as shown in Table 3. The same configuration was given for the Windows 2008 R2 Desktop in both OpenStack VDI and AWS WorkSpace. Parameters: The following are the parameters considered for this experiment: CPU Utilization: It is measured in terms of processor time and user time in percentage. Memory Utilization: It is calculated based on available Megabytes Average IO size: It is measured in terms of Average Disk bytes per transfer Average Latency: It is measured in terms of Average Disk sec per transfer. Throughput: It is measured in terms of Disk bytes per sec. IOPS: It is measured in terms of Disk transfers per sec This experiment was divided into several stages. At each stage specific test case was performed and the performance of the Windows 2008 R2 virtual desktop running in OpenStack VDI and Aws WorkSpace was calculated Test 1 - Full load on CPU A bash script was written to completely use the processing power of the virtual desktop. Next performance metrics were calculated from the counter values generated from Perfmon. The script was provided in the appendix section Test 2 Full load on Memory 15

25 For test 2 memory for the virtual desktops was put under stress. A script was written which completely uses the memory of the instance Test 3 Write temp file In this test, Heavy load tool was used. The tools creates a temporary folder and continues writes functions with variable speed are written in the file Test 4 Simulation of Disc Accesses Simulate Disc access checks the reliability of the disk by giving number of hits on files and folders on all local disks. During this test case the counter values from Perfmon are stored with a sample period of ten second to calculate various performance metrics. 3.7 Assumptions For measuring the performance of the Desktop as a Service, we have chosen AWS Workspaces. It is a managed service and hence we do not have access to the underlying hardware. So, we ensured that the hardware specifications (measured internally) of the server running on Workspaces are very similar to the one used in OpenStack deployment. For measuring the boot up times we have tried to consider OpenStack instance type which is almost similar to the AWS. 16

26 4 RESULTS The following section presents the results of the experiments conducted. Brief discussion on the results is also presented thereafter. We have considered metrics such as CPU Utilization, Memory Utilization, Throughput, Average Latency, Average IO Size and IOPS under multiple scenarios. 4.1 Section 1 Section 1 details the results for Experiment 1. This section was divided in to two cases. Case1 deals with the Instance boot uptimes. In this experiment we measure the VM boot up time for two different OS.VM boot up time was calculated for two scenarios. In the first scenario, experiment was conducted while considering individual instances boot up time. The image which was considered was single VM, launched on top of OpenStack and AWS. And in the second 5 instances were launched in parallel, one at a time Case 1: Serial Launch In this case the boot up times are taken only for a single instance. For booting the Fedora instance, the time taken was much lower for OpenStack instance when compared to the EC2 instance. However in the case of Windows Server 2008 R2, the margin of difference was lower between the two. Table 5: VM's Serial Launch - Boot up Time It can be clearly interpreted that the average boot up time for a single instance is less in OpenStack when compared with EC2 instance Case 2: Parallel Launch For booting the EC2 public cloud instance, the average boot up time was lower when compared to the OpenStack instance. The boot up times for starting the Windows server 2008 R2 instance was nearly the same for the two. And we can also observe that the time taken for booting 10 instances in parallel took almost similar time compared to the average boot up time to launch a single instance. 17

27 4.2 Section 2 Table 6: VM's Parallel Launch - Boot up Times The following section presents the results of the experiments conducted. Brief discussion on the results is also presented thereafter. We have considered metrics such as CPU Utilization, Memory Utilization, Throughput, Average Latency, Average IO Size and Average IOPS under multiple scenarios. In figures mentioned in this section, series1 and series2 depicts the values for OpenStack VDI and AWS WorkSpace respectively Scenario 1: Full CPU Load In this scenario, we tried to increase the load for CPU to reach its maximum and evaluated several metrics to understand the behavior as described above. From the Figure it can be clearly seen that the CPU usage and memory usage for both Windows server 2008 R2 Desktop running on top of OpenStack and AWS VDI are constant though the usage was higher on AWS instance. While for analyzing throughput and average IO size, several spikes as can be seen. Throughput for AWS instance had decreasing nature while there was an increment for OpenStack instance. For IO size, OpenStack instance had near consistent values. Average Latency was lower in case of AWS instance and for IOPS, it was lower initially and the values gradually spiked up. From the graphs in Scenario 1 it can be clearly interpreted that OpenStack VDI solution performance in all the test cases was better compared to AWS workspace except for the average latency. 18

28 Figure 8: Graphs for Full Load CPU test case (Series 1 represents OpenStack and Series2 represents AWS) In this case the average latency, virtual desktop on OpenStack VDI had a very high value. Figure 9: Graphs for Full Load CPU test case (Series 1 represents OpenStack and Series2 represents AWS) Scenario 2: Full Memory Load In this scenario, we maintained the memory at full level and the metrics were later measured to identify their behavior. 19

29 Figure 10: Graphs for Full Load Memory test case (Series 1 represents OpenStack and Series2 represents AWS) From the above graphs, we can infer that there were drastic variations for CPU. With respect to the memory, constant values were attained after the first few sample times. Throughput and Average IO had few spikes only for few sample times. Average Latency was very low for AWS instance when compared to the Open stack instance. The behavior of IOPS was random, however it was still higher for the Open stack instance when compared to the AWS instance. In this test bed no general conclusion can be drawn. OpenStack behaved better in terms of throughput and IOPS while AWS workspace environment showed much better results for memory usage and average disk latency. We can clearly observe that the value for average latency is very high in OpenStack VDI. Figure 11: Graphs for Full Load Memory test case (Series 1 represents OpenStack and Series2 represents AWS) 20

30 4.2.3 Scenario 3: Write Test File In order to understand the behavior of the VM when faced with dwindling disk space, a heavy load tool was used. It created a temporary folder and then writes into the folder with variable speed. Figure 12: Graphs for Write Temp File test case (Series 1 represents OpenStack and Series2 represents AWS) From the above graphs, we can infer that there was variable CPU usage though it was less for OpenStack instance. With respect to memory, usage was initially very high for Open stack instance but went on decreasing while it was consistent for AWS instance. The Average IO for AWS instance was much higher when compared to Open stack instance. It was on contrary to the behavior for throughput where it was lower for AWS instance. Figure 13: Graphs for Write Temp File test case (Series 1 represents OpenStack and Series2 represents AWS) 21

31 The average latency for OpenStack instance was higher when compared to the AWS instance. Marginally, we can see several spikes for both the instances in terms of IOPS behavior. From above results we can easily infer that OpenStack performed much better in this test scenario though the average latency in OpenStack instance was much higher than in AWS. Even the throughput for AWS instance was better compared to OpenStack though the value was negligible Scenario 4: Simulation of Disk access In this scenario, heavy load simulates a number of hits and folders on all the disks available. It is done to identify how much load the hard disk can take and checks its reliability Figure 14: Graphs for Simulate Disc Access test case (Series 1 represents OpenStack and Series2 represents AWS) The CPU Usage for OpenStack instance was steadily higher when compared to the AWS instance. Memory usage was constant for the both the instances. Average IO and Throughput was low for both the instances when compared to the OpenStack instances. 22

32 Figure 15: Graphs for Simulate Disc Access test case (Series 1 represents OpenStack and Series2 represents AWS) Average Latency values were consistent for both the instance after initial spikes for OpenStack instance. The behavior for IOPS is largely random for both the instances. From the graphs in Scenario 4 we can easily conclude that OpenStack instance performed much better compared to that of AWS. There was considerably high difference in all the performance metrics. We can clearly observe that when the disk is put under stress OpenStack instance performs much better than the AWS instance. 23

33 5 ANALYSIS AND DISCUSSION 5.1 Related Work Although none of the works match exactly with this research work, the works similar or works which aided this work are presented in this section. There were several works that have been carried out previously in the study of cloud based amazon web services public cloud and Open stack private cloud individually. Some authors have a general insights on the problem those are[35] [36]. The right Desktop as a service solution remains an open field of study analyzing, optimizing, and understanding the performance of DaaS on cloud is still an active area of research. Ming Mao [37] et al conducted a study on startup time of cloud VMs across three realworld Public cloud providers Amazon EC2, Windows Azure and Rackspace. The main focus of the work has been on carrying out performance evaluation on various VM start up times. Coming to our work, Though the performance of the VM startup process has been mentioned in several research papers, they just served as part of the work to evaluate the overall performance for a cloud provider. But in our research we have considered boot up time to measure the performance of Desktop as a service solution in cloud. Shinichiro Kibe[38] et al. did an empirical study on Desktop as a Service in educational cloud. They had discussed on the performance features that affect DAAS and provided various solutions to overcome the lags for building a DAAS Solution. This paper provided clear insights on transforming from traditional desktops to desktop as a service. I. Voras [39] et al. has conducted a research on Evaluating Open-Source Cloud Computing Solutions. In this paper the author has compared various open source cloud solutions such as OpenStack, OpenNebula, and Eucalyptus and more, he also discussed the criteria that can be used to evaluate the stability, performance and features of open source clouds. Keith R. Jacksoet [40]et al did a research on Performance Analysis of High Performance Computing Applications on the Amazon Web Services Cloud. The author has done an attempt to run HPC applications in AWS ec2 and conventional data center according to Overall results indicate that EC2 is six times slower than a typical mid-range Linux cluster, and twenty times slower than a modern HPC system. The application performance is one of the tradeoff points to consider when shifting to cloud. There are very few works which were focused on Desktop as a service in cloud. Also much work has not been progressed in this field of study. Which this thesis has major focus on. 5.2 Answering Research Questions 1. How do the Amazon EC2 and OpenStack nova compute Instance perform in terms of boot up times? With respect to boot up times, instances were booted on Amazon EC2 instance and OpenStack Nova with different operating systems like Fedora and Windows 2008 R2. Initially one instance was booted up and then we increased the number of instances to five in parallel. It was noticed in our observation that boot up times for multiple instances was not in multiples of the time taken for booting one instance, rather it was much less. Once an instance is launched, remaining instances take much less time together as they are initiated based on the image of the first instance. Also the boot up times for launching OpenStack Nova instances was much less when compared to launch an EC2 instance. 24

34 2. How do the Desktop as a Service offerings based on AWS and in house OpenStack perform in multiple scenarios in terms of metrics like CPU Utilization, Average Latency, and Throughput etc.? For assessing the performance of the DaaS offerings, we considered metrics like CPU Utilization, Memory Utilization, Throughput, IOPS, Average Latency and processor queue length. Also for measuring these, multiple scenarios were considered such as Full CPU Load, Full Memory Load, Full writes to Disk. Insert and Select throughput were measured for both the databases and are discussed in detail with graphs under Part 1 in the results section. Performance of the databases is evaluated when data volumes are increased in terms of total time taken and average response time. From the results sections it can be interpreted that the CPU usage and memory utilization for all the test cases was less in the virtual desktop hosted on OpenStack VDI/DaaS than in AWS workspace. The average disk latency was considerably high in OpenStack VDI environment than in AWS workspace in all the test scenarios, though in test case stimulate disk access., average latency for OpenStack and AWS workspace environment was similar. The average IO size for all the test cases was higher in virtual desktop on top of OpenStack than in AWS workspace environment. There are no general conclusion drawn for IOPS as it was almost similar in both the environments. Conclusions based on the test beds run on virtual desktops on OpenStack VDI and AWS workspace are: For test cases of full load on CPU and memory there was no general conclusion. Both the environments behaved almost similar. But the average latency of the desktop running on OpenStack VDI was higher than the AWS workspace environment. In the test case of write temp file and simulate disk access OpenStack performed much better than the desktops on AWS workspace. But the average latency even in these scenarios was higher in OpenStack VDI/DaaS. 25

35 6 CONCLUSION AND FUTURE WORK The purpose of this thesis work was to explore the desktop as a service solutions in cloud. In order to do that, we chose to evaluate Amazon Workspaces and an in house solution deployed on top of OpenStack. The impact on various metrics such as CPU, Memory and Network was studied so that we could arrive as how to improve the service offerings. It was concluded that OpenStack VDI performed better in most of the test scenarios. In terms of boot up time there was no drastic change compared with AWS but OpenStack instance boot up times were much less. In terms of instance performance OpenStack instance performance was better. Though there were some cases where AWS performance was good but it was negligible. Another conclusion was that AWS performed better in terms of average latency whereas OpenStack instance showed better CPU and memory performance in all the test scenarios. 6.1 Future Work The scenario of DaaS is rapidly increasing. One of the main parameter which is seen extremely important is the quality of service. It depends on different factors such as latency and response times. This can be done by comparing various display protocols used in VDI solutions. Also future experiments can be performed by finding the number of virtual desktops that can be launched per host. And also the reliability of the offline VDI solutions for various technologies can be performed. 26

36 7 REFERENCES [1] K. Beaty, A. Kochut, and H. Shaikh, Desktop to cloud transformation planning, in Parallel & Distributed Processing, IPDPS IEEE International Symposium on, 2009, pp [2] What is Desktop as a Service (DaaS)? - Definition from WhatIs.com. [Online]. Available: DaaS. [Accessed: 09-Sep-2015]. [3] VDI still on the rise, Wikibon Blog.. [4] M. Armbrust, A. Fox, R. Griffith, A. D. Joseph, R. Katz, A. Konwinski, G. Lee, D. Patterson, A. Rabkin, I. Stoica, and others, A view of cloud computing, Commun. ACM, vol. 53, no. 4, pp , [5] What is public cloud? - Definition from WhatIs.com. [Online]. Available: [Accessed: 09-Sep- 2015]. [6] J. Peng, X. Zhang, Z. Lei, B. Zhang, W. Zhang, and Q. Li, Comparison of several cloud computing platforms, in Information Science and Engineering (ISISE), 2009 Second International Symposium on, 2009, pp [7] Amazon Web Services (AWS) - Cloud Computing Services. [Online]. Available: [Accessed: 09-Sep-2015]. [8] Microsoft Azure: Cloud Computing Platform and Services. [Online]. Available: [Accessed: 09-Sep-2015]. [9] Desktop-as-a-Service - OpenStack Architecture Design Guide - current. [Online]. Available: [Accessed: 09-Sep-2015]. [10] AWS Amazon WorkSpaces. [Online]. Available: [Accessed: 09-Sep-2015]. [11] VDI Virtual Desktop Infrastructure with Horizon (with View) VMware India. [Online]. Available: [Accessed: 09- Sep-2015]. [12] What is VDI?, Citrix.com. [Online]. Available: [Accessed: 09-Sep-2015]. [13] K. Scarfone, Guide to security for full virtualization technologies. DIANE Publishing, [14] U. Gurav and R. Shaikh, Virtualization: a key feature of cloud computing, in Proceedings of the International Conference and Workshop on Emerging Trends in Technology, 2010, pp [15] P. Mell and T. Grance, The NIST definition of cloud computing, [16] Q. Zhang, L. Cheng, and R. Boutaba, Cloud computing: state-of-the-art and research challenges, J. Internet Serv. Appl., vol. 1, no. 1, pp. 7 18, [17] Cloud Computing, Customer Relationship Management, (CRM) - Salesforce.com India, Salesforce.com. [Online]. Available: [Accessed: 09-Sep-2015]. [18] App Engine - Run your applications on a fully-managed Platform-as-a-Service (PaaS) using built-in services Google Cloud Platform. [Online]. Available: [Accessed: 09-Sep-2015]. [19] What Is Amazon EC2? - Amazon Elastic Compute Cloud. [Online]. Available: [Accessed: 09- Sep-2015]. [20] T. Dillon, C. Wu, and E. Chang, Cloud computing: issues and challenges, in Advanced Information Networking and Applications (AINA), th IEEE International Conference on, 2010, pp [21] J. Varia and S. Mathew, Overview of amazon web services, Jan-2014,

37 [22] What is Amazon WorkSpaces? - Amazon WorkSpaces. [Online]. Available: [Accessed: 09-Sep-2015]. [23] M. Bist, M. Wariya, and A. Agarwal, Comparing delta, open stack and Xen Cloud Platforms: A survey on open source IaaS, in Advance Computing Conference (IACC), 2013 IEEE 3rd International, 2013, pp [24] OpenStack components - OpenStack Architecture Design Guide - current. [Online]. Available: [Accessed: 09-Sep-2015]. [25] F. Wuhib, R. Stadler, and H. Lindgren, Dynamic resource allocation with management objectives Implementation for an OpenStack cloud, in Network and service management (cnsm), th international conference and 2012 workshop on systems virtualiztion management (svm), 2012, pp [26] Cinder - OpenStack. [Online]. Available: [Accessed: 09-Sep-2015]. [27] Welcome to Swift s documentation! swift dev20 documentation. [Online]. Available: [Accessed: 09-Sep-2015]. [28] Keystone - OpenStack. [Online]. Available: [Accessed: 09-Sep-2015]. [29] Neutron - OpenStack. [Online]. Available: [Accessed: 09-Sep-2015]. [30] Horizon: The OpenStack Dashboard Project horizon b4.dev50 documentation. [Online]. Available: [Accessed: 09-Sep-2015]. [31] A quick overview of OpenStack technology - Thoughts On Cloud. [Online]. Available: [Accessed: 09-Sep-2015]. [32] New Blueprint Serves as Guide for Building OpenStack VDI and DaaS Press Release Leostream. [Online]. Available: [Accessed: 09-Sep-2015]. [33] AnandK@TWC, Perfmon or Performance Monitor in Windows 10 / 8 / 7, The Windows Club.. [34] Amazon WorkSpaces 導 入 支 援 サービス, 株 式 会 社 プログマインド. [Online]. Available: [Accessed: 09-Sep-2015]. [35] A. E. S. Ahmed, A. K. Alsammak, and E. Algizawy, A New Approach to Manage and Utilize Cloud Computing Underused Resources, Int. J. Comput. Appl., vol. 76, no. 11, pp , [36] X. Wen, G. Gu, Q. Li, Y. Gao, and X. Zhang, Comparison of open-source cloud management platforms: OpenStack and OpenNebula, in Fuzzy Systems and Knowledge Discovery (FSKD), th International Conference on, 2012, pp [37] M. Mao and M. Humphrey, A performance study on the vm startup time in the cloud, in Cloud Computing (CLOUD), 2012 IEEE 5th International Conference on, 2012, pp [38] S. Kibe, T. Koyama, and M. Uehara, The evaluations of desktop as a service in an educational cloud, in Network-Based Information Systems (NBiS), th International Conference on, 2012, pp [39] I. Voras, B. Mihaljević, M. Orlić, M. Pletikosa, T. Pavić, K. Zimmer, V. Paunović, S. Tomić, and others, Evaluating open-source cloud computing solutions, in MIPRO, 2011 Proceedings of the 34th International Convention, 2011, pp [40] K. R. Jackson, L. Ramakrishnan, K. Muriki, S. Canon, S. Cholia, J. Shalf, H. J. Wasserman, and N. J. Wright, Performance analysis of high performance computing applications on the amazon web services cloud, in Cloud Computing Technology and Science (CloudCom), 2010 IEEE Second International Conference on, 2010, pp

38 [41] performance - How can I produce high CPU load on Windows? - Super User. [Online]. Available: [Accessed: 04-Oct-2015]. 29

39 8 APPENDIX 8.1 Survey Results 30

40 31

41 32