Providing Scientific Software as a Service in Consideration of Service Level Agreements

Transcription

1 Providing Scientific Software as a Service in Consideration of Service Level Agreements Oliver Niehörster 1, Georg Birkenheuer 1, André Brinkmann 1, Dirk Blunk 4, Brigitta Elsässer 2, Sonja Herres-Pawlis 2, Jens Krüger 2, Julia Niehörster 3, Lars Packschies 4, and Gregor Fels 2 1 Paderborn Center for Parallel Computing, Universität Paderborn, Germany 2 Department Chemie, Universität Paderborn, Germany 3 Department Agricultural Sciences, Universität Hohenheim, Germany 4 Universität zu Köln, Germany Abstract Software-as-a-Service (SaaS) is an interesting concept for providing scientific applications. Service providers in this field have to either run their own data centre or to outsource the applications to a third party provider. In both cases, they want to optimally utilize resources without violating service level agreements (SLAs) negotiated with their customers. This work analyzes the properties of scientific applications. We found that there are essential differences between scientific and business applications. We also build up an own cloud and analyze Gromacs 4, a common scientific application. This shows that for fulfilling an SLA, it is often possible to provide different configurations, using either a few high performance machines or multiple smaller machines. Furthermore, experiences of the performance behavior gained in physical environments must not necessarily apply in virtual environments. Fulfilling SLAs also depends significantly on the mapping of the virtual machines to physical hosts. 1 Introduction Cloud computing provides a convenient access to virtualized data centers over the Internet at different levels (see Figure 1). Infrastructure-as-a-Service (IaaS) providers SaaS PaaS End Users Application Developers IaaS IT- Architects Fig. 1: Different interfaces to cloud computing.

2 offer the service to run complete virtual infrastructures (VIs) with many connected virtual machines (VMs). Hypervisors are used for the hardware abstraction, enabling customers to deploy their own environments, including the operating systems and software solutions of their choice. Plattform-as-a-Service (PaaS) providers offer pre-configured environments for applications developers, including web server packages as well as cloud-libraries. Software-as-a-Service (SaaS) providers target the end-user of applications. Today SaaS providers concentrate on business and office software and offer online word processors, Content-Management, Customer-Relationship-Management, or Human-Resource-Management applications. In our work we focus on scientific software. We present the properties of different scientific applications with the focus on the mentioned aspects. We have started a survey at different scientific institutes. The scientists have to categorize and describe the applications they use in their research. For a proof of concept we have built up a private cloud and provide Gromacs-asa-Service to our users. Gromacs was chosen because the survey shows that it has all required properties mentioned before. 2 Related Work Cloud computing providers and software vendors are already offering a huge set of solutions on the different cloud computing levels. Amazon with its EC2-infrastructure currently runs the most prominent IaaS-infrastructure, while Eucalyptus and OpenNebula provide solutions to build IaaS-environments [4][8][9]. The MapReduce programming paradigm or the BigTable environment, a compressed database system built on Google File System (GFS), as well their open source equivalents offer application designers powerful tools on the PaaS-level [3][2][6]. Both, enterprise business solutions like Salesforce and scientific solutions, like Mathematica or Matlab are providing software, which are tailored to their end users. Keahey and Freeman describe a first evaluation project with the subject to provide IaaS offers to a scientific community. They build a science cloud testbed based on the Nimbus CloudKit at the universities of Chicago and Florida and offer it to the scientists [7]. Another project, called CUMULUS, with the focus on building a scientific cloud is presented by Wang et al. [11]. They merge existing grid infrastructures with cloud technologies by building a frontend service that unifies OpenNebula and GLOBUS. Typical use cases mentioned by them are the setup of training centers and providing virtual infrastructures or complete virtual computing centers to a scientific community. Cacheiro et al. analyze the behaviour of different scientific applications in cloud environments [1]. They do low-level benchmarks where the performance of specific system components was evaluated as well as performance benchmarks on application level. Analyzed are common scientific applications like Gromacs [10] and Gaussian. Their results show that the performance in virtual environments is very similar to the one in physical with the two exceptions: I/O operations and message passing interface (MPI) latency.

3 3 The Survey Generally we try to answer the question whether or not it is possible to provide scientific applications in consideration of SLAs and if there are technical reasons of today s absence of scientific SaaS offerings. We started a survey to find general properties of scientific applications. The interviewed scientists named and classified their applications. We focus on the temporal behavior and the ability to change the number or the configuration of the compute nodes during runtime as well as the kind of status providing. In a second part we asked more precise questions on these subjects. 3.1 Categorization At first the respondents classified their applications in different service categories coping with the main fields of behavior of activities. Continuously running applications are driven by a business case where the resource user sells another service and permanently waits for customers. This business case is widespread in Web 2.0 and Internet shopping. The requirements for a quality of service (QoS) here are the availability of the application and the number of requests per second that the web server is able to handle. This work labels this class of activities as service. Scientific software, in contrast, is mostly based on atomic activities, which are executed to provide a certain result and disappear when finished. They are typically used for parameter studies or simulations, which are the daily work of scientists. The first class of atomic activities is characterized in a way that they should calculate the result as fast as possible and meet a given deadline. This deadline driven activities can occur as single execution i.e. a simulation, bulks of batch jobs, or even workflows with time constrains. We will call this class of activities batch job. In contrast to the batch jobs, a second class of atomic activities has no deadline. The jobs in this class cope with problems, where it is uncertain if and in case yes when they will end. In this scenario, a user buys resources according to the time he guesses the job might take. The only QoS requirement here is that the infrastructure should calculate as fast as possible compared to a reference installation. This class of activities is labelled as experiments. The QoS requirements are very different for these three use cases and the service level objectives (SLOs) have to be based on different criteria. The service provider as well as the customer have to be able to monitor these criteria and available monitoring tools mostly depend on the executed application. There can be applications, which have their own key performance indicators (KPIs) and for instance implement a progress bar that informs the user about the computation speed and let him calculate the estimated end time of the job. This can be used to check a deadline aligned with the customer. Another group of applications may not give a direct feedback, but can be evaluated by a benchmark program. In this case, the SLOs are formulated based on the benchmark results. Initially, the provider performs the benchmark in an offline phase before the SLA negotiation. The provider measures the performance he can provide compared to the customer needs, service price, and provisioning costs.

4 For a third group of applications, a mapping between system parameters can be used to formulate an estimation function. SaaS can support all these three categories: service, batch job, and experiment. However, the service activity category is the typically use case in a cloud environment for business applications and this work will only use it as comparison with the other types. The results of this first part of the survery are shown in Figure 2. Kind/Method Live progress status Permanent Service Batch Job Plabso4, R, Gromacs, NWChem Benchmark Scien;fic Gromacs Scien;fic Es9ma9on Func9on Business Gaussian Unknown Experiment Gromacs Gromacs Gaussian SAS, ASReml Fig. 2: Classification of the used applications. 3.2 Further Questions We asked further questions to get a deeper understanding about the applications denoted and classified in chapter 3.1. The questions can be classified in the four topics Parallelism, Input, Progress and Checkpoint and Restart. The application behaviour mainly depends on input parameters. They affect whether the calculation will be a short, a long running job, or if it will not finish at all. We ask how the inputs are given. Possibilities are interactive inputs during runtime or input streams generated by other applications. In all mentioned applications the inputs are initially given. The input size ranges from only a few kilobytes (for example in R) up to 2 GB (in ASRempl and SAS). Questions of the topic Parallelism focus especially on the support of horizontal scaling. Scientific applications are often designed to run in parallel. The number of parallel instances is mostly one of the start parameters. Gromacs, NWChem and Gaussian handle the distribution of jobs themselves. In Plabsoft, SAS and R the user has to distribute the jobs manually. Applications of the first kind of software have their own load-balancer and scheduling algorithms specifying the parallel behaviour. The users indicate that this behaviour does not change during the runtime. Non application that supports the adding of nodes during runtime was mentioned. Figure 2 shows that Plabsoft, R, Gromacs and NWChem provide progress status during the calculation time. We asked detailled questions aimed at this. The answers

5 show that the status information is mostly not reliable. In NWChem the progress can be seen at the written output data and an experienced user can guess the residual runtime. In Gromacs the estimated end-time alternates extremely at job start and after about 500 iterations the calculation begins to converge. Checkpointing is another important feature of scientific applications. A common SLO in scientific calculations is a deadline until the calculation must be finished. Scheduling algorithms can make use of this feature to fulfill such SLOs. Another important field of application is doing backups by saving intermediate results. There are applications writing checkpoints continuously (R and NWChem) or on user requests (R and Gromacs). 4 Analyzing The overall goal is to provide scientific software as a service in consideration of SLAs. Properties like an online status providing, the support of checkpointing, and the mentioned properties of a batch-job simplify the compliance of SLAs based on deadlines. In a proof of concept we choose Gromacs 4 to evaluate the feasibility of being an SSaaS provider. We have set up an own private Cloud and we have written scripts that automate the building of different virtual infrastructures in which we analyzed Gromacs. We did horizontal and vertical scalings and we measured the performance with a typical benchmark program. Our Cloud consists of three physical hosts. The first one is a single Core 2 GHz Xeon with 4 Cores and 6 GB RAM. The second and third system have two Xeon CPUs with 4 Cores running at 2, 27 GHz and 12 GB RAM. They run a 64 Bit Linux kernel. The used Cloud infrastructure is based on Eucalyptus [8], offering the popular Amazon EC2 cloud interface [12], which also manages the storage and network infrastructure. Eucalyptus has an hierarchical architecture where the systems have different roles. Our first physical system acts as cloud-, cluster-, and also as nodecontroller. The other computers are node-controller. Running multiple instances in the cloud requires that the cluster controller includes a mapping algorithm that places the VMs on physical hosts. Eucalyptus implements the algorithms greedy and round-robin. We used round-robin that mapped a new virtual instance to the first host in the roundrobin list of nodes having enough available resources and iteratively walks thought this list. Currently there is no live migration. The initial mapping will not change during runtime. Running Gromacs 4 in parallel requires MPI installed and configured on all instances. We used OpenMPI [5] and the the number of MPI processes equals the number of compute nodes multiplied by the number of CPU cores. Figure 3 shows the results of folding dipalmitoylphosphatidylcholine (DPPC) in water in nano seconds per day. Comparing the different performance values reached by a single node of different types (see Table 2 for the type definitions) suggest good vertical scaling of Gromacs. Increasing the number of compute cores also results in a performance rise for instances of type m1.small or type c1.medium. First, adding more nodes of type m1.large or type m1.xlarge also tends to a higher performance. But this performance enhancement is not monotonic in the number of nodes.

6 Performance [ns/d] m1.small c1.medium m1.large m1.xlarge c1.xlarge #Nodes Fig. 3: The performance of Gromacs with different number of nodes. Number Mapping Req/sec Name vcpus RAM(MB) Disk(GB) 1 (0,1,0) m1.small (1,0,1) c1.medium (1,1,1) m1.large (1,2,1) m1.xlarge c1.xlarge Tab. 1: Mapping of the m1.xlarge instances to the cloud hosts. Tab. 2: The available instance types in the cloud. We analyzed this behavior by regarding virtual machines of type m1.xlarge. Table 1 shows the performance values and also the mapping of the instances to the cloud hosts. As in Figure 3, it can be seen that the performance decreases from three to four nodes. The reason is that there are no more free resources for the hypervisor on the second cloud node. This node has eight physical cores and, in the case of four instances, it runs two VMs each with four virtual CPUs. The highest performances values are reached by two machines of type c1.xlarge or three m1.large instances. Considering lower performance values depicts that sometimes there are many different combinations that compute nearly equally fast. The combinations of one c1.xlarge, two m1.large, two or three m1.xlarge and three c1.medium instances result in nearly the same performance. In benchmarks in physical environments we figured out that increasing the number

7 of cores does not necessarily tend to a higher performance. These results are presented at our website 1. Even there, it is not possible to predict the performance behaviour. The previous experiments show that in virtual environments this problem becomes more complex because it is also influenced by the mapping of the VMs. Finding the most efficient configuration fulfilling SLAs requires a deep understanding of the application. Experiences that are gained in physical environments must not necessarily apply in virtual environments. It might be very beneficial for a SaaS provider to test several configurations at its IaaS service provider to get information about the most cost efficient configuration necessary to achieve a predefined performance. 5 Conclusion With the goal to become a scientific SaaS provider we try to find software offerings that are demanded by the scientific community. We started a survey among the customers of our data center. The results show that there are many differences between business and scientific applications. Many scientific applications have properties complicating the offering. Especially the fulfillment of SLAs is very difficult. For a proof of concept we build up a private cloud and analyze Gromacs 4. Gromacs 4 has properties like an online progress status, parallelism, and checkpointing support that simplifies the service providing in consideration of SLAs based on deadlines. Our experiments show that the mapping policy of virtual machines to the cloud hosts extremely influences application performance and that increasing the assigned resources of an instance does not always result in higher performance. We observe that the instances on a physical system influence each other and also influence the hypervisor. Therefore, performance between different virtual machines is not completely isolated from each other. The experiments show that in some cases it is even possible to increase performance by reducing the number of compute nodes. We have been also able to show that different combinations of numbers of nodes and instance types can achieve the same performance. In terms of costs, this is very important for a scientific SaaS provider. Sometimes it is cheaper to run multiple slow instances than running a single fast instance. Finding out the most computationally efficient number of compute nodes for a particular service is not easy, but only with this optimum the scientific SaaS provider is able to operate an efficient data center in terms of costs, energy and performance. The provider will select the cheapest combination of node type and node number that fullfills SLAs concluded with his customers. In our future work we will focus on additional scientific applications. At the same time we have started to work on algorithms that guarantees SLAs automatically and we are analyzing different policies for the mapping of the VMs to the physical hosts. Solving the network performance issues in virtualized environments is another subject we 1 P C 2 BenchmarkCenter: benchmarkcenter/

8 work on. We evaluate VM-bypass and SR-IOV approaches to get high-speed infiniband device access in VMs. References 1. J. L. Cacheiro, C. Fernandez, E. Freire, S. Diaz, and A. Simon. Providing grid services based on virtualization and cloud technologies. In Proceedings of the 4 th Workshop on Virtualization in High-Performance Cloud Computing (VHPC), Delft, Netherlands, Aug F. Chang, J. Dean, S. Ghemawat, W. Hsieh, D. Wallach, M. Burrows, T. Chandra, A. Fikes, and R. Gruber. Bigtable: A distributed storage system for structured data. In Proceedings of the 7 th Conference on Operating Systems Design and Implementation (OSDI), pages , Seattle, Washington, USA, Nov J. Dean and S. Ghemawat. Mapreduce: Simplified data processing on large clusters. In Proceedings of the 6 th Conference on Operating Systems Design and Implementation (OSDI), pages , San Francisco, California, USA, Dec G. DeCandia, D. Hastorun, M. Jampani, G. Kakulapati, A. Lakshman, A. Pilchin, S. Sivasubramanian, P. Vosshall, and W. Vogels. Dynamo: amazon s highly available key-value store. In Proceedings of the 21 st ACM Symposium on Operating Systems Principles (SOSP), pages , Stevenson, Washington, USA, Oct E. Gabriel, G. E. Fagg, G. Bosilca, T. Angskun, J. J. Dongarra, J. M. Squyres, V. Sahay, P. Kambadur, B. Barrett, A. Lumsdaine, R. H. Castain, D. J. Daniel, R. L. Graham, and T. S. Woodall. Open MPI: Goals, concept, and design of a next generation MPI implementation. In Proceedings, 11th European PVM/MPI Users Group Meeting, pages , Budapest, Hungary, September S. Ghemawat, H. Gobioff, and S.-T. Leung. The google file system. In Proceedings of the 19 th ACM Symposium on Operating Systems Principles (SOSP), pages 29 43, Lake George, NY, USA, Oct K. Keahey and T. Freeman. Science clouds: Early experiences in cloud computing for scientific applications. In Proceedings of the 1 th Workshop on Cloud Computing and its Applications (CCA), Chicago, USA, Oct D. Nurmi, R. Wolski, C. Grzegorczyk, G. Obertelli, S. Soman, L. Youseff, and D. Zagorodnov. The eucalyptus open-source cloud-computing system. In Proceedings of the 9 th IEEE International Symposium on Cluster Computing and the Grid (CCGrid), Shanghai, China, May B. Sotomayor, R. S. Montero, I. M. Llorente, and I. Foster. Capacity leasing in cloud systems using the opennebula engine. In Proceedings of the Workshop on Cloud Computing and its Applications 2008 (CCA), Chicago, Illinois, USA, Oct D. Van Der Spoel, E. Lindahl, B. Hess, G. Groenhof, A. Mark, and H. Berendsen. Gromacs: fast, flexible, and free. Journal of Computational Chemistry, 26(16): , Dec L. Wang, M. Kunze, and J. Tao. The cumulus project: Build a scientific cloud for a data center. In Proceedings of the 1 th Workshop on Cloud Computing and its Applications (CCA), Chicago, USA, Oct A. Weiss. Computing in the clouds. networker, 11(4):16 25, Dec