Melaie Siebehaar, Olga Wege, Roy Has, Hasa Terca, Ralf Steimetz: Verifyig the Availability of Cloud Applicatios. I: Proceedigs of the 3rd Iteratioal Coferece o Cloud Computig ad Services Sciece (CLOSER 2013), May 2013. Verifyig the Availability of Cloud Applicatios Melaie Siebehaar, Olga Wege, Roy Has, Hasa Terca, ad Ralf Steimetz Multimedia Commuicatios Lab (KOM), Techische Uiversita t Darmstadt, Germay Email: firstame.lastame@kom.tu-darmstadt.de Keywords: Cloud Computig, Service Level Agreemets, Verificatio, Moitorig, Performace, Availability Abstract: Cloud-based services provide a high level of flexibility ad elimiate large up-frot IT ivestmets by tradig capital expediture for operatioal expediture. However, performace, availability, ad security still remai domiat barriers whe decidig whether to move to the cloud or ot. Although cloud providers already try to tackle these issues by offerig SLAs ad correspodig moitorig solutios, the ability of these solutios to cotrol the performace of cloud-based services is still cosidered as usatisfactory by cosumers. I this paper, we preset a approach for verifyig availability guaratees from a cosumer s perspective, sice availability is oe of the very few performace parameters that is cosidered i the SLAs of today s cloud providers. The aim of our research is to facilitate the verificatio of performace guaratees idepedetly from a cloud provider, which will help to icrease cloud service adoptio i the future. 1 INTRODUCTION Cloud computig promises to provide a high level of flexibility whe usig cloud-based services. Highly cofigurable computig resources are provided odemad ad with miimal maagemet effort over the Iteret (Mell ad Grace, 2011) similar to utilities like electricity or water (Buyya et al., 2009). However, this also icludes a shift of resposibility to the cloud provider ad thus, a loss of cotrol for the cloud cosumer. I order for a cloud cosumer to still maitai cotrol, cloud providers typically offer so-called service level agreemets (SLAs). Basically, a service level agreemet represets a cotract betwee a cloud provider ad a cloud cosumer ad specifies certai quality levels a cloud provider is willig to provide (e.g., miimum values for performace parameters such as availability ) ad the pealties i case of violatig the specified guaratees. However, this solutio does ot seem to be sufficiet. Accordig to a cloud market maturity study coducted by the Cloud Security Alliace ad ISACA i the secod quarter of 2012, there is oly a low degree of cofidece o cosumer side, that providers effectively moitor performace agaist SLAs (CSA ad ISACA, 2012). (Patel et al., 2009) also state that cosumers may ot completely trust these measuremets ad that cloud providers ofte put the burde of reportig SLA violatios o their customers. Although cloud providers ofte implemet particular moitorig solutios ad provide some moitorig iformatio to their customers, these solutios caot be perceived to provide a sufficiet ad idepedet evidece base for reliably detectig ad documetig SLA violatios from a cosumer s perspective. Whe solely relyig o provider-specific moitorig solutios, cloud providers ca modify some moitorig data or restrict the provided iformatio so that it becomes very hard to prove SLA violatios. This raises the questio how compliace with SLAs ca be verified from a cosumer s perspective. Such a solutio ot oly requires to obtai reliable data of a cloud-based service, but also requires to provide a holistic view of the edto-ed performace of a cloud-based service to cosumers. I this paper, we preset such a approach for verifyig the availability of cloud applicatios from a cosumer s perspective, sice availability is oe of the very few performace parameters that are part of the SLAs of today s cloud providers. The remaider of the paper is structured as follows. Sectio 2 discusses related approaches to our work. Sectio 3 describes the curret SLA ladscape, itroduces some basic iformatio about availability ad presets a taxoomy for dowtimes of cloud applicatios. Sectio 4 describes our approach for availability verificatio ad Sectio 5 presets the correspodig prototypical implemetatio as well as some experimetal results. The paper closes with a coclusio ad future directios i Sectio 6. The documets distributed by this server have bee provided by the cotributig authors as a meas to esure timely dissemiatio of scholarly ad techical work o a o-commercial basis. Copyright ad all rights therei are maitaied by the authors or by other copyright holders, ot withstadig that they have offered their works here electroically. It is uderstood that all persos copyig this iformatio will adhere to the terms ad costraits ivoked by each author's copyright. These works may ot be reposted without the explicit permissio of the copyright holder.
2 RELATED WORK Although several moitorig approaches i the field of cloud computig have bee proposed so far, oly a few approaches exist which address the problem of SLA verificatio from a cosumer s perspective. (Chazalet, 2010) presets a geeric framework that does ot deped o a specific cloud service model. However, the framework focuses either o server-side or cliet-side moitorig. I (Haberkor ad Trivedi, 2007), the authors preset a geeric approach for moitorig highavailability systems that cosist of differet compoets. Agai, o cliet-side moitorig is cosidered. A performace model based o rutime moitorig data is suggested by (Shao ad Wag, 2011). Availability is calculated based o the umber of successful requests. Hece, a sufficiet umber of requests is required i order to obtai accurate results. (Michlmayr et al., 2009) apply cliet-side ad server-side moitorig for SLA violatio detectio. It is ot clear, how the authors combie the results from both moitors to determie the overall performace. Agai, availability is oly calculated based o the umber of successful requests. (Mastelic et al., 2012) preset a geeric approach for moitorig applicatio level metrics i resourceshared cloud eviromets. Availability ad clietside moitorig are ot part of their work. I cotrast, our approach allows to verify availability from a cosumer s perspective ad to achieve visibility of the etire cloud service delivery chai. 3 AVAILABILITY OF CLOUD APPLICATIONS 3.1 SLA Ladscape ad Availability I compariso to, e.g., Web services, cloud services exhibit a higher complexity due to the three differet service models (Mell ad Grace, 2011), software as a service (SaaS), platform as a service (PaaS), ad ifrastructure as a service (IaaS), ad the fact that cloud applicatios o the upper SaaS layer ofte comprise several differet compoets. The complexity further icreases due to the utilizatio of virtualizatio so that surroudig coditios may chage i the backgroud without beig oticed by cosumers. Ufortuately, curret cloud SLAs do ot completely cover this ew iheret complexity. There is ofte a gap betwee lower-level moitorig data collected by providers ad higher-level guaratees provided to cosumers (CSCC, 2012). Hece, besides the demad for more specific SLAs, also correspodig meas for moitorig higher-level metrics of cloudbased services must be provided to cosumers. The paper at had focuses o the availability of cloud applicatios, sice availability is busiess critical ad oe of the very few performace guaratees that are curretly offered by cloud providers (e.g., Elastic Compute Cloud (EC2) by (Amazo, 2008)). I order to develop a availability moitorig approach, availability must be defied i a cloud computig cotext. (Jai, 1991) basically defies the availability of a system as the fractio of the time the system is available to service users requests. He further states that it is ofte more reasoable to use the mea uptime (MTTF), because small uptime ad dowtime combiatios may result i highavailability values although the service caot be delivered. Usig MTTF ad MTTR as the mea dowtime results i the followig formula: MT T F availability = (1) MT T F + MT T R Sice errors o differet layers or failures of compoets ca lead to dowtimes of the cloud applicatio, differet types of availability must be combied i order to measure the overall availability. Hece, the reasos for dowtimes must be aalyzed first i order to derive a appropriate defiitio. 3.2 Reasos for Dowtimes I order to successfully ivoke a specific fuctioality, i the followig deoted as service, provided by a cloud applicatio, cosumer s require workig IT systems o-premise ad a proper etwork coectivity to the cloud provider. Furthermore, due to the variety of resources ivolved i service delivery, may reasos for dowtimes exist. I the followig, we preset a taxoomy for dowtimes of cloud applicatios from a cosumer s perspective (Figure 1). Cosumer- Side Provider- Side ISP-Side Locatio Software Virtual Machie Physical Machie Network Failure Overload Outage Migratio Maiteace Dowtime Resource Icidet Itet Sigificace Uplaed Plaed Attack SLA Violatio No SLA Violatio Figure 1: Taxoomy for dowtimes of cloud applicatios First of all, dowtimes ca be distiguished accordig to the locatio, where icidets happe. There-
fore, we describe their locatio accordig to the followig spheres of cotrol: cosumer, Iteret service provider (ISP), ad cloud provider. Furthermore, differet types of resources ca be resposible for causig dowtimes due to several icidets. Hece, we added the two categories resource ad icidet to our taxoomy. Besides the actual dowtimes, also their pre- ad postcoditios must be take ito accout i order to determie if a SLA violatio occurred. Basically, dowtimes ca happe with or without itet, ad eve with crimial itet whe a system is uder attack. Depedig o the egotiated terms, all three types of itet ca either be covered by SLAs or ot. For example, SLAs ca specify a maximum legth of time for plaed dowtimes such as scheduled maiteace, uplaed dowtimes ca either refer to failures of physical machies or to emergecy maiteace, ad eve attacks ca happe due to egligece of cosumers or due to a isufficiet amout of implemeted security mechaisms by providers. Therefore, dowtimes must always be cosidered i cojuctio with the egotiated SLAs. Moreover, our solutio must be able to attribute icidets to their root cause, sice cloud providers ca oly make guaratees with respect to their ow IT systems. 4 A HYBRID APPROACH FOR AVAILABILITY VERIFICATION We will ow elaborate o how to use the kowledge about dowtimes preseted before i order to develop a approach for verifyig the availability of cloud applicatios from a cosumer s perspective. I this paper, we will focus o uplaed dowtime oly, sice plaed dowtimes will be usually aouced by cloud providers i advace ad the detectio of attacks is ot i the scope of our work. The ext sectio presets the requiremets for such a approach. 4.1 Requiremets First of all, our approach should be able to detect all relevat dowtimes precisely without affectig the overall performace ad should be still applicable whe the umber of users ad compoets chages. A SLA violatio is cosidered to be relevat either if the duratio of a sigle dowtime or the aggregatio of several dowtimes may exceed the threshold defied i the respective SLA. Furthermore, our solutio should be able to compute the overall availability. Fially, a trusted third party could geerally provide the compoets of our moitorig approach to cosumers i order to esure reliability. 4.2 Desig Now, cosiderig all the requiremets stated above, we propose a hybrid moitorig approach that combies cosumer- ad cloud-side moitorig. I additio, we make use of a broker actig as a coordiatig etity that collects ad aggregates all data (Figure 2). Cosumer-side Moitor A Broker Cosumer-side Moitor B Cloud-side App VM Moitor Moitor VM 1 cloud applicatios VM 2 push to pull from Figure 2: Overview of the moitorig framework Cosumer-Side Moitorig: The cosumer-side moitor ivokes predefied services (e.g., specified i the SLAs) that are essetial for the proper fuctioig of a cloud applicatio usig a periodical pull model. The moitorig frequecy ca be adapted to the required resolutio of a applicatio s availability. We cosider a cloud applicatio to be uavailable if oe of the essetial services fails (i.e., o/icorrect respose). I order to determie the overall availability, the cosumer-side moitor commuicates with the broker usig a evet-based push approach. Wheever a cosumer-side moitor is started or the start or the ed of a dowtime is detected, a message is set to the broker. Failures i the etwork could also prevet cosumers from detectig all dowtimes. Hece, we assume that a eterprise uses two differet moitors at differet etwork domais. Sice a high moitorig frequecy (e.g., i case of may cosumers) will affect system performace, additioal cloud-side moitorig must be cosidered. Cloud-Side Moitorig: For our approach we assume that a cosumer has access to the VM where a cloud applicatio is hosted. Sice a moitor placed o this VM 1 would ot be able to report ay dowtimes if the VM 1 crashes, we eed a additioal VM 2 withi the provider s data ceter i order to place our cloud-side moitor. Nevertheless, we also require access to the VM 1 i order to check the status of predefied processes that are essetial for ruig the cloud applicatio. Therefore, a lightweight software compoet (VM moitor) must be istalled o the VM 1. We apply a periodical pull model i order to ivoke the VM moitor from the cloud-side moitor ad a evet-based push model to sed data to the broker.
4.3 Broker ad Availability Calculatio The broker maitais a dowtime list for each moitor ad periodically computes the overall availability. For this purpose, the broker first determies the overlap of the dowtimes reported by both cosumer-side moitors i order to separate cloud applicatio dowtimes from etwork errors ad afterwards, merges the cosolidated dowtimes with the dowtimes of the cloud-side moitor. For the latter, we assume that wheever two detected dowtime itervals from cosumer- ad cloud-side overlap, these dowtime itervals belog to the same outage. I this case, the broker decides which dowtime iterval better reflects the real dowtime by evaluatig the followig coditios. T represets the differece betwee the reciprocals of the moitorig frequecies (i.e., T = 1/ f ) of both moitors ad dc ad d p are the duratios of the dowtime moitored at cosumer- ad provider-side, respectively. > T cosumer-side moitor (2) (dc d p ) = T cloud-side moitor < impossible T The coditios above result from the differece i precisio of both moitors. They express that wheever the differece betwee the duratios is ot caused by the iaccuracy of the cosumer-side moitors (case 2), the dowtime of some essetial services of the applicatio must exceed the dowtime of the uderlyig processes moitored at cloud-side (case 1). Fially, the broker obtais a list of dowtimes ad calculates the overall availability based o the calculatio proposed by (Haberkor ad Trivedi, 2007) as described i the followig. For the calculatio, we itroduce a set of variables (Table 1). Figure 3: Us at dowtime (Haberkor ad Trivedi, 2007) Figure 4: Us at uptime (Haberkor ad Trivedi, 2007) Whe calculatig the availability durig a uptime, Us has to be calculated as follows: 0 Us = (Ts Ds ) + u (4) I cotrast, whe calculatig the availability durig a dowtime, Us ca be determied as follows: 00 0 Us = (Ts Ds ) = (Ts di d ) (5) i=1 Fially, the overall availability ca be calculated usig Equatio 1 (Haberkor ad Trivedi, 2007): MT T F = 5 Us m ad MT T R = Ds (6) EXPERIMENTS Table 1: Variables for calculatig the overall availability Ts Ds Us m d1,..., d u1,..., um total service time aggregated dowtime aggregated uptime umber of dowtime itervals umber of uptime itervals completed dowtime itervals completed uptime itervals The aggregated dowtime Ds ad uptime Us of a cloud applicatio ca the be calculated as follows: Ds = di ad Us = Ts Ds (3) i=1 Sice Us varies depedig o whether the overall availability is calculated durig a dowtime or a up0 00 time (Figures 3 ad 4), two differet cases Us ad Us must be distiguished (Haberkor ad Trivedi, 2007). Our approach has bee prototypically implemeted usig the (Kaltura, 2012) video platform, that we deployed o a VM i our blade ceter. O this VM, we also istalled our VM moitor ad o a secod VM, we deployed our cloud-side moitor. Furthermore, we placed cosumer-side moitor A as well as the broker o a local desktop computer ad cosumerside moitor B o a laptop. The techical specificatio is show i Table 2. Table 2: Techical setup of the implemetatio App VM CetOS 51 2 vcpu2 2.13 GHz 4 GB Cloud Mo. VM Wi. 7 2 vcpu 2.13 GHz 2 GB Mo. A PC Wi. 7 4 CPU 2.67 GHz 4 GB Mo. B Laptop Wi. 7 1 CPU 2 GHz 4 GB
We used Java as programmig laguage to implemet all compoets of our moitorig framework. The commuicatio betwee the differet moitorig compoets is realized by TCP/IP sockets, except for the ivocatio of the cloud services provided by the Kaltura cloud applicatio. These service ivocatios are based o REST ad HTTP. For cloud-side moitorig, we used httpd, mysql, ad memcached as essetial processes resposible for a proper Web server fuctioality, database access, ad memory cachig. O cosumer-side, we used a small sample video file that we uploaded to the Kaltura platform ad that was also stored locally at cosumer-side. Based o this sample video file, we periodically set a first request to the Kaltura platform to retrieve the metadata of this video file ad a secod request to dowload the video file. I doig so, our solutio verified the storage access ad trasmissio capabilities of the platform. 5.1 Setup The experimets have bee performed usig moitorig itervals of 3 miutes for the cosumer-side moitors ad 5 secods for the cloud-side moitor. Every 60 secods, the broker calculates the overall availability. The timeouts at cosumer-side ad cloud-side for receivig resposes are set to 15 secods ad 4 secods, respectively. I order to simulate outages of the cloud applicatio as well as etwork impairmets, we have used the wide area etwork emulator (WANem, 2011). I the first two experimets, we have simulated dowtimes i order to evaluate the detectio rate of our framework ad i the last experimet, we have simulated etwork impairmets i order to evaluate the behaviour uder real etwork coditios. 5.2 Short, Periodical Dowtimes I this experimet, we simulated short, periodical dowtimes over a period of 15 miutes with each dowtime ad uptime lastig 20 secods ad 60 secods, respectively. We have repeated this experimet for 5 times. While the cloud-moitor detected all 11 dowtimes i each ru, the cosumer-side moitors oly detected 1.8 dowtimes o average. The results (Figure 5) show that our moitorig framework is basically able to detect all dowtimes, but also poit out the lower precisio of cosumer-side moitorig. All i all, our moitorig framework oly achieved a deviatio of 0.826% from the real availability due to the accurate cloud-side moitorig. However, the ext experimet will show that the cloud-side moitor 1 CetOS: free Liux based o Red Hat Eterprise 2 vcpu: virtual CPU assiged to a VM Figure 5: Differece i precisio of both moitor types ot always delivers reliable results, which emphasizes the eed for a hybrid approach. 5.3 Higher Precisio at Cosumer-Side The secod experimet simulates oly a sigle dowtime of 6 miutes durig a total service time of 11 miutes. Agai, this experimet has bee repeated for 5 times. From the perspective of the cloud-side moitor, the failure of the cloud applicatio is corrected after 78 secods o average durig each ru. Figure 6 shows that the cosumer-side moitors show a higher precisio tha the cloud-side moitor. Figure 6: More precise detectio by cosumer moitor Although our framework chose the correct dowtime, i.e., the result from the cosumer-side moitors i order to calculate the availability, the resultig deviatio of 6.03% from the real value is quite high, so that the moitorig itervals for cosumer-side moitorig should be further decreased. 5.4 Network Impairmets This experimet was coducted to evaluate our moitorig framework uder differet etwork coditios. For this purpose, we applied WANem to iduce several impairmets ito the etwork. As a prerequisite for our experimet, we defied six differet etwork quality classes ragig from a etwork without ay impairmets T 0 to a etwork T 1500 with a delay of 1500ms, 10% packet loss, ad 10% corrupted packets. Differet tests, each lastig 530 secods ad simulatig a dowtime of 130 secods, were coducted at each quality level. The impairmets were iduced i all etworks iside ad outside the cloud. Although the actual availability of 75.471% of the cloud applicatio did ot chage durig the tests, Figure 7 shows that the differece betwee the actual availability ad the moitored availability cosiderably decreases (to
56.63%) with a icreasig amout of etwork impairmets. This experimet shows that our moitorig approach is very sesitive to failures i the etwork, which have to be addressed i future work. Figure 7: Precisio decreases with etwork impairmets 6 CONCLUSION AND OUTLOOK I this paper, we itroduced a taxoomy for dowtimes of cloud applicatios ad preseted a hybrid approach for verifyig the availability of cloud applicatios from a cosumer s perspective. The approach combies cosumer- ad cloud-side moitorig i order to be able to attribute dowtimes to their root cause ad to distiguish SLA violatios from other types of dowtimes. Our framework periodically provides a updated overall availability to cosumers. The results of our experimets reveal that a hybrid approach is ideed required i order to allow for a precise calculatio of the overall availability. However, the experimets also reveal that i case of etwork impairmets, the detectio accuracy decreases. Hece, we will develop approaches to icrease the robustess of our moitorig framework i future work. Furthermore, we will coduct experimets i real cloud eviromets ad we will explore how to determie a appropriate ratio betwee the moitorig frequecies o cosumer- ad cloud-side. ACKNOWLEDGEMENTS The work preseted i this paper was partially fuded by the Germa Federal Miistry of Educatio ad Research (BMBF) uder grat o. 01 C10S05 i the cotext of the Software-Cluster project SWINNG (www.software-cluster.org). I additio, this work is supported i part by E-Fiace Lab Frakfurt am Mai e.v. (http://www.efiacelab.com). The authors assume resposibility for the cotet. REFERENCES Amazo (2008). Amazo EC2 Service Level Agreemet. http://aws.amazo.com/ec2-sla/, [last access: 3 December 2012]. Buyya, R., Yeo, C. S., Veugopal, S., Broberg, J., ad Bradic, I. (2009). Cloud Computig ad Emergig IT Platforms: Visio, Hype, ad Reality for Deliverig Computig as the 5th Utility. Future Geeratio Computer Systems, 25(6):599 616. Chazalet, A. (2010). Service Level Checkig i the Cloud Computig Cotext. I Proceedigs of the 3rd IEEE Iteratioal Coferece o Cloud Computig, pages 297 304. CSA ad ISACA (2012). Cloud Computig Market Maturity. Study Results. Cloud Security Alliace ad ISACA. http://www.isaca.org/kowledge- Ceter/Research/Documets/2012-Cloud- Computig-Market-Maturity-Study-Results.pdf, [last access: 28 November 2012]. CSCC (2012). Practical Guide to Cloud Service Level Agreemets. Cloud Stadards Customer Coucil. http://www.cloudstadardscustomercoucil.org/ 04102012.htm, [last access: 30 November 2012]. Haberkor, M. ad Trivedi, K. (2007). Availability Moitor for a Software Based System. I Proceedigs of the 10th IEEE High Assurace Systems Egieerig Symposium (HASE 2007), pages 321 328. Jai, R. (1991). The Art of Computer Systems Performace Aalysis. Wiley. Kaltura (2012). Kaltura Ic. http://corp.kaltura.com/, [last access: 3 December 2012]. Mastelic, T., Emeakaroha, V. C., Maurer, M., ad Bradic, I. (2012). M4Cloud - Geeric Applicatio Level Moitorig for Resource-shared Cloud Eviromets. I Proceedigs of the 2d Iteratioal Coferece o Cloud Computig ad Services Sciece (CLOSER 2012), pages 522 532. Mell, P. ad Grace, T. (2011). The NIST Defiitio of Cloud Computig. http://csrc.ist.gov/publicatios/ istpubs/800-145/sp800-145.pdf, [Last access: 28 November 2012]. Michlmayr, A., Roseberg, F., Leiter, P., ad Dustdar, S. (2009). Comprehesive QoS Moitorig of Web Services ad Evet-Based SLA Violatio Detectio. I Proceedigs of the 4th Iteratioal Workshop o Middleware for Service Orieted Computig (MW- SOC 2009), pages 1 6. Patel, P., Raabahu, A., ad Sheth, A. (2009). Service Level Agreemet i Cloud Computig. Techical report, Koesis Ceter, Wright State Uiversity, USA. Shao, J. ad Wag, Q. (2011). A Performace Guaratee Approach for Cloud Applicatios Based o Moitorig. I Proceedigs of the 35th Aual Computer Software ad Applicatios Coferece Workshops (COMPSACW 2011), pages 25 30. WANem (2011). The Wide Area Network Emulator. Performace Egieerig Research Cetre, TATA Cosultacy Services. http://waem.sourceforge.et/, [last access: 5 December 2012].