Economic Models for Cloud Service Markets Pricing and Capacity Planning

Transcription

1 Economc Models for Cloud Servce Markets Prcng and Capacty Plannng Ranjan Pal 1 and Pan Hu 2 1 Unversty of Southern Calforna, USA, rpal@usc.edu 2 Deutsch Telekom Laboratores, Berln, Germany, pan.hu@telekom.de Abstract. Cloud computng s a paradgm that has the potental to transform and revolutonalze the next generaton IT ndustry by makng software avalable to end-users as a servce. A cloud, also commonly known as a cloud network, typcally comprses of hardware (network of servers) and a collecton of softwares that s made avalable to end-users n a pay-as-you-go manner. Multple publc cloud provders (ex., Amazon) co-exstng n a cloud computng market provde smlar servces (software as a servce) to ts clents, both n terms of the nature of an applcaton, as well as n qualty of servce (QoS) provson. The decson of whether a cloud hosts (or fnds t proftable to host) a servce n the long-term would depend jontly on the prce t sets, the QoS guarantees t provdes to ts customers, and the satsfacton of the advertsed guarantees. In the frst part of the paper, we devse and analyze three nter-organzatonal economc models relevant to cloud networks. We formulate our problems as non co-operatve prce and QoS games between multple cloud provders exstng n a cloud market. We prove that a unque pure strategy Nash equlbrum (NE) exsts n two of the three models. Our analyss paves the path for each cloud provder to know what prces and QoS level to set for end-users of a gven servce type, such that the provder could exst n the cloud market. A cloud provder servces end-user requests on behalf of cloud customers, and due to the uncertanty n user demands over tme, tend to overprovson resources lke CPU, power, memory, storage, etc., n order to satsfy QoS guarantees. As a result of over-provsonng over long tme scales, server utlzaton s very low and the cloud provders have to bear unnecessary wasteful costs. In ths regard, the prce and QoS levels set by the CPs drve the end-user demand, whch plays a major role n CPs estmatng the mnmal capacty to meet ther advertsed guarantees. By the term capacty, we mply the ablty of a cloud to process user requests,.e., number of user requests processed per unt of tme, whch n turn determne the amount of resources to be provsoned to acheve a requred capacty. In the second part of ths paper, we address the capacty plannng/optmal resource provsonng problem n sngle-tered and mult-tered cloud networks usng a techno-economc approach. We develop, analyze, and compare models that cloud provders can adopt to provson resources n a manner such that there s mnmum amount

2 2 of resources wasted, and at the same tme the user servce-level/qos guarantees are satsfed. Keywords: cloud markets; competton; Nash equlbrum; capacty; sngle-ter; mult-ter 1 Introducton Cloud computng s a type of Internet-based computng, where shared resources, hardware, software, and nformaton are provded to end-users n an on demand fashon. It s a paradgm that has the potental to transform and revolutonalze the IT ndustry by makng software avalable to end-users as a servce [1]. A publc cloud typcally comprses of hardware (network of servers) and a collecton of softwares that s made avalable to the general publc n a pay-as-you-go manner. Typcal examples of companes provdng publc clouds nclude Amazon, Google, Mcrosoft, E-Bay, and commercal banks. Publc cloud provders usually provde Software as a Servce (SaaS), Platform as a Servce (PaaS), and Infrastructure as a Servce (IaaS).The advantage of makng software avalable as a servce s three-fold [1], 1) the servce provders beneft from smplfed software nstallaton, mantenance, and centralzed versonng, 2) end-users can access the software n an anytme anywhere manner, can store data safely n the cloud nfrastructure, and do not have to thnk about provsonng any hardware resource due to the lluson of nfnte computng resources avalable on demand, and 3) end-users can pay for usng computng resources on a short-term bass (ex., by the hour or by the day) and can release the resources on task completon. Smlar beneft types are also obtaned by makng both, platform as well as nfrastructure avalable as servce. Cloud economcs wll play a vtal role n shapng the cloud computng ndustry of the future. In a recent Mcrosoft whte paper ttled Economcs of the Cloud, t has been stated that the computng ndustry s movng towards the cloud drven by three mportant economes of scale: 1) large data centers can deploy computatonal resources at sgnfcantly lower costs than smaller ones, 2) demand poolng mproves utlzaton of resources, and 3) mult-tenancy lowers applcaton mantenance labor costs for large publc clouds. The cloud also provdes an opportunty to IT professonals to focus more on technologcal nnovaton rather than thnkng of the budget of keepng the lghts on. The economcs of the cloud can be thought of havng two dmensons: 1) ntra-organzaton economcs and 2) nter-organzaton economcs. Intra-organzaton economcs deals wth the economcs of nternal factors of an organzaton lke labor, power, hardware, securty, etc., whereas nter-organzaton economcs refers to the economcs of market competton factors between organzatons. Examples of some popular factors are prce, QoS, reputaton, and customer servce. In ths paper, we focus on nter-organzatonal economc ssues. Multple publc cloud provders (ex., Amazon, Google, Mcrosoft, etc.,) coexstng n a cloud computng market provde smlar servces (software as a

3 3 servce, ex., Google Docs and Mcrosoft Offce Lve) to ts clents, both n terms of the nature of an applcaton, as well as n qualty of servce (QoS) provson. The decson of whether a cloud hosts (or fnds t proftable to host) a servce n the long-term would (amongst other factors) depend jontly on the prce t sets, the QoS guarantees t provdes to ts customers 3, and the satsfacton of the advertsed guarantees. Settng hgh prces mght result n a drop n demand for a partcular servce, whereas settng low prces mght attract customers at the expense of lowerng cloud provder profts. Smlarly, advertsng and satsfyng hgh QoS levels would favor a cloud provder (CP) n attractng more customers. The prce and QoS levels set by the CPs thus drve the end-user demand, whch, apart from determnng the market power of a CP also plays a major role n CPs estmatng the mnmal resource capacty to meet ther advertsed guarantees. By the term capacty, we mply the ablty of a cloud to process user requests,.e., number of user requests processed per unt of tme. The estmaton problem s an mportant challenge n cloud computng wth respect to resource provsonng because a successful estmaton would prevent CPs to provson for the peak, thereby reducng resource wastage. The competton n prces and QoS amongst the cloud provders entals the formaton of non-cooperatve games amongst compettve CPs. Thus, we have a dstrbuted system of CPs (players n the game), where each CP wants to maxmze ts own profts and would tend towards playng a Nash equlbrum 4 (NE) strategy (.e., each CP would want to set the NE prces and QoS levels.), whereby the whole system of CPs would have no ncentve to devate from the Nash equlbrum pont,.e., the vector of NE strateges of each CP. However, for each CP to play a NE strategy, the latter should mathematcally exst. In the frst part of the paper, we address the mportant problem of Nash Equlbrum characterzaton of dfferent types of prce and QoS games relevant to cloud networks, ts propertes, practcal mplementablty (convergence ssues), and the senstvty analyss of NE prce/qos varatons by any CP on the prce and QoS levels of other CPs. Our problem s mportant from a resource provsonng perspectve as mentoned n the prevous paragraph, apart from t havng obvous strategc mportance on CPs n terms of sustenance n the cloud market. In the second part of our paper we develop and analyze models that wll be useful to cloud provders to provson resources n a manner such that there s mnmum amount of resources wasted, and at the same tme the user servce-level/qos guarantees are satsfed. 3 A cloud provder generally gets requests from a cloud customer, whch n turn accepts requests from Internet end-users. Thus, typcally, the clents/customers of a cloud provder are the cloud customers. However, for modelng purposes, end-users could also be treated as customers. (See Secton 2) 4 A group of players s n Nash equlbrum f each one s makng the best decson(strategy) that he or she can, takng nto account the decsons of the others.

4 4 1.1 Related Work In regard to market competton drven network prcng, there exsts research work n the doman of multple ISP nteracton and tered Internet servces [2][3], as well as n the area of resource allocaton and Internet congeston management [4][5][6]. However, the market competton n our work relates to optmal capacty plannng and resource provsonng n clouds. There s the semnal work by Songhurst and Kelly [7] on prcng schemes based on QoS requrements of users. Ther work address mult-servce scenaros and derve prcng schemes for each servce based on the QoS requrements for each, and n turn bandwdth reservatons. Ths work resembles ours to some extent n the sense that the prce and QoS determned can determne optmal bandwdth provsons. However, t does not account for market competton between multple provders and only focus on a sngle servce provder provdng multple servces,.e., the paper addresses an ntra-organzaton economcs problem. However, n ths paper, we assume sngle-servce scenaros by multple servce provders. In a recent work [8], the authors propose a queueng drven game-theoretc model for prce-qos competton amongst multple servce provders. The work analyzes a duopolstc market between two servce provders, where provders frst fx ther QoS guarantees and then compete for prces. Our work extends the latter cted work n the followng aspects: (1) we generalze our model to ncorporate n servce provders, (2) we address two addtonal game models whch are of practcal mportance,.e., prce-qos smultaneous competton and prces fxed frst, followed by QoS guarantees competton, (3) we provde an effcent technque to compute multple equlbra n games, and (4) our models explctly characterze percentle performance of parameters, whch s specfc to cloud networks provsonng resources on a percentle bass. We also want to emphasze the fact that research on prce/qos competton amongst organzatons s not new n the economcs doman. However, n ths paper we model networkng elements n prce/qos games va a queueng theoretc approach and analyze certan prce/qos games that are manly characterstc of Internet servce markets. Recent research efforts on cloud resource provsonng have devsed statc and dynamc provsonng schemes. Statc provsonng [19][20] s usually conducted offlne and occurs on monthly or seasonal tmescales 5, whereas dynamc provsonng [21][22] dynamcally adjusts to workload fluctuatons over tme. In both the statc and the dynamc case, VM szng s dentfed as the most mportant step, where VM szng refers to the estmaton of the amount of resources to be allocated to a VM or jontly to many VMs [23]. However, none of the above cted works have accounted for external factors such as cloud provder prce competton, n determnng the optmal capacty of a cloud provder for a gven tme-slot. Market competton between cloud provders s a vtal factor n capacty plannng because cloud provders set prces to prmarly to make profts and the prces they set nfluence demands from end-users, and user demands 5 Several cloud management softwares lke VMWare Capacty Planner, CapactyIQ, and IBM WebSphere CloudBurst adopt ths functonalty.

5 5 drve the provsonng of optmal capactes. Other factors lke schedulng polces (ex., FCFS, Processor Sharng, etc.) employed by cloud provders, as well as the number of ters a web applcaton needs for servce, also contrbute to optmal capacty provsonng. Recent works on cloud network provsonng have accounted for parameters lke schedulng and mult-ter servces [24], but do not provde any analytcal results on the mpact of these parameters on optmally provsoned capacty, nor do they evaluate the optmal provsoned capacty. In contrast wth exstng approaches, we take a techno-economc approach to evaluatng the optmal provsoned capacty and provde theoretcal nsghts for our problem. Our optmal provsoned capacty s metrczed by the number of user requests processed per unt of tme. However, ths noton of capacty can be mapped to physcal resource capacty metrcs lke bandwdth, CPU, etc. Our proposed models ams to focus on how certan techncal and economc parameters nfluence optmal provsoned capacty of a cloud provder, as well as other competng cloud provders, whch s mportant when t comes to network desgn. 1.2 Contrbutons Statement Our proposed theory analyzes a few basc nter-organzatonal economc models through whch cloud servces could be prced under market competton. The evoluton of commercal publc cloud servce markets s stll n ts ncepton. However, wth the ganng popularty of cloud servces, we expect a bg surge n publc cloud servces competton n the years to come. The models proposed n ths paper take a substantal step n hghlghtng relevant models to the cloud networkng communty for them adopt so as to approprately prce current and future cloud servces. In practce, scenaros of prce and/or QoS competton between organzatons exst n the moble network servces and ISP markets. For example, AT&T and Verzon are competng on servce,.e., Verzon promses to provde better coverage to moble users than AT&T, thereby ncreasng ts propensty to attract more customers. Smlarly, prce competton between ISPs always exsted for provdng broadband servces at a certan gven bandwdth guarantee. Regardng our work, we also want to emphasze 1) we do not make any clams about our models beng the only way to model nter-organzatonal cloud economcs 6 and 2) there s a dependency between ntra-organzatonal and nter-organzatonal economc factors, whch we do not account n ths paper due to modelng smplcty. However, through our work, we defntely provde readers wth a concrete modelng ntuton to go about addressng problems n cloud economcs. To the best of our knowledge, we are the frst to provde an analytcal model on nter-organzatonal cloud economcs. Our Contrbutons - We make the followng contrbutons n ths paper. 6 We only model prce and QoS as parameters. One could choose other parameters (n addton to prce and QoS, whch are essental parameters) and a dfferent analyss mechansm than ours to arrve at a dfferent model.

6 6 1. We formulate a separable end-user demand functon for each cloud provder w.r.t. to prce and QoS levels set by them and derve ther ndvdual utlty functons (proft functon). We then defne the varous prce-qos games that we analyze n the paper. (See Secton 2.) 2. We develop a model where the QoS guarantees provded by publc CPs to end-users for a partcular applcaton type are pre-specfed and fxed, and the cloud provders compete for prces. We formulate a non-cooperatve prce game amongst the players (.e., the cloud provders) and prove that there exsts a unque Nash equlbrum of the game, and that the NE could be practcally computed (.e., t converges). (See Secton 3.) 3. We develop a non-cooperatve game-theoretc model where publc cloud provders jontly compete for the prce and QoS levels related to a partcular applcaton type. We show the exstence and convergence of Nash equlbra (See Secton 4). As a specal case of ths model, we also analyze the case where prces charged to Internet end-users are pre-specfed and fxed, and the cloud provders compete for QoS guarantees only. The models mentoned n contrbutons 3 and 4 drve optmal capacty plannng and resource provsonng n clouds, apart from maxmzng CP profts. (See Secton 4.) 4. We conduct a senstvty analyss on varous parameters of our proposed models, and study the effect of changes n the parameters on the equlbrum prce and QoS levels of the CPs exstng n a cloud market. Through a senstvty analyss, we nfer the effect of prce and QoS changes of cloud provders on ther respectve profts, as well as the profts of competng CPs. (See Sectons 3 and 4.) 7 5. We develop an optmzaton framework for sngle-tered and mult-tered cloud networks to compute the optmal provsoned capacty once the equlbrum prce and QoS levels for each CP have been determned. (See Secton 5.) 2 Problem Setup We consder a market of n competng cloud provders, where each provder servces applcaton types to end-users at a gven QoS guarantee. We assume that end-users are customers of cloud provders n an ndrect manner,.e., Internet end-users use onlne softwares developed by companes (cloud customers), that depend on cloud provders to servce ther customer requests. Each CP s n competton wth others n the market for servces provded on the same type of applcaton w.r.t functonalty and QoS guarantees. For example, Mcrosoft and Google mght both serve a word processng applcaton to end-users by provdng 7 We study Nash equlbrum convergence as ts proves the achevablty of an equlbrum pont n the market. We emphasze here that the exstence of Nash equlbrum does not mply achevablty as t may take the cloud market an eternty to reach equlbrum, even though there may exst one theoretcally.

7 7 smlar QoS guarantees. Here, the word processng applcaton represents a partcular type. For a gven applcaton type, we assume that each end user sgns a contract wth a partcular CP for a gven tme perod 8, and wthn that perod t does not swtch to any other CP for gettng servce on the same applcaton type. Regardng contracts between a CP and ts end-users, we assume that a cloud customer forwards servce requests to a cloud provder on behalf of endusers, who sgn up wth a cloud customer (CC) for servce. The CP charges ts cloud customer, who s turn charges ts end-users. We approxmate ths two-step chargng scheme by modelng a vrtual one-step scheme, where a CP charges end-users drectly 9. In a gven tme perod, each CP postons tself n the market by selectng a prce p and a QoS level s related to a gven applcaton type. Throughout the paper, we assume that the CPs compete on a sngle gven type 10. We defne s as the dfference between a benchmark response tme upper bound, rt, and the actual response tme rt,.e., s = rt rt. For example, f for a partcular applcaton type, every CP would respond to an end-user request wthn 10 seconds, rt = 10. The response tme rt may be defned, ether n terms of the expected steady state response tme,.e., rt = E(RT ), or n terms of φ- percentle performance, rt (φ), where 0 < φ < 1. Thus, n terms of φ-percentle performance 11, P (RT < rt (φ)) = φ. We model each CP as an M/M/1 queueng system, where end-user requests arrve as a Posson process wth mean rate λ, and gets servced at a rate µ. We adopt an M/M/1 queueng system because of three reasons: 1) queueng theory has been tradtonally used n request arrval and servce problems, 2) for our problem, assumng an M/M/1 queueng system ensures tractable analyses procedures that entals dervng nce closed form expressons and helps understand system nsghts n a non-complex manner, wthout sacrfcng a great deal n capturng the real dynamcs of the actual arrval-departure process, and 3) The Markovan nature of the servce process helps us generalze expected steady state analyss and percentle analyss together. Accordng to the theory of M/M/1 queues, we have the followng standard results [17]. rt = 1 µ λ, (1) 8 In ths paper, the term tme-perod refers to the tme duraton of a contract between the CP and end-users. 9 We assume here that prces are negotated between the CP, CC, and end-users and there s a vrtual drect prce chargng connecton between the CP and ts end-users. We make ths approxmaton for modelng smplcty. 10 In realty, each CP may n general servce several applcaton types concurrently. We do not model ths case n our paper and leave t for future work. The case for sngle applcaton types gves nterestng results, whch would prove to be useful n analyzng the multple concurrent applcaton type scenaro. 11 As an example, n cloud networks we often assocate provsonng power accordng to the 95th percentle use. Lkewse, we could also provson servce capacty by accountng for percentle response tme guarantees.

8 8 and rt (φ) = ln( 1 1 φ ) µ (φ) λ, (2) µ = λ + 1 rt, (3) µ (φ) =λ + ln( 1 1 φ ) rt (φ) Equatons 2 and 4 follow from the fact that for M/M/1 queues, P (RT < rt (φ)) = φ =1 e (µ λrt(φ)). Wthout loss of generalty, n subsequent sectons of ths paper, we conduct our analyss on expected steady state parameters. As mentoned prevously, due to the Markovan nature of the servce process, the case for percentles s exactly smlar to the case for expected steady state analyss, the only dfference n analyss beng due to the constant, ln( 1 1 φ ). Thus, all our proposed equlbrum related results hold true for percentle analyss as well. Each cloud provder ncurs a fxed cost c per user request served and a fxed cost ρ per unt of servce capacty provsoned. c arses due to the factor λ n Equaton 3 and ρ arses due to the factor 1 rt n the same equaton. In ths sense, our QoS-dependent prcng models are queueng-drven. A cloud provder charges pr to servce each end-user request, where pr ɛ [pr mn, pr max ]. It s evdent that each CP selects a prce that results n t accrung a non-negatve gross proft margn. The gross proft margn for CP s gven as pr c ρ, where c + ρ s the margnal cost per unt of end-user demand. Thus, the prce lower bound, pr mn, for each CP s determned by the followng equaton. (4) pr mn = c + ρ, =1,..., n (5) We defne the demand of any CP, λ, as a functon of the vectors pr = (pr 1,..., pr n ) and s =(s 1,..., s n ). Mathematcally, we express the demand functon as λ = λ (pr, s) =x (s ) y pr j α j (s j )+ j β j pr j, (6) where x (s ) s an ncreasng, concave, and thrce dfferentable functon n s satsfyng the property of non-ncreasng margnal returns to scale,.e., equal-szed reductons n response tme results n progressvely smaller ncreases n end-user demand. The functons α j are assumed to be non-decreasng and dfferentable. A typcal example of a functon fttng x (s ) and α j (s j ) s a logarthmc functon. We model Equaton 6 as a separable functon of prce and QoS vectors, for ensurng tractable analyses as well as for extractng the ndependent effects of prce and QoS changes on the overall end-user demand. Intutvely, Equaton 6 states that QoS mprovements by a CP result n an ncrease n ts end-user demand, whereas QoS mprovements by other compettor CPs result n a decrease

9 9 n ts demand. Smlarly, a prce ncrease by a CP results n a decrease n ts end-user demand, whereas prce ncreases by other competng CPs result n an ncrease n ts demand. Wthout loss of practcal generalty, we also assume 1) a unform ncrease n prces by all n CPs cannot result n an ncrease n any CP s demand volume, and 2) a prce ncrease by a gven CP cannot result n an ncrease n the market s aggregate end-user demand. Mathematcally, we represent these two facts by the followng two relatonshps. y > j β j,=1,..., n (7) and y > j β j,=1,..., n (8) The long run average proft for CP n a gven tme perod, assumng that response tmes are expressed n terms of expected values, s a functon of the prce and QoS levels of CPs, and s gven as P (pr, s) =λ (pr c ρ ) ρ rt s, (9) The proft functon for each CP acts as ts utlty/payoff functon when t s nvolved n prce and QoS games wth other competng CPs. We assume n ths paper that the proft functon for each CP s known to other CPs, but none of the CPs know the values of the parameters that other competng CPs adopt as ther strategy. Problem Statement: Gven the proft functon for each CP (publc nformaton), how would each advertse ts prce and QoS values (wthout negotatng wth other CPs) to end-users so as to maxmze ts own proft. In other words, n a compettve game of profts played by CPs, s there a stuaton where each CP s happy wth ts (prce, QoS) advertsed par and does not beneft by a postve or negatve devaton n the values of the advertsed par. In ths paper, we study games nvolvng prce and QoS as the prmary parameters,.e., we characterze and analyze the exstence, unqueness, and convergence of Nash equlbra. Our prmary goal s to compute the optmal prce and QoS levels offered by CPs to ts end-users under market competton. Our analyss paves the path for each cloud provder to 1) know what prce and QoS levels to set for ts clents (end-users) for a gven applcaton type, such that t could exst n the cloud market, and 2) practcally and dynamcally provson approprate capacty for satsfyng advertsed QoS guarantees, by takng advantage of the property of vrtualzaton n cloud networks. The property of vrtualzaton entals each CP to allocate optmal resources dynamcally n a fast manner to servce end-user requests. Usng our prcng framework, n each tme perod, cloud provders set the approprate prce and QoS levels after competng n a

10 10 Symbol U = P pr pr pr c λ ρ rt rt C φ s s s x () α j() Meanng Utlty functon of CP Prce charged by CP per end-user Prce vector of CPs Nash equlbrum prce vector Cost ncurred by CP to servce each user Arrval rate of end-users to CP cost/unt of capacty provsonng by CP response tme upper bound guarantee response tme guarantee by CP Capacty cost of CP for provsonng ts user demands percentle parameter QoS level guarantee provded by CP to ts users QoS vector of CPs Nash equlbrum QoS vector ncreasng, concave, and a thrce dfferentable functon non-decreasng and dfferentable functon Table 1. Lst of Symbols and Ther Meanng game; the resultng prces drve end-user demand; the CPs then allocate optmal resources to servce demand. Remark. We decded to not analyze a compettve market,.e., where CPs are prce/qos takng and a Walrasan equlbrum results when demand equals supply, because a compettve market analyss s manly applcable when the resources traded by an organzaton are neglgble wth respect to the total resource n the system [9][10]. In a cloud market ths s defntely not the case as there are a few cloud provders and so the resource traded by one s not neglgble wth respect to the total resources traded n the system. Therefore we analyze olgopolstc markets where CPs are prce/qos antcpatng. We consder the followng types of prce-qos game models n our work. 1. CP QoS guarantees are pre-specfed; CPs compete wth each other for prces, gven QoS guarantees. (Game 1) 2. CPs compete for prce and QoS smultaneously. (Game 2) 3. CP prce levels are pre-specfed; CPs compete for QoS levels. (Game 3). Game 3 s a specal case of Game 2 and n Secton 4, we wll show that t s a Game 2 dervatve. Lst of Notatons: For reader smplcty, we provde a table of most used notatons related to the analyss of games n ths paper. 3 Game 1 - Prce Game In ths secton we analyze the game n whch the QoS guarantees of CPs are exogenously specfed and the CPs compete for prces. Game Descrpton

11 11 Players: Indvdual cloud provders; Game Type: Non-cooperatve,.e., no nteracton between CPs; Strategy Space: Choosng a prce n range [pr mn, pr max ]; Player Goal: To maxmze ts ndvdual utlty U = P Our frst goal s to show that ths game has a unque prce Nash equlbrum, pr (an nstance of vector pr), whch satsfes the followng frst order condton P pr = y (pr c ρ )+λ,, (10) whch n matrx notaton can be represented as M pr = x(s)+z, (11) where M s an n n matrx wth M =2y, M j = β j, j, and where z = y (c + ρ ). We have the followng theorem and corollary regardng equlbrum results for our game. The readers are referred to the Appendx for the proofs. Theorem 1: Gven that the QoS guarantees of CPs are exogenously specfed, the prce competton game has a unque Nash equlbrum, pr, whch satsfes Equaton 11. The Nash equlbrum user demand, λ, for each CP evaluates to y (pr c ρ ), and the Nash equlbrum profts, P, for each CP s gven by y (pr c ρ ) 2 ρ. rt s Corollary 1: a) pr and λ are ncreasng and decreasng respectvely n each of the parameters {c,ρ,=1, 2,..., n}, and b) pr s j = 1 λ y s j =(M 1 ) j x j (s j) l j (M 1 ) l x lj (s j). Corollary 1 mples that 1) under a larger value for CP s degree of postve externalty δ, t s wllng to make a bolder prce adjustment to an ncrease n any of ts cost parameters, thereby mantanng a larger porton of ts orgnal proft margn. The reason s that competng CPs respond wth larger prce themselves, and 2) there exsts a crtcal value 0 s 0 j rt such that as CP j ncreases ts QoS level, pr and λ are ncreasng on the nterval [0,s 0 j ), and decreasng n the nterval [s 0 j, rt). Senstvty Analyss: We know the followng relatonshp P s j =2y (pr c ρ ) pr s j (12) From t we can nfer that CP s proft ncreases as a result of QoS level mprovement by a competng CP j f and only f the QoS level mprovement results n an ncrease n CP s prce. Ths happens when P ncreases on the nterval [0,s 0 j ] and decreases on the remanng nterval (s0 j, rt]. In regard to proft varaton trends, on ts own QoS level mprovement, a domnant trend for a CP s not observed. However, we make two observatons based on the holdng of the followng relatonshp P s j =2y (pr c ρ ) pr s j ρ (rt s ) 2 (13)

12 12 If a CP ncreases ts QoS level from 0 to a postve value and and ths results n ts prce decrease, s equlbrum profts become a decreasng functon of ts QoS level at all tmes. Thus, n such a case s better off provdng mnmal QoS level to ts customers. However, when CP s QoS level ncreases from 0 to a postve value resultng n an ncrease n ts prce charged to customers, there exsts a QoS level s b such that the equlbrum profts alternates arbtrarly between ncreasng and decreasng n the nterval [0,s b ), and decreases when s s b. Convergence of Nash Equlbra: Snce the prce game n queston has a unque and optmal Nash equlbra, t can be easly found by solvng the system of frst P order condtons, pr = 0 for all. Remark. It s true that the exstence of NE n convex games s not surprsng n vew of the general theory, but what s more mportant s whether a realstc modelng of our problem at hand results n a convex game. Once we can establsh that our model results n a convex game, we have a straghtforward result of the exstence of NE from game theory lterature. Ths s exactly what we do n the paper,.e., to show that our model s realstc and ndeed leads to a convex game thus leadng further to the exstence of NE. 4 Game 2 - Prce-QoS Game In ths secton, we analyze the game n whch the CPs compete for both, prce as well as QoS levels. In the process of analyzng Game 2, we also derve Game 3, as a specal case of Game 2, and state results pertanng to Game 3. Game Descrpton Players: Indvdual cloud provders; Game Type: Non-cooperatve,.e., no nteracton between CPs; Strategy Space: prce n range [pr mn s ; Player Goal: To maxmze ts ndvdual utlty U = P We have the followng theorem regardng equlbrum results. 4yρ Theorem 2: Let rt 3, pr max ] and QoS level (x ) 2, where y = mn y,ρ = mn ρ, x = max x (0). There exsts a Nash equlbrum (pr,s ), whch satsfes the followng system of equatons: P pr = y (pr c ρ )+λ =0,, (14) and satsfes the condton that ether s (pr ) s the unque root of x (s )(pr ρ c ρ )= 1 f pr (rt s ) 2 c + ρ (1 + rt 2 x (0)) or s (pr )=0otherwse. Conversely, any soluton of these two equatons s a Nash equlbrum. Senstvty Analyss: We know that s (pr ) depends on x (s ) and pr. Thus, from the mplct functon theorem [11] we nfer that the QoS level of CP ncreases wth the ncrease n ts Nash equlbrum prce. We have the followng

13 13 relatonshp for pr >c + ρ (1 + 1 rt 2 x (0)), s (pr )= x (s ) x (s > 0, (15) )(pr c ρ ) ρ (rt s ) 2 whereas s (pr 1 ) = 0 for pr < c + ρ (1 + We also notce that for rt 2 x (0)). 1 pr >c + ρ (1 + rt 2 x (0)), s ncreases concavely wth pr. The value of s (p ) obtaned from the soluton of the equaton x (s )(pr c ρ )= f ρ (rt s ) 2 1 pr c + ρ (1 + can be fed nto Equaton 15 to compute the prce rt 2 x (0)), vector. The system of equatons that result after substtuton s non-lnear n vector pr and could have multple solutons,.e., multple Nash equlbra. Inferences from Senstvty Analyss: Games 1, 2, and 3 gves us non-ntutve nsghts to the prce-qos changes by ndvdual CPs. We observe that the obvous ntutons of equlbrum prce decrease of competng CPs wth ncreasng QoS levels and vce-versa do not hold under all stuatons and senstvty analyss provde the condtons under whch the counter-result holds. Thus, the ntrcate nature of non-cooperatve strategy selecton by ndvdual CPs and the nterdependences of ndvdual strateges on the cloud market make cloud economcs problems nterestng. Convergence of Nash Equlbra: Snce multple Nash equlbra mght exst for the prce vectors for the smultaneous prce-qos game, the tatonnement scheme [9][12] can be used to prove convergence. Ths scheme s an teratve procedure that numercally verfes whether multple prce equlbra exst, and unqueness s guaranteed f and only f the procedure converges to the same lmt when ntal values are set at pr mn or pr max. Once the equlbrum prce vectors are determned, the equlbrum servce levels are easly computed. If multple equlbra exst the cloud provders select the prce equlbra that s component-wse the largest. Regardng the case when CP prce vector s gven, we have the followng corollary from the result of Theorem 2, whch leads us to equlbrum results of Game 3, a specal case of Game 2. Corollary 2. Gven any CP prce vector, pr f, the Nash equlbrum s(pr f ) s the domnant soluton n the QoS level game between CPs,.e., a CP s equlbrum QoS level s ndependent of any of ts compettors cost or demand characterstcs and prces. When s (pr f ) > 0, the equlbrum QoS level s ncreasng and concave n pr f, wth s (prf )= x (s). x (s)(prf c ρ) 2ρ (rt s ) 3 We observe that Game 3 beng a specal case of Game 2 entals a unque Nash equlbrum, whereas Game 2 entals multple Nash equlbra.

14 14 5 Optmzaton Framework for Capacty Provsonng In ths secton, we develop optmzaton models for optmally provsonng capacty n both, sngle-ter as well as mult-ter cloud networks. As mentoned n prevous sectons, the term capacty has a queueng-theoretc noton to t and s the servce rate of a queueng system processng user requests,.e., t s the number of user requests processed per unt of tme. The capacty measure can be translated to allocatng hardware and other system resources optmally so as to satsfy user QoS demands. In the followng subsectons, we frst deal wth the capacty analyss n sngle ter clouds, whch s followed by the analyss n mult-ter cloud networks. 5.1 Sngle-Ter Case We model each CP as an M/M/1 queueng system wth frst-come, frst-serve (FCFS) schedulng, where end-user requests arrve as a Posson process wth mean rate λ, and gets servced at a rate µ. We adopt an M/M/1 queueng system because of three reasons: 1) queueng theory has been tradtonally used n request arrval and servce problems, 2) for our problem, assumng an M/M/1 queueng system ensures tractable analyses procedures that entals dervng nce closed form expressons and helps understand system nsghts n a non-complex manner, wthout sacrfcng a great deal n capturng the real dynamcs of the actual arrval-departure process, and 3) The Markovan nature of the servce process helps us generalze expected steady state analyss and percentle analyss together. We assume that each CP adopts the FCFS schedulng polcy because they serve a sngle class of end-users wth the same QoS level guarantees. The metrc for end-user satsfacton n queueng systems s response/watng tme. The response tme rt may be defned, ether n terms of the expected steady state response tme,.e., rt = E(RT ), or n terms of φ-percentle performance, rt (φ), where 0 <φ< 1. Thus, n terms of φ-percentle performance 12, P (RT < rt (φ)) = φ. Accordng to the theory of M/M/1 queues, we have the followng standard results [17]. rt = 1 µ λ, (16) rt (φ) = ln( 1 1 φ ) µ (φ) λ, (17) µ = λ + 1 rt, (18) 12 As an example, n cloud networks we often assocate provsonng power accordng the 95th percentle use. Lkewse, we could also provson servce capacty by accountng for percentle response tme guarantees.

15 15 and µ (φ) =λ + ln( 1 1 φ ) rt (φ) (19) Equatons 20 and 22 follow from the fact that for M/M/1 queues, the followng result holds, P (RT < rt (φ)) = φ =1 e (µ λrt(φ)) (20) The nverse of rt (rt (φ)) s s (s (φ)), whch s the advertsed QoS level guarantee of CP to ts end-users. Thus, we observe from Equatons 21 and 22 that the queueng servce rate (capacty) s lnear n λ and s (s (φ)). Snce C s proportonal to µ (µ (φ)), we nfer that C s lnear n λ and s (q (φ)). Our am n ths paper s to fnd the optmal µ(µ (φ)) for each CP such that ts advertsed QoS level guarantees to ts end-users are satsfed, wthout wastng any resources. Assumng that t takes a cost of ρ for CP to provson a sngle unt of servce capacty, we have the followng optmzaton problems consderng the expected value and percentle value of response tme respectvely. mn ρ µ subject to and subject to 1 µ λ rt mn ρ µ (φ) log( 1 1 φ ) µ (φ) λ rt (φ) 5.2 Mult-ter Case In order to model the mult-ter case, we model a gven cloud network for CP as a network of queues. Each queue n the network acts as a M/M/1 queue servng end-user requests n a FCFS manner. We assume that the queueng network s an open Jackson network [17]. We also assume the queueng network for any CP s dstnct from other CP queung networks,.e., for CP, there s no queue n ts network that serves any other CP j, j. Each queue s representatve of a ter n a cloud network and s represented as a vertex/node n the open Jackson network. The departure process of one ter/level s an arrval process for the next ter. We defne the followng notatons n relaton to our analyss of queueng networks for CP V - set of n vertces n an open Jackson network for CP. π j - fracton of end user requests that start servcng at node j.

16 16 p jk - probablty that a user request moves to node k after gettng servce from node j. P - matrx of p jk values and s sub-stochastc n nature,.e., Lt n (P ) n =0 µ j ρ j - servce rate of node j ɛ V - capacty cost per unt of servce rate at node j. Ω - vector of aggregate arrval rates for CP. The vector of arrval rates for each CP s expressed as recursve expresson of the form Ω = λ π +(P ) T Ω (21) Solvng the above equaton, we get Ω = λ δ, (22) where the vector δ =(I (P ) T ) 1 π. Accordng to queueng theory results regardng networks of queues, we get the followng for expressons for each CP (for the expected value case of response tme) 13 and By Lttle s Law, we have Ω j E[requests atnode j] = µ j Ω j E[total number system requests] =λ s 1 = j ɛ V δ j µ j λ Ω j j ɛ V δ j µ j Ω j (23) (24) (25) We now prove through the followng theorem that even n mult-ter cloud networks, C s lnear n λ and s, for each CP. Ths fact regardng lnearty s mportant when t comes to the ease of analyzng prce-qos games. Theorem 3. The capacty provsonng cost, C, for each cloud provder n a mult-ter cloud network s lnear n ther user arrval rate and the advertsed QoS level guarantee. 13 Due to the Markovan nature of the servce process, the case for general percentles s exactly smlar to the case for expected steady state analyss. The expressons reman nearly the same apart from a constant factor multplcaton.

17 17 Proof. Each cloud provder s wllng to mnmze ther capacty costs. Thus t selects µ =(µ j : j ɛ V ) such that t s the soluton of the followng constraned optmzaton problem mn j ɛ V ρ jµ j subject to j ɛ V δ j µ j λ δ j s 1 Applyng Karush-Kuhn-Tucker (KKT) condtons [25] for optmalty, we have ρ j = γδ j (µ j(opt) λ δ j )2, j ɛ V, (26) where γ s the Lagrange multpler. From the prevous equaton we get µ j(opt) λ δj = δ j γ ρ, j ɛ V (27) j The mnmum cost of CP evaluates to j ɛ V ρ j µ j(opt), whch s of the form A 1 λ + A 2 s, where A1 = ρ jδj (28) j ɛ V and A2 =( j ɛ V δ j ρ j )2 (29) Thus, the capacty provsonng cost per CP n a mult-ter cloud network s lnear n ther user arrval rate and the advertsed QoS level guarantee. We emphasze that the theorem holds (due to the Markovan nature of the servce tmes) when we consder the response-tme as a percentle parameter, rather than an expected value. Q.E.D. Optmzaton Problems: We have the followng two optmzaton problems for mult-ter networks consderng the expected value and percentle value of response tme respectvely. mn j ɛ V ρ jµ j subject to and j ɛ V δ j µ j λ δ j mn s 1 j ɛ V ρ jµ j(φ)

18 18 subject to 1 log( 1 φ ) δj µ j ɛ V j (φ) λ δj s 1 (φ) The optmzaton problems for the sngle-ter and mult-ter cases provde a framework va whch resources can be provsoned n the cloud n a manner so as to mnmze over-provsonng n a dynamc manner. 6 Concluson and Future Work In the frst part of the paper, we developed nter-organzatnal economc models for prcng cloud network servces when several cloud provders co-exst n a market, servcng a sngle applcaton type. We devsed and analyzed three prce- QoS game-theoretc models relevant to cloud networks. We proved that a unque pure strategy Nash equlbrum (NE) exsts n two of our three QoS-drven prcng models. In addton, we also showed that the NE s converge;.e., there s a practcally mplementable algorthm for each model that computes the NE/s for the correspondng model. Thus, even f no unque Nash equlbrum exsts n some of the models, we are guaranteed to fnd the largest equlbra (preferred by the CPs) through our algorthm. Regardng convergence to Nash equlbra, t s true that t could take a long tme for convergence of Nash equlbra (computng NE s PPAD Complete [18]), however n 95% of the cases n practcal economc markets, NE s acheved n a decent amount of tme. Our prce-qos models can drve optmal resource provsonng n cloud networks. The NE prce and QoS levels for each cloud provder drves optmal end-user demand n a gven tme perod w.r.t. maxmzng ndvdual CP profts under competton. Servcng end-user demands requres provsonng capacty. Once the optmal values are computed, the power of vrtualzaton n cloud networks makes t possble to execute dynamc resource provsonng n a fast and effcent manner n multple tme perods. In ths regard, n the second part of the paper, we developed an optmzaton framework for sngle-tered and multtered cloud networks to compute the optmal provsoned capacty once the equlbrum prce and QoS levels for each CP have been determned. As part of future work, we plan to extend our analyss to the case where cloud provders are n smultaneous competton wth other CPs on multple applcaton types. 7 Appendx Proof of Theorem 1. Proof: For a gven servce level vector s, each CP reserves a capacty of 1 rt = 1. Consder the game G wth proft/utlty functons for each CP rt s represented as P =(x (s ) y p j α j (s j )+ j (β j p j )(pr c ρ ) W, (30)

19 19 where ρ W = rt s 2 P pr pr j Snce = β j, the functon P s supermodular 14. The strategy set of each CP les nsde a closed nterval and s bounded,.e., the strategy set s [pr mn, pr max ], whch s a compact set. Thus, the prcng game between CPs s a supermodular game and possesses a Nash equlbrum [13]. Snce y > j β j, =1,..., n (by Equaton 7), 2 P > 2 P pr 2 j pr pr j and thus the Nash equlbrum s unque. Rewrtng Equaton 11 and usng Equaton 6, we get λ = y (pr c ρ ). Substtutng λ n Equaton 9, we get P = y (pr c ρ ) 2 ρ Q.E.D. rt s Proof of Corollary 1. Proof: Snce the nverse of matrx M,.e., M 1 exsts and s greater than or equal 0[14], from pr = M 1 (x(s)+z) (Equaton 11), we have pr s ncreasng n {c,ρ =1, 2,..., n}. Agan, from Lemma 2 n [14], we have δ y (M 1 ) 0.5 δ < 1, where δ s the degree of postve externalty 15 faced by CP from other CP (prce, QoS) parameters, and t ncreases wth the β coeffcents. Ths leads us to pr c = pr ρ = y (M 1 ) = δ > 0. Therefore, we show n another dfferent way that pr s ncreasng n {c,ρ,=1, 2,..., n}. Snce M 1 exsts and s greater than or equal to 0, we agan have λ c = λ ρ = y ( pr ρ 1) = y ( pr 1) = y (δ 1) < 0, from whch we conclude that λ s decreasng n c {c,ρ,=1, 2,..., n}. Part b) of the corollary drectly follows from the fact that the nverse of matrx M,.e., M 1 exsts, s greater than or equal 0, and every entry of M 1 s ncreasng n β j coeffcents. Q.E.D. Proof of Theorem 2. Proof: To prove our theorem, we just need to show that the proft functon P s jontly concave n (pr,s ). Then by the Nash-Debreu theorem [15], we could nfer the exstence of a Nash equlbra. We know the followng results for all CP P pr = y (pr c ρ )+λ (31) and P θ = x (s )(pr c ρ ) ρ (rt s ) 2 (32) 2 P s pr = Thus, 2 P = 2y pr 2 < 0, 2 P = x s 2 (s )(pr c ρ ) 2ρ < 0, (rt s ) 3 x (s ). We determne the determnant of the Hessan as 2y (x (s )(pr c ρ ) ρ 0 (the suffcent condton for P (rt s ) 2 to be jontly concave n (pr,s )), 14 A functon f : R n R s supermodular f t has the followng ncreasng dfference property,.e., f(m 1,m ) f(m 2,m ), ncreases n m for all m 1 >m 2 n (pr, pr j). The readers are referred to [16] for more detals on supermodularty. 15 A postve externalty s an external beneft on a user not drectly nvolved n a transacton. In our case, a transacton refers to a CP settng ts prce and QoS parameters.

20 20 f the followng condton holds: 4y ρ pr 2 (x (s )) 2 4y 3 ρ rt mn s (x (s )) 2 = 3 4y ρ, (33) (0))2 where the last equalty follows from the fact that x > 0 and x s decreasng. Now snce pr = pr (s ), by Theorem 1 t s n the closed and bounded nterval [pr mn, pr max ] and must therefore satsfy Equaton 15. Agan from Equaton 31, we have P s as s tends to rt, whch leads us to the concluson that s (pr ) s the unque root of x (s )(pr c ρ )= ρ 1 f pr (rt s ) 2 c + ρ (1 + rt 2 x (0)) or s (pr ) = 0 otherwse. Q.E.D. Proof of Corollary 2. Proof: Substtutng pr max = pr mn = pr f nto Theorem 2 leads us to the fact that s(pr f ) s a Nash equlbrum of the QoS level competton game amongst CPs and that t s also a unque and a domnant soluton, snce s(pr f ) s a functon of pr, c, and ρ only. (Followng from the fact that s (pr ) s the unque root of x (s ρ )(pr c ρ )= 1 f pr (rt s ) 2 c + ρ (1 + or rt 2 x (0)) s (pr ) = 0 otherwse.) Q.E.D. (x References 1. M. Armbrust, A. Fox, R. Grffth, A. D. Joseph, R. H. Katz, A. Konwnsk, G. Lee, D. A. Patterson, A. Rabkn, I. Stoca, and M. Zahara. Above the clouds: A Berkeley Vew Of Cloud Computng. Techncal Report, EECS, U. C. Berkeley, S. C. M. Lee and J. C. S. Lu. On The Interacton and Competton Among Internet Servce Provders. IEEE Journal on Selected Areas n Communcatons, 26, S. Shakkota and R. Srkant. Economcs Of Network Prcng Wth Multple ISPs. IEEE/ACM Transactons on Networkng, 14, P. Hande, M. Chang, R. Calderbank, and S. Rangan. Network Prcng and Rate Allocaton Wth Content-provder Partcpaton. In IEEE INFOCOM, L. Jang, S. Parekh, and J. Walrand. Tme-dependent Network Prcng and Bandwdth Tradng. IEEE BoD, J. K. Macke-Mason and H. R. Varan. Prcng Congestble Network Resources. IEEE Journal on Selected Areas n Communcatons, 13, D. Songhurst and F. Kelly. Chargng Schemes For Multservce Networks. 15th Internatonal Teletra?c Congress, P. Dube, R. Jan, and C. Touat. An Analyss of Prcng Competton For Queued Servces Wth Multple Provders. ITA Workshop, H. R. Varan. Mcroeconomc Analyss. Norton, M. E. Wetzsten. Mcroeconomc Theory: Concepts and Connectons. South Western, W.Rudn. Prncples of Mathematcal Analyss. Mc.Graw Hll, K.Arrow. Handbook of Mathematcal Economcs. North Holland, X. Vves. Nash Equlbrum and Strategc Complementartes. J. Mathematcal Economcs, 19, F. Bernsten and A. Federgruen. Compartve Statcs, Strategc Complements, and Substtutes n Olgopoles. Journal of Mathematcal Economcs, 40, 2004.

21 15. D.Fudenberg and J.Trole. Game Theory. MIT Press, D. M. Topks. Supermodularty and Complementarty. Prnceton Unversty 17. D. Bertsekas and R. Gallager. Data Networks. Prentce Hall Inc., C. Daskalaks, P. W. Goldberg, and C. H. Papadmtrou. The Complexty of Computng A Nash Equlbrum. SIAM Journal of Computng. 39(1), D.Gmach, J.Rola, L.Cherkasova, and A.Kemper. Capacty Management and Demand Predcton for Next Generaton Data Centers. IEEE Internatonal Conference on Web Servces, T.Wood, L.Cherkasova, K.Ozonat, and P.Shenoy. Proflng and Modelng Resource Usage of Vrtualzed Applcatons. ACM Internatonal Conference on Mddleware, D.Kusc and N.Kandasamy. Rsk-Aware Lmted Lookahead Control for Dynamc Resource Provsonng n Enterprse Computng Systems. IEEE ICAC, P.Padala, K.G.Shn, X.Zhu, M.Uysal, Z.Wang, S.Snghal, A.Merchant, and K.Salem. Adaptve Control of Vrtualzed Resources n Utlty Computng Envronments. ACM SIGOPS, X.Meng, C.Isc, J.Kephart, L.Zhang, and E.Boulett. Effcent Resource Provsonng n Compute Clouds va VM Multplexng. ACM ICAC, J.Dejun, G.Perre, and C-H.Ch. Autonomous Resource Provsonng for Mult- Servce Web Applcatons. ACM WWW, S. Boyd and L. Vanderberghe. Convex Optmzaton. Cambrdge Unversty Press,