Optmal Schedulng n the Hybrd-Cloud Mark Shfrn Faculty of Electrcal Engneerng Technon, Israel Emal: shfrn@tx.technon.ac.l Ram Atar Faculty of Electrcal Engneerng Technon, Israel Emal: atar@ee.technon.ac.l Israel Cdon Faculty of Electrcal Engneerng Technon, Israel Emal: cdon@ee.technon.ac.l Abstract The emergng hybrd cloud archtecture allows organzatons to optmze ther computaton needs and costs by mantanng ther prvate computatonal nfrastructure at hgh utlzaton and meetng peak requrements by offloadng selected tasks to the publc cloud. Consequently, there s a need to devse effcent systems equpped wth onlne task cloudburstng algorthms that optmze the overall cost whle mantanng adequate qualty of servce. Such algorthms must take nto account the dfference n communcaton and computatonal requrements assocated wth dfferent tasks. For example, t s clear that when two tasks have the same local computatonal requrements, the one wth the lower cloudburstng cost s a better canddate to be off-loaded and sent to the cloud. In ths paper, we address the case n whch arrvng tasks have the same computatonal cost but dfferent communcaton costs. We desgn schedulng system based on onlne decson algorthms drven by the user s local nfrastructure constrants. We model the onlne schedulng problem as Markov Decson Process (MDP) problem and provde optmal polces for schedulng tasks ether locally or remotely. We further explore the usage of MDP n dfferent scenaros and prove the structural propertes of the optmal polces n order to ncorporate them nto the decson engne. The desgn of the practcal schedulng system s supported by the analytcal results and numercal evaluatons. We demonstrate the practcal computatonal advantage of threshold type polces and provde an nsght nto ther dependence on system parameters. I. INTRODUCTION The emergence of cloud computng s changng the ways n whch organzatons address ther nformaton technology (IT) needs. Cloud computng s a new way of ncreasng computatonal and storage capacty wthout nvestng n new nfrastructure, tranng new personnel, or lcensng new software. In theory, the cloud s aglty and elastcty make t the best IT nfrastructure soluton. However, many practcal ssues, such as lmted network speeds, lack of strct SLA guarantees and cloud standards, lmt the full adopton of a cloud model [2]. Consequently, there s a topcal trend to leverage the best of both worlds by keepng the mnmal essental IT nfrastructure whle adoptng the publc cloud ether when t s more cost effectve or when t s dctated by performance requrements. One of the terms whch are frequently used to descrbe ths paradgm s a hybrd cloud. Ths term refers to organzatons that own some of the computaton nfrastructure, whle also utlzng the servces of a cloud provder to augment or supplement the prvate nfrastructure. Another topcal term assocated wth a hybrd cloud s cloudburstng. Accordng to ths concept, n case the prvate data center runs out of computng resources, the organzaton bursts (.e. offloads) workload to an external cloud on an on-demand bass. The nternal computng resource s the Prvate Cloud, whle the external cloud s typcally a Publc Cloud, for whch the organzaton gets charged on a payper-use bass. Effectve cloudburstng offers the organzaton a good trade-offs between overall computaton costs, avalablty and performance. We term the onlne decson process whch determnes whether to locally process the tasks at the prvate cloud or to cloudburst them to the publc cloud as a schedulng process. The hybrd cloud s envronment poses new research challenges assocated wth effectve task schedulng. If all the computatonal tasks were dentcal, a smple schedulng mechansm would track the backlog of tasks watng for local executon; f the backlog crossed a certan value assocated wth the maxmal allowed latency or maxmal space lmt, arrvng tasks would be drected to the cloud. As the IT tasks are heterogeneous n terms of computaton, communcaton and storage requrements, the cost-effectve desgn of such task schedulng mechansms becomes more challengng. Therefore, the goal of ths paper s to explore the modelng of these new problems and to fnd the algorthmc soluton to them. To that end, we frst propose a user-level schedulng system whch comprses task schedulng algorthms. We assume that the goal of the cloud user s to mnmze the cost of servng the arrvng tasks by usng the combnaton of a resource lmted prvate cloud and cloudburstng whle meetng a predefned QoS level. The model defnton nvolves fndng schedulng algorthms to handle the flow of several task types wth dfferent resource consumpton requrements. Such algorthms must take nto account the dfference n communcaton and n computatonal requrements among dfferent task types. The dfference n requrements mpose dfferent cloudburstng costs. Our schedulng system s equpped wth optmal onlne decson algorthms for modelng and solvng several assocated MDP problems. The new system s desgned to handle the task nflux whch conssts of several task types wth dfferent cloudburstng costs. We show that the optmal schedulng polcy has a threshold structure. That s, the optmal decson s to drect the tasks to a prvate cloud n case the number of already present tasks s below a certan threshold. Otherwse, the tasks must be cloudburst to the publc cloud. The schedulng rule can be easly mplemented due to the threshold structure of the optmal polcy. Hence, the proof to structural propertes of the optmal polcy s provded. To the best of our knowledge, ths s the frst work that proposes optmal schedulng polces utlzng control theory and MDP methods for the hybrd cloud envronment, n whch tasks can be ether served n an exstng prvate cloud or cloudburst onto the publc cloud. The optmzaton of a schedulng process naturally lends tself to an optmzaton of queung system whch can be tackled as an MDP problem. The drect technques of fndng the optmal polcy for the correspondng MDP are related to the methods of value teraton, [7],[22],[31]. Ths process s known to become computatonally consumng once the state space of a problem ncreases. Consequently, the mere understandng that 978-3-901882-50-0 c 2013 IFIP 51
the optmal polcy has a smple structure such as a threshold type, facltates the computatonal effort of the schedulng process. Our proposed system s enhanced wth estmaton technques whch can be appled n order to calculate the optmal polcy assocated wth MDP problems wth sgnfcant reducton n complexty. Partcularly, when system parameters change, polcy adjustment can be effectvely appled. To summarze, the contrbuton of ths paper s two-fold: A user-level schedulng system provded wth the MDP based technques, whch calculates the decson whether to serve heterogeneous tasks n the prvate cloud or to cloudburst them onto the publc cloud. The dervaton of the optmal schedulng polcy for the scenaro presented above. We present and prove several structural propertes of the optmal polcy,.e. the threshold-type polcy. The threshold polcy facltates the onlne polcy calculaton (adjustment) wth reduced complexty. The correspondng algorthm s provded. A. Related Work We refer to two related subtopcs: cloudburstng decson systems that are related to the practcal problem addressed n our paper and works whch address MDP threshold polces that are related to our analytcal results. Cloudburstng: Only few works have addressed cloudburstng as a decson problem. An onlne schedulng system n a hybrd cloud s presented n [26]. Unlke ths work, where the addressed performance metrc s system response tme whch s heurstcally mnmzed, the objectve of our system s an optmzaton of the combned cost. [19],[20],[25] suggest a software soluton to the nteracton between the users and a cloud, but do not present effcent schedulng algorthms. [23] develops a rate-lmtng archtecture for the cloud. [13] presents the soluton to the permanent mgraton of tasks to the cloud, whle our paper addresses dynamc cloudburstng. Some works whch present task schedulng combned wth resource allocaton n other systems can be appled to the cloud computng, e.g. [6],[11]. The methods used n the aforementoned papers are heurstc and therefore suboptmal, whle we consder the optmal schedulng n hybrd cloud by usng the soluton to the correspondng MDP. The schedulng of resources nsde the publc cloud constraned by cost-aware metrcs s formulated as a lnear programmng (LP) problem n [27]. The system closest to ours s descrbed n [30]. However, smlarly to [27], t employs LP methods. MDP and threshold polces: The prmary tools that we utlze are Markov Decson Process and proofs to the structural propertes of the underlyng optmal polcy. Regardng the dervaton of threshold-type polcy for networkng systems, numerous notable works can be found, e.g. [12],[10],[33],[29],[8],[18]. Other works are related to the analytcal methods used n our paper. Specfcally, tools from flud and dffuson approxmatons have been appled to address models that are closely related to those we ntroduce here. In partcular, a sgnfcant amount of work has been done n recent years on models wth a large number of servers (see [3],[4],[5],[14] and references theren). The approach presented n those papers extends the task nflux model beyond the Posson dstrbuton constrant, whch s the assumpton throughout ths paper. Arrval processes of other than Posson dstrbuton pose dfferent analytcal challenges and are left for the future work. Fg. 1. Cloudburstng Schedulng System The rest of the paper s organzed as follows. In secton II, we descrbe the user perspectve hybrd cloud schedulng system. We provde a detaled descrpton of each component of the system. In secton III, we concentrate on the MDP soluton and on ts structural propertes, where the cases of a sngle server for all tasks and a dedcated server for each task type are treated separately. Consequently, we show the correspondng one-dmensonal or mult-dmensonal thresholds. The fnal secton s devoted to numercal results and practcal models. II. USER PERSPECTIVE HYBRID CLOUD SCHEDULING SYSTEM We start wth presentng a user perspectve schedulng system n a hybrd cloud envronment. The system controls the user envronment, whch contans a prvate cloud wth lmted computatonal resources and an access to a publc cloud. Partcularly, t performs a real-tme schedulng of the nflux of heterogeneous tasks. The block representaton of the system s llustrated n Fgure 1. The components of the system nclude the analyss of nput processes, the computaton of schedulng polcy by solvng the MDP problem, polcy adjustment, the schedulng unt whch mplements the optmal polcy, and the cloud toolkt whch mantans the control channel wth the publc cloud. We elaborate n detal the objectve of each unt and show how dfferent unts nterconnect. Relevant detals about the confguraton of the prvate and the publc clouds, as well as the underlyng assumptons, are brought theren too. Prvate cloud confguraton: We assume that the computatonal capabltes of a prvate cloud are lmted and fxed. Clearly, n case there s a performance ndependence among dfferent task types, the problem can be effectvely splt nto separate ndependent problems. However, n a practcal system, there s a common resource shared by all tasks regardless of the task type. We express ths lmtaton by usng a fnte shared task buffer, and that can be vewed ether as a watng delay constrant or as a task capacty constrant. We vew two basc server settngs (confguratons) of a prvate cloud: 52 2013 IFIP/IEEE Internatonal Symposum on Integrated Network Management (IM2013)
Sngle-server model. Specfcally, a prvate cloud has a sngle server resource whch serves all the tasks n a FIFO order wth equal servce rates. In ths case, tasks of dfferent types compete wth each other over the computatonal resources. Mult-server model. Here, we assume that each task type can only be processed by a dedcated server, but the tasks share a common storage nfrastructure. In ths case, tasks of dfferent types compete wth each other over the storage resources. In order to smplfy the mathematcal treatment of the MDP, we assume no mantenance (holdng) cost n a prvate cloud. However, t can be easly shown that convex, e.g. lnear holdng cost nflcts no complcaton regardng the optmal soluton and no change n the structure of the optmal polcy. Input process profler: The task nput s fed to the schedulng unt and tracked by a specal unt whch performs nput process analyss (proflng), denote t as (IP). The man objectve of the IP s task proflng,.e. an estmaton of the arrval task rate correspondng to each task type. Throughout the paper, we assume that the arrval rate of each task type can be estmated as constant over long tme perods,.e the arrval processes are quas-constant. Schedulng polcy computaton: The schedulng polcy computaton s provded by a desgnated unt. The nput to ths unt s the average arrval rates of the tasks whch are fed by the IP, as well as the prces correspondng to each task type. The output of ths unt s the optmal polcy structure whch s calculated by solvng the correspondng MDP. As demonstrated n the next secton, the optmal polcy s a threshold polcy. Thus, the decson rule whch s provded for the schedulng unt s rather smple to mplement. To ths end, sub-unts of ths unt are Polcy Evaluaton (PE) sub-unt and Polcy Adjustment (PA) sub-unt. The PE sub-unt mplements the value teraton of the MDP resultng n a numercally calculated value functon. Then t fnds the optmal polcy whch corresponds to the value functon. (See [7] for MDP defnton and soluton). The PA sub-unt comes n use once a partcular change n parameters occurs, such as an update of prcng for the cloudburstng of one of the task types onto the publc cloud, or a change n arrval rate. Consequently, a polcy estmaton technque mght be appled, whle an old polcy s used n order to fnd the updated polcy. The mplementaton nvolves the detals of the value teraton procedure and s referred to n secton IV. The utlzaton of the PA sub-unt s especally effectve for the mult-dmensonal case, n whch the state space of the MDP s comparatvely large, and the value teraton process s a resource-consumng procedure. The computatonal effectveness of the polcy estmaton procedure s nfluenced by the fact that the optmal polcy has a threshold type structure. Therefore, provdng the proof to the threshold type structure of the optmal polcy s mportant. For smple settngs, n whch the state space s small, the operaton of the PA component s optonal, and thus the PE component may be trggered every tme a new decson rule s needed. Schedulng Unt: The Schedulng Unt (SU) conssts of two sub-unts: the Orchestrator whch mplements the schedulng rule and the Cloud Toolkt (CT) whch allocates the tasks desgnated for the publc cloud. Note that ths separaton s schematc, and the unt mght be mplemented as a sngle ntegrated block. The nput receved by the Orchestrator from the MDP computaton s the optmal polcy,.e the schedulng rule. The SU mplements a real-tme decson accordng to ths rule, dependng on the current state of the prvate cloud. Consequently, the Orchestrator tracks the current state of a prvate cloud and keeps updates of the state upon ether any task beng sent to the prvate cloud or any task accomplshment. Those tasks whch are to be cloudburst are passed to the allocaton n the publc cloud. The msson of the CT can be fulflled wth a few known tools, such as OpenNebula [21] or Eucalyptus [20]. CT tracks avalable cloud resources and makes allocatons of VMs on the on-demand bass accordng to the type of the task beng cloudburst. Consequently, there s a complance wth the cloudburstng process and the VM allocaton performed by the CT. The prcng model of cloud computng s the focus of the ongong research, e.g. [15] and [32]. Therefore, we assume no precsely defned SLA pattern assocated wth a specfc publc cloud. The allocaton mght be performed wthn or out of the exstng long-term SLA wth pay-as-you-go charges. Moreover, the on-demand VM allocaton can be performed wthn more than one cloud provder. The general prcng model that we adopt s a charge for VM usage on the tme bass. The examples of such prcng patterns can be found at Amazon EC2 [1] and at Fujtsu Global Cloud Platform [9]. Note that MDP formulaton assumes that the cloudburst tasks are charged n a per-task manner. Therefore, a CT must be able to translate the cost of each task type nto the MDP language. Provded that the servce and the communcaton requrements of the task types are clearly estmated, ths translaton can be done by an ad-hoc calculaton. Any updates n task prcng are transferred by the CT to the MDP computaton part block, whch recalculates the polcy accordngly and outputs a new decson rule to the Orchestrator. III. THRESHOLD STRUCTURE OF THE OPTIMAL POLICY Ths secton addresses the threshold structure of the optmal polcy. The prvate cloud s modeled as a server (e.g.a server cluster) that processes an nflow of tasks of several types. As mentoned before, we dstngush between two possble settngs, where all the task types are served by a common FIFO server, or by FIFO servers dedcated to each task type. We prove the structural propertes of the optmal polcy for the both cases. The dffculty posed by both problems s the fnte buffer, and s not extensvely addressed n prevous works. For example, the problem analyzed n [8] consders maxmzaton objectve, therefore t dffers from our settngs. As s elaborated below, the sngle server model dctates a one-dmensonal state space, regardless of the number of task types. The dmenson of the state space of the mult-server model s equal to the number of task types,.e. the number of dedcated servers. We present two dfferent proofs of exstence of threshold-type optmal polces for the both settngs. In the mult-dmensonal case the proof, whch s presented n III-B s based on theorems proved n [16], whle for the one-dmensonal case we present a novel proof usng dfferent technques. A. Threshold polcy of sngle-server case In the followng, we present the detals of the sngleserver model. Arrvng tasks that are backlogged for local executon are queued. Such tasks cannot be cloudburst any longer. Servce s FIFO and non-nterruptble. The number of tasks awatng servce never exceeds a certan lmt denoted as B. Therefore, a new arrval occurrng whle there are already B tasks n the queue s sent to the cloud. As mentoned above, the tasks nflux s heterogeneous, meanng that dfferent communcatng costs are assocated wth each task type f handed to the cloud. Denote by k the number of task types. A 2013 IFIP/IEEE Internatonal Symposum on Integrated Network Management (IM2013) 53
A control process U s sad to be admssble f () t s adapted to the fltraton generated by ({A },X); and () under U, the process X(t) satsfes the constrant X(t) B for all t. (5) Fg. 2. MDP model chart The frst condton expresses the requrement that control decsons are made based on events from the past and present, so that the decson maker has no access to nformaton from the future. The second condton addresses the buffer lmt: If for some t one has X(t) =B and a task of type arrves then U (t) must be set to 1. The class of all admssble control processes s denoted by U. The value functon for the optmal control problem s defned as V (x) = nf J(x, U), x {0, 1,..., B}. (6) U U cost C (ncludes processng and communcaton) s ncurred whenever a type- task s sent to the cloud. The types are labeled n such a way that C 1 C 2 C k > 0. (1) For the sake of smplcty, we assume that the task processng tme dstrbuton at the prvate cloud s ndependent of the task type. Upon each arrval, a decson s to be made (assumng the buffer s not full) whether to accept the task to the prvate cloud or to offload t to the publc cloud and pay the correspondng cost. Denote by A and R, =1,...,k, the countng processes for arrval and, respectvely, remote offloadng (cloudburstng), for type. Namely, A (t) represents the number of tasks of type that have arrved up to tme t, R (t) represents the number of tasks of type sent to the cloud up to tme t. All countng processes mentoned n ths paper are assumed to have rght-contnuous sample paths. Further, A are modeled as ndependent Posson processes of ntenstes λ, respectvely, and the servce tme dstrbuton s assumed to be exponental of rate μ>0, ndependent of the task type. Next, for each, let U (t) be a process takng values n {0, 1}, descrbng the control decsons determnng R from A. Namely, U (t) =1f and only f a type- task arrvng at tme t s sent to the cloud. As a result, we can wrte R (t) = t 0 U (s)da (s) (2) The total number of tasks beng enqueued n the prvate cloud s denoted by X(t). The ntal condton of X s denoted by x {0, 1,..., B}. The process X s gven by X(t) =x + A (t) R (t) D(t), (3) where D denotes the departure process, countng the number of completed tasks (of all types). The logcal dagram whch descrbes the MDP model s shown n Fgure 2. The total dscounted cost assocated wth a control process U(t) = (U 1 (t),...,u k (t)) s gven by [ J(x, U) =E 0 e γs where γ>0 s a dscount factor. ] C dr (s), (4) Ths s a problem of contnuous tme Markov decson processes. For such a problem a prncpal tool s the characterzaton of the functon V as the soluton to a Bellman equaton. Usng ths tool we can show that a polcy of threshold type s optmal. Remark 3.1: It s natural to work wth an average cost rather than a dscounted one. However, t s well known that, provded γ>0 s suffcently small, the optmal polces wth and wthout dscount, are the same. We later detal about what s known as Blackwell optmalty. Denotng δ = (μ + γ + λ ) 1, the value functon unquely solves the Bellman equaton (see e.g., [31] Chapter 8) V (x) =δμv (x 1) + wth boundary condtons V (0) = δμv (0) + δλ mn[v (x)+c,v(x +1)], x {1, 2,...,B 1}, (7) δλ mn[v (0) + C,V(1)], V (B) =δμv (B 1) + δλ [V (B)+C ]. (8) Denotng δ = (γ + λ ) 1, the boundary condton at zero could be wrtten n a more standard form as V (0) = k δ λ mn[v (0) + C,V(1)]. However, the form (8) wll be useful n the analyss. The threshold structure s provded by the followng. Theorem 3.1: There exst constants B 1=b 1 b 2 b 3 b k such that the followng polcy s optmal: U 1(t) =0f and only f X(t) b 1; that s, always accept type-1 tasks unless the buffer s full; For =2,...,k, U (t) =0f and only f X(t) b ; that s, accept type- tasks f and only f the buffer contans b or fewer tasks awatng servce. The rest of ths secton s devoted to the proof of ths result. The frst step wll be to prove that V s nondecreasng and convex. To ths end, consder the operator T, actng n the 54 2013 IFIP/IEEE Internatonal Symposum on Integrated Network Management (IM2013)
space of functons from {0, 1,...,B} to R, defned as TU(x) =δμu(x 1) + TU(0) = δμu(0) + δλ mn[u(x)+c,u(x +1)], x {1, 2,...,B 1} δλ mn[u(0) + C,U(1)], TU(B) =δμu(b 1) + δλ (U(B)+C ), (9) for U : {0, 1,...,B} R. Then the Bellman equaton reads TV = V. Denote U =max U(x) (10) x and let S be the set of functons from {0, 1,...,B} to R that are nondecreasng, convex, and havng slope bounded by C 1, that s U(x +1) U(x) C 1, x {0, 1,...,B 1}. The followng lemma asserts that T preserves S, and moreover, acts on t as a strct contracton. Lemma 3.1: One has TS S. Moreover, there exsts a constant a (0, 1) such that TU TW a U W for every U, W S. Proof: To prove the frst asserton, let U S be gven. Then for 2 x B 1, TU(x) TU(x 1) = δμ(u(x 1) U(x 2)) + δλ {mn[u(x)+c,u(x +1)] mn[u(x 1) + C,U(x)]}. (11) Hence, usng the nondecreasng property of U, TU(x) TU(x 1) 0. A smlar calculaton for x =1and x = B gves TU(x) TU(x 1) 0 as well, and the nondecreasng property of TU follows. To show that the slope of TU s bounded by C 1,weuse agan (11). Snce U satsfes such a condton, t follows that U(x 1) U(x 2) C 1 and that each of the expressons n curly brackets s bounded by C 1. Snce δμ + δλ 1, t follows that TU(x) TU(x 1) C 1. A smlar calculaton for x =1and x = B gves an analogous result, and t follows that the slope of TU s bounded by C 1. To prove that TU s convex, we use the fact that f W s any convex functon mappng {0, 1,...,B} to R and C a constant, then the functon Z : {0, 1,...,B} R defned by { mn[w (x)+c, W (x +1)] f x B 1, Z(x) = W (B)+C f x = B, s also convex. The elementary proof of ths fact s omtted. Denote the transformaton mappng W to Z by T C. That s, Z = T C W. Then TU can be wrtten as δμũ + δλ Z, where Z = T C U, and { U(x 1) f x>0, Ũ(x) = U(0) f x =0. Owng to the fact that U s convex and nondecreasng, Ũ s seen to be convex. It follows that TU s convex, as the sum of k +1 convex functons. We have thus shown TU S. Snce U S s arbtrary, ths proves TS S. To prove the second asserton, let U, W S. Consder frst x {1, 2,...,B 1}. By (9), denotng a b = max(a, b), a b =mn(a, b) and usng the nequalty (a b) (c d) a c c d, we have TU(x) TW(x) δμ U(x 1) W (x 1) + δλ [ U(x) W (x) U(x +1) W (x +1) ] a U W, (12) where a = δμ + k δλ. By the defnton of δ, a<1. For x =0and x = B, the calculaton s smlar, and gves the same result, namely TU(x) TW(x) a U W. We conclude that TU TW a U W. Proof of Theorem 3.1. We use the contracton mappng prncple (see e.g. [24, Theorem V.18]). The set S, equpped wth the metrc ρ(u, W )= U W s a complete metrc space. The map T : S S s a strct contracton, as shown n the above lemma. As a result, T has a unque fxed pont. That s, there exsts a unque U S for whch TU = U. Recall that V s the unque soluton to the same equaton n the space of all functons from {0, 1,...,B} to R. As a result, V = U. Ths shows V S, namely, that V s nondecreasng and convex. In order to show the threshold property of the polcy, we employ the method from [31], whch bulds on convexty. One can read off an optmal feedback control from the Bellman equaton (16), as follows. Gven 0 x B 1, f a class- arrval occurs when X(t) =x, send t to the cloud (and pay C ) f and only f V (x)+c <V(x +1). (13) Snce V s convex, V (x+1) V (x) s nondecreasng n x, and so, f (13) holds for some (, x), t also holds for (, x ) for all x<x B 1. In other words, class- task acceptance occurs f and only f X(t) b for sutable constants b. The orderng of b, as alluded to n the statement of the theorem, s also clear by ths argument. It remans to show that b 1 = B 1. By the above dscusson, t suffces to show that V (x)+c 1 V (x+1) for all 0 x B 1. Ths, however, follows from the fact that the slope of V s bounded by C 1,asV S. B. Threshold polcy n a mult-server settng We now turn to the settng of dedcated servers. Snce the buffer s common for all k types of tasks the state space s k-dmensonal. The process X s gven by X(t) = X (t) =x + A (t) R (t) D (t), (14) where D denotes the departure process, countng the number of completed tasks of type. There s no change n the defntons of A, R, U. The control process U = {U 1,U 2,,U k } s sad to be admssble f () t s adapted 2013 IFIP/IEEE Internatonal Symposum on Integrated Network Management (IM2013) 55
to the fltraton generated by ({A },X); and () under U, the process X(t) satsfes the constrant X(t) = X (t) B for all t. (15) The defntons of J(x, U) and V (x) undergo no change. Denote the servce rates of the dedcated servers as μ, {1,.., k}, and δ =( μ + γ + λ ) 1. The value functon unquely solves the Bellman equaton whch s gven as follows: V (x) =δ μ V (x e ) + + δλ mn[v (x + e )+C,V(x + e )] + ( x B) + Y (16) where x s the ntal state vector, e s a vector of sze k wth all zeroes at coordnates j and 1 at coordnate and Y s some suffcently large constant. The last component stands for the constrant of no schedulng beyond the buffer lmt B. Ths formulaton covers all the boundary condtons and wll be useful for the proof (see chpt. 10 of [16] for ths method of notaton). The schedulng to a prvate cloud s performed then U =0, and the cloudburstng s done then U =1.We term the property of swtchng of U from 0 to 1 n adjacent states x, x, x<x, as an ncreasng of U n x. The polcy structure s provded by the followng. Theorem 3.2: For any task type and any state {x j },j {1,...,k}, U s nondecreasng n x j. We prove ths result relyng on theory presented n [16]. To ths end, consder the operator T TU(x) =δ μ V (x e ) + + δλ mn[v (x + e )+C,V(x + e )] + ( x B) + Y (17) for U : {0, 1,..., } R. Then, the Bellman equaton reads TV = V. Note that we do not restrct the state space to {0, 1,...,B}, because addton of the last component n equaton (18) assures that any state for wth k X > B s unreachable. Consder the norm defned as n (10), and let S be the set of functons from {0, 1,..., } to R that are nondecreasng, convex and supermodular. We stck to the defnton of convexty and supermodularty of Chapter 6 n [16] as follows: f(x) s convex n x f 2f(X) f(x + e )+f(x e ) f(x) s supermodular n x,x j f f(x + e + e j) f(x + e j) f(x + e ) f(x), j {1,...,k} The followng lemma asserts that T preserves S, and moreover, acts on t as a strct contracton. Lemma 3.2: One has TS S. Moreover, there exsts a constant a (0, 1) such that TU TW a U W for every U, W S. Proof: To prove the frst asserton, let U S be gven. Next we splt T to the separate operators, assocated wth Fg. 3. Example of 2-d threshold polcy departure, controlled arrval (admsson) and wth the constant Y. We rewrte (18) as follows: TU(x) =δ T D1() U(x)+δ T CA() U(x)+T Y U(x) (18) To prove that TU s convex, nondecreasng and supermodular we use Theorem 7.2 and Theorem 7.3 of [16], whch prove all these propertes for the operators T CA() and T D1() correspondngly. It s easy to see that the operator T Y preserves the propertes as well. The proof of the second asserton s smlar to that of n theorem 3.1 and thus omtted. Proof of Theorem 3.2. Smlarly to the proof of 3.1, we use the contracton mappng prncple (see e.g. [24, Theorem V.18]) to show that V S, namely, that V s nondecreasng, convex and supermodular. In order to show the nondecreasng property of the polcy, we agan employ the method from [31], whch bulds on convexty. In a smlar way, one sees that snce V s convex, V (X + e ) V (X) s nondecreasng n x, and snce V s supermodular, V (X + e ) V (X) s nondecreasng n x j, j. Consequently, class- task acceptance s nondecreasng n number of tasks of any type present n the prvate cloud. The optmal polcy demonstrated n Theorem 3.2 mples geometrcal propertes of schedulng rules, so that there exsts a contnuous regon where the tasks are scheduled to the prvate cloud. The example of the optmal polcy of schedulng n a system of two task types wth two dedcated servers s demonstrated n Fgure 3. There are two lnes whch form the regons where the tasks of the correspondng type are scheduled to the prvate cloud. The ntersecton of the regonsstands for the states where both task types are accepted to the prvate cloud. C. Connecton to average cost objectves We have demonstrated propertes of solutons to hybrd cloud optmzaton problems usng MDPs wth dscounted cost/reward crtera. For the long-run goals t makes sense to optmze average cost crtera. The relaton between the two types of cost s well-understood (Blackwell optmalty, see e.g. [7, Chapter 4.1]). In partcular, when the dscount factor γ s suffcently close to zero, the optmal polcy for dscounted cost s also optmal for the long-run average cost 56 2013 IFIP/IEEE Internatonal Symposum on Integrated Network Management (IM2013)
(bd.). Consequently, our results are vald for analogous MDP formulatons wth long-run average cotsts/rewards. IV. NUMERICAL EVALUATION AND POLICY ADJUSTMENT Ths secton addresses the mplementaton of optmal schedulng polcy computaton. The optmal soluton and ts dependence on the system parameters are numercally evaluated. In the absence of any optmal polcy, the numercal calculaton process, denoted as basc value teraton, s performed as follows: TABLE I. User Hybrd Cloud - Threshold Examples prvate cloud Buffer sze Prces Thresholds 35 0.288, 0.576, 1.152 6, 31, 35 35 0.185, 0.196, 0.2 14, 34, 35 18 0.049, 0.288, 1.61 7, 16, 18 18 0.025, 0.74, 1.379, 2.605-1, 9, 17, 18 1) Intalze the vector of the value functons of the sze correspondng to the state space. 2) Iteraton step: Use the contractng property of the correspondng Bellman equaton to perform teratons for the vector of the value functons. 3) Calculate the dfference wth the results of the prevous teraton. If the dfference s smaller than a predefned error level, go to the fnal step, otherwse go to the teraton step. 4) Calculate the optmal polcy for each state by applyng the part of the Bellman equaton whch contans the mn operator. (Equaton 16). Snce the fnal value functon s unknown, the ntal values at step 1) may be arbtrarly chosen. Note that step 4) does not need to be performed for the entre vector of the value functons, but untl the threshold s found. The latter stems from the threshold property of the optmal polcy. In ths secton, we present an algorthm whch facltates the mplementaton of the PA unt, as defned n secton II. Note that we nvoke the PA once there s a change n one of the parameters, e.g. the prcng of one of the task types. In order to effectvely adjust the new schedulng polcy whch corresponds to ths change, we am to use the prevous value functons and the prevous polcy. The usage of an approxmate value functon s amed to overcome the curse of dmensonalty, the known phenomenon whch refers to the complexty of the MDP n systems wth large state spaces. The approach s generally addressed n [17], where the approxmate representaton of the value functon and the sold understandng of the system are exploted for the effectve soluton. In order to fulfl our objectve, we study the nfluence of varatons n system parameters on the value functons and on the optmal polcy. We gve selected examples of threshold polces for sample systems n order to explore the mpact of system parameters. Next, we formulate a computatonally-effectve algorthm for the polcy adjustment. A. Numercal results for the schedulng model We frst observe the thresholds for dfferent systems. For smplcty, we focus on sngle-server systems but our conclusons are vald for mult-server settngs as well. We use current real-world prcng data, taken from leadng cloud provders. The examples for one-dmensonal models are presented n Table I. Each lne refers to a dfferent case, and the prces are taken from the Amazon EC2 prcng lst, [1]. In partcular, the frst case reflect sper-hour prces of hgh-memory Reserved Instances (RI. The three prces refer to extra large, double extra large, and quadruple extra large, all lghtly loaded nstances (note that the prces are hgher for lghtly loaded resources), where each nstance s selected to ft the correspondng task type. The second lne represents prces for the standard extra large RI, standard large RI, hgh-memory extra large RI, for medum,lght and heavy loads correspondngly. The frst two cases n table I demonstrate the dependence of the thresholds on the prce dfferences. As the dfference between the prces Fg. 4. Load mpact smulaton decreases, the thresholds of the cheaper task types ncrease. The two other cases represent dfferent task settngs whch nclude standard, hgh-memory, cluster GPU, hgh-cpu, RI under dfferent loads. Note that a threshold value of 1 means that the task s always cloudburst to the publc cloud. 1) Impact of the load: Next, we consder the mpact of μ the local load (.e. at the prvate cloud) ρ = K on the thresholds and on the value functon. The settng of λ three task types prced as n the thrd case n table I, and of the prvate cloud buffer of sze B = 100 s tested under a varable load startng at 0.3 and ascendng to 6.0. The results are seen n Fgure 4. Whle the value functon ncreases wth the load, the thresholds of the tasks wth the lowest prce decrease. The latter stems from the fact that the system prefers to reserve the space for the most expensve task type. Note that the orderng of the thresholds stays constant. We conclude that the optmal polcy for the hgher/lower load can be estmated once the optmal polcy for the lower/hgher load s known. 2) Varatons n task prces: We evaluate the effect of the varatons n task prces. The prce for the certan task type can be altered due to both the change n prcng offered by the publc cloud (e.g. dynamc prcng on Amazon, [1]), and to the tasks specfcaton (e.g. tasks need to change from standard VM to advance VM, [9]). Consder the evaluaton of a system wth three task types. The frst and thrd tasks are as n frst case of table I. The prcng of the second task type vares, such that ts ntal value s equal to the prcng of the frst task whle the fnal one s hgher than that of the thrd. The load ρ<1 s fxed. The results are shown n Fgure 5. The threshold of the second task type gradually ncreases untl t surpasses the threshold of the thrd task type. The value functon ncreases as well, whle the threshold of the cheapest task decreases. The ntensty of the varatons n the threshold hghly depends on the load. We observe ths n the same settng but wth the load ρ =5, where λ >μfor all. In ths case, the thresholds alternate rght once the prces become equal. Ths s due to the fact that for the hgh load most of the tasks are 2013 IFIP/IEEE Internatonal Symposum on Integrated Network Management (IM2013) 57
Fg. 5. Value functon and the thresholds vs. type-2 prce, moderate load Clearly, V (x)+c V (x+1) for all x<b(). Next, suppose that the prcng of task type ncreases. We estmate the new threshold as, say, b () =b()+1. Therefore, we ncrease the value functons by an amount proportonal to the ncrease n the prce of task type n order to get a new approxmated vector of V (x), so that V (x) +C V (x +1) for all x<b (). Then, the polcy teraton s contnued by usng the values of V (x). The algorthm for polcy adjustment s summarzed as follows: 1) Observe the optmal soluton for the ntal system and estmate the new thresholds n accordance wth the parameter whch has been altered. 2) Estmate the new value functon based on the estmaton performed n the prevous stage. 3) Contnue the calculaton of the new polcy as n basc polcy teraton usng the new estmaton n the ntalzaton step. Although the approxmaton of the new value functon s heurstc, the eventual convergence to the optmal soluton n step 3) s guaranteed. We could observe that the speed of the convergence s several orders faster than when estmaton step s not used. The actual mprovement n complexty ncreases wth the space state. One understands from [17] that a smlar method can be appled when the state-space of the new system s ncreased, e.g. when an addtonal task type appears or when the prvate cloud buffer s augmented. In these cases, the estmaton step 2) mght be less straghtforward, although the complexty reducton s sgnfcant. We leave the detals for the future work. Fg. 6. Value functon and the thresholds vs. type-2 prce, hgh load cloudburst. However, f the arrval rate of the type 2 tasks s sgnfcantly lower than μ (see Fgure 6), the decrease n the threshold of the type 3 tasks s moderate. Ths s due to the gven low arrval rate and to the observed small cloudburstng probablty. The latter causes low cloudburstng ntensty. For the same reason, the value functon ncreases very slowly once C 2 >C 3. We conclude that the mpact of the system parameters hghly depends on the settng and on the nterdependence between the dfferent parameters. Consequently, the system dynamcs must be well understood, and the update n parameters must be carefully examned. See [28] for more numercal results, examples and dscusson. B. Adjustment of the threshold-type polcy Based on the observatons above, we propose a computaton-effectve algorthm for fndng the optmal polcy for the system whch undergoes changes n ts parameters. As an example, consder a sngle-server system whose thresholds and the vector of the value functons are known. Observe some arbtrary task type whose threshold s equal to b() <B. V. CONCLUSION In ths paper, the problem of schedulng and cloudburstng n a hybrd-cloud envronment s analyzed. We present the system desgn for the optmal schedulng n a hybrd cloud and the correspondng analytcal framework based on Markov Decson Processes. We present a novel approach to optmzaton problems n cloud computng by means of control theory and MDP methods. The proposed system s logcally dvded nto unts whch perform tasks of a optmal polcy computaton, polcy adjustment and schedulng. Our work presents the proofs to the threshold-type structure of the optmal polces. Our numercal results are based on current prcng data from leadng cloud provders. The results demonstrate the threshold structure of the optmal polcy. We explot the latter for defnton of polcy adjustment algorthm. Motvated by the current work, we analyze, n a work n progress, an extended verson of the model, wth general class-dependent servce tme dstrbutons, under heavy traffc dffuson approxmatons. Our results show further structural propertes of the optmal control problem n ths asymptotc regme. In partcular, the problem s shown to undergo a dmensonalty reducton, n the form of a state space collapse. Ths consttutes an addtonal powerful tool for treatng schedulng problems n cloud computng. ACKNOWLEDGMENTS Research supported n part by the ISF (Grant 1315/12), the US-Israel BSF (Grant 2008466), and the Technon fund for promoton of research. 58 2013 IFIP/IEEE Internatonal Symposum on Integrated Network Management (IM2013)
REFERENCES [1] Amazon ec2 prcng. Amazon web servces. [Onlne]. Avalable: http://aws.amazon.com/ec2/prcng/ [2] M. Armbrust, A. Fox, R. Grffth, A. Joseph, R. Katz, A. Konwnsk, G. Lee, D. Patterson, A. Rabkn, I. Stoca et al., Above the clouds: A vew of cloud computng, Berkeley, Tech. Rep., 2010. [3] M. Armony, Dynamc routng n large-scale servce systems wth heterogeneous servers, Queueng Systems, vol. 51, no. 3, pp. 287 329, 2005. [4] R. Atar, A. Mandelbaum, and M. Reman, Schedulng a mult class queue wth many exponental servers: Asymptotc optmalty n heavy traffc, The Annals of Appled Probablty, vol. 14, no. 3, pp. 1084 1134, 2004. [5] R. Atar, A. Mandelbaum, and G. Shakhet, Smplfed control problems for multclass many-server queueng systems, Mathematcs of Operatons Research, vol. 34, no. 4, pp. 795 812, 2009. [6] M. Bennan and D. Menasce, Resource allocaton for autonomc data centers usng analytc performance models, n Autonomc Computng. Proceedngs. Second Internatonal Conference on, 2005, pp. 229 240. [7] D. Bertsekas, Dynamc programmng and optmal control. Athena Scentfc Belmont, MA, 1995, vol. 1, no. 2. [8] E. Çl, E. Örmec, and F. Karaesmen, Structural results on a batch acceptance problem for capactated queues, Mathematcal Methods of Operatons Research, vol. 66, no. 2, pp. 263 274, 2007. [9] Fujtsu global cloud platform. Fujtsu. [Onlne]. Avalable: http://globalcloud.us.fujtsu.com/portal/ctrl/top [10] H. Ghonem and S. Stdham, Control of arrvals to two queues n seres, European Journal of Operatonal Research, vol. 21, no. 3, pp. 399 409, 1985. [11] D. Guo and L. Bhuyan, A QoS aware multcore hash scheduler for network applcatons, n INFOCOM, Proceedngs IEEE, 2011, pp. 1089 1097. [12] B. Hajek, Optmal control of two nteractng servce statons, Automatc Control, IEEE Transactons on, vol. 29, no. 6, pp. 491 499, 1984. [13] M. Hajjat, X. Sun, Y. Sung, D. Maltz, S. Rao, K. Srpandkulcha, and M. Tawarmalan, Cloudward bound: plannng for benefcal mgraton of enterprse applcatons to the cloud, ACM SIGCOMM Computer Communcaton Revew, vol. 40, no. 4, pp. 243 254, 2010. [14] J. Harrson and A. Zeev, Dynamc schedulng of a multclass queue n the halfn-whtt heavy traffc regme, Operatons Research, vol. 52, no. 2, pp. 243 257, 2004. [15] D. Kondo, B. Javad, P. Malecot, F. Cappello, and D. Anderson, Costbeneft analyss of cloud computng versus desktop grds, n Parallel & Dstrbuted Processng. IEEE Internatonal Symposum on, 2009, pp. 1 12. [16] G. Koole, Monotoncty n Markov reward and decson chans: Theory and applcatons. Now Pub, 2007, vol. 1. [17] G. Koole and P. Nan, On the value functon of a prorty queue wth an applcaton to a controlled pollng model, Queueng Systems, vol. 34, no. 1, pp. 199 214, 2000. [18] V. Kulkarn and T. Tedjanto, Optmal admsson control of markovmodulated batch arrvals to a fnte-capacty buffer, Stochastc Models, vol. 14, no. 1-2, pp. 95 122, 1998. [19] H. Lu and D. Orban, Grdbatch: Cloud computng for large-scale datantensve batch applcatons, n Cluster Computng and the Grd. 8th IEEE Internatonal Symposum on, 2008, pp. 295 305. [20] D. Nurm, R. Wolsk, C. Grzegorczyk, G. Obertell, S. Soman, L. Youseff, and D. Zagorodnov, The eucalyptus open-source cloud-computng system, n Cluster Computng and the Grd. 9th IEEE/ACM Internatonal Symposum on, 2009, pp. 124 131. [21] OpenNebula. OpenNebula.org Project. [Onlne]. Avalable: http://www.opennebula.org/ [22] M. Puterman, Markov decson processes: Dscrete stochastc dynamc programmng. John Wley & Sons, Inc., 1994. [23] B. Raghavan, K. Vshwanath, S. Ramabhadran, K. Yocum, and A. Snoeren, Cloud control wth dstrbuted rate lmtng, n ACM SIGCOMM Computer Communcaton Revew, vol. 37, no. 4, 2007, pp. 337 348. [24] M. Reed and B. Smon, Methods of modern mathematcal physcs: Functonal analyss. Gulf Professonal Publshng, 1980, vol. 1. [25] M. Saleh and R. Buyya, Adaptng market-orented schedulng polces for cloud computng, Algorthms and Archtectures for Parallel Processng, pp. 351 362, 2010. [26] Y. Seung, T. Lam, L. L, and T. Woo, Cloudflex: Seamless scalng of enterprse applcatons nto the cloud, n INFOCOM, Proceedngs IEEE, 2011, pp. 211 215. [27] U. Sharma, P. Shenoy, S. Sahu, and A. Shakh, Kngfsher: Cost-aware elastcty n the cloud, n INFOCOM, Proceedngs IEEE, 2010, pp. 206 210. [28] M. Shfrn, R. Atar, and I. Cdon, To cloud or not to cloud: Optmzng cloudburstng costs, Tech. Rep. [Onlne]. Avalable: http://webee.technon.ac.l/publcaton-lnk/ndex/d/603 [29] S. Stdham Jr, Optmal control of admsson to a queueng system, Automatc Control, IEEE Transactons on, vol. 30, no. 8, pp. 705 713, 1985. [30] R. Van den Bossche, K. Vanmechelen, and J. Broeckhove, Costoptmal schedulng n hybrd aas clouds for deadlne constraned workloads, n Cloud Computng (CLOUD), IEEE 3rd Internatonal Conference on, 2010, pp. 228 235. [31] J. Walrand, An ntroducton to queueng networks. Prentce Hall Englewood Clffs, NJ, 1988, vol. 165. [32] H. Wang, Q. Jng, R. Chen, B. He, Z. Qan, and L. Zhou, Dstrbuted systems meet economcs: prcng n the cloud, n Proceedngs of the 2nd USENIX conference on Hot topcs n cloud computng. USENIX Assocaton, 2010, pp. 6 6. [33] R. Weber and S. Stdham Jr, Optmal control of servce rates n networks of queues, Advances n appled probablty, pp. 202 218, 1987. 2013 IFIP/IEEE Internatonal Symposum on Integrated Network Management (IM2013) 59