Modelling Exogenous Variability in Cloud Deployments

Transcription

1 Modllng Exognous Varablty n Cloud Dploymnts Gulano Casal 1 Mrco Trbaston 2 g.casal@mpral.ac.u trbaston@pst.f.lmu.d 1 : Impral Collg London, London, Untd Kngdom 2 : Ludwg-Maxmlans-Unvrstät, Munch, Grmany ABSTRACT Dscrbng xognous varablty n th rsourcs usd by a cloud applcaton lads to stochastc prformanc modls that ar dffcult to solv. In ths papr, w dscrb th blndng algorthm, a novl approxmaton for quung ntwor modls mmrsd n a random nvronmnt. Random nvronmnts ar Marov chan-basd dscrptons of tmvaryng opratonal conons that volv ndpndntly of th systm stat, thrfor thy ar natural dscrptors for xognous varablty n a cloud dploymnt. Th algorthm adopts th prncpl of solvng a sparat transnt-analyss subproblm for ach stat of th random nvronmnt. Each subproblm s thn approxmatd by a systm of ordnary dffrntal quatons formulatd accordng to a flud lmt thorm, mang th approach scalabl and computatonally nxpnsv. A valdaton study on svral hundrd modls shows that blndng can sav up to two ordrs of magntud of computatonal tm compard to smulaton, nablng ffcnt xploraton of a dcson spac, whch s usful n partcular at dsgn-tm. 1. INTRODUCTION Cloud applcatons run n shard nvronmnts whr prformanc and avalablty of computatonal rsourcs may chang ovr tm du to contnton from othr usrs. Ths uncrtants ma t dffcult to rason at dsgn-tm on th prformanc and avalablty that an applcaton wll offr onc dployd. Hnc, a currnt rsarch challng s to dntfy smpl modllng tchnqus that can ffctvly account for such varablty n prformanc and avalablty prdctons. In ths papr, w propos a mthodology for ffcnt analyss of quung-basd prformanc modls that nclud a dscrpton of xognous varablty, modlld by a contnuous-tm Marov chan. Consdr, for nstanc, a dcson problm for a mult-tr wb applcaton dployd (or to b dployd on a cloud platform. A common prformanc modllng approach s to rprsnt ach softwar srvr or vrtual machn as a quu and valuat what-f scnaros by solvng th rsultng stochastc quung ntwor modl usng tchnqus such as approxmat man valu analyss [11]. Howvr, upon ncludng a dscrpton of th surroundng random nvronmnt varablty, t s common for th rsultng modl to bcom ntractabl by xact algorthms. Dcomposton mthods Copyrght s hld by author/ownr(s. tacl ths problm by assumng a mard sparaton btwn th tm constants of th systm (.g., th rqust srvc tms and thos of th random nvronmnt (.g., man tm to falur of a rsourc [13]. Undr ths assumptons, on can obtan a soluton that s asymptotcally xact as th random nvronmnt vnts bcom ncrasngly lss frqunt [1]. Howvr, svral classs of applcatons dployd on th cloud do not satsfy ths assumptons. Ths nclud, for xampl, data-ntnsv applcatons (.g., MapRduc whr rqust srvc tms may last hours and thus rsourc fluctuatons happn rpatdly throughout a rqust xcuton prod [5]. Smlarly, startup tms of vrtual machns may vary from tns of sconds to svral mnuts, thus thy can ntroduc sgnfcant ntrfrnc wth runtm xcuton whn scalng rsourcs [4]. To cop wth th lac of robustnss and scalablty of xstng approxmaton mthods, w ntroduc th blndng algorthm, a smpl approxmaton for quung ntwors n random nvronmnts. Blndng stmats man prformanc mtrcs of th modl by valuatng a dffrnt transnt analyss subproblms for ach stat of th random nvronmnt. Followng ths prncpl, th statonary dstrbuton of th orgnal modl s obtand as a mxtur of th transnt bhavors of ach sub-modl n solaton. Ths solutons ar rlatd to ach othr by avragng th ntal conons provdd to th transnt analyss subproblms, whch s a sgnfcant dffrnc compard to smulaton as w xplan n Scton 6. Du to th lac of tractabl xact solutons for th transnt probablty dstrbuton of a quung ntwor, w us an approxmaton basd on ordnary dffrntal quatons (ODEs n th sns of Kurtz [14]. Crucally, undr ths formulaton th sz of th problm dos not dpnd on th numbr of rqusts, thus t dos not suffr from th stat spac xploson problm; t only grows wth lnar complxty n th numbr of statons. Furthrmor, th approxmaton s provably mor accurat for ncrasng job populaton szs. As a rsult, blndng can analyz problms that ar prohbtv to solv wth dscrt-stat modls du to th stat spac xploson problm. Blow w summarz th othr major contrbutons of th prsnt papr, also wth rspct to an arlr vrson of th blndng algorthm whch was proposd n [6] for xponntal quung ntwors n two-stag random nvronmnts: W formulat a gnralzd blndng algorthm to study random nvronmnts wth an arbtrary numbr of stats. In th cas of a two-stag nvronmnt and xponntal quus, ths algorthm spcalzs to th on n [6]. W formally charactrz th conncton btwn th

2 rlatv mdan slowdown xprmnt numbr (a m1.mdum rlatv mdan slowdown xprmnt numbr (b c1.mdum Fgur 1: Exampl of xognous varablty n rspons tms for two m1.mdum and c1.mdum VMs runnng Apach OFBz [7]. Each pont dscrbs a 1-mnuts xprmnt. blndng algorthm and th xact soluton of th modl undr study. W gnralz th class of quung ntwors supportd by th mthod by allowng Coxan srvc-tm dstrbutons, whch nabl ffctv approxmatons of mprcal srvc tm dstrbutons. Th papr s organzd as follows. Scton 2 llustrats xognous varablty n a ral cloud dploymnt usng an xprmntal datast collctd on Amazon EC2 and shows lmtatons of dcomposton approachs. Our rfrnc modl s gvn n Scton 3 and n Scton 4, rspctvly. W dfn a flud approxmaton for th consdrd quung modls n Scton 5 and ovrvw ts man proprts. Ths s usd n Scton 6 to dvlop th blndng algorthm. A numrcal valdaton of th blndng algorthm s gvn n Scton 7. Conclusons ar gvn n Scton MOTIVATION 2.1 Random Envronmnt Varablty W dscuss th rsults of an xprmntal campagn that w hav prformd on Amazon EC2 Irland usng th dmo -commrc componnt of Apach OFBz, an opn sourc ntrprs rsourc plannng framwor [3]. Th am of th xprmntal campagn s to llustrat tmporal corrlaton btwn prformanc dgradatons that may b attrbutd to xognous factors arsng n th cloud dploymnt nvronmnt. Th OFBz dmo applcaton s smlar to TPC- W [8] and RUBS [9], but th framwor s mor complx snc t faturs svral hundrds Java classs and rls on tchnologs that ar common n producton systms, such as Groovy dynamc scrptng, a tmplat ngn, and AJAX on clnt sd. W hav strssd th applcaton usng th OFBnch sut proposd n [7]. Th worload conssts of a CPU-bound mx of rqusts. Th OFBz applcaton s shppd n ts basc form wth an mbddd Gronmo srvr [2] and an mbddd Drby n-mmory databas [1], whch w run togthr nsd a sngl VM. W hav dployd OFBz on svral tns of c1.mdum and m1.mdum VMs on Amazon EC2 Irland n th u-wst-1a avalablty zon. Each VM runs an ndpndnt copy of th applcaton. Smultanously, w hav run th clnts, on for ach applcaton, nsd VMs nstantatd n th sam avalablty zon. Each xprmnt uss a sngl Frfox clnt coupld wth a closd-loop worload gnrator. Th tm btwn submsson of succssv rqusts from a clnt s xponntally dstrbutd wth a man of 1 scond. Snc thr s a sngl clnt, w us a short warmup prod of 2 mnuts. Th bnchmar dscards th sssons xcutd by th clnt durng th warmup phas of ach xprmnt, snc ths suffr hgh rspons tms du to cach msss. Aftr th warmup, th xprmnt contnus at stady-stat for 1 mnuts bfor th srvr s shut down and rstartd for th followng xprmnt. Each rstart tas approxmatly 15 mnuts to rgnrat th databas and boot th softwar. Thus, succssv xprmnts nsd ach VMs ar ndpndnt. Each xprmnt s rpatd 5 tms, for a total of about 15 hours of xprmnts. Notc that w fx n all xprmnts th random numbr gnrator sd to rplcat th sam squnc of rqusts, ncludng th clnt thn tms, th squnc of rqust typs, and th sam random data. Fgur 1 dpcts th xprmntal rsults. Th mtrc plot s th avrag rspons tm of th rqusts xprncd by th clnt, hnc t ncluds also ntwor latncy. Howvr, th bandwh btwn srvr and clnt s normally abov 85-9 MBt/sc, as rcordd va prf at th bgnnng of ach xprmnt, and t dos not chang sgnfcantly throughout th xprmntal campagn. Th mtrc shown n th fgurs s dfnd as follows. W consdr th rspons tms of th pag wth th hghst ht rat,.., th hompag. Thn, w comput th mdan of ts rspons tms across all th xprmnts and us ths a basln to dtrmn th rlatv mdan slowdown wthn ach xprmnt. Th rsults n th fgur show that th dvaton from ths basln normally dos not xcd 1-15%, but t may last a varabl amount of tm, from mnuts to hours. Snc th xprmnts ar ndpndnt and th ntwor s largly ovrprovsond, t sms rasonabl to attrbut ths slowdown to th VM tslf. As a furthr ndcaton of ths, w hav studd th sam mtrc for othr pags and found narly dntcal trnds. Notc that our rsults ar qut consstnt wth th ons rportd n [5], whch ndcat for Amazon EC2 Irland a CPU varablty of approxmatly 7-8% at 1 hour tmscals. Howvr, compard to th study n [5], Fgur 1 provds adonal nformaton about th tmporal corrlaton of th dgradatons, suggstng that a statbasd modl could b usful to dscrb such prturbatons. 2.2 Lmtaton of Dcomposton Approachs W now show th lmtatons of dcomposton modllng approachs n dscrbng fluctuatons n rsourc capacts such as thos llustratd n th prvous xampl. For th sa of llustraton, w consdr a prformanc modl of a wb applcaton rprsntd as a ntwor of quus. Ths s also th class of modls that w focus on n th rst of ths papr, vn though w xpct that th proposd approach s suffcntly gnral to b xtndd to othr modl classs. W consdr a quung ntwor modl for a vry basc applcaton dploymnt nvolvng two VMs runnng n paralll, havng S 1 = S 2 = 4 cors ach, and a clnt populaton of N = 1 usrs that ssus rqusts wth an avrag thn tm of Z = 7s n btwn succssv rqust arrvals by th sam clnt. A rqust s randomly snt to any of th two VMs. W assum a two-stat random nvronmnt that producs fluctuatons smlar to Fgur 1,.., for 2% of th tm VM2 offrs a dgradd prformanc. Srvc dmands of rqusts ar xponntally dstrbutd, opratng n th

3 Fgur 2: Lmtaton of classc dcomposton approachs. Th fgur shows n th x-axs th rato of rqust srvc dmands at two VMs for a prod of dgradd prformanc lastng 2% of th obsrvaton tm, smlarly to Fgur 2. normal rgm wth th sam man as n th xprmnts n Fgur 1,.., 85ms. Durng th prformanc dgradaton, th dmand at VM2 ncrass by a constant factor f. Our what-f analyss conssts n studyng th systm prformanc as a functon of th f dgradaton factor. Fgur 2 llustrats th prdctons. Th xact soluton s obtand by formulatng ths modl as a contnuous-tm Marov chan (CTMCs and solvng t drctly by numrcal mthods. Th dcomposton approach, nstad, solvs ndpndntly of ach othr th two CTMCs obtand by cononng on th random nvronmnt stat bng th normal rgm or th dgradd rgm. Fnally, th blndng algorthm antcpats th tchnqu dvlopd n th nxt scton. As w s from th fgur, dcomposton s unabl to corrctly charactrz ths modl as th dgradaton factor ncrass. Ths s bcaus th largr th dgradaton, th mor th modl wll xhbt larg and frqunt fluctuatons of prformanc mtrcs ovr tm, whch s an vnt dffcult to modl by dcomposton. Convrsly, th mthod w propos n ths papr s mor approprat to valuat ths what-f scnaro, bsds provdng an ovrall mor scalabl algorthm than CTMC-basd tchnqus. In th nxt sctons, w provd dtals about our rfrnc modls and ntroduc formally th blndng algorthm, whos computatonally proprts ar fnally valuatd n Scton 7 aganst smulaton. 3. MODEL AND NOTATION In th squl, w dfn stat th currnt conon of th quung ntwor modl that rprsnts,.g., a cloud applcaton, as opposd to th trm stag, whch nstad dscrbs th conon of th random nvronmnt,.g., th cloud dploymnt or a srvc offrd by th platform. 3.1 Random Envronmnt Th class of random nvronmnts consdrd n ths papr volvs accordng to a contnuous-tm Marov chan, jumpng btwn stags, and spcfd by th followng st of paramtrs: E: numbr of stags;, h = 1,..., E: stag ndxs; α h : rat of jump from stag to stag h; α = h α h: total outgong rat from stag ; p h = α h /α : jump probablty from stag to h. Consdr for xampl th datast shown n Fgur 1. Upon dfnng a quung modl for th runnng applcaton, w can assum to dscrb rqust srvc tms by th rspons tms obtand n th sngl-usr xprmnts n Fgur 1. Th hstogram of th man srvc tms across th xprmnts can b dcomposd nto E bns to dscrb th ffct of th random nvronmnt on th man srvc tms. Th probablty p h s thn th probablty that a prod wth man srvc tms charactrzd by bn s followd by a prod wth man srvc tms charactrzd by bn h; th man duraton of th prods assocatd to bn and h s α 1, rspctvly. 3.2 Systm Modl W dnot by M th numbr of quung statons and by N th numbr of crculatng rqusts n th modl; both valus ar assumd nvarant across stags. All statons hav unboundd buffr szs. Srvc tm procsss ar ndpndnt and dntcally dstrbutd random varabls followng a Coxan dstrbuton, whch s a popular paramtrc modl ncludng xponntal, hypr-xponntal, and Erlang dstrbutons as spcal cass [13]. W shall rfr to th stats of Coxan srvc procsss as phass. Whn th random nvronmnt s n stag, th quung ntwor s paramtrzd as follows:, j = 1,..., M: staton ndxs; r,j: routng probablty from staton to staton j; S : numbr of srvrs at staton ; K : numbr of phass n th Coxan procss at ; ( = 1,..., K : phas ndx for staton ; µ, : srvc rat n phas at staton ; φ, : probablty for rqusts at to complt aftr srvc phas. W now furthr spcfy how statons ar affctd by random nvronmnt transtons. In our mthodology w assum that upon stag chang, ach staton stat s altrd accordng to a rst rul. That s, for a jump from stag to h, w dfn a mappng matrx R h, whch s 1 n lmnt (, j f and only th th stat n stag s mappd to th jth stat n stag h, othrws. Not that R h s n gnral rctangular and that multpl stats can jump to th sam stat j. Also w assum R h to b such that thr s no loss or gnraton of mass upon stag transton and that th numbr of statons and srvrs dos not chang wth th actv stag. Throughout ths papr, w llustrat ths rul by consdrng th mappng whr th Coxan srvc procss of th staton s rstartd upon an nvronmnt stag chang, manng that all rqusts n srvc ar nstantanously movd bac to phas 1 at that staton. Othr rst ruls ar possbl, howvr ths has th advantag that th cononal dstrbuton gvn th currnt stat s always nown, but t may ovrstmat th ral srvc tms whn th stag jump rat s larg. 3.3 Contnuous-Tm Marov Chan From th abov dfntons, th quung ntwor modl may b dscrbd by a contnuous-tm Marov chan (CTMC havng stat vctor (n,, whr = 1,..., E s th currnt stag of th random nvronmnt. Th dstrbuton of rqusts across statons s dscrbd by n = (n 1, n 2,..., n M,

4 n = (x, s,1,..., s,k bng th stat vctor of staton, whr: 1 s, s th numbr of srvrs that ar currntly procssng rqusts n phas, thus K =1 s, = S. If a srvr s dl, t s countd wthn s,1 togthr wth th srvrs procssng rqusts n phas 1. Not that for a dlay staton s, = +. 2 x = q + n,1 s th sum of th numbr of rqusts q watng n th quu buffr, and th numbr of rqusts n,1 that ar rcvng srvc n phas 1. Ths rprsntaton provds advantags for th flud analyss dvlopd n Scton 5. Th CTMC dfnd on th proposd stat spac may b spcfd usng a collcton of transton rat functons whch gv, for a gnrc stat (n,, all transton rats at whch th procss movs to a dffrnt stat (n, h. Ths ar gvn n Tabl 1. Th transtons 1 (rsp. dscrb th rats of job compltons at staton at th 1st (rsp. th Coxan phas that ar followd by a dpartur to staton j; th transtons f ph 1 (rsp. f ph dscrb th rat of rqusts advancmnt from th 1st to th 2nd Coxan phas at staton ; f nv dscrb th random nvronmnt changng stag. Th nfntsmal gnrator of th modl as a whol may b wrttn as Q 1 Q 1 Q 1E Q = Q 1 Q Q E ( Q E1 Q E Q E whr Q = Q α I, Q s th nfntsmal gnrator of th quung ntwor modl consdrd n solaton wth stag- paramtrzaton, I s th dntty matrx, and for h, Q h = α h R h. Lt π = (π 1,..., π,..., π E b th vctor of qulbrum probablts for th modl, whr π rfrs to stag. Assumng that th CTMC s rgodc and rrducbl, th qulbrum probablts may b obtand by drct numrcal soluton of th global balanc quatons πq =, π1 = 1. Unfortunatly, du to th combnatoral growth of th stat spac, drct numrcal solutons ar prohbtv n most cass of practcal ntrst. Ths motvats th dvlopmnt of th blndng algorthm proposd n ths papr. 4. EMBEDDED PROCESS To cop wth th stat spac xploson ssu, w propos to charactrz th qulbrum bhavor of th modl 1 Not that th n staton stat vctor can b radly mappd to a canoncal stat dscrpton (n, n,1, n,2,..., n,k, whr n, s th numbr of rqusts rcvng srvc n phas and n s th total numbr of rqusts n th quu, accordng to th followng transformaton: K { mn(x, s n = x + s,, n, =,1, = 1 s,, = 2,..., K =2 2 In mplmntatons, ths valu may b st qual to th total job populaton N. focusng on th ntwor stat obsrvd mmdatly aftr stag transtons n th random nvronmnt. Th assocatd mbddd probablts hav statonary vctor dnotd by η = (η 1,..., η,..., η E, whch rprsnts th probablty dstrbuton mbddd at th arrval nstant of any random nvronmnt vnt. Thus, ach η vctor, aftr normalzaton to sum to unty, may b sn as th ntry probablty vctor n stat at qulbrum. In ths scton, w frst show that th η vctors ar rlatd by xprssons that dscrb th transnt voluton of th quung ntwor stat durng th sojourn tm wthn ach stag. Th blndng algorthm, prsntd n th nxt sctons, lvrags on ths charactrzaton by approxmatng th transnt voluton usng Kurtz s flud lmt thorm and thn couplng ths mthod wth an outr tratv approxmaton amd at approxmatng th random nvronmnt ffcts. Avrag stady-stat prformanc ndxs ar obtand usng ntrmdat rsults of th flud analyss, as w show n Scton 5. Th man contrbuton of ths scton s th followng charactrzaton of th mbddd probablts η. Thorm 1. In th CTMC wth gnrator (1, th mbddd probablts satsfy at qulbrum th balanc E ( + p h h=1 h η h Q h t α h αht R h = η (2 for all stags = 1,..., E, whr p h = α h /α h s th probablty of jump from stag h to stag. Th proof s gvn n th Appndx. Thorm 1 njoys a smpl probablstc ntrprtaton. Lt us obsrv that: η h Q h t s th transnt probablty vctor at tm t for th ntwor ntalzd wth probablty η h n stag h. α h α ht s th xponntal dnsty for th tm to th nxt random nvronmnt vnt. η h Q h t R h s th qulbrum ntry probablty vctor n stag aftr spndng t tm unts n stag h. Thus, th summaton ovr all sourc stags h and jump tms t n (2, wghtd for th jump probablts p h, smply provds th statonary ntry vctor η n stag. Th mportanc of (2 ls n th fact that, dspt not bng drctly computabl du to stat-spac xploson, th balanc rlats qulbrum prformanc mtrcs, va th vctors η, wth transnt analyss, va th ntgrand η h Q h t whch s a CTMC. Ths lads us to propos th us of flud analyss mthods, whch ar ffctv n transnt analyss, for th computaton of qulbrum prformanc mtrcs n random nvronmnts, as w dscrb nxt. 5. FLUID LIMITS W now lvrag on th thory of flud lmts dvlopd by Kurtz [14] to dfn an approxmat transnt analyss mthod for quung ntwor modls. Our ntnt s to obtan an approxmaton of th ntgrands of (2 cononal on th random nvronmnt bng n stag h. Du to ths cononng, n th rmandr of ths scton, w smplfy

5 Transton rat Dstnaton stat 1 (n,,, j = φ 1rjµ 1 mn(x, s 1 n = (n 1,..., n,..., n j,..., n M, n = (x 1, s 1, s 2,..., s K n j = (x j + 1, s j1, s j2,..., s jk (n,,, j = φ rjµ s n = (n 1,... n,..., n j,..., n M, n = (x, s 1 + 1,..., s 1,..., s K n j = (x j + 1, s j1, s j2,..., s jk f ph 1 (n,, = (1 φ 1µ 1 mn(x, s 1 n = (n 1,... n,..., n M, n = (x 1, s 1 1, s 2 + 1,..., s K f ph (n,, = (1 φ µ s n = (n 1,... n,..., n M, n = (x, s 1,..., s 1, s (+1 + 1,..., s K f nv (n, = α h ( n = (n 1,..., n M, h n = x + K =2 s, s 1 + K =2 s,,..., } {{ } K 1 Tabl 1: Transton rat functons. Sourc stat s (n, = (n 1,... n,..., n M,. notaton by omttng stag ndxs. 3 W also consdr arbtrary ntal conons, postponng to Scton 6 th dfnton of th ntal conons usd to approxmat (2. For quung ntwors, flud lmts dscrb th bhavor of th undrlyng Marov procss as th populaton of rqusts N and th numbr of srvrs S at ach quu grow smultanously to nfnty. Such growth s assumd to b proportonal to a scal paramtr V that prsrvs th ratos of srvrs to rqusts and whch has th ntrprtaton of th systm sz. Hnc, snc capacty grows at th sam rat of th rqust populaton growth, ths avods saturaton n th asymptotc rgm. In th lmt rgm, th ntwor also mantans th sam topology, routng matrx, and srvc rats of th orgnal modl. Formally, a flud lmt wth th abov proprts s obtand by tang a famly of CTMCs {Y V (t, V N} ndxd by th scal paramtr V. Each vctor Y V (t dscrbs th ntwor stat at tm t accordng to th stat dscrptor n, dfnd as n Scton 3, and wth transton rats as n Tabl 1. Lt Y b th ntal stat of th CTMC at tm t =. Th CTMC Y 1(t, whr V = 1, s thn th on wth stats rachabl from Y through th rats n Tabl 1. Smlarly, Y 2(t, whr V = 2, s th CTMC wth stats rachabl from th ntal stat 2Y through th sam rats. Gnralzng, Y V (t s th CTMC wth stat spac rachabl from V Y, for all V. Th flud approxmaton of ntrst hr conssts n th lmt bhavor of th normalzd CTMC Y V (t/v whn V grows asymptotcally larg, wth ntal stat Y. A ncssary conon to apply th lmt thorm s that all transton rats njoy th so-calld dnsty-dpndnt form, whch nformally stats that all CTMC transton rats ar proportonal to V onc rwrttn n trms of th normalzd stat dscrptor n/v. For nstanc, mang xplct th normalzd stat dscrptor n/v n th 1 rats ylds 1 (n,, j = φ 1r jµ 1 mn(n, s 1 = V φ 1r jµ 1 mn(n /V, s 1/V = V 1 (n/v,, j. 3 Not that our transnt analyss appls also to Coxan quung ntwors n th spcal cas of a (non-random nvronmnt wth a sngl stag. (3 whch satsfs th dnsty-dpndnt form 1 (n,, j = V 1 (n/v,, j for all statons, j. Smlarly, t can b shown that all th othr transton rats of Tabl 1 can b xprssd n such a form, xcpt th last on that s approxmatd drctly by th blndng algorthm wthout rsortng to flud approxmatons. In Kurtz s thory, th dnsty-dpndnt form mpls that a sampl path of th normalsd CTMC tas ncrasngly small stps, of ordr O(1/V, at ncrasngly larg rats, of ordr O(V. In th lmt V +, ths allows us to rlat a sampl path to a contnuous trajctory, soluton of a systm of ordnary dffrntal quatons dfnd n trms of th dnsty-dpndnt transton rats of th CTMC, as dscussd n th nxt subscton. 5.1 Flud Approxmaton Lt us frst assocat a jump vctor to ach transton rat functon, n such a way to rcord th componnt-ws dffrnc btwn sourc and dstnaton stats n th CTMC. For nstanc, th jump vctor for 1 (n,,, j s δ dp 1 (, j, =(,...,, n n,,...,, n j n j,,...,, T and whn th stag s omttd t s smlarly dfnd but wth th last lmnt rmovd. Stmmng from th das dscrbd abov, th flud approxmaton ssntally nvolvs rplacng th stat dscrptor n wth a tm-varyng dtrmnstc vctor y(t and nvstgatng th nonlnar ODE systm dy(t = K M,j=1 =1 K M + f ph =1 =1 (y(t,, jδ dp (, j (y(t, δph( (4 wth ntal conon Y. Usng Kurtz s thorm [14, Thorm 3.1] and th fact that th transton rats ar Lpschtz contnuous vrywhr, whch mpls global xstnc and unqunss of th ntal valu problm (4, t can b shown that, for any fnt T, t holds lm P { sup t T Y V (t/v y(t } =. (5 V Ths mpls that a sampl path (of fnt lngth s asymp-

6 totcally undstngushabl n probablty, from th unqu soluton to th ODE systm (4. Furthrmor, n our modl th flud lmt can b shown to mply convrgnc n man [12]: lm E { Y V (t/v y(t } =, t V whch provds th motvaton for consdrng th approxmaton E{Y V (t} V y(t that rlats th xpctd quu lngth to th r-scald dffrntal trajctory. Ths s th ntrprtaton hrn usd and th ODE soluton wll thus b compard aganst man prformanc ndcs of th quung ntwor. 5.2 Exampl Lt us consdr a two-staton quung ntwor wth N crculatng rqusts, wthout a random nvronmnt. Staton 1 has S 1 srvrs wth xponntal srvc tms (K 1 = 1 phas, rat, staton 2 has S 2 srvrs wth a Coxan dstrbuton (K 2 = 2 phass, rats µ 2,1 and µ 2,2, complton probablts φ 2,1 and φ 2,2 = 1. Staton 1 has slf-routng probablty r 1,1 = 1 p, for < p < 1, whras th othr routng probablts ar r 1,2 = p, r 2,1 = 1, r 2,2 =. Th stat dscrptor n th CTMC that modls ths systm s n = (x 1, S 1,1, x 2, S 2,1, S 2,2, and w st th ntal conon to Y = (N, S 1,, S 2,. As a rsult of ths assumptons, th dfntons n Scton 3 spcalz to: 1 (n, 1, 1 = (1 p mn(x 1, S 1,1, 1 (n, 1, 2 = p mn(x 1, S 1,1, 1 (n, 2, 1 = φ 2,1µ 2,1 mn(x 2, S 2,1, 2 (n, 2, 1 = µ 2,2S 2,2, havng jump vctors f ph 1 (n, 2 = (1 φ 2,1µ 2,1 mn(x 2, S 2,1, δ dp 1 (1, 1 = (,,,, T, δ dp 1 (1, 2 = ( 1,, +1,, T, δ dp 1 (2, 1 = (+1,, 1,, T, δ dp 2 (2, 1 = (+1,,, +1, 1 T δ ph 1 (2 = (,, 1, 1, +1 T. To dvlop th flud approxmaton, w dfn th dtrmnstc stat vctor (x 1(t, S 1,1(t, x 2(t, S 2,1(t, S 2,2(t whch rplacs n n th abov transton rat functons xprssons. Thn, (4 provds th nonlnar ODE systm dx 1(t = p mn(x 1(t, S 1,1(t + µ 2,2S 2,2(t + φ 2,1µ 2,1 mn(x 2(t, S 2,1(t; (6 ds 1,1(t = ; (7 dx 2(t = µ 2,1 mn(x 2(t, S 2,1(t + p mn(x 1(t, S 1,1(t; (8 ds 2,1(t ds 2,2(t = (1 φ 2,1µ 2,1 mn(x 2(t, S 2,1(t = µ 2,2S 2,2(t + µ 2,2S 2,2(t; (9 + (1 φ 2,1µ 2,1 mn(x 2(t, S 2,1(t (1 Quu lngth at staton = 1. =.6 = t (a Plot of Y 1(t. Quu lngth at staton = 1. =.6 = t (b Plot of Y 3(t. Fgur 3: Comparsons btwn th xpctd valus and th flud approxmaton of th quu lngth at staton 1 for th xampl of Scton 5.2 wth p = 1., φ 2,1 =, φ 2,2 = 1, µ 2,1 = µ 2,2 = 2., and {.6,.8, 1.}. Y = (1,,, 4,. (a CTMC Y 1(t; (b CTMC Y 3(t. Sold lns: ODE solutons; dashd lns: CTMC xpctd valus. wth th ntal conon Y = (N, S 1,, S 2,, whr w choos arbtrarly to ntalz th systm wth all rqusts n staton 1. Th ntrprtaton of th ndvdual ODE trms s smlar to th dscrt modl. For xampl, (6 (7 modl th stat of staton 1. Ngatv fluxs modl job dparturs, whl postv fluxs modl job arrvals. Null ODEs such as (7 ar assocatd only to statons wth xponntal srvrs and ndcat that th numbr of srvrs avalabl to srv n phas 1 rmans constant and, n ths xampl, qual to th ntal valu S 1. Not that th sum of all drvatvs (6 (1 s zro, whch prsrvs th proprty of consrvaton of mass for closd ntwors. To numrcally llustrat th accuracy of th flud approxmaton for transnt analyss of quung ntwors, lt us consdr agan th xampl n Scton 5.2 and assum th followng paramtrs: p = 1., φ 2,1 =, φ 2,2 = 1, µ 2,1 = µ 2,2 = 2.. That s, w consdr a tandm ntwor wth no slf-routng probablts whr staton 2 has Erlang- 2 dstrbutd srvc tms wth man 1.. Thr dffrnt nstancs ar studd by varyng th man srvc tms at staton 1, whch s an xponntal dlay staton n ths xampl snc th ntal stat vctor s Y = (1,,, 4,. Fgur 3 shows som rprsntatv plots of th CTMC xpctd valus and thr flud approxmatons durng th transnt rgms for qual to.6,.8, and 1.. Fgur 3(a plots th quu-lngth procsss at staton 1 for th frst CTMC, Y 1(t, of th famly nducd by th ntal conon Y. Th ODE solutons (sold lns ar solvd by standard numrcal ntgraton whras th CTMCs wr smulatd wth tght convrgnc crtra to obtan smooth trajctors for th xpctatons (dashd lns. Th fgur shows a gnrally accptabl qualty of th approxmaton whch hr tnds to ncras wth. Th cas =.6 has a maxmum rror of about 8% (at t 4. although th trnd of th transnt dynamcs s stll capturd wll. It s ntrstng to not that ths conons ar qut far away from th many rqusts/many srvrs paramtrzaton consdrd by th lmt thorm, nvrthlss th flud approxmaton s stll usabl for transnt analyss. Ths s furthr bacd up by Fgur 3(b whch consdrs th CTMC Y 3(t,.., a total populaton of 3 rqusts and 12 srvrs n staton 2. Hr, th maxmum rror s rducd to about

7 4%, and gnrally bhavs wll n all th cass consdrd. Summarzng, th xampls llustrat that flud lmt approxmatons provd an nxpnsv way of valuatng th transnt of a quung ntwor. Ths fatur s at th bass of th blndng approxmaton w ntroduc n Scton 6 for valuatng quung modls wth random nvronmnts. 5.3 Contnuous Mappng W now turn bac to motvatng th flud lmt as an approxmaton for th rght-hand sd of (2. Lt us dnot by L h (t th stochastc procss dfnd by η h Q h t. Each ntgral n th nnr summaton of (2 may b rwrttn as + L h (tα h α ht. (11 whr L h (t = η h Q h t. Snc ths dos not njoy a closdform soluton n gnral, t rqurs numrcal ntgraton ovr a fnt rang [, T ], whr T s a fnt thrshold for numrcal convrgnc. Our goal s to show how transnt analyss ovr any rang [, T ] may b approxmatd n trms of th flud modl prsntd n th prvous subsctons. Snc th procss L h (t dscrbs th bhavour of th ntwor n stag h whn no stag transtons ar possbl, t has an asymptotc bhavour accordng to Scton 5.1. Thrfor, w may conclud that L h (t/v convrgs n probablty, for all t T, to y h (t, whr V s th scalng factor and y h (t s th flud modl assocatd wth th ntwor paramtrzaton at th h-th stag of th random nvronmnt. W wrt such a convrgnc compactly as L h (t/v y h (t. By th contnuous mappng thorm [12], t holds that L h (tα h α ht /V y h (tα h α ht (12 for all t T and for any α h. Thus, th flud approxmaton y h (t allows us to valuat V y h (tα h α ht as an approxmat ntgrand of (2. Ths provds th y computatonal advantag, ovr th orgnal xprsson, of rqurng only th soluton of a small st of ODEs n plac of th transnt of an ntractably larg CTMC. 6. THE BLENDING ALGORITHM Th blndng algorthm s an tratv approxmaton tchnqu that trs to addrss th stat spac xploson ssu of an xact CTMC analyss. Th approach approxmats (2 by solvng n a cyclc fashon a st of E ODE systms smlar to (4. Each ODE systm rprsnts th transnt bhavor of th quung ntwor modl cononal on th random nvronmnt bng n a gvn stag. At ach cycl, a nw st of ntal conons, basd on th rsults at th prvous cycl, s provdd to th E ODE systms. Th algorthm s stoppd aftr C tratons f thr a usr-spcfd maxmum numbr of tratons s rachd or f th ntal conons assgnd to th ODEs convrg to a constant valu wthn a tolranc. Th psudocod s summarzd n Algorthm 1. For ach cycl c = 1,..., C and stag = 1,..., E, th blndng algorthm solvs th ODE systm (4 wrttn as dy c(t = K M,j=1 =1 + K M =1 =1 (y c(t,, jδ dp (, j f ph (y c(t, δ ph (. (13 Each trajctory y c (t dscrbs th transnt bhavor of th quung ntwor aftr t tm unts snc last ntrng stag, subjct to ntal conons y c(, = 1,..., E. At th frst traton, c = 1, y c ( s ntalzd smlarly to th xampl n Scton 5.2. That s, th ntr job populaton N s splt n a balancd mannr across statons and all srvr stat varabls ar st to s,1 = S and to s, =, for phass > 1. At th followng tratons th blndng algorthm uss drctly (2 to updat th ntal conons as y c+1 ( = h + p h h y h c (tα h αht R h (14 for c = 2,..., C, whr h = h ωα,h/ j ωjαj, s a normalzng constant, and ω dnots th qulbrum probablty of stag n th random nvronmnt. Ths normalzng constant accounts for th fact that y h c vctors ar probablty dstrbutons, whras th η h vctors n (2 ar not normalzd to sum to on. Th rato that dfns h s smply th rato of normalzng constants for yc+1( and yc h (, whch only dpnd on th random nvronmnt CTMC and thrfor ar ndpndnt of c. Th algorthm trmnats at th frst traton c whr th maxmum mprovmnt ovr th prvous traton s lss than a usr-provdd convrgnc tolranc δ max (.g., δ max = 1 6 or whn c xcds th maxmum numbr of tratons C. Rmars. Th blndng algorthm dffrs from tchnqus such as stochastc smulaton n th followng sns. In smulaton, th systm stat volvs contnuously and th ffct of random nvronmnt ffcts s manly to chang th rat of th dffrnt vnts n th smulaton and possbly to nstantanously chang on or mor stat varabls. In ordr to produc a mor ffcnt valuaton of th modl, blndng frst consdrs wth (13 all th possbl sampl paths cononal on th modl bng n stag. Thn, th nformaton from all ths trajctors s avragd n th ntal conons (14 provdd n nput to th transnt subproblms at th nxt traton. Thus, nstad of prsrvng th stat upon stag jump, blndng ffctvly avrags all such stats nto th ntal stat varabl. Ths allows to smplfy th modl valuaton at th xpns of som approxmaton rror. 6.1 Man Prformanc Indxs Th approxmat computaton of man prformanc ndxs s basd on th convrgnc n man of Kurtz s flud lmt. Dnot by π th qulbrum probablty of stag obtand by solvng th random nvronmnt CTMC dfnd by th rats α h. Lt (x (t j, s j1(t, s j2(t,..., s jk j (t b th trajctory of th stat of staton j as dtrmnd durng th fnal cycl of th blndng algorthm. Thn th man quu-lngth s radly computd as E[Q j] = E ( K + j π mn(x (t j, s j1(t + s j(t. =1 =2

8 Algorthm 1 Blndng algorthm psudo-cod Input: modl paramtrs ɛ: numrcal tolranc δ max: convrgnc tolranc C > 1: maxmum numbr of tratons Output: approxmat y (t, Algorthm: Intalz traton countr: c = 1 Intalz y c ( for ach stag = 1,..., E. rpat c = c + 1 for = 1,..., E do Solv th ODE systm (13 for y c+1 (t, t T, wth ntal valus y c ( and tolranc ɛ. nd for for = 1,..., E do Comput y c+1 ( by (14. nd for untl max y c+1( y c( 1 < δ max and c C M = 2, E = 1, S = 1 M = 2, E = 1, S = 5 Runtm [s] Runtm [s] Dst. Error SIM BLE Error SIM BLE Erl Exp Cox Tabl 3: Valdaton rsults: ntwors wth 2 statons and 1 stags. M = 1, E = 2, S = 1 M = 1, E = 2, S = 5 Runtm [s] Runtm [s] Dst. Error SIM BLE Error SIM BLE Erl Exp Cox Tabl 4: Valdaton rsults: ntwors wth 1 statons and 2 stags. M = 2, E = 2, S = 1 M = 2, E = 2, S = 5 Runtm [s] Runtm [s] Dst. Error SIM BLE Error SIM BLE Erl Exp Cox Tabl 2: Valdaton rsults: ntwors wth 2 statons and 2 stags. Th man throughput s obtand as E[X j] = E + π µ j,1 mn(x (t j, s j1(tφ,1. =1 Both stmats ar obtand smply by tm-avragng th occupancy masurs n th staton stat vctor. In th cas of th throughput, ths ar scald by th dpartur rat from staton j. Snc n th stady-stat th dpartur rat must b qual n all stags, th throughput n th abov xprsson s gvn as th dpartur rat from phas 1. From ths mtrcs, th man rspons tm pr vst at staton j s radly obtand as E[T j] = E[Q j]/e[x j]. Fnally, not that n cas of dlay srvrs whr S j = +, th xprssons abov contnu to b fnt thans to our assumpton that all srvrs ar ntalzd n phas 1, thus mn(x (t j, s j1(t = x (t j < + and also K j =2 s j(t < + snc th job populaton s assumd fnt and hnc th numbr of actv srvrs nvr xcds N. 7. NUMERICAL VALIDATION To study th rror bhavor of blndng, w hav prformd an xprmntal campagn combnng a larg st of ntwors wth randomly gnratd confguratons, organzd nto groups whr a subst of thr paramtrs was pt fxd n ordr to strss crucal parts of th algorthm. Ovrall, w hav consdrd 6 groups of ntwors wth randomly gnratd paramtrs wth dffrnt numbr of quus (M, stags n th random nvronmnt (E, and numbr of srvrs n ach quu (S. Th modl paramtrs ar drawn from unform dstrbutons wth rangs [., 5.] for th rats of th random nvronmnt. Th populaton s qual to N = 5 jobs. Srvc-tm dstrbutons wr vard n a controlld form. For ach ntwor confguraton, w hav consdrd 3 modls wth an xponntal, Erlang-3 (squard coffcnt of varaton SCV = 1/3, and two-stag Coxan dstrbuton wth balancd phas probablts (SCV = 1, all wth th sam gvn mans sampld at random from a unform dstrbuton wth rang [.1, 1.]. Varyng th srvc-tm dstrbutons allows us to study th mpact of a paramtr whch cannot b capturd by an ordnary flud modl of a ntwor wthout random nvronmnt. For ach group w hav analyzd 3 modls, thus 18 modls wr consdrd ovrall. Each modl was analyzd usng Gllsp s stochastc smulaton algorthm. Both smulaton and th blndng algorthm wr mplmntd n MATLAB, n ordr to us th sam nvronmnt to compar thr runtms. Th mplmntaton of blndng altrnats xcuton of a non-stff solvr (od45 and a stff solvr (od15s untl both mthods cannot furthr mprov th (uncononal man quu-lngths of th random nvronmnt modl. W dfn th mnmal mprovmnt n Algorthm 1 to b δ max =.1. Th maxmum numbr of tratons s st to C = 1. Furthrmor, to spd-up convrgnc, th frst traton of th algorthm trmnats by sttng y 2 ( = y 1 ( such that th blndng algorthm prforms at th frst traton a dcomposton approxmaton computd wth th flud ODEs and ntalzd wth a balancd job dstrbuton across quus. Ths s advantagous partcularly for stags wth long holdng tms, snc t allows th systm to mmdatly accumulat mass on th bottlnc staton cononal on th currnt stag. Ths would b slowr usng blndng snc svral tratons would b

9 ndd to rallocat th balancd dstrbuton on th bottlnc statons. Th rsults ar summarzd n Tabls 2-4. For ach modl wth a populaton of N jobs, th accuracy rror s xprssd as follows: Error = max 1 N Q BLE Q SIM N whr Q BLE and Q SIM ar th stmats of th stady stat quu lngth at staton as computd by blndng and by smulaton, rspctvly. Each tabl shows th avrag rrors and xcuton tms for a st of 1 randomly gnratd ntwors, wth controlld srvc tm dstrbutons and srvr multplcts. Th followng obsrvatons ars from th rsults: Blndng s systmatcally fastr than smulaton. In th ntwors wth two statons, th dffrnc n runtms s two ordr of magntuds on avrag, but t tnds to dcras wth largr ntwor topologs. In th groups of ntwors wth 1 statons (s Tabl 4, an analyss of th bhavor of blndng rvals that ncrasd tm of th analyss s manly causd by a largr runtm pr sngl traton; ths s du to th fact that th systm of ODEs ncrass lnarly wth th numbr of statons and that largr problms may lad to stffnss for whch quatons ar computatonally mor dffcult to solv. Th numbr of tratons, not rportd n th tabl, ar found to b qut smlar n all groups. Th accuracy of blndng s adquat n all th cass consdrd, wth an xpctd tndncy to dcras wth bggr ntwor topologs and mor stags of th random nvronmnt. By comparng th rrors across th sam row n vry tabl, w fnd that ncrasng th numbr of srvrs at ach staton, whl png all th othr paramtrs th sam, lads to an apprcabl ncras n th qualty of th approxmaton. Ths s mportant for practcal purposs, snc modrn mult-cor machns oftn rqur to consdr quus wth multpl srvrs. For such modls, w thrfor xpct our approxmaton to prform accuratly. Th srvc-tm dstrbutons hav an mpact on th accuracy rror, but ths s modl-dpndnt and not asy to ntrprt. For nstanc, th cass wth Erlangdstrbutd srvc tms lad to th largst rrors n Tabl 3, whras thy njoy th smallst rrors n th groups of ntwors of Tabl 4. Summarzng, our valdaton campagn rvals that blndng s rmarably fastr than smulaton, an mportant proprty n partcular for optmzaton-basd studs whr thousands of modls nd to b valuatd n ordr to dtrmn a good local optmum. W hav prformd ntal xprmnts n ths drcton, suggstng that global optmzaton problms usd to solv load-balancng problms that cannot b solvd n 2 days usng smulaton at ach traton of th solvr ar nstad solvd n lss than 2-3 hours usng blndng. Approxmaton rrors ar accptabl n all cass and mportantly thy mprov sharply as th numbr of srvrs n ach staton ncrass by a fw unts, a stuaton that s typcal n applcatons. 8. CONCLUSION Blndng s a nw tchnqu for th analyss of quung ntwor modls n Marov random nvronmnts. Ths class of modls can asly fnd applcaton n dscrbng th prformanc of cloud softwar systms, whr uncrtants about th dploymnt nvronmnt can b asly modlld n trms of a Marov random nvronmnt. A numrcal valuaton confrms that th ths approach capturs trnds n prformanc ndxs that ths othr tchnqus do not captur.a vry attractv fatur of our approach s scalablty, th analyss bng basd on approxmatons obtand wth ordnary dffrntal quatons whch ar nsnstv to th job populatons. W hav found numrcally that th accuracy mprovs wth ncrasng systm szs, whch mas our tchnqu partcularly approprat for th analyss of massvly paralll systms. Du to th good computatonal proprts, t s radly applcabl n dcson problms that rqur th valuaton of th modl ovr larg paramtr spacs. Svral possbl xtnsons of ths wor may b consdrd, for nstanc xtnsons to mult-class worloads and opn modls. Acnowldgmnt Th wor of Gulano Casal s partally supportd by th Europan projct MODAClouds (FP and by an EPSRC Small Equpmnt Fundng grant. Mrco Trbaston s supportd by th Europan projct ASCENS, REFERENCES [1] Apach Drby projct. [2] Apach Gronmo projct. [3] Apach OFBz projct. [4] M. Mao, M. Humphry. A Prformanc Study on th VM Startup Tm n th Cloud. In Proc. IEEE Cloud, , 212. [5] J. Schad, J. Dttrch, J.-A. Quané-Ruz. Runtm Masurmnts n th Cloud: Obsrvng, Analyzng, and Rducng Varanc. In Proc. of th VLDB Endowmnt, Vol. 3, No. 1, 21. [6] G. Casal, M. Trbaston. Flud Analyss of Quung n Two-Stag Random Envronmnts In QEST, pags IEEE Computr Socty, 211. [7] J. Moschtta, G. Casal. OFBnch: an Entrprs Applcaton Bnchmar for Cloud Rsourc Managmnt Studs. In Proc. of MICAS worshop - Managmnt of Rsourcs and Srvcs n Cloud and Sy Computng, Sp 212. [8] D. F. García and J. García. TPC-W E-commrc bnchmar valuaton. Computr, 36(2:42 48, Fb. 23. [9] C. Amza, A. Ch, A. L. Cox, S. Elnty, R. Gl, K. Rajaman, E. Cccht, and J. Margurt. Spcfcaton and mplmntaton of dynamc Wb st bnchmars. In Proc. of IEEE 5th Annual Worshop on Worload Charactrzaton, Oct. 22. [1] P. Courtos. Dcomposablty: Quung and Computr Systm Applcatons Acadmc Prss, Nw Yor, [11] D. A. Mnascé and V. A. F. Almda. Scalng for E-Busnss: Tchnologs, Modls, Prformanc, and Capacty Plannng. Prnctc Hall, 2.

10 [12] P. Bllngsly. Probablty and Masur. Wly, 3rd on, [13] G. Bolch, S. Grnr, H. d Mr, and K. S. Trvd. Quung Ntwors and Marov Chans. 2nd d., John Wly and Sons, 26. [14] T. G. Kurtz. Solutons of ordnary dffrntal quatons as lmts of pur Marov procsss. J. Appl. Prob., 7(1:49 58, Aprl 197. [15] G. Horváth, P. Buchholz, and M. Tl. A MAP fttng approach wth ndpndnt approxmaton of th ntr-arrval tm dstrbuton and th lag corrlaton. In Proc. of th 2nd Conf. on Quanttatv Evaluaton of Systms (QEST, pags , 25. [16] M. F. Nuts. Structurd Stochastc Matrcs of M/G/1 Typ and Thr Applcatons. Marcl Dr, Nw Yor, but ths radly mpls (2 by rwrtng th formula for bloc = 1,..., E as H π h α h R h = γ 1 η h=1 and usng (18 for th π h vctors. APPENDIX Proof of Thorm 1. From th CTMC gnrator Q n (1, w dfn two rat matrcs H = dag(q 1,..., Q,..., Q E, M = Q H, wth th followng ntrprtatons. H s a sub-gnrator ncludng rats of jump not assocatd wth a random nvronmnt transton, whras th rmanng rats ar ncludd n matrx M and ffctvly mard for obsrvaton. That s, (H, M dfns a mard pont procss upon th CTMC gnrator whr arrvals corrspond to stag transtons at qulbrum. Snc all transtons ar Marovan, th procss (H, M may b sn as an nstanc of Nuts s Marovan arrval procss [16], whch s a hddn Marov modl wth phas-typ dstrbutd ntr-obsrvaton tms. From basc proprts of Marovan arrval procsss (.g., [15, Sc. 2], th mbddd dstrbuton at stag transton nstants satsfs η = η( H 1 M, η1 = 1, (15 whr ( H 1 M s a stochastc matrx rprsntng th dscrt-tm Marov chan (DTMC mbddd at stag transton nstants. W now rlat η wth th qulbrum dstrbuton π of Q as follows. It s stablshd that th rlaton btwn tmstatonary dstrbuton and qulbrum dstrbuton of th mbddd procss s π = γ 1 η( H 1, γ = η( H 1 1, (16 whr th normalzng constant γ has th probablstc ntrprtaton of man ntr-obsrvaton tm n th mard procss. Explotng th bloc dagonal form of H, w can thn smplfy (16 as π = γ 1 η (α I Q 1 (17 for all stags = 1,..., E. W now ma th obsrvaton that th matrx nvrs n (17 s smply th Laplac transform L(s of th matrx xponntal Q t valuatd at s = α. W can thrfor wrt π = γ 1 α 1 + Obsrv now that, by (16 and (15 η Q t α αt (18 πm = γ 1 η( H 1 M = γ 1 η