Efficient Redundancy Techniques for Latency Reduction in Cloud Systems

Transcription

1 Efficient Redundancy Techniques fo Latency Reduction in Cloud Systems 1 Gaui Joshi, Emina Soljanin, and Gegoy Wonell Abstact In cloud computing systems, assigning a task to multiple seves and waiting fo the ealiest copy to finish is an effective method to combat the vaiability in esponse time of individual seves, and thus educe aveage latency. But adding edundancy may esult in highe cost of computing esouces, as well as an incease in queueing delay due to highe taffic load. This wok povides a fundamental undestanding of when and how edundancy gives a cost-efficient eduction in latency. Fo a geneal task sevice time distibution, we compae diffeent edundancy stategies, fo e.g. the numbe of edundant tasks, and time when they ae issued and canceled. We get the insight that the log-concavity of the task sevice distibution is a key facto in detemining whethe adding edundancy helps. If the sevice distibution is log-convex, then adding maximum edundancy educes both latency and cost. And if it is log-concave, then less edundancy, and ealy cancellation of edundant tasks is moe effective. We also pesent a heuistic stategy that achieves a good latency-cost tade-off fo an abitay sevice distibution. This wok also genealizes and extends some esults in the famously had analysis of fok-join queues. A. Motivation I. INTRODUCTION An inceasing numbe of applications ae now hosted on the cloud. Some examples ae steaming NetFlix, YouTube), stoage Dopbox, Google Dive) and computing Amazon EC2, Micosoft Azue) sevices. A majo advantage of cloud computing and stoage is that the lage-scale shaing of esouces povides scalability and flexibility. A side-effect of the shaing of esouces is the vaiability in the latency expeienced by the use due to queueing, pe-emption by othe jobs with highe pioity, seve outages etc. The poblem becomes futhe aggavated when the use is executing a job with seveal paallel tasks on the cloud, because the slowest task becomes the bottleneck in job completion. Thus, ensuing seamless, low-latency sevice to the end-use is a challenging poblem in cloud systems. One method to educe latency that has gained significant attention in ecent yeas is the use of edundancy. In cloud computing, unning a task on multiple machines and waiting fo the ealiest copy to finish can significantly educe the latency [1]. Similaly, in cloud stoage systems equests to access the content can be assigned to multiple eplicas, such that it is only sufficient to download one eplica. This can help educe latency significantly. Howeve, edundancy can esult in inceased use of esouces such as computing time, and netwok bandwidth. In famewoks such as Amazon EC2 and Micosoft Azue which offe computing as a sevice, the seve time spent is popotional to the money spent in enting the machines. In this wok we aim to undestand this tade-off between latency and computing cost and popose scheduling policies that can achieve a good tade-off. Ou analysis also esults in some fundamental advances in the analysis of queues with edundant equests. B. Pevious Wok Systems Wok: One of the ealiest instances of exploiting edundancy to educe latency is the use of multiple outing paths to send packets in netwoks. See [2] fo a detailed suvey. A simila idea has also been ecently studied in systems [3]. In lage-scale cloud computing famewoks such as MapReduce [4], the slowest tasks of G. Joshi gaui@mit.edu) and G. Wonell gww@mit.edu) ae with the Depatment of Electical Engineeing and Compute Science, MIT, Cambidge 2139, USA. Emina Soljanin emina@bell-labs.com) is with Bell Labs Alcatel-Lucent, Muay Hill NJ 7974, USA. This wok was suppoted in pat by NSF unde Gant No. CCF , AFOSR unde Gant No. FA , and a Schlumbege Faculty fo the Futue Fellowship.

2 TABLE I: Oganization of main latency-cost analysis esults pesented in the est of the pape. We fok each job into tasks at all n seves full foking), o to some subset out of n seves patial foking). A job is complete when any k of its tasks ae seved. Full foking to all n seves Patial foking to out of n seves k = 1 Replicated) Case Section IV: Compaison of stategies with and without ealy task cancellation Section V: Effect of and the choice of seves on latency and cost Geneal k Section VI: Bounds on latency and cost, and the divesity-paallelism tade-off Section VII: Heuistic edundancy stategy fo cost-efficient latency eduction TABLE II: Latency-optimal and cost-optimal edundancy stategies fo the k = 1 eplicated) case. Canceling edundancy ealy means that instead of waiting fo any 1 task to finish, we cancel edundant tasks as soon as any 1 task begins sevice. Log-concave sevice time Log-convex sevice time Latency-optimal Cost-optimal Latency-optimal Cost-optimal Cancel edundancy ealy o keep it? Patial foking to out of n seves Low load: Keep Redundancy, High load: Cancel ealy Low load: = n fok to all), High load: = 1 fok to one) Cancel ealy Keep Redundancy Keep Redundancy = 1 = n = n a job staggles) become a bottleneck in its completion. Seveal ecent woks in systems such as [5], [6] exploe staggle mitigation techniques whee edundant eplicas of staggling tasks ae launched to educe latency. Although the use of edundancy has been exploed in systems liteatue, thee is little wok on the igoous analysis of how it affects latency, and in paticula the cost of esouces. We now eview some of that wok. Exponential Sevice Time: In distibuted stoage systems, easue coding can be used to stoe a content file on n seves such that it can be ecoveed by accessing any k out of the n seves. The latency expeienced by the use can be educed by foking a download equest to n seves and waiting fo any k to espond. In [7], [8] we found bounds on the expected latency using the n, k) fok-join model with exponential sevice time. This is a genealization of the n, n) fok-join system, which was actively studied in queueing liteatue [9] [11] aound two decades ago. In ecent yeas, thee is a enewed inteest in fok-join queues due to thei application to distibuted computing famewoks such as MapReduce. Anothe elated model with a centalized queue instead of queues at each of the n seves was analyzed in [12]. The latency behavio with heteogeneous job classes fo the eplicated k = 1) case is pesented in [13]. Othe elated woks include [14], [15]. Geneal Sevice Time: Few pactical systems have exponentially distibuted sevice time. Fo example, studies of download time taces fom Amazon S3 [16], [17] indicate that the sevice time is not exponential in pactice, but instead a shifted exponential. Fo sevice time distibutions that ae new-wose-than-used [18], it is shown in [19] that it is optimal to fok a job to maximum numbe of seves. The choice of scheduling policy fo newwose-than-used NWU) and new-bette-than-used NBU) distibutions is also studied in [2] [22]. The NBU and NWU notions ae closely elated to the log-concavity of sevice time studied in this wok. The Cost of Redundancy: If we assume exponential sevice time then edundancy does not cause any incease in cost of seve time. But since this is not tue in pactice, it is impotant to detemine the cost of using edundancy. Simulation esults with non-zeo fixed cost of emoval of edundant equests ae pesented in [21]. The total seve time spent on each job is consideed in [23], [24] fo a distibuted system without consideing queueing of equests. In [25] we pesented an analysis of the latency and cost of the n, k) fok-join with and without ealy cancellation of edundant tasks. 2

3 C. Ou Contibutions In this wok, we conside a geneal sevice time distibution, unlike exponential sevice time assumed in many pevious woks. We analyze the impact of edundancy on the latency, as well as the computing cost total seve time spent pe job). Incidentally, ou computing cost metic also seves as a poweful tool to compae diffeent edundancy stategies in the high taffic egime. Table I gives the diffeent scenaios consideed fo latency-cost analysis in the est of the pape. This analysis gives us the insight that the log-concavity and espectively, the log-convexity) of F, the tail distibution of sevice time, is a key facto in choosing the edundancy stategy. Hee ae some examples, which ae also summaized in Table II. Suppose we fok a job to queues at n seves, and wait fo any 1 out of n tasks to be seved. An altenate stategy is to cancel the edundant tasks ealy, as soon as any 1 task eaches the head its queue. We show that ealy cancellation of edundancy can educe both latency and cost fo log-concave F, but it is not effective fo log-convex F. Suppose we fok each job to only a subset out of the n seves, and wait fo any one of the tasks to finish. Then we can show that foking to moe seves lage ) is both latency and cost optimal fo log-convex F. But fo log-concave F, lage educes latency only in the low taffic egime, and always inceases the computing cost. Using these insights, we also develop a heuistic stategy to decide 1) how many seves to fok to, and 2) when to cancel the edundant tasks, fo an abitay sevice time that may be neithe log-concave no log-convex. D. Oganization In Section II we descibe the fok-join system model and the latency and cost metics. Section III gives the key concepts used in this wok. The oganization of the main esults in the est of the pape is given in Table I. In Section IV and Section V we conside the eplicated k = 1) case, and get insights into choosing the best edundant queueing stategies, depending on log-concavity of the distibution. In Section IV we compae the stategies with and without ealy cancellation of edundancy, and in Section V we conside patial foking to a subset of the seves. In Section VI we detemine bounds on latency and cost fo any k, genealizing some of the fundamental esults on fok-join queues. We also demonstate a divesity-paallelism tade-off in choosing k. In Section VII we pesent a heuistic stategy, dawing fom the insights fom the latency-cost analysis in pevious sections. Section VIII summaizes the esults and povides futue pespectives. Some popeties and examples of logconcavity ae given in Appendix A. The poofs of the k = 1 and geneal k cases ae defeed to Appendix B and Appendix C espectively. A. Fok-Join Model and its Vaiants II. PROBLEM FORMULATION Conside a distibuted system with n statistically identical seves. Each incoming job is foked into n tasks, assigned to fist-come fist-seve queues at each of the n seves. The n tasks ae designed in such a way that completion of any k tasks is sufficient to complete the job. When any k tasks ae seved, the emaining tasks ae canceled. We efe to this system as the n, k) fok-join system, defined fomally as follows. Definition 1 n, k) fok-join system). Each incoming job is foked into n tasks that join a fist-come fist-seve queue at each of the n seves. When any k tasks finish sevice, all emaining tasks ae canceled and abandon thei queues immediately. Fig. 1 illustates the 3, 2) fok-join system. The job exits the system when any 2 out of 3 tasks ae complete. The k = 1 case coesponds to a eplicated system whee a job is sent to all n seves and we wait fo one of the eplicas to be seved. Geneal k aise in appoximate computing algoithms, o in content download fom a coded distibuted stoage system. Instead of waiting fo k tasks to finish, we could cancel the edundant tasks ealie when any k tasks each the heads of thei queues, o ae in sevice. A simila idea has been poposed in systems wok [6]. We efe to this vaiant as the n, k) fok-ealy-cancel system defined fomally as follows. 3

4 1 task seved 2 tasks seved Stat of sevice Stat of sevice Abandon Abandon Abandon Fig. 1: The 3, 2) fok-join system. When any 2 out of 3 tasks of a job ae seved as seen fo the blue job on the ight), the thid task abandons its queue and the job exits the system. Fig. 2: The 3, 2) fok-ealy-cancel system. When any 2 out of 3 tasks of a job ae in sevice, the thid task abandons seen fo the blue job on the left, and geen job on the ight). Definition 2 n, k) fok-ealy-cancel system). Each incoming job is foked to the n seves. When any k tasks ae in sevice, we cancel the edundant tasks immediately. If moe than k tasks stat sevice simultaneously, we peseve any k chosen unifomly at andom. Fig. 2 illustates the 3, 2) fok-ealy-cancel system. Ealy cancellation can save the total time spent pe job computing cost) because the edundant tasks ae canceled befoe the seves stat woking on them. But it could esult in an incease in latency because we have to wait fo all k emaining tasks to finish, the slowest of which could take a long time to finish. Thus, we lose the divesity advantage of having to waiting only fo a subset of tasks to complete. In anothe vaiant defined fomally below, we fok a job to out of the n seves. We efe to this as the n,, k) patial fok-join system defined as follows. Definition 3 n,, k) patial fok-join system). Each incoming job is foked into > k out of the n seves. When any k tasks finish sevice, the edundant tasks ae canceled immediately and the job exits the system. The seves can be chosen accoding to diffeent scheduling policies such as andom, ound-obin, least-wokleft etc. Patial foking can save on the computing cost as well as the netwok cost, which is popotional to the numbe of seves each job is foked to. In this wok we develop insights into the best choice of and the scheduling policy to achieve a good latency-cost tade-off. Othe vaiants of the fok-join system include a combination of patial foking and ealy cancellation, o delaying invocation of some of the edundant tasks. Although not studied in detail hee, ou analysis techniques can be extended to these vaiants. In Section VII we popose a heuistic algoithm to find a edundancy stategy that is a combination of patial foking and ealy cancellation. B. Aival and Sevice Distibutions Conside that jobs aive to the system at ate pe second, accoding to a Poisson pocess. The Poisson assumption is equied only fo the exact analysis and bounds of latency E [T ] defined below). All othe esults on cost E [C], and the insights into choosing the best edundancy stategy hold fo any aival pocess. Afte a task of the job eaches the head of its queue, the time taken to seve it can be andom due to vaious factos such as disk seek time, netwok congestion, and shaing of computing esouces between multiple pocesses. We model it by the sevice time >, with cumulative distibution function F x) and assume that it is i.i.d. acoss equests and seves. Dependence of sevice time acoss seves can be modeled by adding a constant to sevice time. Fo example, some ecent wok [16], [17] on analysis of content download fom Amazon S3 obseved that is shifted exponential, whee is popotional to the size of the content and the exponential pat is the andom delay in stating the data tansfe. In this pape we use F x) = P > x) to denote the tail pobability function of. We use k:n to denote the k th smallest of n i.i.d. andom vaiables 1, 2,..., n. 4

5 C. Latency and Cost Metics Using moe edundant tasks lage n) educes latency, but geneally esults in additional cost of computing esouces. We now define the metics of the latency and esouce cost whose tade-off is analyzed in the est of the pape. Definition 4 Expected Latency). The expected latency E [T ] is defined as the expected time fom when a job aives, until when k of its tasks ae complete. Since we need to wait fo the fist k out of n tasks to be complete, we expect that E [T ] will decease as k educes o n inceases). Definition 5 Expected Computing Cost). The expected computing cost E [C] is the expected total time spent by the seves seving a job, not including the time spent in the queue. In computing-as-a-sevice famewoks, the expected computing cost is popotional to money spent on enting machines to un a job on the cloud. Although we focus on E [C] as the cost metic in this pape, we note that edundancy also esults in a netwok cost of making Remote-Pocedue Calls RPCs) made to assign tasks of a job, and cancel edundant tasks. It is popotional to the numbe of seves each job is foked to, which is n fo the n, k) fok-join model descibed above. In the context of distibuted stoage, edundancy also esults in inceased use of stoage space, popotional to n/k. The tade-off between delay and stoage is studied in [7], [8]. III. PRELIMINARY CONCEPTS We now pesent some peliminay concepts that ae vital fo undestanding the esults pesented in the est of the pape. A. Using E [C] to Compae Systems Intuitively, edundancy affects latency and cost in two opposing ways. Redundancy povides divesity of having to wait only fo a subset of tasks to finish, thus educing sevice time. But the edundant time spent by multiple seves can inceasing the waiting time in queue fo subsequent jobs. The second effect dominates at high aival ates. Thus E [C], the expected total time spent by the seves on each job, it can be used to detemine the sevice capacity of the system. Claim 1 Sevice Capacity in tems of E [C]). Fo a system of n seves with a symmetic foking policy, and any aival pocess with ate, the sevice capacity, that is, the maximum such that E [T ] < is max = n 1) E [C] A symmetic foking policy is defined as follows. Definition 6 Symmetic Foking). In a symmetic foking policy, the tasks of each job ae foked to all o a subset of the n seves such that the expected task aival ate is equal acoss the seves. Most commonly used policies: andom, ound-obin, shotest queue etc. ae symmetic acoss the n seves. of Claim 1: Since the foking policy is symmetic, the mean time spent by each seve pe job is E [C] /n. Thus the seve utilization is ρ = E [C] /n. To keep the system stable such that E [T ] <, the seve utilization must be less than 1. The esult in 1) follows fom this. The system with lowe E [C] has lowe E [T ] when is close to the sevice capacity. Thus, Claim 1 seves as a poweful technique to compae the latency with diffeent edundancy policies systems in the high egime. B. Log-concavity of F When the tail distibution F of sevice time is eithe log-concave o log-convex, we get a cleae undestanding of how edundancy affects latency and cost. Fo example, if we fok a job to all n seves and wait fo any 1 5

6 copy to finish, the expected computing cost E [C] = ne [ 1:n ]. It can be shown that ne [ 1:n ] is non-deceasing non-inceasing) in n when F is log-concave log-convex). Log-concavity of F is defined fomally as follows. Definition 7 Log-concavity and log-convexity of F ). The tail distibution function F is said to be log-concave log-convex) if log P > x) is concave convex) in x fo all x [, ). Fo bevity, when we say is log-concave log-convex) in this pape, we mean that F is log-concave logconvex). Some inteesting popeties and examples of log-concavity ae given in Appendix A. We efe eades to [26] fo a moe detailed study of log-concave distibutions. A canonical log-concave distibution is the shifted exponential, denoted by ShiftedExp, µ). It is an exponential with ate µ, plus a constant shift. An example of a log-convex distibution is the hype-exponential distibution, denoted by HypeExpµ 1, µ 2, p). It is a mixtue of two exponentials with ates µ 1 and µ 2 whee the exponential with ate µ 1 occus with pobability p. Remak 1. Log-concavity of implies that is new-bette-than-used, a notion which is consideed in [19]. Othe names used to efe to new-bette-than-used distibutions ae light-eveywhee in [21] and new-longe-than-used in [22]. Many andom vaiables with log-concave log-convex) F ae light heavy) tailed espectively, but neithe imply the othe. Unlike the tail of a distibution which chaacteizes how the maximum E [ n:n ], behaves fo lage n, log-concavity log-convexity) of F chaacteizes the behavio of the minimum E [ 1:n ], which is of pimay inteest in this wok. C. Relative Task Stat Times The elative stat times of the n tasks of a job is an impotant facto affecting the latency and cost. Let the elative stat times of the tasks be t 1 t 2 t n whee t 1 = without loss of geneality and t i fo i > 1 ae measued fom the instant when the ealiest task stats sevice. Fo instance, if n = 3 tasks stat at times 3, 4 and 7, then t 1 =, t 2 = 4 3 = 1 and t 3 = 7 3 = 4. In the case of patial foking when only tasks ae invoked, we can conside t +1, t n to be. Let S be the time fom when the ealiest task stats sevice, until any k tasks finish. Thus it is the k th smallest of 1 + t 1, 2 + t 2,, n + t n, whee i ae i.i.d. with distibution F. The computing cost C is given by, C = S + S t S t n +. 2) Using 2) we get seveal cucial insights in the est of the pape. Fo instance, in Section V we show that when F is log-convex, having t 1 = t 2 = = t n = gives the lowest E [C]. Then using Claim 1 we can infe that it is optimal to fok a job to all n seves when F is log-convex. IV. k = 1 CASE WITHOUT AND WITH EARLY CANCELLATION In this section we analyze the latency and cost of the n, 1) fok-join system, and the n, 1) fok-ealy-cancel system defined in Section II. We get the insight that it is bette to cancel edundant tasks ealy if F is log-concave. On the othe hand, if F is log-convex, etaining the edundant tasks is bette. A. Latency-Cost Analysis Lemma 1. The latency T of the n, 1) fok-join system is equivalent in distibution to that of an M/G/1 queue with sevice time 1:n. Poof: Conside the fist job that aives to a n, 1) fok-join system when all seves ae idle. Thus, the n tasks of this job stat sevice at thei espective seves simultaneously. The ealiest task finishes afte time 1:n, and all othe tasks ae immediately. So, the tasks of all subsequent jobs aiving to the system also stat simultaneously at the n seves as illustated in Fig. 3. Hence, aival and depatue events, and the latency of an n, 1) fok-join system is equivalent in distibution to an M/G/1 queue with sevice time 1:n. 6

7 3,1) fok-join M/G/1 Queue 1:3 Abandon Fig. 3: Equivalence of the n, 1) fok-join system with an M/G/1 queue with sevice time 1:n, the minimum of n i.i.d. andom vaiables 1, 2,..., n. Theoem 1. The expected latency and computing cost of an n, 1) fok-join system ae given by [ E [T ] = E T M/G/1] E [ ] 1:n 2 = E [ 1:n ] + 3) 21 E [ 1:n ]) E [C] = n E [ 1:n ] 4) whee 1:n = min 1, 2,..., n ) fo i.i.d. i F. Poof: By Lemma 1, the latency of the n, 1) fok-join system is equivalent in distibution to an M/G/1 queue with sevice time 1:n. The expected latency of an M/G/1 queue is given by the Pollaczek-Khinchine fomula 3). The expected cost E [C] = ne [ 1:n ] because each of the n seves spends 1:n time on the job. This can also be seen by noting that S = 1:n when t i = fo all i, and thus C = n 1:n in 2). Fom 4) and Claim 1 we can infe the following esult about the sevice capacity. Coollay 1. The sevice capacity of the n, 1) fok-join system is max = 1/E [ 1:n ], which is non-deceasing in n. In Coollay 2, and Coollay 3 below we chaacteize how E [T ] and E [C] vay with n. Coollay 2. Fo the n, 1) fok-join system with any sevice distibution F, the expected latency E [T ] is noninceasing with n. The behavio of E [C] = ne [ 1:n ] as n inceases depends on the log-concavity of as given by Popety 4 in Appendix A. Using that we can infe the following coollay about E [C]. Coollay 3. If F is log-concave log-convex), then E [C] is non-deceasing non-inceasing) in n. Fig. 4 and Fig. 5 show the expected latency vesus cost fo log-concave and log-convex F, espectively. In Fig. 4, the aival ate =.25, and is shifted exponential ShiftedExp,.5), with diffeent values of. Fo >, thee is a tade-off between expected latency and cost. Only when =, that is, is a pue exponential which is geneally not tue in pactice), we can educe latency without any additional cost. In Fig. 5, aival ate =.5, and is hypeexponential HypeExp.4,.5, µ 2 ) with diffeent values of µ 2. We get a simultaneous eduction in E [T ] and E [C] as n inceases. The cost eduction is steepe as µ 2 inceases. B. Ealy Task Cancellation We now analyze the n, 1) fok-ealy-cancel system, whee we cancel edundant tasks as soon as any task eaches the head of its queue. Intuitively, ealy cancellation can save computing cost, but the latency could incease due to the loss of divesity advantage povided by etaining edundant tasks. Compaing it to n, 1) fok-join system, we gain the insight that ealy cancellation is bette when F is log-concave, but ineffective fo log-convex F. Theoem 2. The expected latency and cost of the n, 1) fok-ealy-cancel system ae given by [ E [T ] = E T M/G/n], 5) E [C] = E [], 6) 7

8 Expected Latency E[T ] n = 1 = = 1 = 1.5 Expected Latency E[T ] µ 2 = 1 µ 2 = 1.5 µ 2 = 2 n = 1 2 n = Expected Computing Cost E[C] 1 n = Expected Computing Cost E[C] Fig. 4: The sevice time +Expµ) log-concave), with µ =.5, =.25. As n inceases along each cuve, E [T ] deceases and E [C] inceases. Only when =, latency educes at no additional cost. Fig. 5: The sevice time HypeExp.4, µ 1, µ 2 ) log-convex), with µ 1 =.5, diffeent values of µ 2, and =.5. Expected latency and cost both educe as n inceases along each cuve. 3,1) fok-ealy cancel M/G/3 Queue Cental Queue Abandon Choose fist idle seve Fig. 6: Equivalence of the n, 1) fok-ealy cancel system to an M/G/n queue with each seve taking time F to seve task, i.i.d. acoss seves and tasks. whee T M/G/n is the esponse time of an M/G/n queueing system with sevice time F. Poof: In the n, 1) fok-ealy-cancel system, when any one tasks eaches the head of its queue, all othes ae canceled immediately. The edundant tasks help find the shotest queue, and exactly one task of each job is seved by the fist seve that becomes idle. Thus, as illustated in Fig. 6, the latency of the n, 1) fok-ealy-cancel system is equivalent in distibution to an M/G/n queue. Hence E [T ] = E [ T M/G/n] and E [C] = E []. The exact analysis of mean esponse time E [ T M/G/n] has long been an open poblem in queueing theoy. A well-known appoximation given by [27] is, [ E T M/G/n] E [] + E [ 2] [ 2E [] 2 E W M/M/n] 7) whee E [ W M/M/n] is the expected waiting time in an M/M/n queueing system with load ρ = E [] /n. It can be evaluated using the Elang-C model [28, Chapte 14]. Using Popety 4 to compae the E [C] with and without ealy cancellation, given by Theoem 1 and Theoem 2 we get the following coollay. Coollay 4. If F is log-concave log-convex), then E [C] of the n, 1) fok-ealy-cancel system is geate than equal to less than o equal to) that of n, 1) fok-join join. In the low egime, the n, 1) fok-join system gives lowe E [T ] than n, 1) fok-ealy-cancel because of highe divesity due to edundant tasks. In the high egime, we can use by Claim 1 and Coollay 4 to imply the following esult about expected latency E [T ]. 8

9 35 3 n, 1) fok-join n, 1) fok-ealy-cancel n, 1) fok-join n, 1) fok-ealy-cancel Expected Latency E[T ] , aival ate of jobs Expected Latency E[T ] , aival ate of jobs Fig. 7: Fo the 4, 1) system with sevice time ShiftedExp2,.5) which is log-concave, ealy cancellation is bette in the high egime, as given by Coollay 5. Fig. 8: Fo the 4, 1) system with HypeExp.1, 1.5,.5), which is log-convex, ealy cancellation is wose in both low and high egimes, as given by Coollay 5. Coollay 5. If F is log-concave, ealy cancellation gives highe E [T ] than n, 1) fok-join when is small, and lowe in the high egime. If F is log-convex, then ealy cancellation gives highe E [T ] fo both low and high. Fig. 7 and Fig. 8 illustate Coollay 5. Fig. 7 shows a compaison of E [T ] with and without ealy cancellation of edundant tasks fo the 4, 1) system with sevice time ShiftedExp2,.5). We obseve that ealy cancellation gives lowe E [T ] in the high egime. In Fig. 8 we obseve that when is HypeExp.1, 1.5,.5) which is log-convex, ealy cancellation is wose fo both small and lage. In geneal, ealy cancellation is bette when is less vaiable lowe coefficient of vaiation). Fo example, a compaison of E [T ] with n, 1) fok-join and n, 1) fok-ealy-cancel systems as, the constant shift of sevice time ShiftedExp, µ) vaies indicates that ealy cancellation is bette fo lage. When is small, thee is moe andomness in the sevice time of a task, and hence keeping the edundant tasks unning gives moe divesity and lowe E [T ]. But as inceases, task sevice times ae moe deteministic due to which it is bette to cancel the edundant tasks ealy. V. PARTIAL FORKING k = 1 CASE) In many cloud computing applications the numbe of seves n is lage. Thus full foking of jobs to all seves can be expensive in the netwok cost of making emote-pocedue-calls to issue and cancel the tasks. Hence it is moe pactical to fok a job to a subset out of the n seves, efeed to as the n,, k) patial-fok-join system in Definition 3. In this section we analyze the k = 1 case, that is, the n,, 1) patial-fok-join system, whee an incoming job is foked to some out n seves and we wait fo any 1 task to finish. The seves ae chosen using a symmetic foking policy Definition 6). Some examples of symmetic foking policies ae: 1) Goup-based andom: This policy holds when divides n. The n seves ae divided into n/ goups of seves each. A job is foked to one of these goups, chosen unifomly at andom. 2) Unifom Random: A job is foked to any out of n seves, chosen unifomly at andom. Fig. 9 illustates the 4, 2, 1) patial-fok-join system with the goup-based andom and the unifom-andom foking policies. In the sequel, we develop insights into the best choice of and foking policy fo a given F. A. Latency-Cost Analysis In the goup-based andom policy, the job aivals ae split equally acoss the goups, and each goup behaves like an independent, 1) fok-join system. Thus, the expected latency and cost follow fom Theoem 1 as given in Lemma 2 below. 9

10 Fok to one of goups, chosen at andom Goup 1 Goup 2 a) Goup-based andom Fok to =2 seves chosen at andom b) Unifom andom Fig. 9: 4, 2, 1) patial-fok-join system, whee each job is foked to = 2 seves, chosen accoding to the goup-based andom o unifom andom foking policies. Lemma 2 Goup-based andom). The expected latency and cost when each job is foked to one of n/ goups of seves each ae given by E [ ] 1: 2 E [T ] = E [ 1: ] + 8) 2n E [ 1: ]) E [C] = E [ 1: ] 9) Poof: Since the job aivals ae split equally acoss the n/ goups, such that the aival ate to each goup is a Poisson pocess with ate /n. The tasks of each job stat sevice at thei espective seves simultaneously, and thus each goup behaves like an independent, 1) fok-join system with Poisson aivals at ate /n. Hence, the expected latency and cost follow fom Theoem 1. Using 9) and Claim 1, we can infe that the sevice capacity maximum suppoted ) fo an n,, 1) system with goup-based andom foking is n max = 1) E [ 1: ] Fom 1) we can infe that the that minimizes E [ 1: ] esults in the highest sevice capacity, and hence the lowest E [T ] in the high taffic egime. By Popety 4 in Appendix A, the optimal is = 1 = n) fo log-concave log-convex) F. Fo othe symmetic foking policies, it is difficult to get an exact analysis of E [T ] and E [C] because the tasks of a job can stat at diffeent times. Howeve, we can get bounds on E [C] depending on the log-concavity of, given in Theoem 3 below. Theoem 3. Conside an n,, 1) patial-fok join system, with a symmetic foking policy. Fo any elative task stat times t i, E [C] can be bounded as follows. E [ 1: ] E [C] E [] if F is log-concave 11) E [] E [C] E [ 1: ] if F is log-convex 12) In the exteme case when = 1, E [C] = E [], and when = n, E [C] = ne [ 1:n ]. To pove Theoem 3 we take expectation on both sides in 2), and show that fo log-concave and log-convex F, we get the bounds in 11) and 12), which ae independent of the elative task stat times t i. The detailed poof is given in Appendix B. In the sequel, we use the bounds in Theoem 3 to gain insights into choosing the best and best scheduling policy when F is log-concave o log-convex. B. Choosing the optimal By Popety 4 in Appendix A, E [ 1: ] is non-deceasing non-inceasing) with fo log-concave log-convex) F. Using this obsevation in conjuction with Theoem 3, we get the following esult about which minimizes E [C]. 1

11 1 8 = 1 = 2 = = 1 = 2 = 3 Expected Latency E[T ] 6 4 Expected Latency E[T ] , the aival ate of download jobs Fig. 1: Fo ShiftedExp1,.5) which is logconcave, foking to less moe) seves educes expected latency in the low high) egime , the aival ate of download jobs Fig. 11: Fo HypeExpp, µ 1, µ 2 ) with p =.1, µ 1 = 1.5, and µ 2 =.5 which is log-convex, foking to moe seves lage ) gives lowe expected latency fo all. Coollay 6 Cost vesus ). Fo a system of n seves with symmetic foking of each job to seves, = 1 = n) minimizes the expected cost E [C] when F is log-concave log-convex). Fom Coollay 6 and Claim 1 we get the following esult on which minimizes the sevice capacity. Coollay 7 Sevice Capacity vs. ). Fo a system of n seves with symmetic foking of each job to seves, = 1 = n) gives the highest sevice capacity when F is log-concave log-convex). Now let us now detemine the value of that minimizes expected latency fo log-concave and log-convex F. When the aival ate is small, E [T ] is dominated by the sevice time E [ 1: ] which is non-inceasing in. This can be obseved fo the goup-based foking policy by taking in 8). Thus we get the following esult. Coollay 8 Latency vs in low taffic egime). Foking to all n seves, that is, = n gives the loweste [T ] in the low egime fo any sevice time distibution F. Since the system with the highe sevice capacity has lowe latency in the high taffic egime, we can infe the following fom Coollay 7. Coollay 9 Latency vs. in high taffic egime). Given the numbe of seves n and symmetic foking of each job to seves, if F is log-concave log-convex) then, = 1 = n) gives lowest E [T ] in the high taffic egime. Coollay 8 and Coollay 9 ae illustated by Fig. 1 and Fig. 11 whee E [T ] is plotted vesus fo diffeent values of. Each job is assigned to seves chosen unifomly at andom fom n = 6 seves. In Fig. 1 the sevice time distibution is ShiftedExp, µ) which is log-concave) with = 1 and µ =.5. When is small, moe edundancy highe ) gives lowe E [T ], but in the high egime, = 1 gives lowest E [T ] and highest sevice capacity. On the othe hand in Fig. 11, fo a log-convex distibution HypeExpp, µ 1, µ 2 ), in the high load egime E [T ] deceases as inceases. Coollay 9 was peviously poven fo new-bette-than-used new-wose-than-used) instead of log-concave logconvex) F in [19], [21], using a combinatoial agument. Ou vesion is weake because log-concavity implies new-bette-than-used but the convese is not tue in geneal see Popety 3 in Appendix A). Using Theoem 3, we get an altenative, and aguably simple way to pove Coollay 9. Coollay 8 and Coollay 9 imply that foking to moe seves lage ) gives lowe E [T ] in the low and high egimes. But we obseve in Fig. 11 that this tue fo all. Fom 8) we can pove thus fo the goup-based andom policy. But the poof fo othe symmetic foking policies emains open. 11

12 4 35 Goup-based andom Unifom Random 6 5 Goup-based andom Unifom Random 3 Expected Latency E[T ] Expected Latency E[T ] , the aival ate of download jobs Fig. 12: Fo sevice distibution ShiftedExp1,.5) which is log-concave, unifom andom scheduling which stagges elative task stat times) gives lowe E [T ] than goup-based andom fo all. The system paametes ae n = 6, = , the aival ate of download jobs Fig. 13: Fo sevice distibution HypeExp.1, 1.5,.5) which is log-convex, goup-based scheduling gives lowe E [T ] than unifom andom in the high egime. The system paametes ae n = 6, = 2. C. Choosing the Foking Policy Fo a given, we now compae diffeent policies of choosing the seves fo each job. By Theoem 3 we know that E [C] E [ 1: ] fo log-convex F fo a symmetic policy with any elative task stat times. Since E [C] = E [ 1: ] when all tasks stat simutaneously, this implies that staggeing the task stat times does not help. Fo log-concave F, since E [C] E [ 1: ], highe divesity in the elative stating times of tasks gives lowe E [T ]. We state this fomally is Coollay 1 below. Coollay 1. Fo a given n and, and task aival distibution at each queue, if F is log-concave log-convex), the symmetic foking policy which esults in moe divesity in the elative task stat times gives lowe highe) latency in the high egime. This phenomenon is illustated in Fig. 12 and Fig. 13 fo the unifom andom and goup-based andom foking policies. Unifom andom foking gives moe divesity in elative stat times than the goup-based andom policy. Thus, in the high egime, unifom andom foking gives lowe latency fo log-concave F, as obseved in Fig. 12. But fo log-convex F, goup-based foking is bette in the high egime as seen in Fig. 13. Fo low, unifom andom foking is bette fo any F because it gives lowe expected waiting time in queue. VI. THE GENERAL k CASE We now move fom the k = 1 eplicated) case to geneal k, whee a job equies any k tasks to complete. In pactice, the geneal k case aises in lage-scale paallel computing famewoks such as MapReduce, and in content download fom coded distibuted stoage systems. In this section we pesent bounds on the latency and cost of the n, k) fok-join and n, k) fok-ealy-cancel systems. The n,, k) patial-fok-join system is not consideed hee, but we pesent a heuistic stategy fo it in Section VII. Fo geneal k, thee is also an inteesting divesity-paallelism tade-off in choosing k. Having lage k means smalle size tasks, and thus smalle expected sevice time pe task. But the divesity in waiting fo k out of n educes as k inceases. We demonstate this tade-off in Section VI-B. A. Latency and Cost of the n, k) fok-join system Unlike the k = 1 case, fo geneal k exact analysis is had because multiple jobs can be in sevice simultaneously fo e.g. blue and geen job in Fig. 1). Even fo the k = n case studied in [1], [11], only bounds on latency ae known. We genealize those latency bounds to any k, and also povide bounds on cost E [C]. The analysis of E [C] can be used to estimate the sevice capacity using Claim 1. 12

13 Expected Latency E[T ] Uppe Bound Simulation Lowe Bound Expected Computing Cost E[C] Uppe Bound Simulation Lowe Bound k, the numbe of seves we need to wait fo Fig. 14: Bounds on latency E [T ] vesus k Theoem 4), alongside simulation values. The sevice time Paeto.5, 2.5), n = 1, and =.5. A tighe uppe bound fo k = n is evaluated using Lemma k, the numbe of seves we need to wait fo Fig. 15: Bounds on cost E [C] vesus k Theoem 5) alongside simulation values. The sevice time Paeto.5, 2.5), n = 1, and =.5. The bounds ae tight fo k = 1 and k = n. Theoem 4 Bounds on Latency). The latency E [T ] is bounded as follows. E [T ] E [ k:n ] + E [ k:n]) 2 + Va [ k:n ]) 21 E [ k:n ]) E [T ] E [ k:n ] + E [ 1:n]) 2 + Va [ 1:n ]) 21 E [ 1:n ]) whee E [ k:n ] and Va [ k:n ] ae the mean and vaiance of k:n, the k th smallest in n i.i.d. andom vaiables 1, 2,, n, with i F. In Fig. 14 we plot the bounds on latency alongside the simulation values fo Paeto sevice time. The uppe bound 13) becomes moe loose as k inceases, because the split-mege system consideed to get the uppe bound see poof of Theoem 4) becomes wose as compaed to the fok-join system. Fo the special case k = n we can impove the uppe bound in Lemma 3 below, by genealizing the appoach used in [1]. Lemma 3 Tighte Uppe bound when k = n). Fo the case k = n, anothe uppe bound on latency is given by, 13) 14) E [T ] E [max R 1, R 2, R n )], 15) whee R i ae i.i.d. ealizations of the esponse time R of an M/G/1 queue with aival ate, sevice distibution F. Tansfom analysis [28, Chapte 25] can be used to detemine the distibution of R, the esponse time of an M/G/1 queue in tems of F x). The Laplace-Stieltjes tansfom Rs) of the pobability density function of f R ) of R is given by, ) ss) 1 E[] Rs) = s 1 s)), 16) whee s) is the Laplace-Stieltjes tansfom of the sevice time distibution f x). The lowe bound on latency 14) can be impoved fo shifted exponential F, genealizing the appoach in [11] based on the memoyless popety of the exponential tail. 13

14 Expected Latency E[T ] k k k =.25 = 1. = k, the numbe of seves we need to wait fo Fig. 16: Divesity-paallelism tade-off. As k inceases, divesity deceases, but paallelism benefit is highe because the size of each task educes. Task sevice time ShiftedExp /k, 1.), and aival ate =.5. Lemma 4 Tighte Lowe Bound fo Shifted Exponential F ). The latency E [T ] is lowe bounded by, ) 2 ) ) nµ + 1 nµ k 1 E [T ] + 1 nµ nµ )) + Theoem 5 Bounds on Cost). The expected computing cost E [C] can be bounded as follows. j=1 1 n j)µ. 17) E [C] k 1)E [] + n k + 1)E [ 1:n k+1 ] 18) k E [C] E [ i:n ] + n k)e [ 1:n k+1 ] 19) Fig. 15 shows the bounds alongside simulation plot of the computing cost E [C] when F is Paetox m, α) with x m =.5 and α = 2.5. The aival ate =.5, and n = 1 with k vaying fom 1 to 1 on the x-axis. We obseve that the bounds on E [C] ae tight fo k = 1 and k = n, which can also be infeed fom 18) and 19). B. Divesity-Paallelism Tade-off In Fig. 14 and Fig. 15 we obseve that the expected latency and cost incease with k. This is because, as k inceases we need to wait fo moe tasks out of n to complete, esulting in highe latency and cost. But in most computing and stoage applications, the sevice time is popotional to the size of the task, which deceases as k inceases. As a esult, thee is a divesity-paallelism tade-off in choosing the optimal k. We demonstate the divesity-paallelism tade-off in Fig. 16 whee sevice time ShiftedExp k, µ), with µ = 1., and k = /k. The constant is popotional to the size of the job, and hence the size of each task is popotional to k = /k. The latency initially educes with k because highe divesity dominates ove lack of paallelism lage value of k ). As k inceases beyond theshold k, the loss of divesity causes an incease in latency. The optimal k that minimizes latency inceases with, because the paallelism due to k = /k dominates ove the divesity advantage of having k < n. We can also obseve the divesity-paallelism tade-off mathemetically in the low taffic egime, fo ShiftedExp /k, µ). If we take in 14) and 13), both bounds coincide and we get, lim E [T ] = E [ k:n] = k + H n H n k, 2) µ whee H n = n 1/i, the nth hamonic numbe. The paallelism benefit comes fom the fist tem in 2), which educes with k. The divesity of waiting fo k out of n tasks causes the second tem to incease with k. The optimal k that minimizes 2) stikes a balance between these two opposing tends. 14

15 C. Latency and Cost of the n, k) fok-ealy-cancel system We now analyze the latency and cost of the n, k) fok-ealy-cancel system whee the edundant tasks ae canceled as soon as any k tasks stat sevice. Theoem 6 Latency-Cost with Ealy Cancellation). The cost E [C] and an uppe bound expected latency E [T ] with ealy cancellation is given by E [C] = ke [] 21) E [T ] E [max R 1, R 2, R k )] 22) whee R i ae i.i.d. ealizations of R, the eponse time of an M/G/1 queue with aival ate k/n and sevice distibution F. The Laplace-Stieltjes tansfom of the esponse time R of an M/G/1 queue with sevice distibution F x) and aival ate is same as 16), with eplaced by k/n. By compaing the cost E [C] = ke [] in 21) to the bounds in Theoem 5 without ealy cancellation, we can get insights into when ealy cancellation is effective fo a given sevice time distibution F. Fo example, when F is log-convex, the uppe bound in 18) is smalle than ke []. Thus we can infe that ealy cancellation is not effective when is log-convex, as we also obseved in Fig. 8 fo the k = 1 case. VII. A HEURISTIC REDUNDANCY STRATEGY We have seen that edundancy is an effective method to educe latency, while efficiently using the computing esouces. In Section IV and Section V, we got stong insights into the optimal edundancy stategy fo log-concave and log-convex sevice time, but it is not obvious to infe the best stategy fo abitay sevice distibutions. We now popose such a heuistic edundancy stategy to minimize the latency, subject to computing and netwok cost constaints. This stategy can also be used on taces of task sevice time when a closed-fom expessions of F and its ode statistics ae not known. A. Genealized Fok-join Model We fist intoduce a geneal fok-join vaiant that is a combination of the patial fok intoduced in Section II, and patial ealy cancellation of edundant tasks. Definition 8 n, f,, k) fok-join system). Fo a system of n seves and a job that equies k tasks to complete, we do the following: Fok the job to f out of the n seves. When any f tasks ae at the head of queues o in sevice aleady, cancel all othe tasks immediately. If moe than tasks stat sevice simultaneously, etain andomly chosen ones out of them. When any k tasks finish, cancel all emaining tasks immediately. Note k tasks may finish befoe some stat sevice, and thus we may not need to pefom the patial ealy cancellation in the second step above. The f tasks that ae canceled ealy, help find the shotest out of the f queues, thus educing waiting time. Fom the tasks etained, waiting fo any k to finish povides divesity and hence educes sevice time. The special cases n, n, n, k), n, n, k, k) and n,,, k) coespond to the n, k) fok-join and n, k) fok-ealycancel and n,, k) patial-fok-join systems espectively, as defined in Section II. B. Choosing Paametes f and We popose a heuistic stategy to choose f and to minimize expected latency E [T ], subject to a computing cost constaint is E [C] γ, and a netwok cost constaint is f max. We impose the second constaint because foking to moe seves esults in highe netwok cost of emote-pocedue-calls RPCs) to launch and cancel the tasks. 15

16 Claim 2 Heuistic Redundancy Stategy). Good heuistic choices of f and to minimize E [T ] subject to constaints E [C] γ and f max ae f = max, 23) = ag min ˆT ), s.t. Ĉ) γ 24) [, max] whee ˆT ) and Ĉ) ae estimates of the expected latency E [T ] and cost E [C], defined as follows: E [ ] k: ˆT 2 ) E [ k: ] + 2n E [ k: ]), 25) Ĉ) E [ k: ]. 26) To justify the stategy above, obseve that fo a given, inceasing f gives highe divesity in finding the shotest queues and thus educes latency. Since f tasks ae canceled ealy befoe stating sevice, f affects E [C] only mildly, though the elative task stat times of tasks that ae etained. So we conjectue that it is optimal to set f = max in 23), the maximum value possible unde netwok cost constaints. Changing on the othe hand does affect both the computing cost and latency significantly. Thus to detemine the optimal, we minimize ˆT ) subject to constaints Ĉ) γ and max as given in 24). The estimates ˆT ) and Ĉ) ae obtained by genealizing Lemma 2 fo goup-based andom foking to any k, and that may not divide n. When the ode statistics of F ae had to compute, o F itself is not explicitly known, ˆT ) and Ĉ) can be also be found using empiical taces of. The souces of inaccuacy in the estimates ae as follows: Since the estimates ˆT ) and Ĉ) ae based on goup-based foking, they conside that all tasks stat simultaneously. Vaiability in elative task stat times can esult in actual latency and cost that ae diffeent fom the estimates. Fo example, fom Theoem 3 we can infe that when F is log-concave log-convex), the actual computing cost E [C] is less than geate than) Ĉ). Fo k > 1, the latency estimate ˆT ) is a genealization of the split-mege queueing uppe bound in Theoem 4. Since the bound becomes loose as k inceases, the eo E [T ] ˆT ) inceases with k. The estimates ˆT ) and Ĉ) ae by definition independent of f, which is not tue in pactice. As explained above, fo f >, the actual E [T ] is geneally less than ˆT ), and E [C] can be slightly highe o lowe than Ĉ). C. Simulation Results We now pesent simulation esults compaing the heuistic given by Claim 2 to othe stategies, including baseline case without any edundancy. The sevice time distibutions consideed hee ae neithe log-concave no log-convex, thus making it had to diectly infe the best edundancy stategy using the analysis pesented in the pevious sections. In Fig. 17 we plot the latency E [T ] vesus computing cost E [C] with sevice time Paeto1, 2.2), and diffeent edundancy stategies. Othe paametes ae n = 1, k = 1, and aival ate =.25. In compaison with the no edundancy case blue dot), the heuistic stategy ed diamond) gives a significant eduction in latency,while satisfying E [C] 5 and f 7. We also plot the latency-cost behavio as = f vaies fom 1 to n. Obseve that using ealy cancellation f > ) in the heuistic stategy gives a slight eduction in latency in compaison with the = f = 4 point. The cost E [C] inceases slightly, but emains less than γ. In Fig. 18 we show a case whee the cost E [C] does not always incease with the amount of edundancy. The task sevice time is a mixtue of an exponential Exp2) and a shifted exponental ShiftedExp1, 1.5), each occuing with equal pobability. All othe paametes ae same as in Fig. 17. The heuistic stategy found using Claim 2 is = f = max = 5, limited by the f max constaint athe than the E [C] γ constaint. VIII. CONCLUDING REMARKS In this pape we conside a edundancy model whee each incoming job is foked to queues at multiple seves and we wait fo any one eplica to finish. We analyze how edundancy affects the latency, and the cost of computing 16

17 2.2 No Redundancy: f = = k 1.2 No Redundancy: f = = k 2. Vaying = f fom k to n Heuistic Stategy = 4 and f = 7 1. Vaying = f fom k to n Heuistic Stategy = f = 5 Expected Latency E[T ] Expected Latency E[T ] Expected Computing Cost E[C] Expected Computing Cost E[C] Fig. 17: Compaing the heuistic stategy with cost constaint γ = 5 and netwok constaint max = 7 to othe edundancy stategies. The sevice time distibution is Paeto1, 2.2). Fig. 18: Compaing the heuistic with cost constaint γ = 2 and netwok constaint max = 5 to othe edundancy stategies. The sevice time distibution is an equipobable mixtue of Exp2) and ShiftedExp1, 1.5). time, and demonstate how the log-concavity of sevice time is a key facto affecting the latency-cost tade-off. Some insights that we get ae: Fo log-convex sevice time, foking to moe seves moe edundancy) educes both latency and cost. On the othe hand, fo log-concave sevice time, moe edundancy can educe latency only at the expense of an incease in cost. Ealy cancellation of edundant equests can save both latency and cost fo log-concave sevice time, but it is not effective fo log-convex sevice time. Using these insights, we also design a heuistic edundancy stategy fo an abitay sevice time distibution. Ongoing wok includes developing online stategies to simultaneously lean the sevice distibution, and the best edundancy stategy. Moe boadly, the poposed edundancy techniques can be used to educe latency in seveal applications beyond the ealm of cloud stoage and computing systems, fo example cowdsoucing, algoithmic tading, manufactuing etc. I. ACKNOWLEDGEMENTS We thank Sem Bost and Rhonda Righte fo helpful suggestions to impove this wok. APPENDI A LOG-CONCAVITY OF F In this section we pesent some popeties and examples of log-concave and log-convex andom vaiables that ae elevant to this wok. Fo moe popeties please see [26]. Popety 1 Jensen s Inequality). If F is log-concave, then fo < θ < 1 and fo all x, y [, ), The inequality is evesed if F is log-convex. P > θx + 1 θ)y) P > x) θ P > y) 1 θ. 27) Poof: Since F is log-concave, log F is concave. Taking log on both sides on 27) we get the Jensen s inequality which holds fo concave functions. Popety 2 Scaling). If F is log-concave, fo < θ < 1, P > x) P > θx) 1/θ 28) 17

18 The inequality is evesed if F is log-convex. Poof: We can deive 28) by setting y = in 27). P > θx + 1 θ)) P > x) θ P > ) 1 θ, 29) P > θx) P > x) θ. 3) To get 3) we obseve that if F is log-concave, then P > ) has to be 1. Othewise log-concavity is violated at x =. Raising both sides of 3) to powe 1/θ we get 28). The evese inequality of log-convex F can be poved similaly. Popety 3 Sub-multiplicativity). If F is log-concave, the conditional tail pobability of satisfies fo all t, x >, The inequalities above ae evesed if F is log-convex. Poof: P > x + t > t) P > x) 31) P > x + t) P > x) P > t) 32) P > x) P > t) 33) = P > x ) x + t) P > t ) x + t), 34) x + t x + t P > x + t) x x+t P > x + t) t x+t, 35) whee we apply Popety 2 to 34) to get 35). Equation 31) follows fom 35). Note that fo exponential F which is memoyless, 31) holds with equality. Thus log-concave distibutions can be thought to have optimistic memoy, because the conditional tail pobability deceases ove time. The definition of the notions new-bette-than-used in [19] is same as 31). By Popety 3 log-concavity of F implies that is new-bette-than-used. New-bette-than-used distibutions ae efeed to as light-eveywhee in [21] and new-longe-than-used in [22]. Popety 4. If is log-concave log-convex), E [ 1: ] is non-deceasing non-inceasing) in. Poof: Setting θ = / + 1 in Popety 2, we get P > x) P > x ) +1)/, 36) + 1 ) ) +1 P > x P > x, 37) + 1 ) ) +1 P > x dx P > x dx, 38) + 1 P > y) dy + 1) P > z) +1 dz, 39) E [ 1: ] + 1)E [ 1:+1 ], 4) whee in 37) we pefom a change of vaiables to x = x. Integating on both sides fom to we get 38). Again by doing change of vaiables y = x / of the left-side and z = x / + 1) on the ight-side we get 39). By using the fact that the expected value of a non-negative andom vaiable is equal to the integal of its tail distibution we get 4). Fo log-convex all the above inequalities ae flipped to show that E [ 1: ] + 1)E [ 1:+1 ]. Remak 2. If is new-bette-than-used a weake notion implied by log-concavity of ), it can be shown that E [] E [ 1: ] fo all 41) 18

19 This is weake than Popety 4 which shows the monotonicity of E [ 1: ] fo log-concave log-convex). Popety 5 Hazad Rates). If F is log-concave log-convex), then the hazad ate hx), which is defined by F x)/ F x), is non-deceasing non-inceasing) in x. Popety 6 Coefficient of Vaiation). The coefficient of vaiation C v = σ/µ is the atio of the standad deviation σ and mean µ of andom vaiable. It is at most 1 fo log-concave, at least 1 fo log-convex, and equal to 1 when is pue exponential. Popety 7 Examples of Log-concave F ). The following andom vaiables have log-concave F : Shifted Exponential Exponential plus constant > ) Unifom ove any convex set Weibull with shape paamete c 1 Gamma with shape paamete c 1 Chi-squaed with degees of feedom c 2 Popety 8 Examples of Log-convex F ). The following andom vaiables have log-convex F : Exponential Hype Exponential Mixtue of exponentials) Weibull with shape paamete < c < 1 Gamma with shape paamete < c < 1 APPENDI B PROOFS OF THE k = 1 CASE Poof of Theoem 3: Using 2), we can expess the cost C in tems of the elative task stat times t i, and S as follows. C = S + S t S t +, 42) whee S is the time between the stat of sevice of the ealiest task, and when any 1 of the tasks finishes. The tail distibution of S is given by PS > s) = P > s t i ). 43) By taking expectation on both sides of 42) and simplifying we get, E [C] = PS > s)ds, 44) u=1 t u tu+1 = u PS > s)ds, 45) u=1 t u tu+1 t u = u PS > t u + x)dx, 46) = u=1 u u=1 tu+1 t u u P > x + t u t i )dx. 47) We now pove that fo log-concave F, E [C] E []. The poof that E [C] E [] when F is log-convex 19

20 follows similaly with all inequalities below evesed. We expess the integal in 47) as, u ) u E [C] = u P > x + t u t i )dx P > x + t u+1 t i )dx, 48) u=1 u=2 u ) u = P > x u=1 u + t u t i dx u ) = E [] + P > x u 1 u + t u t i E [], P > x u + t u+1 t i ) dx ), 49) ) P ) > x u 1 + t u t i dx, 5) whee in 48) we expess each integal in 47) as a diffeence of two integals fom to. In 49) we pefom a change of vaiables x = x /u. In 5) we eaange the gouping of the tems in the sum; the u th negative integal is put in the u + 1 tem of the summation. Then the fist tem of the summation is simply P > x)dx which is equal to E []. In 5) we use the fact that each tem in the summation in 49) is positive when F is log-concave. This is shown in Lemma 5 below. Next we pove that fo log-concave F, E [C] E [ 1: ]. Again, the poof of E [C] E [ 1: ] when F is log-convex follows with all the inequalities below evesed. E [C] = = u=1 u=1 tu+1 t u u u P P E [ 1: ], u P > ux + t u t i ) ) /u > x + ut u t i ) dx ) > x dx + u u=2 u 1 51) ) /u dx, 52) u P > x + ut u t i ) ) ) /u P > x + ut u+1 t i ) dx, ) /u dx P > x + u 1)t u t i ) 53) ) ) u 1 dx, 54) whee we get 52) by applying Popety 2 to 47). In 53) we expess the integal as a diffeence of two integals fom to, and pefom a change of vaiables x = x /u. In 54) we eaange the gouping of the tems in the sum; the u th negative integal is put in the u + 1 tem of the summation. The fist tem is equal to E [ 1: ]. We use Lemma 6 to show that each tem in the summation in 54) is negative when F is log-concave. Lemma 5. If F is log-concave, u ) P > x u + t u t i The inequality is evesed fo log-convex F. u 1 55) ) P > x u 1 + t u t i. 56) 2