STABILITY OF LOAD BALANCING ALGORITHMS IN DYNAMIC ADVERSARIAL SYSTEMS

STABILITY OF LOAD BALANCING ALGORITHMS IN DYNAMIC ADVERSARIAL SYSTEMS ELLIOT ANSHELEVICH, DAVID KEMPE, AND JON KLEINBERG Absrac. In he dynamic load balancing problem, we seek o keep he job load roughly evenly disribued among he processors of a given nework. The arrival and deparure of jobs is modeled by an adversary resriced in is power. Muhukrishnan and Rajaraman (1998) gave a clean characerizaion of a resricion on he adversary ha can be considered he naural analogue of a cu condiion. They proved ha a simple local balancing algorihm proposed by Aiello e. al. (1993) is sable agains such an adversary if he inserion rae is resriced o a (1 ε) fracion of he cu size. They lef as an open quesion wheher he algorihm is sable a rae 1. In his paper, we resolve his quesion posiively, by proving sabiliy of he local algorihm a rae 1. Our proof echniques are very differen from he ones used by Muhukrishnan and Rajaraman, and yield a simpler proof and igher bounds on he difference in loads. In addiion, we inroduce a muli-commodiy version of his load balancing model, and show how o exend he resul o he case of balancing wo differen kinds of loads a once (obaining as a corollary a new proof of he 2-commodiy Max-Flow Min-Cu Theorem). We also show how o apply he proof echniques o he problem of rouing packes in adversarial sysems. Awerbuch e. al. (2001) showed ha he same load balancing algorihm is sable agains an adversary insering packes a rae 1 wih a single desinaion, in dynamically changing neworks. Our echniques give a much simpler proof for a differen model of adversarially changing neworks. Key words. Adversarial load balancing, packe rouing, muli-commodiy flow AMS subjec classificaions. 68W15,68W40 1. Inroducion. Load Balancing. In a disribued nework of compuing hoss, he performance of he sysem can depend crucially on dividing up work effecively across he paricipaing nodes. This ype of load balancing problem has been sudied in many differen models, cenered around he idea ha an algorihm should avoid creaing ho spos ha degrade sysem performance [24]. We consider a basic model of load balancing in a disribued nework, which has formed he basis of a number of earlier sudies [1, 16, 5, 21, 22]. A nework of idenical processors is represened by an undireced graph G = (V, E). There are a number of jobs o be processed in he sysem, absracly represened by uni-size okens. Time progresses in discree seps called rounds; in a given round, each oken is held by one of he nodes, which is viewed as processing he associaed job, and he load on a node is defined o be he number of okens i holds. The goal is o balance he loads, so ha no single node has oo many okens; his can be accomplished by ransmiing okens beween neighboring nodes of he graph, a a rae of one oken per edge per round. We are paricularly ineresed in local algorihms for his problem: raher han using a cenralized approach o coordinae he movemen of okens, each node will simply A preliminary version of his paper appeared in Proc. 34h Annual ACM Symposium on Theory of Compuing, 2002. Deparmen of Compuer Science, Rensselaer Polyechnic Insiue, Troy, NY 12180. Email: eanshel@cs.rpi.edu. This research was suppored in par by an NSF graduae fellowship. Deparmen of Compuer Science, Universiy of Souhern California, Los Angeles, CA 90089. Email: clkempe@usc.edu. This resesarch was suppored by an NSF graduae fellowship. Deparmen of Compuer Science, Cornell Universiy, Ihaca, NY 14853. Email: kleinber@cs.cornell.edu. Research suppored in par by a David and Lucile Packard Foundaion Fellowship, an ONR Young Invesigaor Award, and NSF Faculy Early Career Developmen Award CCR-9701399. 1

2 ELLIOT ANSHELEVICH, DAVID KEMPE, AND JON KLEINBERG compare is load o hose of is neighbors, and decide wheher o move a oken across an edge based on his informaion. This model is clearly very simple in a number of respecs, paricularly in he uniformiy of he processors (nodes) and jobs (okens), and he fac ha any job can be execued on any processor. More subly, i is no even clear in all seings ha balancing load evenly is he opimal sraegy in a disribued nework of processors (see, e.g., [12]). A he same ime, however, he model cleanly capures he basic consrains imposed by an underlying inerconnecion opology in he process of disribuing jobs hrough a nework, as evidenced by he resuls of previous analysis [1, 16, 5, 21, 22]; he simpliciy of he model allows one o reason very clearly abou he effec of hese consrains. Early work on he model focused on he saic version of he problem: each node is given a se of okens iniially, and nodes mus exchange okens as rapidly as possible so ha hey all end up wih approximaely he same number [1, 16, 5, 22]. However, load balancing is a naural seing in which o sudy algorihms designed o run indefiniely; jobs (okens) may arrive and depar from he nework, and a all imes, he algorihm mus mainain an approximaely uniform load across nodes. This is a ype of sabiliy condiion: no load should diverge arbirarily from he average as ime progresses. For a number of differen models, such dynamic algorihms have been sudied in a probabilisic framework, where one assumes an underlying randomized process ha generaes job arrivals and deparures; see, e.g., [13, 20] and he references herein. An Adversarial Model. Moivaed by work in he relaed area of packe rouing [7, 8, 11, 4], Muhukrishnan and Rajaraman proposed an adversarial framework for sudying dynamic load balancing in he oken-based model we have been discussing [21]. Raher han considering a probabilisic process ha generaes okens, hey posi an adversary ha is allowed a he beginning of each round o inroduce okens a some nodes (corresponding o new jobs) and remove okens from ohers (corresponding o jobs ha have finished). Subsequenly, an algorihm is allowed o move okens across edges as above so as o ry o mainain balanced loads. This alernaion of moves by he adversary and algorihm coninues for an arbirary number of rounds. Noe ha by allowing he adversary o conrol he removal of okens as well as heir arrival, one is modeling a wors-case assumpion ha jobs may have arbirary duraion, and he algorihm does no know how much processing ime a job has remaining unil he momen i ends. If we le a denoe he average number of okens per node in he sysem a he beginning of round, and h (v) denoe he number of okens a node v (he heigh of v) a round, hen he goal of a dynamic load balancing algorihm in his model is o keep h (v) close o a for all nodes v and rounds. Formally, we say ha an algorihm is sable agains a given adversary if here is a consan B such ha h (v) a B for all nodes v and rounds. Noe ha sabiliy in his conex imposes a bound on deviaion from he average; i is no required ha he acual number of okens in he sysem remain bounded. As in he case of packe rouing [11, 4], one needs o find a suiable resricion on he adversary: an arbirarily powerful adversary could flood a paricular se of nodes S V wih okens much faser han hese nodes can spread he okens ou o he res of he graph, and hereby preven any algorihm from mainaining sabiliy. This consideraion moivaes he following naural definiion of an adversary [21] wih rae r. For a se S V, le e(s) denoe he se of edges wih exacly one end in S,

STABILITY OF LOAD BALANCING ALGORITHMS 3 and δ (S) he ne increase in okens in se S due o he addiion and removal of jobs in round (noe ha δ (S) could be negaive). If he heighs of nodes in S were o change precisely according o average, hen he ne change in okens in S would be S (a +1 a ). One wans he difference beween hese wo quaniies o be accouned for by he edges in e(s). We say ha he adversary has rae r if for all S V, one has δ (S) S (a +1 a ) r e(s), (1.1) For rae r > 1, here are adversaries agains which no algorihm (wheher online or offline) can be sable. Muhukrishnan and Rajaraman gave a local-conrol algorihm ha is sable agains all adversaries of rae r, for every r < 1. As an open quesion, hey asked wheher here exiss a local-conrol algorihm ha is sable agains all adversaries of rae 1. The presen work. We begin by providing a local-conrol load balancing algorihm ha is sable agains every adversary of rae 1, hereby resolving he open quesion of Muhukrishnan and Rajaraman. In fac, we show ha he following simple rule has his sabiliy propery, for every value of he parameer θ: A any round, if he number of okens on node u exceeds he number of okens on is neighbor v by a leas θ, hen u moves a oken o v. This ype of algorihm was considered in earlier work on he saic model by Aiello e al. [1], as well as by many of he subsequen papers. Seing θ = 2 +1, where is he maximum node degree in G, yields he specific algorihm sudied by Muhukrishnan and Rajaraman. Beyond simply showing he sabiliy of local algorihms a he criical rae r = 1, our analysis is based on a new proof echnique in which a poenial funcion bound is mainained no only for he enire node se V, bu for every subse of V. Compared o [21], we obain significanly improved bounds on he deviaion from he average, and a simpler proof. Specifically, we show ha he maximum possible deviaion from he average is O( n), where n is he number of nodes of G, and his is asympoically opimal in he wors case; he analysis in [21] had esablished a bound of O( 2 n 2.5 (1 r) 1 ) when r < 1. Our analysis also shows sabiliy in a more general model where edges of G can appear and disappear over ime. Following his, we inroduce a muli-commodiy version of his load balancing model. We consider a sysem in which here are k disinc ypes of jobs. The jobs of one given ype induce he same load on each processor; bu he differen ypes of jobs place differen resource requiremens on he nodes, and so we require he load balancing condiion o apply o each ype separaely. Formally, we have he same adversarial model as before wih a nework G and a collecion of okens; bu now he okens are pariioned ino k commodiies and he sabiliy requiremen mus hold when he okens of each commodiy are considered separaely. In a single round, a mos one oken in oal can be sen across any one edge. (This is in keeping wih he sandard muli-commodiy noion ha consrains a nodes mus be saisfied by each commodiy separaely, while shared edge capaciies mus be respeced by he commodiies cumulaively.) We show ha he naural rae condiion on adversaries essenially obained by summing Equaion (1.1) over he commodiies can be relaed in a precise sense o he cu condiion for sandard muli-commodiy nework flow. As a resul, applying well-known resuls on he cu condiion [17, 18, 19], we find ha for every k > 2, here

4 ELLIOT ANSHELEVICH, DAVID KEMPE, AND JON KLEINBERG is a k-commodiy adversary of rae r k 1 agains which no load balancing algorihm can be sable. For k = 2, however, he cu condiion does no pose an obsacle o having algorihms ha are sable all he way up o rae 1. Indeed, we are able o generalize our firs resul o show ha for 2-commodiy load balancing, here is a simple localconrol algorihm ha is sable agains every adversary of rae 1. We also use he relaionship beween adversaries and cu condiions o provide a new proof of Hu s Max-Flow Min-Cu Theorem for 2-commodiy flow [17]. While our proof is no necessarily shorer han oher proofs discovered subsequen o Hu s [19, 23], i is arguably more elemenary: i does no require linear programming dualiy (as in [19]) or even he radiional Max-Flow Min-Cu Theorem for single-commodiy flow (as in [23]). Finally, we furher develop he connecion beween dynamic load balancing and nework flows by exending our analysis o packe rouing in he adversarial model considered by Aiello e al. [2] and Gamarnik [15]. We give an adapive rouing algorihm ha is sable agains adversaries of rae 1 in he case where packes can be injeced a muliple sources bu are desined for a single sink; our algorihm is sable in a dynamic nework model where edges can appear and disappear. A sable algorihm for his version of he problem was previously given in a recen paper of Awerbuch e al. [6], using a differen, bu essenially more general, noion of a dynamic nework; our proof, a direc adapion of he analysis of our single-commodiy load balancing algorihm, is considerably shorer and simpler. Recen Progress. Since he original publicaion of his work, several papers have sudied a model wherein nodes can exchange arbirarily many jobs in one sep, and each node wih non-empy load execues one job in each ime sep. This ype of load-balancing algorihm (ofen called he diffusion algorihm) has been very well sudied, and is known o have many nice properies, alhough is behavior in he presence of an adversary was unknown unil recenly. In his model, he cu condiion becomes rivial; insead, he adversary is allowed o inser a mos n jobs in each ime sep, and sabiliy is defined in erms of an upper bound on he load of all nodes. Berenbrink, Friedezky, and Golberg [9] show ha he work-sealing algorihm, in which only processors wih empy queues reques jobs from ohers, is sable agains adversaries of rae sricly below 1. They assume ha processors can reques jobs from any oher processor. Anagnosopoulos, Kirsch, and Upfal [3] show he same ype of sabiliy for a local proocol ha makes nodes equalize load wih heir neighbors. Mos recenly, Berenbrink, Friedezky, and Marin [10] proved sabiliy agains rae-1 adversaries for a proocol in which nodes exchange load wih all heir neighbors. Boh he proocol and he analysis are very similar o he ones in he presen work. 2. Single-commodiy load balancing. In his secion, we will sudy he load balancing problem for a single commodiy, and in paricular prove ha he naural balancing algorihm is sable a rae 1, hus seling an open quesion from [21]. We firs define precisely he model and he algorihm, and inroduce he necessary noaion. 2.1. Model and Algorihm. The nework is represened by a conneced undireced graph G = (V, E) wih n = V nodes. In each round, he adversary firs adds or removes okens (his is called he Adversary sep). Subsequenly, in he Redisribuion sep, up o one oken can be moved along each edge e E by he algorihm (or up o c e okens in he case of neworks wih edge capaciies). The adversary is limied by he following cu condiion: For a subse S V of

STABILITY OF LOAD BALANCING ALGORITHMS 5 nodes, le δ (S), a, and e(s) be defined as above. Then, he inserion and removal of okens by he adversary during round has o saisfy δ (S) S (a +1 a ) e(s). (2.1) Nodes have queues associaed wih hem, in which hey sore heir okens. The heigh h (v) of a node v is he number of okens in v s queue a he beginning of round. The imbalance is b (v) = h (v) a, i.e., he number of excess (or missing) okens a node v wih respec o he average over he enire nework. h (v) and b (v) denoe he same quaniies afer he Adversary sep of round. I is he decision of he algorihm along which edges o send okens. The goal of any balancing algorihm is o keep he imbalance bounded for all nodes. If an algorihm ensures ha here is an absolue bound B such ha b (v) B for all nodes v and all imes, we call he algorihm sable. We will show ha he following very simple family of local-conrol algorihms is sable agains every adversary respecing he cu condiion. I has a hreshold parameer θ 1, which deermines how aggressively he algorihm balances. Algorihm SCLB θ A each ime, for each edge e = (u, v): If h (u) h (v) + θ, hen send a oken from u o v. If h (v) h (u) + θ, hen send a oken from v o u. This algorihm does no specify wheher okens are sen along an edge (u, v) when h (u) h (v) < θ. All of our subsequen saemens will remain rue independenly of wha he algorihm does in his case. (Noice ha his algorihm only requires local informaion, and can herefore be execued in a disribued fashion in a nework.) 2.2. Sabiliy of he algorihm. Our main heorem in his secion is ha he algorihm SCLB θ is sable agains any adversary respecing he cu condiion of Inequaliy 2.1. We allow for okens o be in he sysem a ime 1, and le H := max v V h 1 (v). Theorem 2.1. For any adversary respecing he cu condiion, and any θ 1, he algorihm SCLB θ is sable, i.e., here is a consan B (depending on H, θ and G), such ha b (v) B for all nodes v a all imes. The inuiion behind our proof is based on he (incorrec) observaion ha he algorihm seems o ensure ha he heigh difference beween adjacen nodes canno grow beyond θ. Hence, he larges difference beween he heighs of any wo nodes should be achieved when G is a simple pah, and he wo nodes are he endpoins of he pah having a heigh difference of abou nθ. I is, however, easy o see ha he heigh difference beween wo adjacen nodes can become more han θ, because he adversary can rearrange he heighs wihin ses o a cerain exen. Imagine, for insance, a long pah wih he adversary adding okens o he firs node of he pah, unil he firs node has heigh nθ, and he heigh of each successive node decreases by θ. Then i is possible o rearrange he okens on he nodes previously having heighs nθ, (n 1)θ and (n 2)θ so ha hey each have (n 1)θ okens, which would resul in he heigh difference of 2θ beween he hird and fourh node of he pah, since he heigh of he fourh node remains (n 3)θ. The adversary can do his by adding wo okens o he node wih heigh (n 2)θ and subracing one from he node wih heigh nθ for θ successive rounds. During each Redisribuion sep of his process, a oken will move from he hird o he fourh

6 ELLIOT ANSHELEVICH, DAVID KEMPE, AND JON KLEINBERG node, bu hese okens will coninue raveling downhill along he pah, so ha he heigh of he fourh node will never grow o be much larger han (n 3)θ. This process obeys he cu condiion, bu aferwards, here are wo adjacen nodes wih almos 2θ heigh difference. Forunaely, alhough here are now more nodes wih large heighs, he adversary had o pay for his rearrangemen by making he highes queue smaller. In an amorized sense, he siuaion has no become worse. These observaions sugges mainaining heigh bounds for each subse of he nodes, and showing ha hese bounds form an invarian. For convenience, we will wrie b (S) = v S b (v) for any se S V of verices (and similarly for oher quaniies like h (S) and δ (S)). Wih denoing he maximum degree of any verex, we wrie γ = 2 + θ. The key invarian is he following: b (S) n j=n S +1 (H + γ j) for all S V. (2.2) γ H..... 1 2 n S Fig. 2.1. An illusraion of Invarian 2.2 Figure 2.1 illusraes he upper bound of his invarian picorially as he sum of he column heighs of he righ-mos S columns. Below, we prove Lemma 2.2, showing ha Inequaliy (2.2) is indeed an invarian over ime for he algorihmsclb θ, agains any adversary respecing he cu condiion. Lemma 2.2. If he adversary respecs he cu condiion, and he invarian (2.2) holds a he beginning of round, hen i holds a he beginning of round + 1. Using his lemma, he proof of Theorem 2.1 is sraighforward. Proof of Theorem 2.1. We prove by inducion ha (2.2) holds a every ime. A ime 1, h 1 (v) H for all v by definiion, so b 1 (S) v S H n j=n S +1 (H + γ j)

STABILITY OF LOAD BALANCING ALGORITHMS 7 for all ses S V. The inducion sep from o + 1 follows from Lemma 2.2, and we can apply he resuling guaranee o he singleon ses {v}, yielding a bound of B = H + γ n. Proof of Lemma 2.2. The proof is by conradicion. Assume ha he invarian (2.2) holds a he beginning of round, bu no a he beginning of round + 1. Le S be a se maximizing n Φ(S) := b +1 (S) (H + γ j). j=n S +1 If several ses achieve he maximum value, le S have minimal size among all hese ses. Firs off, noice ha he choice of S guaranees ha eiher all u S have posiive b +1 (u), or hey all have negaive b +1 (u), and hence b +1 (S) = u S b +1(u). Since (2.2) was assumed o hold a he beginning of round, and fails a he beginning of round + 1, we know ha b +1 (S) > b (S). How can he values h (u) for nodes u S change? Adversary sep: Subsiuing he definiions of b and b, we obain ha for any se S, b (S) = h (S) S a +1 = h (S) + δ (S) S a +1 = b (S) + δ (S) S (a +1 a ) b (S) + δ (S) S (a +1 a ) b (S) + e(s). The wo inequaliies hold because of he Triangle Inequaliy and he cu condiion on he adversary. Redisribuion sep: Fix an edge e = (u, v) wih u S and v / S. Because S maximizes Φ and has minimal size, u has imbalance b +1 (u) > H + γ (n S + 1), since oherwise Φ(S u) Φ(S). In paricular, b +1 (u) > γ. Because v was no included in S, is imbalance b +1 (v) mus eiher have sign opposie o he sign of b +1 (S), or have absolue value b +1 (v) H + γ (n S ). In eiher case, b +1 (u) b +1 (v) > γ, and simply subsiuing he definiion of γ, we also obain h +1 (u) h +1 (v) > 2 + θ. During he Redisribuion sep of round, a mos okens can have moved o or from nodes u and v, so heir heighs can have changed by a mos each, and herefore heir previous heigh difference is a leas h (u) h (v) > θ. Figure 2.2 illusraes he gap of a leas γ beween he queue heighs of nodes in S and ouside S. If b +1 (u) 0, hen h (u) h (v) > θ, so he algorihm SCLB θ moves a oken from u o v along e, and no okens from v o u, hereby decreasing b (u) by 1. On he oher hand, if b +1 (u) < 0, hen h (v) h (u) > θ, and a oken mus be moved from v o u along e, increasing he (negaive) imbalance b (u) by 1. Because b +1 (u) > γ = θ + 2, and herefore b (u) > θ +, he sign of he imbalance does no change during he Redisribuion sep, even if okens were moved o or from u, and hence, b (u) decreased by 1 as a resul of edge e. This holds for every edge e e(s), and using he fac ha he average a does no change during he Redisribuion sep, we obain ha b +1 (S) = u S b +1(u) ( u S b (u) ) e(s) = b (S) e(s).

8 ELLIOT ANSHELEVICH, DAVID KEMPE, AND JON KLEINBERG b (u) +1 γ H+γ(n- S +1) H+γ(n- S )..... Fig. 2.2. The gap of γ beween he queue heighs of nodes in S and ouside S in he case ha b +1 (S) is posiive. S Puing he argumens for he wo seps ogeher, we obain ha b +1 (S) b (S) e(s) b (S). This conradics our assumpion ha b +1 (S) > b (S), and hus complees he proof. Noice ha our bound B = H +γ n is asympoically igh. To see his, consider a simple pah of lengh n. I is cerainly legal for he adversary o inser one oken a node n in every round, and never remove okens. Afer abou θ n2 2 rounds, each node k will conain abou kθ okens, and hence he imbalance of node n is abou θ n 2.3. Capaciies, dynamic neworks, and ime windows. The resul of Theorem 2.1 can be easily exended o he case ha he edges have (ineger) capaciies c e associaed wih hem, and up o c e okens can be sen along e in every round. We assume ha whenever he algorihm decides o send okens from u o v along e, i sends as many as possible, i.e., bounded only by he capaciy c e and he number of okens a u. The cu condiion now requires ha he imbalance creaed by he adversary be resriced by he oal capaciy of he cu. Showing ha he above algorihm is sill sable in his capaciaed version is easy given Theorem 2.1. Corollary 2.3. In he above capaciaed scenario, for any adversary respecing he cu condiion, and any θ 1, he algorihm SCLB θ is sable, i.e., here is a consan B (depending on H, θ, G, and he maximum edge capaciy max e c e ), such ha b (v) B for all nodes v a all imes. Proof. Replace each edge wih capaciy c e by c e parallel edges of capaciy 1. By Theorem 2.1, he algorihm is sable on his new graph, and herefore, i is also sable on he capaciaed graph. Noice ha he consan γ and hence he bound B now depend on he maximum capaciy, and so insead of B = H + 2 n + θn, he new bound becomes B = H + 2 n max e c e + θn, where is he maximum degree of he graph. 2.

STABILITY OF LOAD BALANCING ALGORITHMS 9 Dynamic neworks. Anoher easy exension concerns dynamically changing neworks. Tha is, he se of available edges may change over ime, and we assume ha i is also conrolled by he adversary. For each ime, we have a se E of available edges. The cu condiion on he adversary mus be saisfied a he specific ime when he imbalance is creaed, i.e., δ (S) S (a +1 a ) e (S). Here, e (S) are he edges from E ha have exacly one endpoin in S. Corollary 2.4. In he above dynamically changing nework, for any adversary respecing he cu condiion, and any θ 1, he algorihm SCLB θ is sable, i.e., here is a consan B (depending on H, θ, and G), such ha b (v) B for all nodes v a all imes. Proof. By synacically replacing all erms e(s) wih e (S) in he proof of Lemma 2.2, we obain a proof for he model of dynamically changing neworks, wih exacly he same bound B. Time windows. An exension ofen considered in he conexs of load balancing or packe rouing is o relax he resricion on he adversary by allowing i o violae he cu condiion for a cerain ime, provided ha i hold in he long run. Specifically, a window size W is specified, and i is required ha for any se S and any ime window [, + W), he imbalance creaed on se S over ha ime window be bounded by he oal capaciy, i.e., ( +W 1 r= δ r (S)) S (a +W a ) W e(s). Corollary 2.5. For any adversary respecing he cu condiion on average over a window size W, and any θ 1, he algorihm SCLB θ is sable, i.e., here is a consan B (depending on H, θ, W, and G), such ha b (v) B for all nodes v a all imes. Proof. I is no difficul o see ha by allowing B o depend on W, we can also exend he sabiliy resul o his model once he imbalance grows oo large on a se S, all edges e e(s) will be moving okens so as o reduce he imbalance for every single round of an enire window, so ha he imbalance canno grow furher. 3. Muli-commodiy load balancing. In he previous secion, we considered he problem of balancing loads on processors where he loads were inerchangeable. However, we are also ineresed in he case of differen kinds of loads ha are o be balanced simulaneously. For insance, hink of jobs ha have an emphasis on differen resources of he machine hey are running on. Balancing he differen classes of jobs independenly could be desirable in order o avoid processing ime becoming a boleneck on one machine, and memory size an issue on anoher. In he general muli-commodiy load balancing problem, we have k differen kinds of jobs (or okens), which are sored in separae queues a he nodes. Our goal is o ensure an absolue bound on he deviaion of any queue heigh from he average queue heigh for ha commodiy. Each round is divided ino he same wo seps as before, he Adversary sep and he Redisribuion sep. In analogy o he single-commodiy case, we use he following noaion: For a node v and commodiy i, le h (i) (v) be he number of okens of commodiy i on node v a he beginning of round. Similarly, a (i), b (i) (v), δ (i) (v), h (i) (v), and b (i) (v) are all defined for commodiy i exacly as heir single-commodiy equivalens. The algorihm will now no only have o choose when o send a oken across an edge, bu also which of several available (and conflicing) kinds of okens o send. Our class of algorihms is pracically idenical o he one from [2] and [6], and can be formalized as follows: Algorihm MCLB θ

10 ELLIOT ANSHELEVICH, DAVID KEMPE, AND JON KLEINBERG A each ime, for each edge e = (u, v): Choose i o maximize h (i) (u) h (i) (v). If h (i) (u) h (i) (v) + θ, hen send a oken of commodiy i from u o v. If h (i) (v) h (i) (u) + θ, hen send a oken of commodiy i from v o u. In he case of a single commodiy, his algorihm specializes o SCLB θ. The naural analogue of he cu condiion for a single commodiy is o require ha he adversary saisfy δ (i) (S) S (a (i) +1 a(i) ) e(s) (3.1) i for all node ses S V and imes. This would require ha he oal imbalance for se S creaed by he adversary could be balanced along edges leaving S. Unforunaely, Inequaliy (3.1) is oo weak a resricion i allows he adversary o creae paerns of addiion and removal ha canno be balanced by any algorihm, wheher offline or online. A he end of his secion, we show how o use a reducion from he muli-commodiy flow problem o creae such an adversary wih k 3 commodiies. For he special case k = 2, however, he Max-Flow Min-Cu Theorem sill holds, and in fac, we can show ha he cu condiion is sufficien o ensure ha algorihm MCLB θ is sable. 3.1. Sabiliy for k=2. We le H = max v V {h (1) 1 (v) + h(2) 1 (v)} be he maximum heigh of he queues a any node during he sar of he execuion, and he maximum degree of any verex. This ime, we define γ slighly differenly, namely γ = 2 + 2θ. Theorem 3.1. There is a consan B (depending on H, θ and G), such ha for any adversary respecing he cu condiion, MCLB θ ensures b (i) (v) B a all imes, for all verices v, and commodiies i = 1, 2. Proof. A he sar of he execuion, b (1) 1 (S) + b(2) 1 (S) v S H, by definiion of H. The key Lemma 3.2 esablishes ha for all S V, imes, and commodiies i = 1, 2, b (1) (S) + b (2) (S) n j=n S +1 (H + γ j) (3.2) We can hen apply he resul o all singleon ses {v}, proving he heorem. Lemma 3.2. If (3.2) holds a he beginning of round, i holds a he beginning of round + 1. Proof. The proof is by conradicion. Le S be a se (of minimum size in case of ies) maximizing Φ(S) := b (1) (S) + b (2) (S) n j=n S +1 (H + γ j). In he case of wo commodiies, a node migh be included in S because i conribues a lo o he imbalance in one of he commodiies, alhough is conribuion o he oher commodiy migh acually be negaive. To capure he imbalance conribuion of a node o each commodiy, we define he signed imbalance β (i) (v) := sgn(b(i) +1 (S))

STABILITY OF LOAD BALANCING ALGORITHMS 11 b (i) (v), β(i) (v) := sgn(b(i) +1 (S)) b(i) (v), for every node v V, and every ime sep. Here, sgn denoes he sign of a erm. Noice ha we use he sign of b (i) +1 (S) a all ime seps in he definiion of β (i) (v) (no he sign of b(i) (S) or of b(i) (S)). We can now rewrie he oal imbalance over he se S a ime + 1 as +1 b (1) +1 (S) + b(2) +1 (S) = (β (1) +1 (u) + β(2) +1 (u)). (3.3) u S Again, we show ha he change in he imbalance for se S canno be posiive, by comparing he increase in he imbalance of S during he Adversary sep wih he decrease during he Redisribuion sep, and hus obain a conradicion. Adversary sep: We know ha for each commodiy i = 1, 2, he imbalance on se S afer he Adversary sep is a mos b (i) (S) b (i) (S) + δ (i) (S) S (a (i) +1 a(i) ) by he Triangle Inequaliy. Now, summing over i = 1, 2 and applying he cu condiion on he adversary yields ha b (1) (S) + b (2) (S) b (1) (S) + b (2) (S) + e(s). Redisribuion sep: Fix a node u S, and an edge e = (u, v) of G wih v / S. As in he proof of Lemma 2.2, we can use he definiion of S (as maximizing Φ and being of minimal size) o obain ha he signed imbalances a nodes u and v saisfy (u) + β(2) +1 (u) > β(1) +1 (v) + β(2) +1 (v) + γ. As u and v can lose and gain a mos okens each during he Redisribuion sep, β (1) +1 β (1) (u) + β (2) (u) > β (1) (v) + β (2) (v) + 2θ. (3.4) In paricular, here mus be a commodiy i such ha β (i) (u) > β (i) (v) + θ, and hus also h (i) (u) h (i) (v) > θ. Hence, MCLB θ moved a oken along edge e during he Redisribuion sep (w.l.o.g., i was a oken of commodiy 1). We wan o show ha his oken acually decreased he signed imbalance of node u. Assume ha i did no. This means ha if he oken moved from u o v, hen sgn(b (1) +1 (S)) is negaive, and if he oken moved from v o u, hen sgn(b (1) +1 (S)) is posiive. In eiher case, he signed imbalance for commodiy 1 a node v mus be higher han a node u, and so β (1) (v) β (1) (u) = b (1) (v) b (1) (u) = h (1) (v) h (1) (u). Because MCLB θ maximizes he difference in is choice of commodiy, we obain ha β (1) (v) β (1) (u) = h (1) (v) h (1) (u) h (2) (v) h (2) (u) = β (2) (v) β (2) (u) β (2) (u) β (2) (v). Rearranging his inequaliy yields ha β (1) (v) + β (2) (v) β (1) (u) + β (2) (u), and hus a conradicion wih Inequaliy (3.4). Therefore, every edge (u, v) wih v / S decreases he signed imbalance of u by 1. Summing over all edges and all nodes u S,

12 ELLIOT ANSHELEVICH, DAVID KEMPE, AND JON KLEINBERG gives us β (1) +1 (S)+β(2) +1 (S) β(1) (S)+β (2) (S) e(s). Using Equaion (3.3) and he fac ha β (1) (S) + β (2) (S) b (1) (S) + b (2) (S), we obain b (1) +1 (S) + b(2) +1 (S) b (1) (S) + b (2) (S) e(s). Combining he argumens for he wo seps, b (1) (S) + b (2) (S) increases by a mos e(s) during he Adversary sep, and decreases by a leas e(s) during he Redisribuion sep. Therefore, in oal b (1) +1 (S) + b(2) +1 (S) b(1) (S) + b (2) (S), a conradicion. The resul for wo commodiies can be exended o neworks wih edge capaciies, adversarially changing edge ses, and adversaries wih resricions only for larger window sizes, jus like for he single commodiy case. 3.2. Load balancing and flows. By omiing he adversarial and dynamic naure in he load balancing problem, and forcing he adversary o repea he same paern of oken addiions in every round, we can infer from he sabiliy of he load balancing algorihm he exisence of a muli-commodiy flow. Suppose ha we are given a muli-commodiy flow insance wih edge capaciies c e, source-sink pairs (s i, i ), and demands d i. Le D = max i d i, and le A be he adversary insering, in every round and for each commodiy i, D + d i okens a node s i, D d i okens a node i, and D okens everywhere else. Lemma 3.3. If here is a load-balancing algorihm ha is sable agains he adversary A, hen here is a (fracional) muli-commodiy flow f wih source-sink pairs (s i, i ) and demands d i. Proof. Because we assumed he algorihm o be sable, all imbalances b (i) (v) are always bounded in absolue value by some consan B. Therefore, here are a mos (2B + 1) kn differen combinaions of imbalances for he enire nework, and so here mus exis wo imeseps and such ha < and b (i) (v) = b (i) (v) for all nodes v and commodiies i. For each edge e = (u, v), le σ (i) (u, v) denoe he number of okens of commodiy i sen from u o v in round, and define a flow f by f (i) (u,v) := 1 1 (σ r (i) (u, v) σ r (i) (v, u)). r= Noice ha we define negaive flows, bu only for symmery and ease of noaion. We wan o verify ha f is indeed a feasible muli-commodiy flow for demands (s i, i, d i ). Capaciy consrains: The oal flow along any edge (u, v) is i f (i) (u,v) 1 = 1 i 1 r= 1 r= 1 1 r= = c (u,v). i c (u,v) σ (i) r (u, v) σ (i) r (v, u) σ (i) r (u, v) σ (i) r (v, u)

STABILITY OF LOAD BALANCING ALGORITHMS 13 The firs inequaliy is simply he Triangle inequaliy, and he second inequaliy holds because he balancing algorihm never exceeds he capaciy of any edge wih any of is oken moves, and herefore boh σ r (i) (u, v) and σ r (i) (v, u) lie beween 0 and c (u,v). Flow Conservaion: For any node v and commodiy i, we can wrie ( ) = = (u,v) E (u,v) E r= 1 r= (u,v) E f (i) (u,v) 1 (σ r (i) (u, v) σ r (i) (v, u)) 1 σ r (i) (u, v) r= (u,v) E 1 = h (i) (v) h(i) (v) δ r (i) (v) r= σ (i) r (v, u) = b (i) (v) + a(i) b(i) (v) a (i) ( ) δ (i) (v) = ( )(D δ (i) (v)). The hird equaliy is rue because h (i) (v) h(i) (v) is exacly he number of okens ha enered v during he ime period [, 1], minus he amoun of okens ha lef v during his period. In he las equaliy, we used ha b (i) (v) = b(i) (v) for all i and v, and ha a (i) r = r D for all imes r. Now, if node v is neiher he source nor he sink for commodiy i, hen δ (i) (v) = D, so flow is conserved. If v is he source of commodiy i, hen δ (i) (v) = D + d i, so he oal flow enering node s i is d i. If v is he sink for commodiy i, hen he oal flow enering node i is d i, because δ (i) (v) = D d i by definiion. Hence, f saisfies flow conservaion and all demands. f conserves flow, saisfies all demands, and does no exceed any edge capaciies, so i is a feasible muli-commodiy flow for he given demands. By combining Lemma 3.3 wih he sabiliy of MCLB θ proved in Theorem 3.1, we obain as a corollary an alernae proof of he wo-commodiy Max-Flow Min-Cu Theorem. I only remains o verify ha he adversary A as defined in Lemma 3.3 indeed respecs he cu-condiion. Corollary 3.4 (2-Commodiy Max-Flow Min-Cu). Le G = (V, E) be a graph wih edge capaciies c e, and wo demand pairs (s 1, 1 ), (s 2, 2 ) wih demands d 1, d 2 such ha for any verex se S V, he oal demand of commodiies i {1, 2} wih exacly one of {s i, i } in S is a mos e e(s) c e. Then, here exiss a feasible wo-commodiy flow sending d i unis of flow from s i o i for i = 1, 2. Proof. Define he adversary A as in Lemma 3.3. To show ha he algorihm MCLB θ is sable agains A, we merely have o verify ha A saisfies he cu condiion. Le S V be arbirary. For convenience, we wrie [u S] := 1 if u S, and 0

14 ELLIOT ANSHELEVICH, DAVID KEMPE, AND JON KLEINBERG oherwise. Then, for any ime δ (i) (S) S (a (i) +1 a(i) ) i=1,2 = S D + [s i S] d i [ i S] d i S D i=1,2 = d i [s i S] [ i S]. i=1,2 In he firs equaliy, we used he definiion of he inserion paern for A. The conribuion of commodiy i o his sum is d i if and only if exacly one of s i, i lies in S oherwise, i is 0. Hence, he value of he sum is he oal demand of commodiies i wih exacly one of {s i, i } in S, which by assumpion is bounded by e e(s) c e. Hence, A saisfies he cu condiion. We can herefore apply Theorem 3.1 o obain ha MCLB θ is sable agains A, which in urn implies he exisence of a feasible muli-commodiy flow f for he given insance via Lemma 3.3. The 2-Commodiy Max-Flow Min-Cu Theorem was firs proved by Hu [17], essenially repeaing Ford and Fulkerson s original [14] augmening pahs argumen for wo commodiies. Seymour [23] showed a shor and simple explici reducion o he single-commodiy case. Subsequenly, Linial, London and Rabinovich [19] gave a novel proof using geomeric embeddings and linear programming dualiy. Our proof uses ye differen (and much more elemenary) echniques, and does no rely on he single-commodiy Max-Flow Min-Cu Theorem. Anoher corollary we obain from Lemma 3.3 is he exisence of an adversary respecing he cu condiion for k = 3 commodiies, such ha no algorihm (offline or online) can balance he inserion paern. This is no surprising, since load balancing algorihms (and our algorihm in paricular) aemp o generae a flow from over-loaded nodes o under-loaded nodes, and we know ha he bes flows can be significanly worse han he bes cus for more han 2 commodiies (i.e., he Max- Flow Min-Cu Theorem does no hold for k 3). This means ha he consrain on he adversary is no powerful enough o guaranee he exisence of a good flow and herefore a good algorihm. We make he above discussion precise in he following corollary. Corollary 3.5. There exiss an adversary respecing he cu condiion for k = 3 commodiies, such ha no algorihm (offline or online) is sable agains his adversary. Proof. To prove his, we simply ake a 3-commodiy insance wih a graph G = (V, E) and demand pairs (s i, i ), i {1, 2, 3} (wih demands d i ) such ha for all cus (S, V \S), he oal demand across he cu is a mos he capaciy of he edges crossing he cu, ye here is no (fracional) muli-commodiy flow saisfying all demands. The firs such example for k = 3 was given in [17]. (A simpler well-known example for k = 4 is he complee biparie graph K 2,3 wih a uni demand beween every pair of nodes ha are a a disance of 2 from each oher.) Le A be he adversary defined from his insance as in Lemma 3.3. If any load balancing algorihm were sable agains A, Lemma 3.3 would guaranee a feasible muli-commodiy flow, a conradicion. The above corollary and Lemma 3.3 illusrae he relaion of muli-commodiy flows o load-balancing algorihms, and ell us ha he cu condiion is no enough o guaranee he exisence of a sable algorihm for k 3. In Secion 5 we will discuss an alernaive approach for a resricion on a muli-commodiy adversary.

STABILITY OF LOAD BALANCING ALGORITHMS 15 4. Packe Rouing. There is a naural connecion beween he load balancing problem sudied in he previous secions, and he problem of rouing packes in an adversarial nework. I has been observed previously [2, 6] ha he naural balancing algorihm SCLB θ is also sable for packe rouing. The model for packe rouing differs from he load balancing one in ha afer he Redisribuion sep, here is an addiional Removal sep, during which all packes ha have reached heir desinaion are removed from he nework. Sabiliy of an algorihm is now defined as meaning ha here is an absolue bound on all queue heighs a all imes, i.e., h (i) (v) B for some consan B. In he single-commodiy packe rouing problem, we can again resric he adversary by a cu condiion: he oal number δ (S) of packes insered ino a se S mus be a mos e(s) for any se S no conaining he sink of he packes. If S does conain he sink, hen here is no resricion. In he muli-commodiy case, he adversary specifies a source s i and a sink i for each packe insered, and mus guaranee ha here is a se of edge-disjoin pahs connecing all s i - i pairs. In [6], i was shown ha for he single-commodiy case, he algorihm SCLB θ is sable agains an adversary guaraneeing he exisence of a pah for all packes insered, even when edges dynamically appear and disappear. Aiello e. al. [2] proved ha an algorihm essenially equivalen o MCLB θ is sable for he muli-commodiy packe rouing problem if he pahs specified by he adversary are no only disjoin, bu leave an ε fracion of capaciy for every edge unused over a given window lengh W. Recenly, [?] exended his resul o dynamically changing neworks. Our echniques from Secion 2 can be used o obain an alernae (and simpler) proof for he sabiliy of algorihm SCLB θ in he packe rouing model. Our proof also works for he case of adversarially appearing and disappearing edges, alhough he resricion on he adversary is differen from (and essenially less general han) he one in [6]. We define and H as before, and le γ = 2 + θ. Then, he sabiliy of SCLB θ agains an adversary respecing he cu condiion is guaraneed by he following heorem. Theorem 4.1. For any ime and se S V, h (S) n j=n S +1 (H + γ j). (4.1) Proof. By definiion of H, Invarian (4.1) cerainly holds a ime 1. Assume ha he heorem is wrong, and le be he earlies ime such ha here is a se S violaing (4.1) a ime + 1. Among all such ses, le S be he one maximizing h +1 (S) n j=n S +1 (H +γ j), and break ies for minimal size. Then, we can show as in he proof of Lemma 2.2 ha for all nodes u S, v / S, h +1 (u) > h +1 (v) + γ, and h +1 (u) > H. In paricular, he se S canno conain he sink (because he sink conains no okens afer he Removal sep). Therefore, he adversary can have insered a mos e(s) okens ino S in he Adversary sep. During he Redisribuion sep of round, a oken leaves he se S along each edge e = (u, v) e(s), because even if u had los okens and v gained okens during he Redisribuion sep, h (u) would sill exceed h (v) by a leas θ. Taken ogeher, his shows ha he number of okens in S canno have increased, conradicing he choice of and S.

16 ELLIOT ANSHELEVICH, DAVID KEMPE, AND JON KLEINBERG Exensions. Like he proofs in he previous secions, he proof of Theorem 4.1 can be easily exended o deal wih dynamically changing neworks, edge capaciies, and ime windows in he cu condiion. In addiion, his proof exends o direced graphs G, if we redefine e(s) in he cu condiion o be he edges coming ou of S. We can also show sabiliy for a wider class of single-commodiy balancing algorihms. Specifically, le g : N N be a funcion wih g(x) x for all x. The balancing algorihm SCLB g always sends a oken from u o v (and never sends a oken from v o u) if here is an edge e = (u, v) E and h (u) > g(h (v)). I does no maer wha he algorihm does if neiher h (u) > g(h (v)) nor h (v) > g(h (u)). Exending he above proof only slighly, we obain ha SCLB g is sable for all such funcions g. Of course, he bound B now depends on he rae of growh of g. Our packe rouing resuls also imply he sabiliy of an ineresing load balancing scenario, which is very similar o he one in [10]. Suppose each node processes one job per round, so ha one oken is removed every round from each node wih posiive queue heigh. If he adversary A saisfies he consrain ha he number of okens i adds o S is a mos e(s) + S, hen we can use Theorem 4.1 o show ha he queue heighs are bounded. Consider adding a sink node v o our graph, and add an edge from every node o v. We can now hink of A as adding packes o he new graph, wih v being he packes desinaion. The number of edges coming ou of a se S in he new graph is exacly e (S) = e(s) + S, so A saisfies he condiion needed for Theorem 4.1. I is easy o see ha if he queue heighs are bounded in his new packe rouing scenario, hen hey are also bounded in he original load balancing one. The proofs for Theorem 4.1 and Theorem 2.1 (and he proofs in [2] and [21]) are so similar in naure ha one suspecs a formal reducion from he packe rouing problem o he load balancing problem (which seems more general). However, we have no ye been able o deermine such a reducion. I would cerainly be ineresing, since i would allow us o focus on he load balancing problem in he fuure. As wih he load balancing problem, we can obain a muli-commodiy flow if MCLB θ is sable agains a suiably defined adversary A. The proof is pracically idenical o he one for Lemma 3.3, and we herefore omi i. Lemma 4.2. Le A be an adversary insering d i okens of commodiy i (whose desinaion is i ) ino node s i in every round. If here is a rouing algorihm ha is sable agains his adversary, hen here is a (fracional) muli-commodiy flow f wih source-sink pairs (s i, i ) and demands d i. 5. Conclusions. In his paper, we have shown ha a simple local load-balancing algorihm is sable agains dynamic adversarial addiion and removal of jobs in a nework, so long as he adversary is bounded by a naural exension of he cu condiion in he sense defined in [21]. This seles an open quesion from [21]. Our proof echniques exend o he case of balancing wo commodiies a once, and o rouing packes injeced by an adversary. They yield easier proofs and essenially igh bounds for he general case. In addiion, he sabiliy of he load balancing algorihm for wo commodiies gives a new proof of he wo-commodiy Max-Flow Min-Cu Theorem. This work leaves open a number of ineresing quesions. Mos imporanly, we would like o be able o show sabiliy of he muli-commodiy load balancing algorihm for an arbirary number of commodiies, boh for he problem of rouing packes and balancing loads. If we wan o prove sabiliy of load-balancing algorihms for more commodiies, we will have o use a differen condiion on he adversary. As a consequence of Lemma 3.3, any reasonable resricion on he adversary will have o guaranee he exisence of muli-commodiy flows for all insances where we hope o

STABILITY OF LOAD BALANCING ALGORITHMS 17 prove sabiliy. We herefore sugges he following resricion: Firs, define demands d (i) (v) for commodiy i, node v and ime by d (i) (v) := δ (i) (v) (a (i) +1 a(i) ). Then, he adversary is resriced o moves ha guaranee he exisence of a (fracional) muli-commodiy flow in G saisfying all hese demands. The disadvanage of his condiion is ha i bears no direc relaion o he load balancing problem i arises from observing he insufficiency of he more naural cu condiion raher han from having an acual meaning for he problem of balancing loads. Neverheless, his condiion should be considered he righ resricion on he adversary o measure he qualiy of MCLB θ or oher load balancing algorihms. Alernaively, we migh invesigae wheher he cu condiion is sufficien for balancing muliple loads if we resric our aenion o specific neworks. For example, i is well known ha for rees or cycles, he cu condiion implies he exisence of muli-commodiy flows, and we migh hope ha i would hence be sufficien o prove sabiliy. Acknowledgmens. We hank Marin Pál and Chrisian Scheideler for valuable discussions on he subjecs of load balancing and adversarial packe rouing. REFERENCES [1] W. Aiello, B. Awerbuch, B. Maggs, and S. Rao, Approximae load balancing on dynamic and asynchronous neworks, in Proc. 25h ACM Symp. on Theory of Compuing, 1993. [2] W. Aiello, E. Kushileviz, R. Osrovsky, and A. Rosén, Adapive packe rouing for bursy adversarial raffic, in Proc. 30h ACM Symp. on Theory of Compuing, 1998. [3] A. Anagnosopoulos, A. Kirsch, and E. Upfal, Load balancing in arbirary nework opologies wih sochasic adversarial inpu, SIAM J. Compu., 34 (2005), pp. 616 639. [4] D. Andrews, B. Awerbuch, A. Fernández, J. Kleinberg, F. Leighon, and Z. Liu, Universal sabiliy resuls for greedy conenion resoluion proocols, in Journal of he ACM, vol. 48, 2001, pp. 39 69. [5] F. M. auf der Heide, B. Oeserdiekhoff, and R. Wanka, Srongly adapive oken disribuion, Algorihmica, 15 (1996). [6] B. Awerbuch, P. Berenbrink, A. Brinkmann, and C. Scheideler, Simple rouing sraegies for adversarial sysems, in Proc. 42nd IEEE Symp. on Foundaions of Compuer Science, 2001. [7] B. Awerbuch and F. Leighon, A simple local-conrol approximaion algorihm for mulicommodiy flow, in Proc. 34h IEEE Symp. on Foundaions of Compuer Science, 1993. [8], Improved approximaion algorihms for he muli-commodiy flow problem and local compeiive rouing in dynamic neworks, in Proc. 26h ACM Symp. on Theory of Compuing, 1994. [9] P. Berenbrink, T. Friedezky, and L. Goldberg, The naural work-sealing algorihm is sable, SIAM J. Compu., 32 (2003), pp. 1260 1279. [10] P. Berenbrink, T. Friedezky, and R. Marin, Dynamic diffusion load balancing, in Proc. 32nd Inl. Colloq. on Auomaa, Languages and Programming, 2005, pp. 1386 1398. [11] A. Borodin, J. Kleinberg, P. Raghavan, M. Sudan, and D. Williamson, Adversarial queueing heory, in Journal of he ACM, vol. 48, 2001, pp. 13 38. [12] M. Crovella, M. Harchol-Baler, and C. Mura, Task assignmen in a disribued sysem: Improving performance by unbalancing load, in Proc. ACM Sigmerics Conf. on Measuremen and Modeling of Compuer Sysems, 1998. [13] D. Eager, E. Lazowska, and J. Zahorjan, Adapive load sharing in homogeneous disribued sysems, IEEE Transacions on Sofware Engineering, 12 (1986). [14] L. Ford and D. Fulkerson, Maximal flow hrough a nework, Can. J. Mah, 8 (1956), pp. 399 404. [15] D. Gamarnik, Sabiliy of adapive and non-adapive packe rouing policies in adversarial queueing neworks, in Proc. 31s ACM Symp. on Theory of Compuing, 1999. [16] B. Ghosh, F. Leighon, B. Maggs, S. Muhukrishnan, C. Plaxon, R. Rajaraman,

18 ELLIOT ANSHELEVICH, DAVID KEMPE, AND JON KLEINBERG A. Richa, R. Tarjan, and D. Zuckerman, Tigh analyses of wo local load balancing algorihms, in Proc. 27h ACM Symp. on Theory of Compuing, 1995. [17] T. Hu, Muli-commodiy nework flows, Operaions Research, 11 (1963). [18] F. Leighon and S. Rao, Mulicommodiy max-flow min-cu heorems and heir use in designing approximaion algorihms, Journal of he ACM, 46 (1999), pp. 787 832. [19] N. Linial, E. London, and Y. Rabinovich, The geomery of graphs and some of is algorihmic applicaions, Combinaorica, 15 (1995). [20] M. Mizenmacher, On he analysis of randomized load balancing schemes, in Proc. 9h ACM Symp. on Parallel Algorihms and Archiecures, 1997. [21] S. Muhukrishnan and R. Rajaraman, An adversarial model for disribued dynamic load balancing, in Proc. 10h ACM Symp. on Parallel Algorihms and Archiecures, 1998. [22] D. Peleg and E. Upfal, The oken disribuion problem, SIAM J. on Compuing, 18 (1989). [23] P. Seymour, A shor proof of he wo-commodiy flow heorem, J. of Combinaorial Theory, 26 (1979). [24] B. Shirazi, A. Hurson, and K. Kavi, Scheduling and Load Balancing in Parallel and Disribued Sysems, IEEE Compuer Sociey Press, 1995.