To Coordinate Or Not To Coordinate? Wide-Area Traffic Management for Data Centers

Transcription

1 To Coordinate Or Not To Coordinate? Wide-Area Traffi Management for Data Centers Srinivas Narayana, Joe Wenjie Jiang, Jennifer Rexford, Mung Chiang Department of Computer Siene, and Department of Eletrial Engineering Prineton University {narayana, wenjiej, jrex, ABSTRACT In this paper, we study the impat of oordinating the seletion of data enters for lients ( mapping ) and performing multi-homed network-routing ( routing ) from data enters, two deisions whih are onventionally managed independently. We model their funtional separation and degrees of oordination through an optimization framework, and study the impat of oordination on (i) servie performane, (ii) robustness to traffi variability, and (iii) bandwidth osts. We show that in theory, performing mapping and routing independently an lead to muh lower performane or higher osts than a oordinated deision. In pratie, our traebased evaluations on an operational CDN show that oarsegrained information-sharing between mapping and routing is suffiient for near-optimal request latenies, but not minimal osts. Further, even omplete information-sharing between mapping and routing is not suffiiently robust to traffi variability, as ISP-links an easily be overwhelmed due to traffi burstiness. To address this issue, we design a oordination tehnique whih is muh more robust to traffi variability, and is also provably optimal. 1. INTRODUCTION Organizations running online servies worry about the end-to-end performane their ustomers experiene, sine even small inreases in pereived lateny an have signifiant impat on revenue [1]. To improve performane and reliability, loud servie providers (CSPs) typially run multiple geographially-distributed data enters (DCs), eah peering with multiple ISPs. Plaing servers loser to users redues propagation delay, and path diversity resulting from multi-homing offers performane benefits [2]. However, servie providers also need to onsider operational osts e.g., large servies may send and reeive petabytes of data in a day, leading to signifiant bandwidth osts [3], and an spend tens of millions of dollars annually on eletriity [4]. Client demands and path performane, and even eletriity and bandwidth osts, vary over time. As suh, CSPs should adapt how they diret lient requests to data enters (lient mapping) and response traffi to wide-area paths (network routing). A CSP with its own bakbone network ould diret lients to the nearest ingress point (e.g., a front-end proxy [5]), and then optimize wide-area routing aross all upstream ISPs [3]. However, many CSPs onnet their data enters diretly to the Internet, and rely on mehanisms like DNS or HTTP rediretion to map lient requests to a partiular data enter. Then, the data enter s edge router selets a loal upstream ISP to arry response traffi bak to the lient. Together, these mapping and routing deisions determine whether the CSP offers good performane and balaned loads, at a reasonable ost. Today, request-mapping and response-routing deisions are made independently, whih an in priniple reate situations leading to bad performane and high osts ( 2). For example, the mapping system ould easily diret too many requests to a data enter with limited upstream bandwidth, poor performane [6] or expensive ISP onnetivity, leaving the routing system no ability to retify the problem. This inites the question of whether improved mutual visibility between the mapping and routing systems an indeed boost performane and redue osts in pratie. Hene, we ask to what extent CSPs an benefit from oordination of urrently existing mapping and routing (before building new entralized systems), and to understand whih information is most valuable to share in order to arrive at good olletive deisions. In this paper, we fous on understanding the impliations of oordinating mapping and routing deisions for servie performane, robustness to traffi variability, and ost, when the mapping system has varying levels of visibility into routing. We employ optimization models for mapping-routing shemes with different levels of visibility, and realisti CDN traffi traes for evaluations. Motivated by robustness onsiderations from our evaluations, we propose a mapping-routing oordination sheme, and leverage results from nononvex optimization [7] to show that this sheme provably onverges to an optimum solution. We make the following ontributions: A ase for oordinated mapping and routing. In 2 we present toy examples that illustrate how in- 1

2 dependent ontrol of mapping and routing an lead to lower performane and higher osts. We also show senarios where separately optimizing mapping and routing deisions in an iterative fashion an be detrimental. Optimization formulations to model oordination. In 3 we formulate a set of optimization models that apture different levels of mapping s visibility into routing broadly, per-data enter and per-path information that progress from least to most oordinated mapping shemes. Our formulation also aptures other pratial onsiderations like path performane, link apaities, lient demands, and bandwidth osts. Coarse-grained information-sharing is suffiient for lateny optimization. In 4 we show that sharing per-data enter information namely per-lient smallest lateny and aggregate DC bandwidth with the mapping system is suffiient for near-optimal propagation delays. The reason is that geographi proximity dominates propagation delays i.e., lateny differenes between paths from different DCs greatly outweigh the differenes between paths from the same DC. Even fully oordinated mapping-routing may not be robust to traffi variability. In 5 we show that even with omplete routing-system visibility, several links an have high utilizations due to traffi burstiness between suessive optimization time-intervals, as well as from inherent variability [8] within an interval. Robust, provably optimal mapping-routing with utilization penalties. Our insight ( 6) is that expliitly penalizing high link utilizations helps leave suffiient bandwidth headroom to absorb traffi bursts. This approah avoids a triky tradeoff between performane loss and high utilization if a fixed burst apaity reservation is used, while also ahieving provable optimality. Sharing per-link osts is important for good osts overall. In 7 we show that visibility into perlink bandwidth osts is important to minimize osts. The rest of this paper is organized as follows. 2 reasons why oordinating mapping and routing ould be useful, with toy examples. 3 formulates oordination and its various forms into optimization models. 4 evaluates the impat of oordination on propagation delay, and 5 on robustness to traffi variability. 6 presents a robust, provably optimal mapping-routing sheme. 7 evaluates the impat of visibility on overall osts. 8 disusses why our results are generalizable. 9 disusses the related work and 10 onludes. 2. COORDINATING MAPPING & ROUTING Performing lient-mapping and network-routing independently an miss opportunities to improve performane or redue osts. Through toy examples onsisting of two data enters and one set of users e.g., a single IP prefix, we highlight some hallenges that an hinder independent mapping and routing optimizations from arriving at good olletive deisions. Misaligned objetives an lead to suboptimal deisions. Typially, mapping and routing systems work with different objetives e.g., mapping performs lateny and load-based DC seletion, while routing onsiders end-to-end performane and bandwidth osts to hoose a peer to forward responses. Due to mismathed objetives, overall performane and ost an both suffer. In Fig. 1(a), the routing system at eah DC piks the peer with the least ost per unit bandwidth, while the mapping system piks the DC with the least propagation delay given urrent routing hoies resulting in a situation where the servie neither has globally optimal ost nor optimal lateny. Inomplete visibility an lead to suboptimal deisions. In Fig. 1(b), the mapping and routing systems both optimize lateny, and users are sending traffi at the rate of 5Gb/s. Without information on link loads and apaities, the mapping system direts too muh traffi to the DC on the left, leading to large queueing delays and paket losses. If the mapping system uses information about link apaities, it ould easily diret most traffi to the alternate DC with ample spare apaity, and only slightly worse lateny. Coupled operational onstraints an lead to suboptimal equilibria. Even if mapping and routing have aligned objetives (e.g., minimize lateny) and omplete visibility, optimizing their deisions separately taking turns an still lead to globally suboptimal situations. In Fig. 2(a), mapping and routing are loally optimal given eah other s deisions, as traffi is served through the least lateny paths respeting link apaities. However, the globally optimal traffi alloation is one where all traffi is served by the DC on the right, using both peers. Bad routing deisions for prefixes with no traffi ontribution to a DC an lead to suboptimal equilibria. Consider the senario in Fig. 2(b), where a mapping initialization (e.g., from geo-proximity) direts all traffi to the DC on the left. No traffi from this user reahes the other DC, so all routing deisions here are equally good. Suppose this DC hooses to route traffi through the 100ms path. Then, this set of mapping and routing deisions are loally optimal to eah other preventing the routing system from exploring the globally optimal 50ms path. 3. MODEL In this setion, we introdue our notation and model for (i) mapping and routing deisions, (ii) the servie provider s performane and ost goals, and (iii) optimizing performane using different granularities of informationsharing between mapping and routing deisions. Our notation is summarized in Table 1. 2

3 $5/GB 40ms $1/GB 100ms (a) $10/GB 90ms $12/GB 250ms Route-seleted Alternate route Map-seleted 100ms 1Gb/s 45ms 5Gb/s (b) 10Gb/s 50ms 75ms Route-seleted Alternate route Map-seleted Figure 1: (a) Mapping deision is sub-optimal beause of misaligned objetives (b) Mapping deision is sub-optimal beause routing does not share link-related information. 4Gb/s 90ms 4Gb/s 75ms 4Gb/s 75% 25% (a) 1Gb/s 50ms 3Gb/s 60ms Route-seleted Alternate route Map-seleted 100ms 2Gb/s 60ms 1Gb/s (b) 2Gb/s 100ms 2Gb/s 50ms Route-seleted Alternate route Map-seleted Figure 2: (a) Separate mapping and routing an be ineffiient when oupled operational onstraints exist; (b) Routing deision is suboptimal when some lients send no traffi to a DC. 3.1 Cloud Servie Provider Model Network. A loud servie runs in a set I of data enters. Every data enter i is onneted to the Internet through a set of ISP links, whih we denote by J i. Let C be the set of lients, where eah lient is an aggregate of real users (in our experiments, these are IP prefixes). Clients an be direted to different data enters depending on their loations and the urrent load on the data enters. Suh mapping of users to data enters ours through a set of mapping nodes, suh as authoritative DNS servers or HTTP proxies. One a request is servied at data enter i, the egress router there piks one or more ISP-links j J i for responses. The total traffi that a link j an support is limited by its apaity ap j. Bursting traffi beyond this limit leads to large queueing delays or paket losses either undesirable for delay-sensitive appliations. Periodi deision-reomputation. Mapping and routing deisions are reomputed periodially, e.g., daily, and then applied to the requests that arrives until the next reoptimization. Hene, any parameters to the optimization i.e., traffi and performane metris, are averages over the period of optimization. Repliation. We assume that ontent is fully repliated at all data enters. We believe this is reasonable for read-mostly servies whih tend to tolerate some stale information in pratie. Searh, shopping, and soial networking appliations fall into this ategory. Request-serviing at ingress DCs. Some large servie providers may boune requests between data enters through expansive bakbone links [3]. However, we Symbol I J i C vol α i β j prie j perf j ap j Φ j (r j ) Definition Set of data enters (DC), indexed by i I Set of all outgoing links from DC i, indexed by j J i Set of lient aggregates (prefix), indexed by C Total request rate from lient Fration of lient request mapped to DC i Fration of outgoing lient traffi routed on link j J i Cost ($/request) of routing traffi on link j J i Propagation delay between DC i and lient via link j Bandwidth of link j J i Performane penalty (ongestion ost) as a funtion of total link load r j and apaity ap j Table 1: Summary of key notation. assume that user requests are servied at the first data enter whih they reah. Further, responses re-enter the Internet through a BGP peer of that data enter. Mapping Deisions (hoie of serviing-dc). We denote α i as the proportion of traffi from lient mapped to data enter i. We require that i α i = 1 for all, where α i [0, 1] for tratability of the optimization. This allows for the possibility that different users in the same lient-aggregate may be served by different data enters at a time. Mapping servies suh as DNS or HTTP proxies an ahieve suh flexibility in pratie [9, 10], and are aurate enough to be used in multiple ommerial deployments, e.g., [6]. While the {α i } denote a hoie of data enter, the hoie of a request s ingress ISP is not expliitly ontrolled for in our model we assume that the mapping system only keeps trak of one IP address for eah (data enter, lient) pair at a time, namely the shortest-lateny path to that lient. This allows us to ompare multiple mapping shemes with varying levels of routing-system oordination (Table 2 in 3.4). Sine ingress (request) traffi tends to be muh smaller than responses for typial online servies [3], we ignore any link-bandwidth onsiderations for request traffi. Routing Deisions (hoie of egress-isp). For every data enter i, we denote its response-routing deisions by a set of β j for all j J i, where β j is the fration of response-traffi for lient served over link j J i. Therefore j:j J i β j = 1. Today s BGP routing hooses a single ISP for eah IP prefix, hene routing deisions β j are integers {0, 1}. For tratability of optimizations ( 3.4) however, we relax this onstraint and allow servie providers to freely split traffi aross links, i.e., β j [0, 1]. In pratie, frational routing deisions over lients, e.g., IP prefixes, an be realized by hash-based traffi splitting. Sine eah data enter is a stub AS that does not provide transit servie, routing hanges do not ause BGP onvergene issues. 3

4 3.2 Performane Goals User-pereived request lateny is a key performanemetri of interest for interative servies even small inrements in this metri an have signifiant effets on bottomline revenue [11, 1]. User-pereived lateny depends on a wide variety of fators, inluding roundtrip lateny between users and data enters, requestproessing times within data enters, TCP dynamis, and how information is laid out on a user s browser. In this paper, we fous on optimizing the end-to-end path lateny between users and data enters, whih impats the ompletion time of short TCP flows [12], muh more than metris like bandwidth and paket loss. Fous on propagation delay. The end-to-end path lateny an be deomposed into three parts: path propagation delay, queueing delays along various bottleneked links, and transmission delays of pakets. Out of these three, we fous on the propagation delay, as it has been shown to largely determine Internet latenies for delaysensitive appliations [13] as opposed to queueing delays. Further, transmission delays for short TCP flows are negligible in the wide-area ontext. Average request-delay objetive. We employ average request propagation delay (ms/request) as our performane metri. Path propagation delays depend on whih data enters handle requests, and whih widearea paths deliver traffi. A servie provider obtains the required latenies through ative measurements [14] or Internet path performane predition tools [15]: we use perf j to denote the propagation delay from data enter i to lient, when i piks link j J i to deliver traffi. Then, the performane objetive funtion perf is: perf = vol α i β j perf j / vol, C i I j J i where vol is the total request volume from lient. 3.3 Cost Goals Operational osts are an important onsideration for loud servie providers [4, 16]. For tratability, we assume a ost funtion whih is linear on the link traffi workload. In this paper, we fous on ISP-bandwidth osts although some ISPs employ sophistiated priing funtions (e.g., 95th perentile harging), optimizing a linear ost on every harging interval an redue the monthly 95th perentile osts [3]. We denote prie j as the ost per request on link j J i of data enter i. The ost metri is average ost per request ($/request): ost = vol α i β j prie j / vol C i I j J i 3.4 Information-Sharing Granularity In this setion, we introdue our optimization models to study various degrees of information-sharing on the performane goals of a CSP. Inremental-visibility mapping shemes. For the next two setions ( 4, 5) we assume that the mapping and routing systems share the high-level goal of minimizing lient latenies. However, the mapping system may not have all the information neessary e.g., routing deisions to know exatly what latenies will be experiened by lient traffi sent to a partiular data enter. We introdue different degrees of routing-system visibility into mapping, as follows umulatively adding to prior information as we go along. The resulting mapping shemes are summarized in Table 2, denoting the progression of information-visibility. (For all three mapping shemes, we use a ommon routing with omplete knowledge of mapping deisions, optimizing perf.) A. Shortest lateny between eah data enter and lient. A mapping system onerned with minimizing request latenies needs to know at least the losest data enter to eah lient. Sheme A maps all traffi to the smallest-lateny data enter for eah lient. B. Aggregate link apaities for eah data enter. A data enter annot send traffi beyond the aggregate bandwidth of all its ISP-links. A mapping sheme an ombine this stati, per-data enter information with lient traffi demands to avoid overwhelming data enters. Note that sheme B does not use per-link information suh as latenies or routing deisions, and direts traffi assuming that the smallest lateny path from eah data enter is used. C. Per-path latenies, per-link apaities and routing deisions. Mapping sheme C has visibility into per-path information about the routing system namely, per-link apaities, per-path latenies, and routing deisions for eah prefix. Mapping deisions an potentially be muh better with exat lient-latenies, and apaity onsiderations of the atual links delivering traffi. This mapping sheme optimizes for the performane objetive perf ( 3.2) as it has all the neessary information. The mapping and routing shemes optimize on top of eah other until onvergene. Optimal mapping-routing baseline. To establish a baseline for omparison, we onstrut a entralized sheme that optimizes both mapping and routing deisions, leveraging omplete visibility between the two: 4

5 Sheme Share Share all path Share routing Objetive Constraints Variables link aps? latenies to DCs? deisions? A No No No Average (DC, lient) lateny None {α i } B Yes No No Average (DC, lient) lateny Aggregate DC apaity {α i } C Yes Yes Yes Average (path, lient) lateny Per-link apaity {α i } D Average (path, lient) lateny Per-link apaity {β j } Table 2: Summary of mapping shemes ({A, B, C}) with varying degrees of information-sharing, and the ommon routing sheme (D). All mapping shemes are aware of the losest lateny between (DC, lient) pairs, although A only needs to know the losest DC for eah lient. GLOBAL minimize perf (1a) subjet to vol α i β j ap j, j (1b),i:j J i α i = 1, (1) variables i β j = 1, i, j J i α i 0, i,, β j 0, j, (1d) where onstraints (1b) ensure that traffi on links do not exeed their apaity. The global problem is a linear program on α and β separately, but non-onvex when both are variables. As suh, it annot be solved using standard onvex optimization tehniques. However, Appendix A in our teh report [17] shows that it an be onverted into an equivalent LP and solved effiiently. 4. IMPACT ON PROPAGATION DELAY In this setion, we evaluate the set of mapping shemes from Table 2 with a fous to understand the benefits of greater visibility into the routing system. In this setion, we perform an offline evaluation, i.e., assume that all shemes have perfet knowledge of the traffi and performane for the next optimization interval. We take up the issue of traffi variability and its impat on performane in 5. We introdue our evaluation setup briefly, followed by results. 4.1 Experiment Setup We perform all evaluations using trae-based simulations for CoralCDN [18], a ahing and ontent distribution platform running on Planetlab. Our request-level trae olleted at 229 Planetlab sites running Coral onsists of over 27 million requests and a terabyte of data, orresponding to Marh 31st, These requests arrive from about IP prefixes. For latenies to these prefixes, we use iplane [15], a system that ollets widearea network statistis to destinations on the Internet from Planetlab vantage points. To orrelate the traffi and lateny data, we narrow down our lient set to IP prefixes, ontributing 38% of the traffi (by requests) of the entire trae. We omit further details in the interest of spae, and refer the reader to our teh report (Appendix D, [17]) for omplete details. For our experiments, we luster Planetlab sites in approximately the same metropolitan area and treat them as ISPs onneted to the same data enter, similar to the approah followed in [2]. Our setup onsists of 12 data enters distributed in North Ameria, Europe and Asia with 3-6 ISP onnetions eah. 4.2 Evaluation Results In this setion, we ompare the average request propagation delay (perf) of various shemes over hourly traffi averages. We show that oarse-grained informationsharing (i.e., sheme B) an provide near-optimal propagation delays beause the smallest lateny to a DC is representative of all the path latenies to a lient from that DC, when ompared to paths from other DCs. We argue that this is due to propagation delays being mostly determined by geographi distane, as opposed to routing hoies downstream. Capaity-awareness is ruial to avoid overwhelming peering ISP links. We find that at all hours of the day, sheme A exeeds the aggregate bandwidth apaity of some data enter or the other, so that lient traffi an never be alloated within apaity bounds of all its ISP links. The popular data enters every hour are sometimes overwhelmed by more than two times their aggregate apaity. Indeed, the need for apaity-aware traffi management for ISP links and request-mapping has already been highlighted in existing systems [3, 6, 19]. If links are overutilized, queueing delays and paket losses may start dominating performane, severely undermining the signifiane of propagation delay as a performane metri. For these reasons, heneforth we ignore sheme A from our analysis. Coarse information-visibility is suffiient for nearly optimal propagation delays. In Fig. 3, we show the hourly variation of the average propagation delay with the apaity-aware shemes, and ompare it with 5

6 perf (ms) sheme B 60 sheme C global Hour of the day (UTC) CCDF over requests DC rank DC-level link rank 1e Rank Figure 3: Average request propagation delay on hourly optimizations through the day. Figure 4: Dataenter and ISP link rank CDFs for sheme B. the optimal baseline. Throughout the day, we see that aggregate-apaity aware mapping deisions (i.e., sheme B) an already ahieve propagation delays very lose to optimal within 10ms differene in this experiment without knowledge of per-path latenies or routing deisions. As sheme C is initialized with sheme B, we see that it also is very lose to optimal, the average gap being less than 2ms. Next, we fous on understanding why these observations hold. Mapping and routing deisions are nontrivial even under oarse-grained visibility. Our first hypothesis is that the mapping and routing deisions are just implementing a trivial alloation, i.e., using the smallestlateny DC and ISP for most lients. However, Fig. 4 (for hour 0 data) shows that this is not the ase. We define the rank of a data enter with respet to a lient as its index when the data enters are sorted in inreasing order of their shortest latenies to that lient. For an ISP link, its rank is its index in the ordering of all ISP links belonging to the same data enter. The figure shows that > 30% of requests are alloated to data enters whih are not the losest (rank 2), while > 25% of requests are routed through a path whih is not the shortest from that data enter. This leads us to believe that the shortest lateny from a data enter to a lient is somehow representative of all path-latenies from the data enter to that lient. This is reasonable, sine propagation delays are related to geographi distane. Path-lateny variations between peers of the same data enter an be high. Our intuition is that if lateny variation between peers of a given data enter is small, then the shortest lateny between the data enter and a lient an at as a good proxy for all pathlatenies between that data enter and lient. Therefore, it an be a suffiient statisti for making good mapping deisions. To quantify this variation, we onsider the top four ranked data enters for eah lient (whih over more than 95% of total traffi by vol- ume), and ompute the average differene between all path-latenies from these data enters and the shortestlateny at the same data enter. The CDF of these differenes by traffi volume is shown under all paths in Fig. 5. We find that there is a sizable traffi volume (i.e., 10%) from lients whih have an average path-lateny differene of > 100ms from the shortest at its data enter. This differene is quite signifiant. However, > 95% of traffi only traverses the shortest three paths at eah data enter (Fig. 4), so we perform the averaging only over these paths (urve shortest 3 paths in Fig. 5). The average differene redues signifiantly, e.g., 95% of requests now have an average differene of < 40ms. However, it is not small enough to explain an optimality gap of < 10ms (Fig. 3). Lateny variations between paths from the same DC are low ompared to paths from other DCs. Our hypothesis is that the non-trivial lateny variation between peers of the same data enter observed above is dwarfed by the variation between ISPs aross different data enters so muh that the shortest-lateny from a data enter is an aurate proxy of the latenies from that data enter, when ompared to latenies from other data enters. To quantify this, we ompute the differene between intra-data enter variation (i.e., average differene from shortest-lateny ISP of same DC for top three shortest paths) and inter-data enter variation (i.e., average differene of shortest-latenies of top four ranked DCs) for eah lient. The CDF by traffi volume is shown in Fig. 6. We see that, on average, most requests (i.e., 90%) are from lients whose average path-lateny variations within data enters are less than between data enters, i.e., their differene < 0ms. Indeed, 95% of requests are aptured by lients whose average intra-data enter variation is not more than 10ms higher than the inter-data enter variation. We infer that by mapping lients only based on shortest path-latenies, sheme B is not worse-off on average by > 10ms for 95% of the total traffi. 6

7 CDF over requests all paths shortest 3 paths Avg. diff. from shortest lateny (top 4 DCs, ms) Figure 5: CDF of average lateny variation between ISPs within the same DC. CDF over requests Intra minus inter-dc lateny variation (ms) Figure 6: CDF of the differene between intra- DC and inter-dc lateny variation metris. These lateny patterns arise mostly due to geographi distane. We believe that our high-level inferenes in this setion generalize well, beause of a simple physial reason: geographi proximity is the strongest determinant of propagation delays, muh more so than routing hoies of ISPs in pratie. A servie operator ould ollet and analyze her own network measurements to see how reasonable a proxy geographi distane is for path latenies, as part of deiding how sophistiated the level of oordination needs to be. Hene, although our speifi quantitative results may not be general, we believe the insights are broadly useful. 5. ROBUSTNESS TO TRAFFIC VARIATION In this setion, we ompare how different levels of oordination between mapping and routing help guard against traffi variations. These variations an arise in the form of (i) hanges in reeived traffi volumes between suessive intervals of optimization, i.e., 1 hour in our experiments unless speified otherwise, (ii) requestrate variability within an optimization interval, whih may arise even if the average arrival rate is unhanged from the previous interval, due to inherent variane in request arrival-rates over shorter timesales. We show that even the most fully-oordinated mapping-routing may not be robust to inter-interval traffi variations ( 5.1) due to high link utilizations aused by traffi burstiness. Further, intra-interval variation an also ause performane disruptions if link utilizations start approahing apaities ( 5.2). 5.1 Inter-Interval Traffi Variability To study inter-interval performane variation, we apply mapping and routing deisions online, i.e., ompute deisions from traffi measurements over the previous optimization interval, but apply them to traffi in the urrent interval. We then ompare performane against the offline optimal sheme. We have two key observations: (i) propagation delays obtained from optimizing traffi volumes from the previous interval are near-optimal for the urrent interval. This is beause the median traffi volume hange between optimization intervals is quite small. However, (ii) the most bursty traffi an ause very high link utilizations even in fullyoordinated mapping and routing, if links don t have suffiient spare apaity to absorb bursts. Capaity-aware shemes have nearly optimal average propagation delay when performed online. We find that both sheme B and sheme C in an online senario are within 10ms of the offline optimal traffi alloations throughout the day (graph omitted due to spae onstraint). Hene, inter-interval traffi variability does not signifiantly affet average request propagation delays. Also, fine-grained visibility into routingsystem information is not muh more advantageous. Next, we investigate whether the traffi alloations obtained in an online setting are indeed apaity-aware, i.e., within reasonable link utilizations. Many links are overutilized under both apaityaware shemes more visibility is not neessarily benefiial. Fig. 7 depits the omplementary CDF of the utilization of all links aross hourly optimizations through the day. (We find similar distributions of link utilizations at any given hour of the trae.) We see that links an be badly overutilized by both shemes B and C. Indeed, at any given hour we find that more than 10% of links are overutilized leading to signifiant queueing delays and paket losses. Hene, traffi variability between suessive optimization intervals is suffiiently large to make a apaity-respeting traffi alloation from the previous hour into an overutilizing one in the urrent hour. Next, we understand the traffi variability patterns that ause this observed behavior. Most traffi volume is quite stable, but there is a long tail of very bursty traffi that leads to high link utilizations. We quantify the traffi variation of a lient from the urrent interval to the next 7

8 1 1 CCDF over links, hours sheme B sheme C util. penalty-based Link utilization (%) CDF over lients over traffi volume Median traffi variation (%) Figure 7: CDF of link utilizations over all links and hours, when mapping and routing deisions are applied in an online fashion every hour. as a perentage differene from the urrent interval s traffi a 0% hange implies a perfet predition of the next interval, a negative value implies that the urrent interval s traffi overestimates the next s traffi, and a positive value implies underestimation. We pik every lient s median variation from its timeseries (one sample per hour), and plot the CDF over lients and over traffi volumes in Fig. 8. We find that there are a large number of very bursty lients (urve over lients ), but a large hunk of traffi volume ontribution omes from lients whih are quite stable between eah hour (urve over traffi volume ). However, there is a long tail on the traffi volume distribution whih is the ause of link overutilizations. Indeed, we find that these burstiness patterns in our trae are onsistent aross multiple reoptimization timesales. Even though the median variation is really good (lose to 0%) at all timesales, about 10% of traffi volume in the trae is ontributed by some really bursty prefixes at all timesales. 5.2 Intra-Interval Traffi Variability In this subsetion, we look at how shemes with varying degrees of oordination reat to intra-interval traffi variability. This variability ours beause of the inherently bursty nature of request traffi at multiple timesales [8] hene, traffi volumes used for mapping and routing deisions are neessarily averages of a traffi distribution with a nontrivial variane. As a result, link utilization values resulting from average traffi demands are averages also and links may be overwhelmed with bursty traffi at shorter timesales, leading to performane disruptions even if average traffi demand preditions are aurate for the interval. Naturally, we examine (average) link utilizations over eah optimization interval in an offline setting to understand the robustness of oordinated mapping-routing to intra-interval burstiness. Capaity-aware shemes have high average link Figure 8: CDF of median hourly traffi variation when aggregated over lients, and over traffi volumes. There are some very bursty lients (urve lients ), but most requests are from lients with almost zero median-variation between suessive hours (urve traffi volume ). CDF over links, hours sheme B sheme C util. penalty-based Link utilization (%) Figure 9: CDF of link utilizations over all links and hours in an offline setting. utilizations, and fine-grained oordination is not muh better. Fig. 9 shows that apaity-onstrained shemes tend to push link utilizations up to the maximum allowed apaity for ISP-links orresponding to short-lateny paths to lients. Indeed, for both shemes B and C we see that the median link utilization over all optimization intervals and links is at the maximum allowed per link, whih is 95% of apaity in this ase. As average link utilization approahes apaity, signifiant queueing delays or paket losses may our. In the next setion, we explore how to mitigate the illeffets of traffi burstiness that our even in the most ompletely oordinated mapping and routing shemes. 6. ROBUSTNESS WITH SPARE CAPACITY Our observations from 5 suggest that traffi burstiness aross optimization intervals an ause undesirable link overutilization, as an transient traffi bursts even when the mapping system is ompletely aware of routing- 8

9 system information, and vie-versa. In this setion, we explore a simple tehnique to ahieve robustness to traffi burstiness: enforing some bandwidth headroom while omputing mapping and routing deisions. By leaving some unused slak apaity in eah link while omputing mapping and routing deisions, traffi burstiness an be aommodated muh better. The simpliity of this idea makes it appealing, as opposed to other tehniques suh as AS-level traffi aggregation [20], whih makes it hallenging to distinguish lients within an AS by performane, or employing more sophistiated estimators for traffi arrivals whih may still prove ineffetive against transient bursts. Below, we explore the question of setting a fixed apaity reserve for absorbing bursts but show that we need to navigate a triky tradeoff between the redution in the number of highly-utilized links and lateny inrease whih ours due to traffi taking larger-lateny paths. Then, in 6.1 we introdue the idea of penalizing high utilizations naturally in the objetive funtion to ahieve better robustness, and show in 6.2 that this also provides optimality guarantees. How muh slak apaity should be used? We ask what is a good fration of apaity to reserve to absorb bursty traffi. The answer depends on the volumes of the traffi bursts and link apaities, in pratie. If the fration of spare apaity reserved is too low, then it might not be suffiient to absorb inoming traffi bursts. However, if the fration is too high, then in the event that the link is not bursted, traffi is alloated to longer-lateny paths even when shorter lateny paths are available leading to unutilized apaity and larger request latenies. Indeed, we find that as the fration of apaity reserved for bursts inreases, the fration of overutilized links through the entire trae redues from 24% at no reservations to 4% at 30% apaity reservation. However, the average propagation delay inreases ompared to what ould have been ahieved with the full link apaities, sine traffi is now fored to use longer-lateny paths even if the reserved apaity is not bursted. The average propagation delay gap goes up from 5ms at no reservations to 22ms at 30% reservations for sheme C, and 10ms and 26ms for sheme B respetively. 6.1 Dynami Utilization Penalties Balaning the tradeoff between average request-delay and link-overutilization while piking a fixed apaityreservation an be triky. Instead, we introdue an alternate approah: penalize high link utilizations in the objetive funtion of the optimization with higher penalties for higher utilizations. Intuitively, a penalty on link utilizations dynamially redues link loads by inreasing the value of the objetive funtion, but permits higher utilizations in ases where suh redution would ome at the ost of a large performane loss, i.e., inrease in propagation delay. The utilization penalty. Instead of using hard apaity limits in our optimization models, we relax the link apaity onstraints by introduing a penalty funtion. In partiular, we utilize a piee-wise linear funtion Φ j ( ) (Appendix C, [17]) to apture suh penalties due to high link utilization, and is often used in ISP traffi engineering [21]. We introdue a refined objetive funtion, namely perf Φ, whih replaes the objetives from sheme C and sheme D by perf Φ (α, β) = α i β j vol perf j / vol i,j J i, + ( ) α i β j vol / J (2) i,j J i Φ j where J = i J i, the set of all ISP links. perf Φ an be viewed as an average request-lateny with a link ongestion onsideration. The link apaity onstraints are then removed from both problems. The utilization-penalty based sheme improves robustness to traffi variations over both the other shemes. We evaluate the robustness of the utilizationpenalty based mapping and routing shemes to traffi variability in both online and offline senarios. In the online ase (Fig. 7), we find that utilization penalties an lower the fration of overutilized links from 20% to < 5%. However, the distribution ontinues to be longtailed we believe is due to very high worst-ase traffi variability. In the offline ase (Fig. 9), link utilizations are in general lower than the other shemes, and distributed gradually between 0 and 90%, the step funtion resulting from pieewise-linearity of the penalty funtion. The generally lower utilizations help redue the effet of short-term traffi bursts. In all our runs we find that the average propagation delays of the utilization-penalty based sheme are within 15ms of the global propagation delay optimum. 6.2 Optimality The utilization penalty funtion provides a serendipitous benefit in addition to being more robust alternate mapping-routing optimizations an be made to onverge provably to the global optimum of the new objetive funtion perf Φ over the ombined feasible spae of the mapping and routing variables, bypassing the diffiulties in 2. We provide our key optimality theorem and intuition here, and refer the reader to our teh report (Appendix B, [17]) for the omplete proof. Optimality theory. We denote the transformed mapping and routing optimizations with objetive perf Φ and link apaity onstraints removed, as C Φ and D Φ 9

10 respetively. We also define the transformed baseline GLOBAL Φ in a similar fashion, replaing the objetive funtion with perf Φ and removing link apaity onstraints. Intuitively, we get around the oupled operational onstraint problem in 2 by onverting hard apaity onstraints into soft objetive funtion penalties. We deal with bad routing deisions in the absene of traffi ( 2) by adopting the refinement from [7]. We term a mapping or routing optimization with this refinement as an optimal projetion (Def. 2). Our key optimality result is as follows. Theorem 1. Optimal projetions of C Φ and D Φ onverge to an optimal solution of GLOBAL Φ. Due to spae onstraint, we only provide the proof sketh here and refer the reader to our teh report (Appendix B, [17]) for the omplete proof. The proof proeeds in three steps. We first show that there exists an equilibrium point (α, β ), suh that α is a best response to β, and vie versa. We define the best response as an optimal projetion (Def. 2), and the resulting equilibrium as a Nash equilibrium (Def. 3). We further prove that (α, β ) is also an optimal solution to GLOBAL Φ, extending previous results [7] to models with linear mapping onstraints. We finally show that iterative optimal projetions lead to a Nash equilibrium point, ompleting the proof. We introdue the formalities to assist our proof. First, we need a way to apture the marginal ost f ij of serving lient from data enter i over link j J i : Definition 1. Let the metri f ij be defined as f ij (α i, β j ) = vol perf j / vol ( ) +vol Φ j α i β j vol / J We next define the optimal projetion of mapping and routing strategies, respetively: Definition 2. (i) Given fixed routing deisions β, a set of mapping deisions α is alled an optimal projetion onto mapping spae if α is an optimal solution to C Φ. (ii) Given fixed mapping deisions α, a set of routing deisions β is alled an optimal projetion onto routing spae if, i,, we have f ij (α i, βj ) f ij (α i, βj ) for all j, j J i suh that βj > 0. We next define the notion of Nash equilibrium. Definition 3. A set of mapping and routing deisions (α, β ) is alled a Nash equilibrium if α is an optimal projetion given β, and β is an optimal projetion given α. We state the results whih string the proof together: Theorem 2. There exists a Nash equilibrium. Theorem 3. Let (α, β ) be a Nash equilibrium. Then {α, β } is an optimal solution to GLOBAL Φ. Theorem 4. Optimal projetions of C Φ and D Φ onverges to a Nash equilibrium. Visualization. Consider the toy example illustrated in Figure 10(a). Let α and β be the mapping and routing deisions respetively. Suppose the lient has one unit of demand, and the goal of both mapping and routing is to minimize lateny. The optimal performane is ahieved when all traffi is mapped to the DC at the right, with routing set suh that no link apaity is violated, i.e., α = 1, β = 1/4. However, by separate optimizations, both parties an easily get stuk in a loal optimum. For instane, onsider α = 1/2, β = 1/2, whih are valid mapping and routing settings. Mapping is optimal as inreasing α overshoots the link apaity. Routing is also optimal beause routing 1/2 of the traffi on the 50ms link makes it fully utilized. Figure 10(b) visualizes how the loal optimum is reahed from an initial starting point. The olor-shaded region represents the feasible deision spae. The urvy boundaries reflet the apaity onstraints of the 50ms and 60ms links. The olor oding represents the objetive value, i.e., lateny, where red means high values and blue, low. All points on these boundaries exept the rightmost one are loal optima with higher latenies than the global optimum. In general, the feasible spae determined by link apaity onstraints is non-onvex. Figure 10(b) suggests that loal equilibria only exist on the boundaries that are implied by the link apaity onstraint. Hene, we fous on this onstraint and prove that removing it removes the suboptimality from alternate optimizations. Indeed, when we remove the apaity onstraint and introdue the penalty into the objetive funtion, we arrive at the situation in Fig. 10(). The whole deision spae is now feasible, but the violation of the link apaities will imply a high objetive (shown by dark red olor). Further, there is a gradient of objetive values near the (previous) apaity boundary, transitioning from low to very high values of the objetive funtion. Indeed, this objetive gradient allows alternate mappingrouting optimizations to onverge to the global optimum as shown in Fig. 10(). Note that the global optima shown in Figures 10(b) and 10() are slightly different, as the ongestion ost is not onsidered in the original formulation. As noted previously in 6.1, the differene in optimal propagation delays is within 15ms in all our experiments. 7. IMPACT ON COST 10

11 ap=100 ap= ms 60ms ap=3/4 1-β 1-α α 75ms 50ms ap=1/4 β Available route (a) Example network (b) GLOBAL () GLOBAL Φ Figure 10: An example of loal optima with separate mapping-routing optimization. We evaluate the impat of per-link information visibility for osts. We introdue mapping shemes analogous to shemes B and C namely B.1 and C.1 whih use average osts of all links in a DC and per-link bandwidth osts, respetively. We also introdue the entralized sheme whih optimizes for the average ost metri ( 3), global.1. (The osts per unit bandwidth for links are derived from a loud bandwidth priing plan [22]; we refer the reader to our teh report for details [17]). Visibility into per-link osts provides a signifiant advantage. We find that sheme B.1 an inur signifiantly worse osts (25% higher on average) than sheme C.1. This is reasonable, as prior works have noted that the bandwidth-ost diversity between ISPs an be quite large [3, 23]. We find that C.1 ahieves near-optimal osts most hours of the day, with a few outliers (due to situations like in 2). 8. GENERALIZABILITY A natural onern that may arise from the empirial observations in this paper is whether they generalize beyond the speifi CDN traes that we have used. We argue that even though speifi quantitative results from this work may not hold in general, our insights and methodologies are still broadly useful. Propagation delay omparisons. A servie operator an easily determine through the desribed method and his own network measurements whether propagation delay is indeed the key determinant of lateny variation patterns thereby deiding on a suitable level of oordination. Indeed, some publily known state-ofthe-art mapping systems, e.g., [19] default to geographially losest replias with some load information. Robustness to traffi burstiness. Burstiness of web traffi at various timesales has been observed in multiple environments [8, 20, 24]. Clearly, the proportion of highly utilized or overutilized links is a network bandwidth and workload-dependent property. However, the general traffi burstiness harateristis and their impliations for link load are in agreement with observations from prior works on traffi engineering whih attempt to redue link utilizations, e.g., [21]. Cost omparisons. The large diversity of ISP bandwidth osts has been observed in prior work [3, 23], motivating shemes that attempt to minimize osts by utilizing low-ost links as opposed to high-ost ones. A network operator an easily determine if this diversity plays a key role in her network osts through the evaluations as desribed. Robust, optimal oordination sheme. Our optimality result is independent of any speifis of our dataset, and an provide useful performane-preditability for servie operators. 9. RELATED WORK Joint ontrol. Reently, there has been work on joint ontrol and interation between two parties for traffi engineering. DiPalatino et al. [7] model a game in whih two players have independent ontrol over routing and server seletion, and determine when a soial planner might find optimal utilities. Jiang et al. [25] propose ooperative server seletion and traffi engineering between network and ontent providers who have onfliting objetives. We propose an extension of the optimality result (Appendix B, [17]) inorporating pratial onsiderations suh as DC-level load balaning and apaity onstraints. In addition, we evaluate multiple formulations orresponding to varying degrees of visibility. Wide-area traffi engineering. WhyHigh [6] diagnoses lateny problems with losest-replia mapping, ategorizing their auses into peering, apaities, and ISP traffi engineering. Our work is omplementary to this approah, as it embraes the possibility that 11

12 losest-replia mapping does not always offer best performane, and fousses on the benefits of oordination given the urrent ISP peers and link apaities. Entat [3] proposes to optimize traffi engineering within Mirosoft s bakbone aross all upstream ISPs, when requests enter through the losest ingress point. In ontrast, we onsider the impat of the hoie of ingress point i.e., mapping, whih is ruial for CSPs without a bakbone network. Hene, we also onsider the funtional separation between mapping and routing. Salable networked servies. Volley [26] onsiders the problem of data plaement aross the wide-area, but we assume that ontent is read-mostly and fully repliated. Goldenberg et al. [23] studied multi-homed traffi engineering for stub autonomous systems to optimize ost and performane. DONAR [9] is a deentralized mapping servie that allows ustomized lient poliies and offers better performane to lients. In ontrast to these latter two works, we study a set of distributed solutions oordinating mapping and routing. Network performane measurement. There is a rih body of researh on network path performane estimation, whih is omplementary to our study, for determining the perf j metri. Virtual oordinate systems e.g., [27] estimate lateny based on syntheti oordinates. Others fous on the reduing the overheads for measuring IP address spae, e.g., [28]. Our study employs a path performane predition servie [15] that builds on segments of known Internet paths. 10. CONCLUSION We present the problem of oordinating data enter seletion and response-routing for online servies, whih an allow servie providers to offer better performane to users at lower ost. We provide optimization formulations and use real CDN traffi traes to study varying levels of information visibility, and find that (i) sharing oarse-grained information is suffiient to ahieve good latenies, but (ii) fine-grained information is neessary for minimal osts. Further, (iii) even systems with full information visibility may lak robustness to traffi variations. Finally, we propose a robust oordination tehnique whih retains administrative separation between the two systems, and is also provably optimal. 11. REFERENCES [1] J. Hamilton, The ost of lateny. mvdirona.om/2009/10/31/thecostoflateny.aspx. [2] A. Akella, B. Maggs, S. Seshan, A. Shaikh, and R. Sitaraman, A measurement-based analysis of multihoming, in Pro. ACM SIGCOMM, [3] Z. Zhang, M. Zhang, A. Greenberg, Y. C. Hu, R. Mahajan, and B. Christian, Optimizing ost and performane in online servie provider networks, in Pro. NSDI, [4] A. Qureshi, R. Weber, H. Balakrishnan, J. Guttag, and B. Maggs, Cutting the eletri bill for internet-sale systems, in Pro. ACM SIGCOMM, [5] Y. Chen, S. Jain, V. K. Adhikari, and Z.-L. Zhang, Charaterizing roles of front-end servers in end-to-end performane of dynami ontent distribution, in Pro. Internet Measurement Conferene, [6] R. Krishnan, H. V. Madhyastha, S. Srinivasan, S. Jain, A. Krishnamurthy, T. Anderson, and J. Gao, Moving beyond end-to-end path information to optimize dn performane, in IMC, [7] D. DiPalantino and R. Johari, Traffi engineering vs. ontent distribution: A game theoreti perspetive, in Pro. IEEE INFOCOM, [8] A. Feldmann, A. C. Gilbert, and W. Willinger, Data networks as asades: Investigating the multifratal nature of internet wan traffi, in SIGCOMM, [9] P. Wendell, J. W. Jiang, M. J. Freedman, and J. Rexford, DONAR: Deentralized server seletion for loud servies, in Pro. ACM SIGCOMM, [10] Client subnet in DNS requests, IETF draft. [11] R. Kohavi, R. M. Henne, and D. Sommerfield, Pratial guide to ontrolled experiments on the web: Listen to your ustomers not to the hippo, in Pro. ACM SIGKDD, [12] N. Cardwell, S. Savage, and T. Anderson, Modeling tp lateny, in Pro. IEEE INFOCOM, [13] C. Fraleigh, S. Moon, B. Lyles, C. Cotton, M. Khan, D. Moll, R. Rokell, T. Seely, and C. Diot, Paket-level traffi measurements from the Sprint IP bakbone, IEEE Network, [14] M. Szymaniak, D. Presotto, G. Pierre, and M. van Steen, Pratial large-sale lateny estimation, Computer Networks, vol. 52, pp , May [15] H. V. Madhyastha, T. Isdal, M. Piatek, C. Dixon, T. Anderson, A. Krishnamurthy, and A. Venkataramani, iplane: An information plane for distributed servies, in Pro. OSDI, Nov [16] J. Hamilton, Overall data enter osts. OverallDataCenterCosts.aspx. [17] S. Narayana, J. W. Jiang, J. Rexford, and M. Chiang, Tehnial report. ~narayana/joint-maproute.html. [18] M. J. Freedman, E. Freudenthal, and D. Mazières, Demoratizing ontent publiation with Coral, in Pro. NSDI, [19] Y. Zhu, B. Helsley, J. Rexford, A. Siganporia, and S. Srinivasan, LatLong: Diagnosing wide-area lateny hanges for CDNs. In submission, http: // [20] N. Feamster, J. Borkenhagen, and J. Rexford, Guidelines for interdomain traffi engineering, CCR, [21] B. Fortz, J. Rexford, and M. Thorup, Traffi engineering with traditional IP routing protools, IEEE Communiation Magazine, vol. 40, pp , Ot [22] Amazon EC2 Priing. [23] D. K. Goldenberg, L. Qiu, H. Xie, Y. R. Yang, and Y. Zhang, Optimizing ost and performane for multihoming, in Pro. ACM SIGCOMM, [24] W. E. Leland, M. S. Taqqu, W. Willinger, and D. V. Wilson, On the self-similar nature of ethernet traffi (extended version), IEEE/ACM Trans. Networking. [25] J. W. Jiang, R. Zhang-Shen, J. Rexford, and M. Chiang, Cooperative ontent distribution and traffi engineering in an ISP network, in Pro. ACM SIGMETRICS, [26] S. Agarwal, J. Dunagan, N. Jain, S. Saroiu, A. Wolman, and H. Bhogan, Volley: automated data plaement for geo-distributed loud servies, in Pro. NSDI, [27] F. Dabek, R. Cox, F. Kaashoek, and R. Morris, Vivaldi: A deentralized network oordinate system, in Pro. ACM SIGCOMM, [28] P. Franis, S. Jamin, C. Jin, Y. Jin, D. Raz, Y. Shavitt, and L. Zhang, IDMaps: A global Internet host distane estimation servie, IEEE/ACM Trans. Networking,

13 [29] G. Appenzeller, I. Keslassy, and N. MKeown, Sizing router buffers, in SIGCOMM, [30] A. Bavier, M. Bowman, B. Chun, D. Culler, S. Karlin, S. Muir, L. Peterson, T. Rosoe, T. Spalink, and M. Wawrzoniak, Operating system support for planetary-sale network servies, in NSDI, [31] The RouteViews Projet. 13

14 Symbol J X ij K P i w i ɛ i B i f ij g Definition Set of all outgoing ISP links, i.e., i J i Fration of lient traffi routed through link j J i That is, i,j J i X ij = 1 for all. Cost-performane tradeoff parameter Proportion of total traffi direted to DC i Target traffi split ratio of DC i Tolerane of deviation from target split ratio Request-rate ap plaed on DC i Marginal ost funtion for link j J i, lient Fration of traffi ontributed by lient λ i1, λ i2 Dual optimal variables for the GLOBAL problem µ i1, µ i2 Dual optimal variables for the MAP P ING problem Table 3: Additional notation introdued in this appendix for model generalization and proof. APPENDIX In this appendix, we extend the mapping and routing with omplete information exhange (i.e., shemes C and D) with two onsiderations: (i) joint ost and performane optimization, and (ii) data enter load management. These extended formulations are referred to as MAPPING and ROUTING respetively. We also rewrite the baseline problem GLOBAL. These extensions subsume formulations C and D, and the GLOBAL formulation provided earlier in the paper. Consistent with earlier notation, we denote formulations with link utilization penalties instead of link apaity onstraints using a Φ in the subsript: namely, MAPPING Φ, ROUTING Φ and GLOBAL Φ respetively. The appendixes are organized as follows. Appendix A shows that the extended global problem, whih is non-onvex, is equivalent to a entralized onvex optimization problem. Appendix B proves the optimality of alternating mapping and routing optimizations (with extended formulations), when link utilization penalties are used. Appendix C shows the form of this link utilization penalty funtion Φ. Appendix D desribes the data setup for the evaluations. The additional notation introdued in the appendixes is desribed in Table 3. Model Extensions Jointly optimizing performane and ost. The goals of maximizing performane and minimizing osts are often at odds. Good links that offer lower latenies are usually pried higher, and over-utilizing low-ost links will degrade performane due to inreased ongestion. Therefore, CSPs need to ustomize the ostperformane tradeoff. To strike a balane between ost and performane, we introdue a weight fator K that reflets the amount of additional osts ($/request) that a CSP is willing to pay for one unit of performane improvement (ms/request). That is, the CSP s goal is to minimize the objetive funtion ost + K perf. Note that the sub-ases of performane-only and ost-only optimizations are obtained by appropriate settings of K, namely K = 0 and K = respetively. Data enter load management. For additional flexibility, we allow CSPs to express their preferenes of a traffi split ratio w i for eah data enter i, with a tolerane ɛ i of deviation for other purposes like performane and ost. This allows CSPs to balane workload between data enters proportional to their server resoures, or intentionally shift traffi to under-utilized data enters due to unattrative network loations, e.g., large RTTs to reah the majority of lients. CSPs usually need to plae a ap on the total requests a DC reeives aording to its omputing power, e.g., proportional to the number of servers in that DC. We let CSPs speify suh a request-rate ap, e.g., B i, for every data enter i as its apaity limit. Note that appropriate settings of w i, ɛ i and B i allow us to easily reflet formulations without these onstraints, e.g., setting ɛ i = 1.0, w i [0, 1], and B i > vol. With these extensions, the mapping, routing and global baseline formulations are written down as follows. MAPPING(β) minimize ost + K perf (3a) α i vol subjet to w i ɛ i, i(3b) vol α i vol B i, i (3) variables vol α i β j ap j, j(3d),i:j J i α i = 1,, (3e) i α i 0, i, where onstraints (3b) and (3) represent data enterlevel load management onstraints, namely load splitting and load apping. Constraints (3d) represent perlink bandwidth apaities. ROUTING(α) minimize ost + K perf (4a) subjet to β j = 1, i, (4b) variables j vol α i β j ap j, j(4),i:j J i β j 0, i, j, 14

15 GLOBAL minimize ost + K perf (5a) subjet to vol α i β j ap j, j (5b),i:j J i vol α i vol w i ɛ i, i (5) vol α i B i, i (5d) variables α i = 1, i β j = 1, i, j J i α i 0, i,, β j 0, j, (5e) (5f) A. A CENTRALIZED APPROACH TO THE JOINT PROBLEM We present an equivalent onvex formulation of the joint mapping and routing optimization problem that aepts an effiient entralized solution. As written above (5), GLOBAL is a non-onvex optimization problem on variables (α i, β j ), and hene annot be solved using standard onvex programming tehniques. It is not lear whether loal optima exist and how bad they are relative to the global optimum. However, we show that (5) an be translated into an equivalent onvex optimization problem by introduing a new set of variables X ij, whih represents the fration of lient traffi that is mapped to data enter i through link j. There is a diret onnetion between the two sets of variables: X ij = α i β j, (i, j, ) We an rewrite (5) into the following onvex program: GLOBAL X minimize subjet to variables ( ) X ij vol priej + K perf j ij (6a) vol,j J i vol X ij vol w i ɛ i, i(6b) vol X ij B i, i (6),j J i vol X ij ap j, j (6d),i:j J i X ij = 1, (6e) ij X ij 0 i, j, As we show in the theorem below, the optimal solutions of GLOBAL and GLOBAL X are related, and the two problems have the same optimal objetive. Theorem 5. GLOBAL and GLOBAL X are equivalent. They have the same optimal objetive values. Further, their optimal solutions an be derived from eah other. Proof. First, the optimal value of GLOBAL X is a lower-bound on that of GLOBAL. Consider any feasible solution {α i, β j } of (5). We onstrut X ij = α i β j, whih is also feasible for problem (6). It an be readily verified that two solutions ahieve the same objetive value. Seond, the optimal value of GLOBAL is a lower-bound on that of GLOBAL X. Consider any feasible solution {X ij } of (6). We onstrut α i = j J i X ij, β j = X ij /α i if α i > 0 and β j = 1/ J i otherwise. It an be verified that {α i, β j } is feasible for GLOBAL, and ahieves the same objetive value as GLOBAL X. We establish a one-to-one mapping between variables of (5) and (6), and hene they obtain the same optimal objetives and solutions. It is also easily heked that the equivalene ontinues to hold between the two new optimization problems formed when GLOBAL and GLOBAL X are refined by inluding link utilization penalties in the objetive instead of hard apaity onstraints (5b) and (6d). B. OPTIMALITY OF DISTRIBUTED MAP- PING AND ROUTING We present the proof of optimality of alternating mapping and routing optimizations with link utilization penalties referred to as MAPPING Φ and ROUTING Φ. These are obtained from (3) and (4) above respetively by replaing the objetive funtion by ost + K perf Φ, and removing the link apaity onstraints (3d) and (4). At a high level, the algorithm proeeds as follows: given routing deisions β as input, mapping nodes solve (3) and optimize over α, and given mapping deisions α as input, edge routers solve (4) and optimize over β. The two steps are arried out iteratively until solutions onverge. We allow mapping and routing problems to be solved at different time-sales, with the only requirement that (3) does not start until (4) is fully solved, and vie versa. This is easily ensured by having eah omponent wait to reeive inputs from the other before solving its own loal optimization problem. However, we showed in Figure 2(b) that in general the alternate projetion algorithm may still lead to suboptimal equilibria, when some lients send no traffi to a data enter. To overome this issue, we introdue a refinement to routing alled optimal projetion, in addition to the optimality of (4). We soon prove the optimality of suh a refined routing strategy. In pratial omputation of routing deisions, we introdue an approximation to (4) by inrementing an infinitesimally small demand to a lient-server pair with zero traffi, i.e., α i δ if α i = 0, where δ is a small positive onstant [7]. This approximation is often used to ease 15

16 the omputation suh that standard optimization tehniques an be applied. However, lients do not need to send real traffi, making this approah pratially appealing. We redefine the metri f ij (previously Def. 1) and optimal projetion (previously Def. 2) orresponding to the extended optimization formulations. Note that the notion of Nash equilibrium (Def. 3) is reinterpreted through the modified definition of optimal projetion. Definition 4. Let the metri f ij be redefined as f ij (α i, β j ) = vol ( priej + K perf j ) / vol ( ) +vol Φ j α i β j vol / J Definition 5. (i) Given fixed routing deisions β, a set of mapping deisions α is alled an optimal projetion onto mapping spae if α is an optimal solution to MAPPING Φ. (ii) Given fixed mapping deisions α, a set of routing deisions β is alled an optimal projetion onto routing spae if, i,, we have f ij (α i, βj ) f ij (α i, βj ) for all j, j J i suh that βj > 0. We prove the results stringing the proof together in the subsetions below. Even though we follow the definitions and proofs of onvergene and optimality in [7], we annot diretly apply their results beause of the presene of data enter load onstraints (3b) and (3). B.1 Optimal Projetion implies Optimality Lemma 1. If β is an optimal projetion given α, then β is also an optimal solution to ROUTING Φ. Proof. To show that β is an optimal solution to the routing problem (i.e., (4) with refined objetive), we hek the KKT onditions for (4). These onditions hold if and only if α i f ij α i f ij, j, j J i and β j > 0. This is true beause when α i > 0, the definition of optimal projetion implies the above ondition. When α i = 0, the optimality ondition holds too. It is easy to hek that an optimal solution to (4) is not neessarily an optimal projetion. B.2 Existene of Nash Equilibrium Theorem 6. There exists a Nash equilibrium. Proof. We show this by onstrution. Let X be an optimal solution to GLOBAL X with refined objetive funtion ost + K perf Φ and without apaity onstraint (6d). We onstrut a set of mapping and routing deisions as follows: αi = j J i Xij, β j = Xij /α i if α i > 0. If α i = 0, we set β j = 1 for some j = argmin j Ji f ij, and set βj = 0 for j J i \{j }. By Theorem 5, it is easy to hek that (α, β ) is also an optimal solution to GLOBAL Φ. Therefore, α must be an optimal solution to MAPPING Φ (β ), as otherwise we an find a better α to improve the global objetive funtion. By definition, α is an optimal projetion given β. Similarly, β is an optimal solution to ROUTING Φ (α ). The KKT onditions for ROUTING Φ imply that the definition of optimal projetion holds when αi > 0. When α i = 0, our hoies of βj stritly follows the optimal projetion definition. Combining the two ases shows β is an optimal projetion given α. Therefore, (α, β ) is a Nash equilibrium. B.3 Optimality of Nash Equilibria Theorem 7. Let (α, β ) be a Nash equilibrium. Then {α, β } is an optimal solution to GLOBAL Φ. Proof. Construt a set of variables Xij = α i β j for (i, j, ). We show that {Xij } is an optimal solution to GLOBAL X (when refined with link utilization penalties instead of hard link apaity onstraints), given that {αi, β j } is a Nash equilibrium. It is easy to hek that {Xij } are feasible for (refined) GLOBAL X. Then, it suffies to show that the KKT onditions for GLOBAL X hold with our hoie of {Xij } and appropriate hoies of other parameter, e.g., Lagrange multipliers, involved in the optimality onditions (whih we disuss shortly). In a slight abuse of notation below, we write f ij (X) = vol (prie j + K perf j )/ vol + Φ j ( X ijvol ) vol / J. Here we onsider the mapping onstraint (3b) only and (3) should follow similarly. We write down the KKT onditions for the refined GLOBAL X. There exist dual optimal variables {λ i1 0, λ i2 0} i suh that f ij + (λ i1 λ i2 ) g f i j + (λ i 1 λ i 2) g, if X ij > 0, λ i1 X ij g w i ɛ i = 0, i j λ i2 X ij g w i + ɛ i = 0, i j where g = vol / vol. The first ondition originates from the stationarity requirement, and an be interpreted as follows: when a link j J i (link j of data enter i) is used to reah lient, e.g., X ij > 0, its assoiated marginal ost must be no greater than the marginal ost of any other link j J. Note that the marginal ost onditions are similar to those in [7], but slightly more ompliated due to the presene of dual variables and onstraints. The seond and third optimality onditions are omplementary slakness for the dual variables and orresponding onstraints. 16

17 Next we onsider a Nash equilibrium (α, β ). We an write down the KKT optimality onditions for MAPPING Φ and ROUTING Φ, respetively, beause the definition of optimal projetion implies the optimality to the two optimization problems. Following the same notations, we have: (i) KKT optimality onditions for MAPPING Φ : there exist dual optimal variables {µ i1 0, µ i2 0} i suh that βjf ij + (µ i1 µ i2 )g j J i βj f i j + (µ i 1 µ i 2)g, if αi > 0, j J i ( ) (8) µ i1 αig w i ɛ i = 0, i ( ) µ i2 αig w i + ɛ i = 0, i Similarly, the first ondition is the stationarity requirement, and the seond and third onditions are the omplementary slakness for the dual variables. (ii) KKT optimality onditions for ROUTING Φ : f ij f ij for j, j J i and β j > 0, (i, ) where α i > 0 (9) It follows that the values of f ij are equal for all j suh that β j > 0 and α i > 0. By our onstrution, Xij = α i β j. We next show that {Xij } satisfy the optimality onditions (7), given the KKT onditions (8)-(9) established for {αi, β j }. Without loss of generality, onsider Xij > 0 for some (i, j, ) and j J i, and we have αi > 0, β j > 0. From ROUTING Φ optimality onditions (9), we know that f i j takes the same value for all j J i when β j > 0. By our hoie of β j > 0, we have β j f i j = f ij β j = f ij (10) j J i j J i Consider any link j in data enter i, i.e., j J i. There exists at least one link j J i suh that β j > 0. Applying the optimality onditions (9) on (i, j, ), we have f i j f i j (11) Further, by an argument similar to (10), we have f i j = β j f i j (12) KKT optimality onditions (7) are satisfied with suh hoies. Consider Xij > 0 for some (i, j, ) and j J i, we write down its assoiated marginal ost f ij, and ompare it against that of any other link j J i for the same lient. We have f ij + (λ i1 λ i2 ) g = f ij + (µ i1 µ i2 ) g (hoie of dual variables) = j J i β jf i j + (µ i1 µ i2 ) g by (10) β j f i j + (µ i 1 µ i 2) g by (8) j J i = f i j + (µ i 1 µ i 2) g by (12) f i j + (µ i 1 µ i 2) g by (11) = f i j + (λ i 1 λ i 2) g (hoie of dual variables) So we arrive at the onlusion that {Xij } are optimal solutions to (refined) GLOBAL X. Therefore, {αi, β j } are also optimal solutions to GLOBAL Φ as they attain the same optimal objetive values. B.4 Convergene to Nash Equilibrium Theorem 8. Alternate projetion of MAPPING Φ and ROUTING Φ onverges to a Nash equilibrium. Proof. Consider the funtion obj := ost + K perf Φ, whih is the ommon objetive for both MAPPING Φ and ROUTING Φ problems. By the definition of optimal projetions, the sequene of objetive values of obj is dereasing under alternating optimizations of mapping and routing. In addition, the objetive value is lower-bounded by the optimal value of GLOBAL X. Therefore, there exists a limit point of the sequene. Similarly, we an argue that the (α, β) sequene also has a limit point, sine the feasible spae of {α, β} is ompat and ontinuous. It is not diffiult to hek that the limit point must be be a Nash equilibrium, as otherwise we an find another feasible solution that is within an ɛ-ball of the the limit point, whih gives a lower objetive value than the limit of obj, whih ontradits our assumption at the beginning. j J i To show that (7) holds for {Xij }, it suffies to find a set of parameters {λ i1 0, λ i2 0} for all i suh that those equalities and inequalities in (7) hold. We laim that the dual optimal variables {µ i1, µ i2 } in (8) an be used as a hoie of {λ i1, λ i2 }. We then show that the C. PERFORMANCE PENALTY FUNCTION ISP traffi engineering models link ongestion ost with a onvex inreasing funtion Φ(r j ) on the link traffi load, i.e., r j. The exat shape of the funtion is not important, and we use the same pieewise linear ost 17

18 funtion as in [21], given below: r j 0 r j /ap j < 1/3 3r j 2/3 ap j 1/3 r j /ap j < 2/3 10r j 16/3 ap j 2/3 r j /ap j < 9/10 Φ j (r j, ap j ) = 70r j 178/3 ap j 9/10 r j /ap j < 1 500r j 1468/3 ap j 1 r j /ap j < 11/ r j 16318/3 ap j 11/10 r j /ap j < In our evaluations, we sale this funtion suh that 100% utilization gives a 250ms queuing lateny, orresponding to some standard router buffer sizes [29]. D. EXPERIMENT SETUP We evaluate the benefits of the joint mapping-routing approah using trae-based simulations for CoralCDN [18], a ahing and ontent distribution platform running on top of Planetlab. Planetlab [30] is a servie deployment platform whih spans more than 800 sites geographially distributed in six ontinents. We orrelate the traffi data with lateny measurements from iplane [15] whih ollets various wide-area network statistis to destinations on the Internet from a few hundred vantage points (hosted on Planetlab nodes). All data orresponds to Marh 31st, Emulating data enters by lustering Coral nodes: Most Planetlab sites are single-homed i.e., have just one ISP onneting them to the Internet. Therefore, it is hallenging to demonstrate the benefits of our sheme on Coral, beause the only deision that ontrols the lateny and ost of user requests is the mapping deision. Instead, we adopt the approah followed in [2] where we treat multiple Planetlab sites in the same general metropolitan area as different egress links of a single data enter loated in that area. The density of Planetlab sites around ertain ities makes suh lustering of sites into data enters possible. We handpiked 12 data enters loated in North Ameria, Europe, Asia and South Ameria, with a minimum of three Coral nodes in eah data enter and a maximum of six. We show the loations of the data enters in Figure 11. Figure 11: Loation of the 12 emulated data enters. Routable prefixes as lient aggregates: We aggregate users into routable Internet prefixes (from a RouteViews [31] FIB dump), for two reasons. First, routing deisions at data enter egress routers happen at this granularity, hene it is a natural hoie. Seond, the aggregation of IP prefixes in routing tables enables us to deide on reahability and lateny information for a larger number of lient IPs from the same set of initial lateny measurements. The number of routable IP prefixes is large ( 350, 000) but only of these send any traffi to Coral. Traffi data from CoralCDN: We use a worldwide request trae from CoralCDN whih lists a request timestamp, the Planetlab site at whih it was reeived, a lient IP address, and the number of bytes involved in the transfer. Overall, this trae ontains 27 million requests and over a terabyte of data. For our experiments, we aggregate the requests made by lients into their most speifi IP prefix from the RouteViews dump, and average their request rates for eah hour. Lateny measurements from iplane: We extrat round trip lateny (in milliseonds) from the iplane logs, whih ontain traeroutes made to a large number of IP addresses from various vantage points at Planetlab sites. We only use lateny information from vantage points whih are also egress links in the data enters we piked in Figure 11, whih limits us to 49 vantage points. We assign the lateny to eah destination IP address to the orresponding most-speifi IP prefix from the RouteViews dump, assuming that the lateny is representative. If there is lateny data for multiple IP addresses from the same prefix, we use their average value. The logs provide only one set of lateny values for the entire day, and we assume that the paths are lightly loaded when the probes our hene treating these measured latenies as path propagation delays. Piking a set of workable lient prefixes: The set of destination IP prefixes for whih lateny data is available is not uniform aross vantage points. Hene, we only retain data for destinations whih are reahable from at least one vantage point belonging to eah data enter. This allows us to perform mapping of any lient to any data enter. Next, we interset this set of destinations with those lient prefixes whih send traffi to Coral CDN at any time through the day. We are left with a set of about prefixes, whih onstitute about 39% of the total traffi trae by requests and 38% by bytes. Capaity estimations: The apaities of links onneting Planetlab sites to the Internet are not publily available, and even if they are, Planetlab mahines are shared aross multiple servies eah possibly with bandwidth aps. Instead of attempting to determine apaity or bandwidth aps from ground truth (this information is not publily available to our knowledge), we 18