Measuring Bottleneck Bandwidth of Targeted Path Segments

Transcription

1 Measuring Bottleneck Bandwidth of Targeted Path Segents Khaled Harfoush Deartent of Couter Science North Carolina State University Raleigh, NC Azer Bestavros Couter Science Deartent Boston University Boston, MA 2215 John Byers Couter Science Deartent Boston University Boston, MA 2215 Abstract Accurate easureent of network bandwidth is crucial for network anageent alications as well as flexible Internet alications and rotocols which actively anage and dynaically adat to changing utilization of network resources. Extensive work has focused on two aroaches to easuring bandwidth:easuring it ho-by-ho, and easuring it end-toend along a ath. Unfortunately, best-ractice techniques for the forer are inefficient and techniques for the latter are only able to observe bottlenecks visible at end-to-end scoe. In this aer, we develo end-to-end robing ethods which can easure bottleneck bandwidth along arbitrary, targeted subaths of a ath in the network, including subaths shared by a set of flows. We evaluate our technique through extensive ns siulations, then rovide a coarative Internet erfoance evaluation against ho-by-ho techniques. We also describe a nuber of alications which we foresee as standing to benefit fro solutions to this roble, ranging fro network troubleshooting and caacity rovisioning to otiizing the layout of alication-level overlay networks, to otiized relica laceent. I. INTRODUCTION Measureent of network bandwidth is crucial for anyinternet alications and rotocols, eseciallythose involving the transfer of large files and those involving the delivery of content with real-tie QoS constraints, such as streaing edia. Soe secific exales of alications which can leverage accurate bandwidth estiation include end-syste ulticast and overlaynetwork configuration rotocols [6], [19], [1], content location and deliveryin eer-to-eer (P2P) networks [33], [3], network-aware cache or relica laceent olicies [2], [31], and flow scheduling and adission control olicies at assively-accessed content servers [7]. In addition, accurate easureents of network bandwidth are useful to network oerators concerned with robles such as caacity rovisioning, traffic engineering, network troubleshooting and verification of service level agreeents (SLA s). Bandwidth Measureent: Two different easures used in end-to-end network bandwidth estiation are bottleneck bandwidth, or the axiu transission rate that could be achieved between two hosts at the endoints of a given ath in the absence of anycoeting traffic, and available bandwidth, the ortion of the bottleneck bandwidth along a ath that could be acquired bya given flow at a given instant in tie. Both of these easures are iortant, and each catures different relevant roerties of the network. Bottleneck bandwidth is a static baseline easure that alies over long tie-scales (u to the tie-scale at which network aths change), and is indeendent of both the articular traffic dynaics at a tie instant. Available bandwidth rovides a dynaic easure of the load on a ath, or ore recisely, the residual caacity of a ath. Additional alication-secific inforation ust then be alied before aking eaningful use of either easure; for exale, the rate aroriated byan additional TCP flow is quite different than the unused caacityalong a ath [1], [11]. While easures of available bandwidth are certainlyore useful for control or otiization of rocesses oerating at short tie scales, rocesses oerating at longer tie scales (e.g. server selection or adission control) will find estiates of both easures to be helful, while any network anageent alications (e.g. caacityrovisioning) are concerned riarilywith bottleneck bandwidth. In this aer, we focus on easuring bottleneck bandwidth. Catalyst Alications: To exelifythe tye of alications that can be leveraged bythe identification of shared bottleneck bandwidth (or ore generally, the bottleneck bandwidth of an arbitrary, targeted subath), we consider the two scenarios illustrated in Figure 1. In the first scenario, a client ust select two out of three sources to use to download data in arallel. This scenario ayarise when downloading content in arallel fro a subset of irror sites or ulticast sources [4], [32], [12], or fro a subset of eer nodes in P2P environents [3]. In the second scenario, an overlaynetwork ust be set u between a single source and two destinations. This scenario ayarise in ad-hoc networks and end-syste ulticast systes [6], [19]. For the first scenario illustrated in Figure 1 (left), the greedy aroach of selecting the two servers whose aths to the client have the highest end-to-end bottleneck bandwidth naely, servers A and B is not otial, since the aggregate bandwidth to the client would be liited bythe shared 3Mbs bottleneck bandwidth fro servers A and B to the client. To be able to select the air of servers yielding the axiu aggregate bandwidth of 5Mbs naelya and C or B and C the client needs to easure the shared bottleneck bandwidth between airs of servers. Siilarly, in the second scenario illustrated in Figure 1 (right), the identification of

2 A B C 5 3 X 2 B A Fig. 1. Leveraging shared bandwidth easureent for otiizing arallel downloads (left) and overlaynetwork organization (right). Nueric labels reresent bottleneck bandwidth of ath segents in Mbs. the best set of routes for distributing content fro source A to destinations B and C hinges on our abilityto deterine the bottleneck bandwidth of the shared ortion of the AB and AC aths (as well as the end-to-end bottleneck bandwidth of ath BC). Secifically, it is better to use the AB + BC links to rovide 3Mbs to client B and 2Mbs to client C, rather than the AB + AC links for 1.5Mbs to each (assuing fair sharing). Paer Scoe, Contributions, and Organization: In this aer we roose an efficient end-to-end easureent technique that yields the bottleneck bandwidth of an arbitrary subath of a route between a set of end-oints. Bysubath, we ean a sequence of consecutive network links between anytwo identifiable nodes on that ath. A node i on a ath between a source s and a destination d is identifiable if it is ossible to coerce a acket injected at the source s to exit the ath at node i. One can achieve this by(1) targeting the acket to i (if i s IP address is known), or (2) forcing the acket to sto at i through the use of TTL field (if the hocount fro s to i is known), or (3) bytargeting the acket to a destination d,such that the aths fro s to d and fro s to d are known diverge at node i. Our ethods are uch less resource-intensive than existing ho-by-ho ethods for estiating bandwidth along a ath and uch ore general than end-to-end ethods for easuring bottleneck bandwidth. In articular, our ethod rovides the following advantages over existing techniques: (1) it has ore robust filtering, (2) it can estiate bandwidth on links not visible at end-to-end scoe, and (3) it can easure the bandwidth of fast links following slow links. The reainder of this aer is organized as follows. In Section II, we review existing literature. In Section III, we develo a basic robing toolkit, corising existing ethods and our new ideas. We coose several of these tools together in Section IV to easure the bottleneck bandwidth along arbitrarysubaths, as well as that shared bya set of flows. In Sections V and VI, we resent results of siulation and Internet validation exerients, showing the effectiveness of our constructions. For detailed roofs of the leas and corollaries in this aer, we refer the reader to [13]. 2 C II. RELATED WORK One wayof classifying bandwidth estiation techniques is based on whether theyconduct ho-by-ho [16], [25], [9], [23], [24] or end-to-end [21], [2], [5] easureents. Ho-byho techniques relyon increentallyrobing routers along a ath and tiing their ICMP relies, whereas end-to-end techniques base their bandwidth estiation on end-host relies only. The techniques we resent in this aer belong to this latter class, albeit at a granularityfiner than that achievable using existing end-to-end techniques. Another classification of bandwidth easureent techniques is based on whether they easure the bottleneck bandwidth [16], [25], [9], [21], [5], [23], [24] or the available bandwidth [2], [5], [1], [11] of a ath. The techniques we resent in this aer are aied at easuring bottleneck bandwidth. In classifying bandwidth easureent techniques, one can also look at the robing ethodologyeloyed naely, the nuber and sizes of ackets in a robe. Probe structures considered in the literature include: single acket robing [2], [16], [25], [9], acket bunch robing, eloying a grou of ackets sent back-to-back [5], unifor acket-air robing, eloying two back-to-back ackets of the sae size [21], [5], and non-unifor acket-air robing, eloying two backto-back ackets of different sizes [23], [24]. The robing techniques we will roose can be classified as acket-bunch robes with non-unifor acket sizes. Finally, one can classify bandwidth estiation techniques into active andassive techniques. Activetechniques, corising ost of the work in the literature, send robes for the sole urose of bandwidth easureent. Passive techniques rely on data ackets for robing as exelified in Lai and Baker s nettier tool [24], which uses a acket-air technique at the transort level to assivelyestiate bottleneck link bandwidth. The techniques we roose in this aer are alied actively. The robing constructions ost closelyrelated to ours are the acket-air [22] and tailgating [23] constructions. We discuss relevant technical roerties of these constructions, which we eloyand build uon in Section III. III. PROBING TOOLKIT In this section, we describe basic constructs of our robing sequences and corresonding terinology. With each robing construct, we describe its roerties and oint to its usefulness as a building block for the end-to-end easureent of subath bottleneck bandwidth, which we describe in Section IV. A. Basic Definitions For the uroses of this aer, a robe is a sequence of one or ore ackets transitted fro a coon origin. We say that anycontiguous subsequence of ackets within a robe are transitted back-to-back if there is no tie searationbetween transission of the individual ackets within the subsequence. As detailed in the related work section, back-to-back ackets have been widelyused in estiating the end-to-end bandwidth of a connection [2], [21], [5], [24], [23]. A ulti-destination robe is one in which the constituent ackets of the robe

3 do not all target the sae destination IP address. Multidestination robes have begun to see wider use as eulations of notional ulticast ackets anyof the sae end-to-end inferences that can be ade with ulticast ackets can be ade with ulti-destination unicast robes (albeit with added colexity) [], [14]. A unifor robe is one in which all of the constituent ackets are of the sae size; likewise, a nonunifor robe consists of ackets of different sizes. Finally, we saythat an individual acket is ho-liited if its TTL is set to an artificiallysall value so as not to reach the ostensible destination. Ho-liited ackets can be used to trigger an ICMP resonse fro an interediate router and in other ways that we describe later in the aer. Throughout the aer we use various robing techniques that relyon sending sequences of robes. The robing techniques differ in the nuber of ackets constituting a robe, the size and the ath traversed byeach robe acket. They also differ in the host collecting the robing resonses and the function used bythis host to erfor the required estiation. Each acket transitted within a robe is araeterized byits size s() in bytes and its final destination, D(). In the event that a acket is ho-liited, it has a third araeter, its axiu ho-count, h(). To denote a robe, we refer to each robe acket with a distinct lowercase letter, and reresent the sequential order in which theyare transitted fro the robing host bywriting the fro left to right. We denote interacket sacing with square braces. As an exale, [q][q][r] would denote transission of a air of identical two-acket robes followed bya single acket robe which has different characteristics; ackets in each of the twoacket robes are transitted back-to-back while robes are transitted saced-out. We use the ter interarrival tie of ackets and q at a link to denote the tie elased between the arrival of the last byte of and the arrival of the last byte of q at that link. Siilarily, we use the ter interdearture tie to denote the tie elased between the transission of the last byte of and the transission of the last byte of q. Bythese definitions, the interarrival tie of ackets and q at a given link is the sae as the interdearture tie of ackets and q at the receding link on the ath. B. Existing Probing Methods and Proerties One of the essential techniques that we build uon is the use of acket-airs, originallyused bykeshav [22], and subsequentlyrefined bycarter and Crovella [5], Paxson [2], [], [29] and Lai and Baker [24], to deterine bottleneck bandwidth on a network ath. Packet-air techniques relyon the following roerty, which holds under an assued network odel discussed later in this section. Lea 1: Packet-Pair Proerty. Consider a ath of n hysical links L 1,L 2,...,L n with base bandwidths b 1,b 2,...,b n resectively. If a robe of the for [] is injected at L 1, with D() =L n, then the interarrival tie of the two constituent ackets of this robe at L n is units of tie. s() in k b k An iortant corollaryto Lea 1 is that the bottleneck bandwidth across a set of links (in k b k ) can be estiated through easureent of acket interarrival ties and knowledge of acket sizes. Another closelyrelated technique also used in our constructions is acket-tailgating. This technique was introduced by Lai and Baker in [23] and evaluated within their nettier tool [24] to estiate the bottleneck bandwidth of all hysical links along a ath. The acket-tailgating technique hinges on the following roerty[24], which forulates the condition under which a non-unifor acket-air reains back-to-back over a sequence of hysical links. Lea 2: Tailgating Proerty. Consider a ath of n hysical links L 1,L 2,...,L n with base bandwidths b 1,b 2,...,b n resectively. If a robe of the for [q] is injected at L 1, with s() D() =D(q) =L n and if k n, s(q) b k+1 b k,then[q] will reain back-to-back along the entire ath. The two basic roerties selled out in Leas 1 and 2, as well as the constructions and analyses we resent later in this aer, are conditioned on a set of basic assutions about the network. These assutions, which are coon to ost robing studies (e.g., [2], [5], [23], [24]), are enuerated below: (1) Routers are store-and-forward and use FIFO queueing. (2) Probing hosts can inject back-to-back ackets into the network. (3) Host clock resolution is granular enough to enable accurate tiing easureents. (4) Analytic derivations assue an environent free fro cross-traffic. Assution 1 is needed to ensure that robe acket orderings are reserved. Assutions 2 and 3 are easilyenforcable using roer kernel caabilities. Assution 4, while necessaryfor analysis, is tyicallydiscarded in exeriental (siulation or ileentation) settings to establish the robustness of the constructions in realistic settings. C. Ensuring Back-to-Back Queuing at a Given Link We now describe the first of our constructions a construction that allows us to establish conditions that guarantee that all constituent ackets of a robe will queue u back-to-back at a given interediate link along a given ath. We do so through the use of a (tyically large) acer acket, which leads the robe into the network. Definition 1: A aced robe is a robex sentback-to-back behind a large acer acket of the for [X]. The acer acket has a destination D() at an interediate oint in the network. It leads the aced robe (its followers) u through this link as art of their tri and all the followers queue behind in the queue at router D(). At this oint, is droed. The following lea exresses the condition guaranteeing that robe reains back-to-back at the acer acket s final destination. Lea 3: Let L be a sequence of n hysical links L 1, L 2,...L n with caacitybandwidths b 1, b 2,...b n resectively. Also, let [q 1...q ] be a robe consisting of a set of aced

4 ackets which are injected back-to-back behind a acer acket, where D() =L k and D(q i )=L n. A sufficient condition for all follower ackets q i to queue behind at link L k is in 1 w ( s()+ w 1 j=1 s(q j) w j=1 s(q j) ) b k in i k b i IV. SUBPATH BANDWIDTH MEASUREMENT USING CARTOUCHE PROBING In this section we resent our ain robing structures, which enable us to achieve our stated goal of estiating the bottleneck bandwidth for an arbitraryath segent. In articular, given a ath consisting of a sequence of links L 1,...,L n with base bandwidths b 1,...,b n, our goal is to estiate the bottleneck bandwidth of an arbitrarysequence of links along that ath, i.e. estiate in i k j b k, for arbitrary i and j such that i j n. We use the shorthand b i,j to denote the bottleneck bandwidth in the interval between links i and j inclusive. We roceed byfirst deonstrating how to estiate the bottleneck bandwidth over a refixof a ath and overa suffix of a ath. Techniques for handling these two easier cases, which are often useful in their own right, will rovide insight as to how to aroach the general roble. A. Estiating Bandwidth Over a Prefix of the Path We begin byestiating the bottleneck bandwidth along a ath refix, i.e. inferring b 1,j. Since the acket-air technique described in Section III rovides an estiate for b 1,n, it follows that if b 1,j b j+1,n,thenb 1,j = b 1,n,givingusa solution. But when b 1,j >b j+1,n, the acket-air technique will end u estiating b j+1,n. The underlying reason for this is that acket-air techniques relyon the reservation of acket interarrival ties induced at the bottleneck. So while the acket-air roertygives an interarrival tie j at L j of j = s() b 1,j, the interarrival tie at L n is n = s() b 1,n.This suggests a otential solution, naelyreserving j unaltered to the end-host so that the end-host ayinfer b 1,j. Indeed, Lea 4 gives us the condition we ust satisfyto ensure such reservation. To do so, we need to generalize the acket-air construction (selled out in Lea 1) to yield an interarrival tie that is large enough to satisfythe constraints set by Lea 4. Lea 5: Consider a ath of n hysical links L 1, L 2,...L n with caacitybandwidths b 1, b 2,...b n resectively. If a robe of the for [{} (r+1) ] is injected at L 1 and destined towards L n then the interarrival tie ( j ) between the first and the last robe ackets at the end of everyhysical link D. Preserving Packet Interarrival Ties Over a Subath Our next construction allows us to tackle another challenge, which is to soe extent coleentaryto acing naely how we can ensure that the interarrival tie of two ackets at a secific link L i along a ath can be reserved as these ackets traverse additional links en route to their coon destination r s() L j,for1 j n, is in L n. The abilityto reserve acket sacing over the subath 1 k j b k. Based on the above lea, one can generalize the acketair technique byusing a robe structure consisting of a L i L i+1...l n enables us to easure such sacing reotely(at L n ). The following Lea establishes a necessarycondition sequence of ackets of the sae size, wherebyall ackets for the reservation of acket sacing over a sequence of links. excet the first and the last are droed at the end of L j. Lea 4: Preservation of Sacing. Consider a ath of Byincluding enough ackets in this sequence, the interarrival n hysical links L 1,L 2,...,L n with base bandwidths tie between the first and last ackets j at the end of L j can b 1,b 2,...,b n resectively. If a robe of the for [][] is be ade large enough to be reserved as these two ackets injected at L 1 with D() =L n and an interarrival tie of traverse links L j+1,...l n. Indeed, Lea 5 shows that if, then will be reserved over all links L i if and onlyif s() in we use r +1 ackets, then j would be r s() b 1 k n b k. 1,j. To satisfy the acket interarrival reservation condition, it turns out that Lea 4 shows that in order to avoid skewing the interarrival tie ( i ) through subath L i+1,...l n, the condition s() we need the condition r b1,j b i j+1,n to be satisfied. That is, we would need as anyrobe ackets as the ratio between b 1,j in (i+1) k n b k ust hold. and b j+1,n, which akes this aroach iractical. A better aroach to reserve the interarrival ties of robe ackets at an internal link L j as these ackets traverse subsequent links L j+1...l n is to use sall ackets as arkers that deliit easureent boundaries. Sall ackets have lower transission delays and thus are less suscetible to variation in their interarrival ties. Using this iroved idea, we are now readyto resent a basic robing structure that incororates all the features needed for an end-to-end inference of the bottleneck bandwidth of a ath refix. Definition 2: A cartouche [{q} r 1 ] over the set of links L 1,...L j,...,l n is a sequence of r +1heterogenous acket-airs in which s() s() =s(q), D() =D(q) = L j,andd() =L n. We refer to the first acket () in each air as the agnifier acket, the second acket ( or q) in each air as the arker acket, and r as the cartouche size. With the excetion of the first and last arker ackets, all ackets of the cartouche are targeted to L j, which is called the egress link of the cartouche. L n, the destination of the first and last arker ackets is called the target of the cartouche. Figure 2 shows the coosition and rogression of a cartouche of size r injected at link L i towards a target end-host L n = A with link L j as its egress link. Lea 6: Let L be a sequence of n hysical links L 1, L 2,...L n with base bandwidths b 1, b 2,...b n resectively. Given a cartouche of the for [{q} r 1 ] over L with L j as its egress link, let t f and t l be the tie that the final byte of the first and last arker ackets are received at link L j, resectively, then t l t f = r(s()+s()) in 1 k j b k.

5 L1 L2 L3 L4 L5 r+ 1 airs q q i q i i j tl tf q j j q 2(s()+s()) b1,3 tl tf A A A Fig. 2. A cartouche of size r consisting of back-to-back acket airs of the for [{q} r 1 ] injected at L i (left). Cartouche constituents are sread out over tie until theyarrive at L j (iddle), where onlythe first and last arkers continue on to target A (right) with an interarrival tie t l t f = r(s()+s()). in 1 k j b k Lea 6 rovides the ost iortant roertyof cartouche robing. It defines the interarrival tie for the first and last arker ackets over everyhysical link u to the cartouche egress link. Figure 3 illustrates this through a secific exale. Corollary 1: Let L be a sequence of n hysical links L 1, L 2,...L n with base bandwidths b 1, b 2,...b n, resectively. Given a cartouche of the for [{q} r 1 ] over L with L j as its egress link, let j be the interarrival tie between the arkers at the end of L j,then j will be reserved b 1,j b j+1,n over L j+1,...l n if and onlyif Corollary1 follows directlyfro Lea 6 and Lea 4 to derive a sufficient condition for the reservation of arkers interarrival ties uon exit fro the cartouche egress link L j and throughout the sequence L j+1,...l n. Note that with s() = bytes and s() =4 bytes, reservation holds r(s()+s()) s(). even when b1,j b j+1,n 3.5r; that is the interarrival tie between the first and last arker ackets holds even when b j+1,n is aroxiately4r ties saller than b 1,j,wherer is the cartouche size. Lea 6 and Corollary1 are all that are needed to rovide a solution to the roble of inferring the bottleneck bandwidth of a ath refix. Secifically, this is done by: (1) sizing a cartouche to satisfythe conditions of Corollary1, (2) setting the cartouche egress link to be L j, (3) injecting the cartouche ackets back-to-back at link L 1, and (4) using the interarrival tie of the first and last arker ackets at link L n as an estiate of their interarrival tie at link L j and using the relationshi given in Lea 6 to estiate b 1,j. B. Estiating Bandwidth over a Path Suffix We now turn our attention to the coleentaryroble of estiating the bottleneck bandwidth of a ath suffix naely, b i,n for an arbitrary i such that 1 < i n. For the case b 1,n the task is trivial since b i,n would be equal to b 1,n, which can be inferred using the acket-air technique. Fig. 3. Illustration of cartouche robing: A cartouche of size r =2is used to easure b 1,3. Here, the axial sacing between arker ackets is introduced at L 2 (which is the slowest link on the subath L 1 L 2 L 3 )and reserved until the arkers reach their target L 5. In the rest of this section, we concentrate on the case when <b 1,n and our aroach to estiating b i,n is to attet to identifythe bottleneck link over the subath of interest L i,...l i+1,...l n and estiate that link s bandwidth. We do so using cartouche trains. Definition 3: A cartouche train over a set of links L 1,...L i,... L j,...l n is a robe consisting of a sequence of l = j i+1 ossiblyoverlaing cartouches of size r each, whose egress links are L i,l i+1,...l j, resectively. Link L i is called the initial egress link of the cartouche train and link L j is called the final egress link of the cartouche train. The nuber l of ossiblyoverlaing cartouches in a cartouche train is called the length of the cartouche train. A cartouche train is coletelydefined byits length l, bythe size r of its constituent cartouches, byits initial (or final) egress link L i (or L j ), and byits target L n.for instance a cartouche train of length 2 and of size 3, whose final egress link is L 5 and whose target is L is given by [ 3 4 q 4 4 q q 5 5 q 5 5 ], whered( w )=D(q w )= L w,w =3, 4, 5, andd() =L. Consider a cartouche train of the for [ i 1 i i+1... n ], whose target as well as final egress link is L n.this cartouche train consists of (n i +1) overlaing cartouches (each of the for [ w 1 w ]). Clearly, D() =L n and D( w ) = L w, which eans that all arker ackets are targeted to L n whereas agnifier ackets are targeted to successive links starting at L i 1. Figure 4 shows the coosition and rogression of such a cartouche train of length l =2transitted fro L 1 and targeted to L 5 with L 4 (L 5 ) as its initial (final) egress link. Note that the structure of the cartouche train over the subath before L 4 (i.e., over L 1, L 2 and L 3 ) resebles that of a cartouche of size r =2. This eans that the interarrival tie between anyair of successive arker ackets just before the initial egress link L 4 is s()+s() b 1,3. Also, due to the waythe

6 Fig. 4. b 4,5. L1 L2 L3 L4 L5 s()+s() s() s() + b1,3 b4 b5 b3 s()+s() s() s() + Illustration of a cartouche train r =1and l =2used to estiate agnifier ackets exit the subath L 4 L 5 on successive links, each of these interarrival ties aybe udated at onlyone secific link. For instance, the interarrival tie at L 3 between arker ackets 1 and 2 can onlybe altered at L 4. Siilarily, the interarrival tie between arker ackets 2 and 3 can onlybe altered at L 5. This keyroertyis forulated in the following lea, which quantifies the interarrival tie ( k n ) between the two arker ackets iediatelyreceding and iediatelyfollowing a agnifier acket egressing at link L k. Lea 7: Let L be a sequence of n hysical links L 1,...L i,...l n with base bandwidths b 1,...b i,...b n resectively, such that b i,n. Let [ i 1 { w : n w i}] be a cartouche train of size r = 1 and length l = n i +1 over L with L i and L n as the initial and final egress links, resectively. If s(w) s() < b k b k 1, the interarrival tie between the two arker ackets iediatelyreceding and iediatelyfollowing the agnifier acket egressing at L k is given by k n = s(w)+s(),otherwise it is given by k n = s(w)+s() s() b k 1. + s(w) b k Corollary 2: Let L be a sequence of n hysical links L 1,...L i,...l n with base bandwidths b 1,...b i,...b n resectively, such that b i,n.let[ i 1 { w : n w i}] be a cartouche train of size r =1and length l = n i +1 over L with L i and L n as the initial and final egress links, resectively. If at least one link k of the subath L i,...l n satisfies the tailgating roertythen k n =ax i k n ( k n) would indicate that L k is the subath bottleneck link and s( b i,n = b k = w) s( w) k n s()+s() b + s() 1,i 1 b k 1 b1,3 b4. k n s(w)+s() Corollary2 sells out how cartouche trains of size r =1 and of length l = n i +1 could be used to estiate b i,n. The aroxiation in b i,n equation reflects ignoring the transission delayof one arker acket over L k 1. The sall size of the arker ackets akes this aroxiation racticallytolerable. Also, notice that if all k n are equal to s( w)+s() then this signifies that links L i,...l n are fast enough that none of the arker ackets is queued behind its agnifier acket. In this case b i,n cannot be inointed and we would then have to relyon an increental ho-by-ho technique such as athchar to estiate b i,n. C. Estiating Bandwidth over an Arbitrary Subath We are now readyto tackle our ain goal of estiating b i,j for arbitrary i, j satisfying 1 i<j n. First, we observe that using cartouche robing we can easure and b 1,j. If >b 1,j then b i,j = b 1,j. Otherwise, we need to find out a wayto easure b i,j. We do so using cartouche trains. Consider a cartouche train of length l = j i +1 targeted at L n with L i (L j ) as the initial (final) egress link. Clearly, the interarrival ties k j i k j between the arker ackets at L j can be used to estiate b i,j. Thus, the roble of easuring b i,j reduces to figuring out a wayof reserving the sacing k j as it arkers travel through links L j+1...l n. This can be readilyachieved using the results of Lea 4 which sets the conditions for the reservation of sacing over a subath. According to Lea 4, in order for the arker interarrival tie k s(q) j at link L j to be reserved, the condition in k j b j+1,n ust be satisfied. Using s()= bytes and s()=4 bytes, the k j sacing is reserved if b1,i 1 b j+1,n 3.5. Notice that this bound is siilar to the one we obtained for cartouches of size r =1. In order to reserve k j over L j+1,...l n even if b1,i 1 > 3.5 we need to agnify k j (as we did for b j+1,n cartouche robing in Section IV-A) using cartouche trains of size r>1. The following Lea sells this out. Lea : Let L be a sequence of n hysical links L 1,...L i,...l j,...l n with base bandwidths b 1,...b i,...b j,...b n resectively. Let <b i,j. Given a cartouche train of length l = j i +1,sizer, and with L i as its initial egress link and L j as its final egress link, if s(w) s() < b k b k 1 then k j = r(s( w)+s()),otherwise k j r(s(w)+s()) where i k j and i 1 w j. + s(w) s() b k, D. Suary of Measureent Procedure We conclude this section with a suaryof our rocedure for easuring the bottleneck bandwidth b i,j. Ste 1: Using a acket-air technique, we easure b 1,n.This will enables us to aroriatelysize the cartouches used in later stes (using the results of Corollary1). Ste 2: Using aroriately-sized cartouches, we easure and b 1,j using the relationshi established in Lea 6. If b 1,j then b i,j = b 1,j and we are done, otherwise we roceed to Ste 3. Ste 3: Using an aroriately-sized cartouche train of length l = j i+1, with initial egress link L i, we estiate b i,j using Lea. If such estiation is ossible, k j > r(s(w)+s()), then we are done, otherwise we conclude that our tool cannot accuratelyeasure b i,j. E. Shared Bottleneck Bandwidth We now extend our robing technique to enable the inference of the bottleneck bandwidth along the sequence of links shared byflows eanating fro the sae server S and destined to

7 two different clients A and B. Estiation of the bottleneck bandwidth over the shared links, is tantaount to couting the bottleneck bandwidth over a ath refix. However, we tyically will not have arioriknowledge of the length of the shared refix, nor the IP address of the branching oint. One otion is to use traceroute [17] on the ath fro S to A, and fro S to B, to deterine this issing inforation, but this ethod is error-rone and inelegant. A ore effective aroach is to use a cartouche robe, but instead of using ho-liited robe ackets as in the revious sections, we use ulti-destination robes instead: Magnifiers are destined to one of the clients, say A, and arkers are destined to the other client B. This way, agnifiers and arkers travel together only over the shared ath and client B can use r(s()+s()) as an estiate of the shared bandwidth, where is the interarrival tie between the arkers at B. V. IMPACT OF CROSS TRAFFIC In Section IV, we resented an analysis of our end-to-end bottleneck bandwidth estiation rocedures. As stated in Section III, the analysis assues an environent free fro cross-traffic, and it is under this idealistic assution that we rove the various roerties of cartouche robing. Clearly, in anyractical setting, cross-traffic cannot be ignored. In this section, we resent results fro extensive siulations intended to characterize the iact of cross traffic on cartouche robing. Our goal in this section is to identify traffic conditions under which cartouche robing is and is not effective. We also deonstrate scenarios in which structural characteristics of the network ath itself iact our results. Prelude: Recall that cartouche robing relies on the reservation of sacing between arker ackets to estiate the bottleneck bandwidth of a ath segent at endoints. In general, cross traffic ayiact arker sacing in two ossible ways: it ay cause arker coression, i.e.interacket sacing between a air of arkers is reduced in transit, or arker decoression, i.e. inter-acket sacing between a air of arkers is increased in transit. Both coression and decoression can result fro the arrival of cross-traffic at a link []. Marker coression is also ossible even in the absence of cross traffic. Recall our constructions in Section IV-A. There, we reserved sacing between the two arkers used to easure the bottleneck bandwidth b i,j as the arkers travel over subsequent links L j+1...l n.butifb j+1,n is sall enough to violate the condition stated in Corollary1, interacket sacing is not reserved. To avoid such coression, which is due entirelyto the static roerties of the ath, the size r of the cartouches eloyed ust be increased so as to satisfy the conditions of Corollary1. To reduce the effects of cross traffic, a bottleneck bandwidth easureent exerient ust be conducted reeatedlyand estiates that ayhave been affected byarker coression or decoression ust be identified and excluded using heuristics []. All of our ethods require the end-host A conducting the exerient to coile a histogra 1 of the frequencyof each estiate it obtains. One sile heuristic is to ick the bin with the largest frequency, i.e. the ode. But with a ore refined understanding of how arker coression or arker decoression affects our bottleneck bandwidth estiation in secific exerients, we develo better alternative heuristics to silyicking the ode. Fro equations in Section IV, one can see that arker coression results in overestiation of bottleneck bandwidth, whereas arker decoression results in underestiation of bottleneck bandwidth. Moreover, as we will subsequently deonstrate, arker coression due to cross traffic is ore revalent in exerients involving ath refixes, whereas arker decoression is ore revalent in in exerients involving ath suffixes and targeted ath segents. This suggests that icking the first odality of a histogra in refix exerients and icking the last odality of a histogra for suffix/subath exerients are better heuristics to use for filtering the effects of cross traffic. For lack of sace, we do not describe in this aer how we autoate ode detection and in the resented results we use the ode as our final estiate. Exeriental Setu: We used the Network Siulator (ns) [27] to siulate a ath L connecting two hosts A and B. L consists of 2 hysical links L 1, L 2,...L 2. Link bandwidth values b 1, b 2,...b 2 and link latencies d 1, d 2,...d 2 were hand-icked to illustrate various scenarios. Link cross-traffic was odeled bya set of aggregated Pareto ON/OFF UDP flows with 1. as the distribution shae araeter,.5 seconds as the average burst and idle ties, and 32Kb/sec as ean flow rate. Packet sizes of cross-traffic flows were also Pareto with 1. as the distribution shae araeter, 2 bytes as the ean, and bytes as the axiu. By varying the nuber of cross-traffic flows over each link we control the level of congestion on that link. Probe transission, tie easureents, logging and estiation functions were all erfored at host A. In all exerients resented in this section, we use the following settings in the construction of cartouches: s() = bytes, s() =s(q) =4bytes. Path Prefix Exerients: As described in Section IV-A, our technique for easuring the bottleneck bandwidth of ath refix relies on sending a sequence of [{q} r 1 ] cartouches fro source A, with L i as the egress link. Host A onitors the interarrival tie of the resonses to the arker ackets, and uses the forula r(s()+s()) to estiate b 1,i. Figure 5 shows the histogras we obtain (at host A) trying to infer b 1, using a sequence of [{q} r 1 ] cartouches of sizes r =1(left) r =2(iddle) and r =3 (right). The setu includes 16 cross-traffic flows over each hysical link in L, with an actual b 1, = Mbs and b 11,2 = 1 Mbs. Exaining the results in Figure 5, we observe that the r =2and r =3cases lead to a correct b 1, =estiate while the r =1case does not. Lea 1 In all our exerients, we use a fixed bin width of 1Mbs for the histogras.

8 Fig. 5. Histogras of estiated b 1, values after using a sequence of [{q} r 1 ] cartouches of diensionality r =1(left) r =2(iddle) and r =3(right). The setu includes 16 cross-traffic flows over each hysical link, actual b 1, =Mbs and b 11,2 =1Mbs. Estiated BW i =, r =2 i =, r =3 i =15, r =2 i =15, r =3 Estiated BW Estiated BW Estiated BW Actual BW Actual BW Actual BW Actual BW Fig. 6. Grids corresonding to ath refix bandwidth estiates b 1,i for different values of i (i =, 15) and using cartouche sizes of r =2, Fig. 7. Histogras of estiated b 15,2 (left) and b,2 (right) values after using a sequence of cartouche trains. The setu includes 16 cross-traffic flows over each hysical link, actual b 15,2 =Mbs and b,2 =Mbs. 6 exlains whythis haens. Using s() = bytes and s() =4bytes the condition b1,i b i+1,n 3.5r ust hold to deliver unerturbed arker interarrival ties to the endhost. Since b1, b 11,2 =, the condition holds onlyfor r =2and r =3, but not for r =1. Thus, when r =1, our cartouches were undersized, resulting in arker decoression and an underestiate of the value of b 1,i. The histogras corresonding to r =1, 2 and 3 show few instances of severe underestiation of b 1,, indicated byshort histogra bars for <b 1, < 3.5. These are exales of arker decoression due to burstycross traffic Notice that this decoression is ore ronounced for r =3.Thisis due to the fact that larger cartouches ilylonger arker interarrivals, which in turn leads to a higher robabilityof cross traffic bursts further searating the arkers. The histogras corresonding to r =2and r =3show instances of overestiation of b 1,. These are exales of arker coression due to burstycross traffic. Notice that our overestiates of b 1, were caed at 77Mbs and 115.5Mbs, for r =2and r = 3, resectively. Again, this is a direct consequence of the inequalityin Lea 6, which in effect secifies an uer bound on the axiu observable value for b 1,i using cartouches of size r. This bound (confired in Figure 5) is r b i+1,n 3.5, which is 3.5Mbs and 77Mbs and 115.5Mbs for r =1, 2 and 3 resectively. Figure 6 reresents 3-diensional grids lotting the estiated b 1,i values (the odes) when we varythe nuber of cross-traffic flows over each link and for different actual b 1,i values, while always keeing b i+1,2 =1Mbs. These lots are given for refix lengths of i =and i =15and for cartouche sizes r = 2 and r = 3. Bycarefullyinsecting the resulting grids, one can ake the following observations. First, the results confir that the axiu ossible b 1,i value we can estiate (given the sizes we icked for,, and q ackets) is 77Mbs and 115.5Mbs for r = 2 and r = 3 resectively. Second, the closer the ratio between b 1,i and b i+1,2 to the value of r, the ore suscetible the cartouche ackets are to interarrival tie alterations due to cross-traffic. This advocates using larger values of r. Third, incorrect estiates due to arker coression are uch ore significant than those resulting fro arker decoression. Finally, refix length (the value of i) does not see to ose anysignificant iact on the accuracyof our techniques. Path Suffix Exerients: As described in Section IV-B, our technique to easure ath suffix bottleneck bandwidth relies on sending a cartouche train of size r =1and of length l equal

9 Fig.. Histogras of estiated b 5, (left) and b 5,15 (right) values after using a sequence of cartouche trains of diension r =2. The setu includes 16 cross-traffic flows over each hysical link, actual b 5, =Mbs and b 5,15 =Mbs. to the suffix length. As before, host A onitors the interarrival tie of the resonses to arkers, then uses the largest between anyair of successive arker ackets, k n,and to estiate b i,n. Estiated BW Actual BW Estiated BW Actual BW Fig. 9. Grids corresonding to ath suffix bandwidth estiates b i,2 for i =15(left) and i =(right). Estiated BW 1 Estiated BW Actual BW Actual BW Estiated BW 1 Estiated BW Actual BW Actual BW Fig.. Grids corresonding to arbitrarysub-ath bandwidth estiates b i,j for i=5, j =(to), j =15(botto), and using r =2(left), r =3(right). Figure 7 shows the histogras we obtain when estiating b 15,2 (left) and b,2 (right) after using a sequence of cartouche trains of length l =5and l =, resectively. The setu includes 16 cross-traffic flows over each hysical link in L, with actual b 15,2 =Mbs and b,2 =Mbs, with the overall ath bottleneck b 1,2 being set to 1Mbs. Clearly, in both cases, the ode reresents the correct bandwidth estiate. The figure also shows that incorrect estiates due to arker decoression are non-negligible, whereas those due to arker coression are virtuallynon-existent. In addition, incorrect estiates due to arker decoression are ore revalent for longer cartouche trains. This is in shar contrast to our results for refix bandwidth estiation, in which we have shown that incorrect estiates due to arker coression are ore revalent, and in which the refix length does not laya significant factor. Figure 9 shows grids fro exerients estiating b 15,2 (left) and b,2 (right) under a range of cross-traffic and actual bandwidth conditions. These grids confir that as cross traffic increases, arker decoression becoes ore significant, and that this is ore revalent for long subaths. Arbitrary Path Segents Exerients: As described in Section IV-C, our technique to easure arbitrarysubath bandwidth relies on sending a sequence of aroriatelysized cartouche trains fro source A. HostA onitors the interarrival tie of the resonses to arkers, thenuses the largest between anyair of successive arker ackets, k n,and to estiate b i,j. Figure shows the histogras we obtain trying to infer b 5, (left) and b 5,15 (right) after using a sequence of cartouche trains of size r =2. The setu includes 16 cross-traffic flows over each hysical link in L, with actual b 5, =, b 1,4 =1, b 11,2 =5 (left) and b 5,15 =, b 1,4 =1, b 16,2 =5 (right). The histogras show that both b 5, and b 5,15 are correctlyestiated. It also shows that understiates (due to arker decoression) are quite revalent. As we observed in our suffix exerients, which uses the sae cartouche train construction, arker decoression is ore ronounced when the subath length l is longer. Figure shows the grids resulting fro exerients to estiate b 5, and b 5,15. These confir that it is desirable to use cartouche trains of the sallest ossible size r and suggest that for long subaths, it is not advisable to silyuse long cartouche trains. A better divide-and-conquer alternative ay be to artition a long subath into segents, to which shorter cartouche trains could be alied. Postlude: We conclude this section with a suaryof our findings regarding the suscetibilityof our constructions to cross traffic. Secifically, we observe that: (1) Marker coression and hence overestiation of b 1,i resents the ost significant hurdle for bottleneck bandwidth estiation of ath refix using cartouches. (2) Marker decoression and hence

10 underestiation of b j,n resents the ost significant hurdle for bottleneck bandwidth estiation of ath suffix (or arbitrary ath segents) using cartouche trains. This difficultycan be alleviated through the use of the sallest cartouches that would satisfythe structural constraints iosed bylea 6, and through a divide-and-conquer aroach. VI. INTERNET MEASUREMENT EXPERIMENTS In this section, we coare cartouche robing erforance and efficiencyto those of existing ho-by-ho techniques (naely char and nettier). We also resent Internet easureent results intended as a roof of concet of the efficacyof the cartouche robing technique. A. Coarison to char and nettier Both char[25] and nettier [24] are ho-by-ho techniques which eans that in order to estiate the bottleneck bandwidth along a ath segent theyneed to increentally run tests to estiate the bandwidth of everyho in the segent. Also, these techniques are cuulative, in the sense that estiates for anyho deend on estiates ade for revious hos in the segent. On the other hand, Cartouche robing directlytargets the bottleneck bandwidth of the segent avoiding the roagation of erroneous results. In coaring against char and nettier, we consider two easures: (1) Tie efficiency: which reflects the tie it takes to return a bandwidth estiate, and (2) Byte efficiency: which is a easure of the aount of traffic injected into the network to get an estiate. In ters of tie efficiency, cartouche robing is ore efficient since it does not have to orchestrate a round of robing for everyho in a segent before returning the final estiate. In ters of byte efficiency, cartouche robing is ore efficient than char, which uses linear regression in its statistical analysis for every ho, thus requiring it to inject quite a bit of extra traffic. 2 Cartouche robing byte efficiency is coarable to that of nettier. 3 Careful insection of the cartouche constructions reveal that the byte requireents of a sequence of cartouche robes is k(r +1)(s()+s()) bytes, where k is the nuber of robes in the sequence, and the byte requireents of a sequence of cartouche trains is k(lr +1)(s()+s()) bytes, where l is the ath segent length. B. Exeriental Setu Two Internet aths (connecting our laboratoryat Boston Universityto Georgia Tech in the US and Ecole Norale Suerieure in France) were handicked to deonstrate the different scenarios that we discussed in section V. Figure 11 shows that these two aths share the first three hos and then diverge. The labels on the figure reflect the ariori-known bottleneck bandwidth of links in our own laboratory, links of Internet2 hos ublished byabilene [26], and links on the far end of the aths obtained through ersonal contacts. 2 In fact, char default settings inject ore than 35MB in the network er ho, whereas cartouche robing tyically injects uch less than 1MB. 3 A cartouche of size r has as anybytes as r +1nettier tailgated airs and a cartouche train of length l and size r has as anybytes as in (r +1)l 1 tailgated airs. 522 wing.bu.edu Fig. 11. The Internet aths used in our exerients. Labels are arioriknown link bandwidths. All units are in Mbs. To evaluate our echaniss, we incororated our cartouche robing functionalityinto PERISCOPE[15]: A linux API roviding a flexible interface to define arbitraryrobing structures. This allows the cartouche robing transissions to be orchestrated fro the Linux kernel, ensuring back-to-back transissions and accurate acket tiestaing. We installed PERISCOPE in our laboratoryon a Pentiu IV rocessor, running RedHat Linux over a Mbs LAN. In all exerients, we use agnifier robe ackets of size bytes and arker ackets of size 6 bytes. Exerients were conducted once er second until we obtain valid robe relies acket reordering and/or acket losses invalidate a rely. Ho Actual BW Cartouche Pchar ? ? TABLE I RESULTS FROM CARTOUCHE PROBING AND char FOR ariori-known LINKS OF THE PATH TO GEORGIA TECH. C. Results Figure 12 shows the histogras we obtained when using cartouche robing to estiate b 1, b 1,3, b 5 for the ath to Georgia Tech and the bandwidth of the closest link for Ecole Norale. The figure shows that the estiated values are close to the ariori-known bandwidth values. Table I coares cartouche robing and char estiates over the arioriknown links of the ath to Georgia Tech. 4 Note that char does not return an estiate for soe links. Aart fro that, the estiates are coarable. While the Cartouche robing estiate for the high-seed OC- link on ho 6 is not accurate, char does not return an estiate for that link. VII. CONCLUSION We have described an end-to-end robing technique caable of inferring the bottleneck bandwidth along an arbitraryath segents in the network, or across the ortion of a ath 4 nettier docuentation does not secifyhow to use the tool to estiate the bandwidth of everyho along a ath; however, the techninque details can be found in [24].

11 Fig. 12. Histogras of estiated bottleneck bandwidth along different segents of the aths to Georgia Tech, and Ecole Norale Suerieure. The histogras (fro left to right) reresent bandwidth estiates for Georgia Tech b 1, b 1,3, b 5 and for the hoe network link of Ecole Norale Suerieure. Dotted lines reresent ariori-known bandwidth values. shared bya set of connections, and have resented results of extensive siulations and reliinaryinternet easureents of our techniques. The constructions we advocate are built in art uon acket-air techniques, and the inferences we draw are accurate under a varietyof siulated network conditions and are robust to network effects such as the resence of bursty cross-traffic. While the end-to-end robing constructions we roosed in this aer are geared towards a secific roble, we believe that there will be increasing interest in techniques which conduct reote robes of network-internal characteristics, including those across arbitrarysubaths or regions of the network. We anticiate that lightweight echaniss to facilitate easureent of etrics of interest, such as bottleneck bandwidth, will see increasing use as eerging networkaware alications otiize their erforance via intelligent utilization of network resources. Acknowledgents: This work was done while Khaled Harfoush was at the Couter Science Deartent of Boston University, and was artially suorted by NSF grants ANI , CAREER ANI-93296, ANI-959, EIA- 2267, and ITR ANI REFERENCES [1] D. Andersen, H. Balakrishnan, M. F. Kaashoek, and R. Morris. Resilient OverlayNetworks. In Proceedings of SOSP 21, Banff, Canada, October 21. [2] J. C. Bolot. End-to-end Packet Delayand Loss Behavior in the Internet. In SIGCOMM 93, ages 29 29, Seteber [3] J. Byers, J. Considine, M. Mitzenacher, and S. Rost. Infored Content DeliveryAcross Adative OverlayNetworks. In Proceedings of ACMSIGCOMM 2, Pittsburgh, PA, August 22. [4] J. Byers, M. Luby, and M. Mitzenacher. Accessing Multile Mirror Sites in Parallel: Using Tornado Codes to Seed U Downloads. In Proceedings of IEEE INFOCOM 99, ages 275 3, March [5] R. L. Carter and M. E. Crovella. Measuring bottleneck link seed in acket switched networks. Perforance Evaluation, 27&2:297 31, [6] Y.-H. Chu, S. Rao, and H. Zhang. A Case for End-Syste Multicast. In ACMSIGMETRICS, Santa Clara, CA, June 2. [7] M. E. Crovella, R. Frangioso, and M. Harchol-Balter. Connection Scheduling in Web Servers. In Proceedings of 1999 USENIX Syosiu on Internet Technologies and Systes (USITS 99), October [] C. Dovrolis, P. Raanathan, and D. Moore. What Do Packet Disersion Techniques Measure? In INFOCOM 1, Anchorage AK, Aril 21. [9] A. Downey. Using Pathchar to Estiate Internet Link Characteristics. In SIGCOMM 99, Boston, MA, August [] N. Duffield, F. Lo Presti, V. Paxson, and D. Towsley. Inferring Link Loss Using Stried Unicast Probes. In IEEE INFOCOM21, Aril 21. [11] M. Goyal, R. Guerin, and R. Rajan. Predicting TCP Throughut Fro Non-invasive Network Saling. In Proceedings of IEEE INFOCOM 2, June 22. [12] K. Hanna, N. Natarajan, and B. Levine. Evaluation of a Novel Two-Ste Server Selection Metric. In 9th International Conference on Network Protocols (ICNP), Riverside, CA, Noveber 21. [13] K. Harfoush. A Fraework and Toolkit for the Effective Measureent and Reresentation of Internet Internal Characteristics. PhD thesis, Boston University, June 22. [14] K. Harfoush, A. Bestavros, and J. Byers. Robust Identification of Shared Losses Using End-to-End Unicast Probes. In th International Conference on Network Protocols (ICNP), Noveber 2. [15] K. Harfoush, A. Bestavros, and J. Byers. PeriScoe: An Active Probing API. In Proc. of the 22 Passive and Active Measureent Worksho, PAM 2, Fort Collins, Colorado, March 22. [16] V. Jacobson. Pathchar: A Tool to Infer Characteristics of Internet Paths. ft://ft.ee.lbl.gov/athchar. [17] V. Jacobson. traceroute. ft://ft.ee.lbl.gov/traceroute.tar.z, 199. [1] M. Jain and C. Dovrolis. End-to-end Available Bandwidth: Measureent Methodology, Dynaics, and Relation with TCP Throughut. In Proceedings of ACMSIGCOMM 2, August 22. [19] J. Jannotti, D. Gifford, K. Johnson, M. F. Kaashoek, and J. O Toole Jr. Overcast: Reliable Multicasting with an OverlayNetwork. In Proceedings of OSDI 2, San Diego, CA, October 2. [2] J. Kangasharju, J. Roberts, and K. W. Ross. Object Relication Strategies in Content Distribution Networks. In Proceedings of WCW 1: Web Caching and Content Distribution Worksho, Boston, MA, June 21. [21] S. Keshav. A Control-Theoretic Aroach to Flow Control. In SIGCOMM 91, Seteber [22] S. Keshav. Congestion Control in Couter Networks. PhD thesis, Universityof California at Berkeley, Seteber [23] K. Lai and M. Baker. Measuring Link Bandwidths Using a Deterinistic Model of Packet Delay. In SIGCOMM, Stockhol, August 2. [24] K. Lai and M. Baker. Nettier: A tool for Measuring Bottleneck Link Bandwidth. In Proceedings of USITS 1, March 21. [25] B. Mah. char. htt:// bah/software/char, 2. [26] The Abilene Network Logical Ma. htt:// January 22. [27] ns: Network Siulator. htt://www-ash.cs.berkeley.edu/ns/ns.htl. [2] V. Paxson. End-to-end Routing Behavior in the Internet. In SIGCOMM 96, Stanford, California, August [29] V. Paxson. End-to-end Internet Packet Dynaics. In SIGCOMM, [] V. Paxson. Measureents and Analysis of End-to-end Internet Dynaics. PhD thesis, U.C. Berkeleyand Lawrence Berkeley Laboratory, [31] P. Radoslavov, R. Govindan, and D. Estrin. Toology-Infored Internet Relica Placeent. In Proceedings of WCW 1: Web Caching and Content Distribution Worksho, Boston, MA, June 21. [32] P. Rodriguez, A. Kiral, and E. Biersack. Parallel-access for Mirror Sites in the Internet. In Proceedings of IEEE INFOCOM, March 2. [33] I. Stoica, R. Morris, D. Karger, F. Kaashoek, and H. Balakrishnan. Chord: A Scalable Peer-to-Peer Looku Service for Internet Alications. In Proceedings of ACMSIGCOMM 1, San Diego, CA, August 21.