Packet-Pair Bandwidth Estimation: Stochastic Analysis of a Single Congested Node

Transcription

1 Packet-Pair Badwidth Estimatio: Stochastic Aalysis of a Sigle ogested Node 1 Seog-ryog Kag, 2 Xiliag Liu, 1 Mi Dai, ad 1 Dmitri Loguiov 1 Texas A&M Uiversity, ollege Statio, TX 77843, USA {skag, dmitri}@cs.tamu.edu, mi@ee.tamu.edu 2 ity Uiversity of New York, New York, NY 116, USA xliu@gc.cuy.edu Abstract I this paper, we examie the problem of estimatig the capacity of bottleeck liks ad available badwidth of edto-ed paths uder o-egligible cross-traffic coditios. We preset a simple stochastic aalysis of the problem i the cotext of a sigle cogested ode ad derive several results that allow the costructio of asymptotically-accurate badwidth estimators. We first develop a geeric queuig model of a Iteret router ad solve the estimatio problem assumig reewal cross-traffic at the bottleeck lik. Noticig that the reewal assumptio o Iteret flows is too strog, we ivestigate a alterative filterig solutio that asymptotically coverges to the desired values of the bottleeck capacity ad available badwidth uder arbitrary (icludig o-statioary) cross-traffic. This is oe of the first methods that simultaeously estimates both types of badwidth ad is provably accurate. We fiish the paper by discussig the impossibility of a similar estimator for paths with two or more cogested routers. I. INTRODUTION Badwidth estimatio has recetly become a importat ad mature area of Iteret research [1], [2], [4], [5], [6], [7], [1], [11], [12], [14], [15], [16], [18], [19], [2], [21], [22], [23], [24], [25], [26], [27]. A typical goal of these studies is to uderstad the characteristics of Iteret paths ad those of cross-traffic through a variety of ed-to-ed ad/or router-assisted measuremets. The proposed techiques usually fall ito two categories those that estimate the bottleeck badwidth [4], [5], [6], [12], [13] ad those that deal with the available badwidth [11], [19], [24], [26]. Recall that the former badwidth metric refers to the capacity of the slowest lik of the path, while the latter is geerally defied as the smallest average uused badwidth amog the routers of a ed-to-ed path. The majority of existig bottleeck-badwidth estimatio methods are justified assumig o cross-traffic alog the path ad are usually examied i simulatios/experimets to show that they ca work uder realistic etwork coditios [4], [5], [6], [7], [12]. With available badwidth estimatio, crosstraffic is essetial ad is usually take ito accout i the aalysis; however, such aalysis predomiatly assumes a fluid model for all flows ad implicitly requires that such models be accurate i o-fluid cases. Simulatios/experimets are agai used to verify that the proposed methods are capable of dealig with bursty coditios of real Iteret cross-traffic [11], [19], [24], [26]. To uderstad some of the reasos for the lack of stochastic modelig i this field, this paper studies a sigle-ode badwidth measuremet problem ad derives a closed-form estimator for both capacity ad available badwidth A = r, where r is the average rate of cross-traffic at the lik. For a arbitrary cross-traffic arrival process r(t), we defie r as the asymptotic time-average of r(t) ad assume that it exists ad is fiite: 1 t r = lim r(u)du <. (1) t t Notice that the existece of (1) does ot require statioarity of cross-traffic, or does it impose ay restrictios o the arrival of idividual packets to the router. While other defiitios of available badwidth A ad the average cross-traffic rate r are possible, we fid that (1) serves our purpose well as it provides a clea ad mathematically tractable metric. The first half of the paper deals with badwidth estimatio uder i.i.d. reewal cross-traffic ad the aalysis of packetpair/trai methods. We first show that uder certai coditios ad eve the simplest i.i.d. cross-traffic, histogram-based methods commoly used i prior work (e.g., [5]) ca be misled ito producig iaccurate estimates of. We overcome this limitatio by developig a asymptotically accurate model for ; however, sice this approach evetually requires ergodicity of cross-traffic, we later build aother model that imposes more restrictio o the samplig process (usig PASTA priciples suggested i [26]), but allows cross-traffic to exhibit arbitrary characteristics. Ulike previous studies [26], we prove that the correspodig PASTA-based estimators coverge to the correct values ad show that they ca be used to simultaeously measure capacity ad available badwidth A. To our kowledge, this is the first estimator that measures both ad A without cogestig the lik, assumes o-egligible, o-fluid cross-traffic i the derivatios, ad applies to o-statioary r(t). Note that while this estimator ca measure A over multiple liks, its iheret purpose is ot to become aother measuremet tool or to work over multi-ode paths, but rather to uderstad the associated stochastic models ad dissemiate the kowledge obtaied i the process of writig this paper. Proceedigs of the 12th IEEE Iteratioal oferece o Network Protocols (INP 4) /4 $ 2. IEEE

2 We coclude the paper by discussig the impossibility of buildig a asymptotically optimal estimator of both ad A for a two-ode case. While estimatio of A may be possible for multi-ode paths, our results suggest that provably-coverget estimators of do ot exist i the cotext of ed-to-ed measuremet. Thus, the problem of derivig a provably accurate estimator for a two-ode case with arbitrary cross-traffic remais ope; however, we hope that our iitial work i this directio will stimulate additioal research ad prompt others to prove/disprove this cojecture. II. STOHASTI QUEUING MODEL I this sectio, we build a simple model of a router that itroduces radom delay oise ito the measuremets of the receiver ad use it to study the performace of packet-pair badwidth-samplig techiques. Note that we depart from the commo assumptio of egligible ad/or fluid cross-traffic ad specifically aim to uderstad the effects of radom queuig delays o the badwidth samplig process. First cosider a uloaded router with o cross-traffic. The packet-pair mechaism is based o a observatio that if two packets arrive at the bottleeck lik with spacig x smaller tha the trasmissio delay of the secod packet over the lik, their spacig after the lik will be exactly. I practice, however, packets from other flows ofte queue betwee the two probe packets ad icrease their spacig o the exit from the bottleeck lik to be larger tha. Assume that packets of the probe traffic arrive to the bottleeck router at times a 1, a 2,...ad that iter-arrival times a a 1 are give by a radom process x determied by the server s iitial spacig. Further assume that the bottleeck ode delays arrivig packets by addig a radom processig time ω to each received packet. For the remaider of the paper, we use costat 1 packet size q for the probig flow ad arbitrarily-varyig packet sizes for cross-traffic. Furthermore, there is o strict requiremet o the iitial spacig x as log as the modelig assumptios below are satisfied. This meas that both isolated packet pairs or bursty packet trais ca be used to probe the path. Let the trasmissio delay of each applicatio packet through the bottleeck lik be q/ =, where is the trasmissio capacity of the lik. Uder these assumptios, packet departure times d are expressed by the followig recurrece 2 : { a1 + ω 1 + =1 d = max(a,d 1 )+ω + 2. (2) I this formula, the depedece of d o departure time d 1 is a cosequece of FIFO queuig (i.e., packet caot depart before packet 1 is fully trasmitted). Furthermore, packet caot start trasmissio util it has fully arrived (i.e., time a ). The value of the oise term ω is proportioal to the umber of packets geerated by cross-traffic ad queued 1 Methods that vary the probig packet size also exist (e.g., [6], [1]). 2 Times d specify whe the last bit of the packet leaves the router. Similarly, times a specify whe the last bit of the packet is fully received ad the packet is ready for queuig. arrival departure Fig. 1. Departure delays itroduced by the ode. time i frot of packet. The fial metric of iterest is iterdeparture delay y = d d 1 as the packets leave the router. The various variables ad packet arrival/departure are schematically show i Figure 1. Eve though the model i (2) appears to be simple, it leads to fairly complicated distributios for y if we make o prior assumptios about cross-traffic. We ext examie several special cases ad derive importat asymptotic results about process y. A. Packet-Pair Aalysis III. RENEWAL ROSS-TRAFFI We start our aalysis with a rather commo assumptio i queuig theory that cross-traffic arrives ito the bottleeck lik accordig to some reewal process (i.e., delays betwee crosstraffic packets are i.i.d. radom variables). I what follows i the ext few subsectios, we show that modelig of this directio requires statioarity (more specifically, ergodicity) of cross-traffic. However, sice either the i.i.d. assumptio or statioarity holds for regular Iteret traffic, we the apply a differet samplig methodology ad a differet aalytical directio to derive a provably robust estimator of capacity ad average cross-traffic rate r. The goal of bottleeck badwidth samplig techiques is to queue probe packets directly behid each other at the bottleeck lik ad esure that spacig y o the exit from the router is. I practice, however, this is rarely possible whe the rate of cross-traffic is o-egligible. This does preset certai difficulties to the estimatio process; however, assumig a sigle cogested ode, the problem is asymptotically tractable give certai mild coditios o cross-traffic. We preset these results below. To geerate measuremets of the bottleeck capacity, it is commoly derived that the server must sed its packets with iitial spacig o more tha (i.e., o slower tha ). This is true for uloaded liks; however, whe cross-traffic is preset, the probes may be set arbitrarily slower as log as each packet i arrives to the router before the departure time of packet i 1. This coditio traslates ito a d 1 ad Proceedigs of the 12th IEEE Iteratioal oferece o Network Protocols (INP 4) /4 $ 2. IEEE

3 (2) expads to: d = { a1 + ω 1 + =1 d 1 + ω + 2. (3) From (3), packet iter-departure times y after the bottleeck router are give by: y = d d 1 = +ω, 2. (4) Notice that radom process ω is defied by the arrival patter of cross-traffic ad also by its packet-size distributio. Sice this process is a key factor that determies the distributio of sampled delays y, we ext focus o aalyzig its properties. Assume that iter-packet delays of cross-traffic are give by idepedet radom variables {X i } ad the actual arrivals occur at times X 1,X 1 + X 2,... Thus, the arrival patter of cross-traffic defies a reewal process M(t), which is the umber of packet arrivals i the iterval [,t]. Usig commo covetio, further assume that the mea iter-arrival delay E[X i ] is give by 1/λ, where λ = r is the mea arrival rate of cross-traffic i packets per secod. To allow radom packet sizes, assume that {S j },j = 1, 2,...are idepedet radom variables modelig the size of packets i cross-traffic. We further assume that the bottleeck lik is probed by a sequece of packet-pairs, i which the delay betwee the packets withi each pair is small (so as to keep their rate higher tha ) ad the delay betwee the pairs is high (so as ot to cogest the lik). Uder these assumptios, the amout of cross-traffic data received by the bottleeck lik betwee probe packets 2m 1 ad 2m (i.e., i the iterval (a 2m 1,a 2m ]) is give by a cumulative reward process: v = M(a ) M(a 1) j=1 S j, =2m, (5) where represets the sequece umber of the secod packet i each pair. For ow, we assume a geeral (ot ecessarily equilibrium) process M(t) ad re-write (4) as: M(a ) M(a 1 ) j=1 y = + v = +, =2m. (6) Sice classical reewal theory is mostly cocered with limitig distributios, y by itself does ot lead to ay tractable results because the observatio period of the process captured by each of y is very small. Defie a time-average process W to be the average of {y i } up to time : W = 1 y i. (7) The, we have the followig result. laim 1: Assumig ergodic cross-traffic, time-average process W coverges to: lim W = + λe[x ]E[S j ] = +E[ω ], (8) S j Sd1 1 mb/s 1 mb/s 1.5 mb/s R 1 R 2 Sd2 1 mb/s Fig mb/s Rcv1 Rcv2 Sigle-lik simulatio topology. where E[x ] is the mea iter-probe delay of packet-pairs. Proof: Process W samples a much larger spa of M(t) ad has a limitig distributio as we demostrate below. Applyig Wald s equatio to (6) [28]: E [y ]= + E[M(a ) M(a 1 )]E[S j ]. (9) The last result holds sice M(t) ad S j are idepedet ad M(a ) M(a 1 ) is a stoppig time for sequece {S j }. Equatio (9) ca be further simplified by oticig that E[M(t)] is the reewal fuctio m(t): E [y ]= + (m(a ) m(a 1 ))E[S j ]. (1) Assumig statioary cross-traffic, (1) expads to [28]: E [y ]= + λe[x ]E[S j ]. (11) Fially, assumig ergodicity of cross-traffic (which implies that of process y ), we ca obtai (11) usig a large umber of packet pairs as the limit of W i (7) as. Notice that the secod term i (8) is strictly positive uder the assumptios of this paper. This leads to a iterestig observatio that the filterig problem we are facig is quite challegig sice the sampled process y represets sigal corrupted by a o-zero-mea oise ω. This is a drastic departure from the classical filter theory, which mostly deals with zero-mea additive oise. It is also iterestig that the oly way to make the oise zero-mea is to either sed probe traffic with E[x ]=(i.e., ifiitely fast) or to have o crosstraffic at the bottleeck lik (i.e., λ = r =). The former case is impossible sice x is always positive ad the latter case is a simplificatio that we explicitly wat to avoid i this work. We will preset our aalysis of packet-trai probig shortly, but i the mea time, discuss several simulatios to provide a ituitive explaatio of the results obtaied so far. B. Simulatios Before we proceed to estimatio of, let us explai several observatios made i previous work ad put them i the cotext of our model i (6) ad (8). For the simulatios i this sectio, we used the s2 etwork simulator with the topology show i Fig. 2. I the figure, the source of probe packets Sd1 seds its data to receiver Rcv1 across two routers R 1 ad R 2. The speed of all access liks is 1 mb/s (delay 5 ms), Proceedigs of the 12th IEEE Iteratioal oferece o Network Protocols (INP 4) /4 $ 2. IEEE

4 frequecy.4.3 frequecy.2 frequecy.1 frequecy values of y() (ms) values of y() (ms) values of y() (ms) values of y() (ms) (a) r =1mb/s (b) r =1.3 mb/s (a) r = 75 kb/s (b) r =1mb/s The histogram of measured iter-arrival times y uder BR cross- Fig. 3. traffic. The histogram of measured iter-arrival times y uder TP cross- Fig. 4. traffic. while the bottleeck lik R 1 R 2 has capacity =1.5 mb/s ad 2 ms delay. Note that there are five cross-traffic sources attached to Sd2. We ext discuss simulatio results obtaied uder UDP cross-traffic. I the first case, we iitialize all five sources i Sd2 to be BR streams, each trasmittig at 2 kb/s ( r =1mb/s total cross-traffic). Each BR flow starts with a radom iitial delay to prevet sychroizatio with other flows ad uses 5-byte packets. The probe flow at Sd1 seds its data at a average rate of 5 kb/s for the probig duratio, which results i 1% utilizatio of the bottleeck lik. I the secod case, we lower packet size of cross-traffic to 3 bytes ad icrease its total rate to 1.3 mb/s to demostrate more challegig scearios whe there is packet loss at the bottleeck. These simulatio results are summarized i Fig. 3, which illustrates the distributio of the measured samples y based o each pair of packets set with spacig x (the results exclude packet pairs that experieced loss). Give capacity =1.5 mb/s ad packet size q =1, 5 bytes, the value of is 8 ms. Fig. 3 shows that oe of the samples are located at the correct value of 8 ms ad that the mea of the sampled sigal (i.e., W ) has shifted to 11.7 ms for the first case ad 14.5 ms for the secod oe. Next, we employ TP cross-traffic, which is geerated by five FTP sources attached to Sd2. The TP flows use differet packet sizes of 54, 64, 84, 1,4, ad 1,24 bytes, respectively. The histogram of y for this case is show i Fig. 4 for two differet average cross-traffic rates r = 75 kb/s ad r =1mb/s. As see i the figure, eve though some of the samples are located at 8 ms, the majority of the mass i the histogram (icludig the peak modes) are located at the values much higher tha 8 ms. Recall from (8) that W of the measured sigal teds to +E[ω ]. Uder BR cross-traffic, we ca estimate the mea of the oise E[ω ] to be approximately =3.7 ms i the first case ad =6.5 ms i the secod oe. The two aive estimates of based o W are = q/w =1, 25 kb/s ad = 827 kb/s, respectively. Likewise, for the TP case, the measured averages (i.e., 12.2 ad 13.3 ms each) of samples y lead to icorrect aive estimates = 983 kb/s ad = 92 kb/s, respectively. I order to uderstad how ad the value of W evolve, we ru aother simulatio uder 1 mb/s total TP cross-traffic ad plot the evolutios of the absolute error ad that of W i Fig. 5. As Fig. 5(a) shows, the absolute error betwee ad coverges to a certai value after 5, samples y, providig a rather poor estimate 1, 1 kb/s. Fig. 5(b) illustrates that W i fact coverges to +E[ω ], where the mea of the oise is W =3.9 ms. Previous work [1], [4], [5], [15], [22] focused o idetifyig the peaks (modes) i the histogram of the collected badwidth samples ad used these peaks to estimate the badwidth; however, as Figs. 3 ad 4 show, this ca be misleadig whe the distributio of the oise is ot kow a-priori. For example, the tallest peak o the right side of Fig. 3 is located at 13 ms ( = 923 kb/s), which is oly a slightly better estimate tha 827 kb/s derived from the mea of y. Moreover, the tallest peak i Fig. 4(a) is located at 14.5 ms, which leads to a worse estimate = 827 kb/s compared to 983 kb/s computed from the mea of y. To combat these problems, existig studies [1], [4], [5], [15], [22] apply umerous empirical methods to fid out which mode is more likely to be correct. This may be the oly feasible solutio i multi-hop etworks; however, oe must keep i mid that it is possible that oe of the modes i the measured histogram correspods to as evideced by both graphs i Fig. 3.. Packet-Trai Aalysis Aother topic of debate i prior work was whether packettrai methods offer ay beefits over packet-pair methods. Some studies suggested that packet-trai measuremets coverge to the available badwidth 3 for sufficietly log bursts of packets [1], [4]; however, o aalytical evidece to this effect has bee preseted so far. Other studies [5] employed packet-trai estimates to icrease the measuremet accuracy of bottleeck badwidth estimatio, but it is ot clear how these samples beefit asymptotic covergece of the estimatio process. We cosider a hypothetic packet-trai method that trasmits probe traffic i bursts of k packets ad averages the iterpacket arrival delays withi each burst to obtai idividual 3 Eve though this questio appears to have bee settled i some of the recet papers, we provide additioal isight ito this issue. Proceedigs of the 12th IEEE Iteratioal oferece o Network Protocols (INP 4) /4 $ 2. IEEE

5 Absolute error (kb/s) Average sample values (ms) frequecy frequecy x x values of y() (ms) values of y() (ms) (a) (b) W (a) k =5 (b) k =1 Fig. 5. (a) The absolute error of the packet-pair estimate uder TP crosstraffic of r =1mb/s. (b) Evolutio of W. Fig. 6. The histogram of measured iter-arrival times Z k based o packet trais ad BR cross-traffic of r =1.3 mb/s. samples {Z}, k where is the burst umber. For example, if k =1, samples y 2,...,y 1 defie Z1 1, samples y 12,...,y 2 defie Z2 1, ad so o. The reaso for excludig samples y 1,y +1,...,y k+1 is because they are based o the leadig packets of each burst, which ecouter large iter-burst gaps i frot of them ad do ot follow the model developed so far. I what follows i this sectio, we derive the distributio of {Z} k as k. laim 2: For sufficietly large k, costat x = x, ad a regeerative processes M(t), packet-trai samples coverge to the followig Gaussia distributio for large : { Z k } D N (, λxv ar[s j λe[s j ]X i ] ) (k 1) 2, + λxe[s j] (12) where D deotes covergece i distributio, N(µ, σ) is a Gaussia distributio with mea µ ad stadard deviatio σ, ad X i are iter-packet arrival delays of cross-traffic. Proof: First, defie a k-sample versio of the cumulative reward process i (5): V k = M(a k ) M(a k( 1)+1 ) j=1 S j, =1, 2,... (13) Process V k is also a coutig process, however, its timescale is measured i bursts istead of packets. Thus, V k determies the amout of cross-traffic data received by the bottleeck lik durig a etire burst. Equatio (13) shows that Z k ca be asymptotically iterpreted as the reward rate of the reward-reewal process V k : Z k = + V k (k 1), (14) where k 1 is the umber of iter-packet gaps i a k-packet trai of probe packets. Assumig M(t) is regeerative ad for sufficietly large k, we have [28]: Z k = + V1 k + o(1). (15) (k 1) Applyig the regeerative cetral limit theorem, costraiig the rest of the derivatios i this sectio to costat x = x, ad assumig E[X i ] < [28]: { } V k ( 1 D N k 1 λxe[s j ], λxv ar[s j λe[s j ]X i ] k 1 (16) ombiig (15) ad (16), we get (12). First, otice that the mea of this distributio is the same as that of samples {y } i (11), which, as was ituitively expected, meas that both measuremet methods have the same expectatio. Secod, it is also easy to otice that the variace of Z k teds to zero as log as Var[X i ] is fiite. laim 3: If Var[X i ] is fiite, the variace of packet-trai samples Z k teds to zero for large k. Proof: Sice λ, x, ad E[S j ] are all fiite ad do ot deped o k, usig idepedece of S j ad X i i (12), we have: ( λxv Var[Z]= k ar[sj ]+λ 3 xe 2 ) 2 [S j ]Var[X i ] (k 1) 2, (17) which teds to for k. As a result of this pheomeo, loger packet trais will produce arrower distributios cetered at +E[ω ]. The BR case already studied i Fig. 3(b) clearly has fiite Var[X i ] ad therefore samples {Z} k must exhibit decayig variace as k icreases. Oe example of this covergece for packet trais with k =5ad k =1is show i Fig. 6. D. Discussio Now we address several observatios of previous work. It is oted i [5] that while packet-pair histograms usually have may differet modes, the histogram of packet-trai samples becomes uimodal with icreased k. This readily follows from (12) ad the Gaussia shape of {Z}. k Previous papers also oted (e.g., [5]) that as packet-trai size k is icreased, the distributio of samples {Z} k exhibits lower variace. This result follows from the above discussio ad (17). Furthermore, [5] foud that packet-trai histograms for large k ted to a sigle mode whose locatio is idepedet of burst size k. Our derivatios provide a isight ito how this process happes ad shows the locatio of this sigle mode to be +E[ω ] i (12). I summary, packet-trai samples {Z} k represet a limited reward rate that asymptotically coverges to a Gaussia distributio with mea E[y ]. Perhaps it is possible to ifer some ). Proceedigs of the 12th IEEE Iteratioal oferece o Network Protocols (INP 4) /4 $ 2. IEEE

6 characteristics of {X i } by observig the variace of {Z k } ad applyig the result i (12); however, sice there are several ukow parameters i the formula (such as Var[X i ] ad E[S j ]), this directio does ot lead to ay tractable results uless we assume a particular process M(t). Sice {Z k } asymptotically ted to a very arrow Gaussia distributio cetered at +E[ω ], we fid that there is o evidece that {Z k } measure the available badwidth or offer ay additioal iformatio about the value of as compared to traditioal packet-pair samples {y }. IV. ARBITRARY ROSS-TRAFFI I this sectio, we relax the statioarity ad reewal assumptios about cross-traffic ad derive a robust estimator of ad r. Assume a arbitrary arrival process r(t) for crosstraffic, where r(t) is its istataeous rate at time t. We impose oly oe costrait o this process it must have a fiite time average r show i (1). The goal of the samplig process is to determie both ad r. Sice r >imply costat packet loss ad zero available badwidth, we are geerally iterested i o-trivial cases of r. Stochastic process r(t) may be reewal, regeerative, a superpositio of ON/OFF sources, self-similar, or otherwise. Furthermore, sice packet arrival patters i the curret Iteret commoly exhibit ostatioarity (due to day-ight cycles, routig chages, lik failure, etc.), our assumptios o r(t) allow us to model a wide variety of such o-statioary processes ad are much broader tha commoly assumed i traffic modelig literature. Next, otice that if the probig traffic ca sample r(t) usig a Poisso sequece of probes at times t 1,t 2,..., the average of r(t i ) coverges to r (applyig the PASTA priciple [28]): r(t 1 )+r(t 2 ) r(t ) 1 t lim = lim r(u)du = r, t t (18) as log as delays τ i = t i t i 1 are i.i.d. expoetial radom variables. I order to accomplish this type of samplig, the seder must emit packet-pairs at expoetially distributed itervals. Assumig that the i-th packet-pair arrives to the router at time t i, it will sample a small segmet of r(t) by allowig g i amout of data to be queued betwee the probes: g i = t i +x i t i r(u)du r(t i )x i, (19) where x i is the spacig betwee the packets i the i-th packetpair. Agai, assumig that y i is the i-th iter-arrival sample geerated by the receiver, we have: y i = + g i = +r(t i)x i. (2) Fially, fixig the value of x i = x, otice that W has a well-defied limit: lim W 1 = lim = + x lim ( + r(t ) i)x i r(t i ) = +x r. (21) I essece, this result 4 is similar to our earlier derivatios, except that (21) requires much weaker restrictios o crosstraffic ad also shows that a sigle-ode model is completely tractable i the settig of almost arbitrary cross-traffic. We ext show how to extract both ad r from (21). A. apacity Observe that (21) is a liear fuctio of x, where r is the slope ad is the itercept 5. Therefore, by ijectig packetpairs with two differet spacigs x a ad x b, oe ca compute the ukow terms i (21) usig two sets of measuremets {yi a} ad {yb i }. To accomplish this, defie the correspodig average processes to be W a ad W: b W a = 1 yi a, W b = 1 yi b. (22) The simplest way to obtai both W a ad W b usig a sigle measuremet is to alterate spacig x a ad x b while preservig the PASTA samplig property. Usig a oe-bit header field, the receiver ca uambiguously sort the iterarrival delays ito two sets {yi a} ad {yb i }, ad thus compute their averages i (22). While samples are beig collected, the receiver has two ruig averages produced by (22). Subtractig W b from W a, we are able to separate r/ from : lim (W a W)= b (x a x b ) r. (23) Next, deote by the followig estimate of at time : = W a W a W b x a. (24) x a x b Takig the limit of (24), we have the followig result. laim 4: Assumig a sigle cogested bottleeck for which time-average rate r exists, coverges to : lim =. (25) Proof: Re-writig (25): lim = + x a r x (x a x b ) r a =, (26) (x a x b ) which is obtaied with the help of (21), (23), ad (24). Our ext result shows a more friedly restatemet of the previous claim. 4 A similar formula has bee derived i [3], [8], [19] ad several other papers uder a fluid assumptio. 5 For techical differeces betwee this approach ad previous work (such as TOPP [19]), see [17]. Proceedigs of the 12th IEEE Iteratioal oferece o Network Protocols (INP 4) /4 $ 2. IEEE

7 TABLE I AVAILABLE BANDWIDTH ESTIMATION ERROR Bottleeck capacity Relative error (mb/s) Model (29) Pathload Spruce IGI % 46.5% 27.9% 84.5% 5 8.3% 4.1% 23.4% 9.% 1 1.1% 4.9% 26.9% 89.% % 38.5% 24.5% 83.1% Relative estimatio error (a) E Relative estimatio error (b) E A orollary 1: Assumig a sigle cogested bottleeck for which time-average rate r exists, estimate = q/ coverges to capacity : lim = lim B. Available Badwidth = lim q W a W x a W b a x a x b q(x a x b ) x a W b x b W a =. (27) Notice that kowig a estimate of i (27) ad usig r i (23), it is easy to estimate the mea rate of cross-traffic: (W a W lim ) b = r, (28) x a x b which leads to the followig result. orollary 2: Assumig a sigle cogested bottleeck for which time-average rate r exists, the followig coverges to the available badwidth A = r: ( lim q xa x b W a + W b ) x a W b x b W a = r = A. (29). Simulatios We cofirm these results ad compare our models with several recet methods spruce [26], IGI [8], ad pathload [12] through s2 simulatios. Sice the mai theme of this paper is badwidth estimatio i heavily-cogested routers, we coduct all simulatios over a loaded bottleeck lik i Fig. 2 with utilizatio varyig betwee 82% ad 92% (the exact value chages depedig o ad the iteractio of TP cross-traffic with probe packets). Delays x a ad x b are set to maitai the desired rage of lik utilizatio. Defie E A = Ã A /A ad E = / to be the relative estimatio errors of A ad, respectively, where A is the true available badwidth of a path, Ã is its estimate usig oe of the measuremet techiques, is the true bottleeck capacity, ad is its estimate. Table I shows relative estimatio errors E A for spruce, IGI, ad pathload. Forpathload, we averaged the low ad high values of the produced estimates Ã. I the IGI case, we used the estimates available at the ed of IGI s iteral covergece algorithm. Also ote that we fed both spruce ad IGI the exact bottleeck capacity, while model (29) ad pathload operated without this iformatio. As the table shows, spruce performs better tha pathload i heavily-cogested cases, which is expected Fig. 7. Evolutio of relative estimatio errors of (27) ad (29) over a sigle cogested lik with = 1.5 mb/s ad 85% lik utilizatio. Relative estiamtio error (a) spruce Relative estimatio error (b) IGI Fig. 8. Relative estimatio errors E A produced by spruce ad IGI over a sigle cogested lik with = 1.5 mb/s ad 85% lik utilizatio. sice it ca utilize the kow capacity iformatio. Iterestigly, however, IGI s estimates are worse tha those of pathload eve though IGI utilizes the true capacity i its estimatio algorithm. A similar result is observed i [26] uder a relatively small amout of cross-traffic (2% to 4% lik utilizatio). Next, we examie models (27), (29) with a large umber of samples to show their asymptotic covergece ad estimatio accuracy. We plot the evolutio of relative estimatio errors E ad E A i Fig. 7. As Fig. 7(a) shows, coverges to a value that is very close (withi 3%) to the true value of. I Fig. 7(b), the available badwidth estimates quickly coverge withi 1% of A. For the purpose of compariso, we ext plot estimatio errors E A produced by spruce ad IGI i Fig. 8. As Fig. 8(a) shows, eve with the exact value of ad after 1, samples, spruce exhibits a error of 27%. Furthermore, IGI s estimates are much worse tha spruce s as illustrated i Fig. 8(b), which plots the evolutio of errors util IGI s iteral algorithm termiates. To better uderstad these results, we ext study how the accuracy of capacity iformatio provided to spruce ad IGI affects their measuremet accuracy. D. Further Aalysis of spruce ad IGI Sice bottleeck capacities of Iteret paths are ot geerally kow, the use of spruce ad IGI may be limited to a small umber of kow paths, uless these methods ca obtai capacity measuremets from other tools such as ettimer [15], [16], pathrate [5], or approbe [13]. These estimators of are usually very accurate whe the routers alog the path are ot highly utilized; however, as lik utilizatio icreases, Proceedigs of the 12th IEEE Iteratioal oferece o Network Protocols (INP 4) /4 $ 2. IEEE

8 Relative estimatio error Relatvie estimatio error spruce IGI mea of y() (ms) Simulatio result y=x mea of y() (ms) Simulatio result y=x Ratio of estimated ad real x() (ms) x() (ms) (a) (b) (a) BR cross-traffic (b) TP cross-traffic Fig. 9. (a) Relative capacity estimatio error E measured by approbe. (b) Relative estimatio errors E A of spruce ad IGI based o the ratio /. Fig. 1. overgece of ω to zero-mea additive oise for large x uder BR ad TP cross-traffic. their results ofte become iaccurate. Hece, a atural questio is how robust are spruce ad IGI to iaccurate values of provided to their estimatio process? We examie this issue below, but first discuss simulatio results of approbe to illustrate the pitfalls that may estimators of experiece over heavy-loaded bottleeck liks. For the approbe simulatio, we use the topology show i Fig. 2 with a sigle bottleeck lik whose utilizatio was kept at 85% usig five TP cross-traffic sources. Fig. 9(a) plots the evolutio of E produced by approbe i this experimet. As the figure shows, approbe s miimum filterig is sesitive to radom queuig delays i frot of the first packet of the pair ad ca evetually coverge to a completely wrog value (6% error). We ext examie how the value of supplied to IGI ad spruce affects their accuracy. We plot the relative estimatio errors i Fig. 9(b), i which the accuracy of both methods deteriorates whe / becomes large. To uderstad the exact effect of o these methods, we have the followig simple aalysis. Accordig to [8], IGI first seds packet trais Z i with iterpacket spacig x i (x i <x j for i<j) to determie the turig poit ˆx. Each packet-trai cosists of k back-to-back packets ad the turig poit is the -th iter-packet spacig x at which the receivig rate of probe traffic starts to match the sedig rate: ˆx = x = 1 k y i. (3) k 1 Subsequetly, IGI computes the average cross-traffic rate r as followig [8]: (y i ) y i >max(,ˆx) r = (31) (k 1)ˆx ad estimates available badwidth by subtractig (31) from its a-priori kow value of : à = r. Notice from (31) that the average cross-traffic r depeds o the capacity value provided to the estimatio algorithm. This depedecy explais the icreased estimatio iaccuracy whe / 1 as illustrated i Fig. 9(b). Similarly, estimatio accuracy of spruce also chages as a fuctio of. For example, eve though spruce provides i=2 better estimates tha IGI with a correct capacity value = = 1.5 mb/s, it exhibits large estimatio errors (over 2%) whe exceeds by 33%. This meas that the accuracy of spruce is also heavily depedet o the performace of the uderlyig bottleeck badwidth estimatio method. To better uderstad this observatio, we ext aalyze spruce s estimatio process. Spruce collects idividual samples A i [26]: ( A i = 1 y i ), (32) where y i is the i-th measured packet spacig at the receiver. The algorithm averages samples A i to obtai a ruig estimate of the available badwidth: A = 1 A i =2 2 y i. (33) q Takig the limits of (33) ad substitutig W from (21) ito (33), we get à = lim A = x r. (34) This brief aalysis shows that the badwidth estimatio mechaism i spruce requires that the seder set its iterpacket spacig x to be =q/. This is possible whe is kow exactly; however, i cases whe is ot correctly estimated, the iitial spacig x caot be set to ad spruce caot coverge to A. Also otice that if r, estimatio errors are geerally small; however, as lik utilizatio icreases, ay deviatio of x/ from 1 will have a sigificat impact o Ã. V. MULTIPLE LINKS I this sectio, we exted our sigle-ode model to the case of multiple cogested routers ad cojecture that it is impossible to derive a closed-form solutio that filters out the oise itroduced by cross-traffic at several liks of a ed-toed path. A. Large Iter-Probe Delays osider the origial model of a router i (2). This time, assume that oe of the probe packets queue behid each other at the bottleeck router. This meas that packet 1 leaves Proceedigs of the 12th IEEE Iteratioal oferece o Network Protocols (INP 4) /4 $ 2. IEEE

9 Sd1 1 mb/s Sd2 1 mb/s Rcv1 1 mb/s R p R 1 R 2 1 mb/s 1 mb/s 1 mb/s Sd3 Rcv3 Rcv2 Trasmissio delay (ms) estimated x1 4 (a) Sigle cogested lik Trasmissio delay (ms) estimated x1 4 (b) Two cogested liks Fig. 11. Multi-lik simulatio topology. Fig. 12. The evolutio of estimated trasmissio delay for a sigle ad two cogested liks. the router before packet arrives, which is expected if iterpacket spacig x at the source is very large compared to the trasmissio delay. Uder these assumptios, d 1 <a ad (2) becomes: d = a + ω +, 1. (35) Hece, iter-departure delays y are: y = a a 1 + ω ω 1, 2. (36) Notice that the first term a a 1 i (36) is the iter-arrival delay x of the probe traffic ad the secod term ω ω 1 ca be modeled as some zero-mea radom oise. This ca be explaied ituitively by oticig that uder the assumptio of large x, each router delays probe packets (o average) by the same amout. The the distace betwee each pair of subsequet packets fluctuates aroud the mea of x. Usig a iductive argumet, it is also easy to show the followig. laim 5: If the iitial spacig x is larger (i a statistical sese) tha queuig delays experieced by packets at each router of a N-hop ed-to-ed path, the mea of the sampled sigal y is equal to E[x ] ad the followig holds for each router j: E[ω (j) ω (j) 1 ]=. (37) To cofirm that the zero-mea model i (37) holds i practice, we ru s2 simulatios with 85% utilizatio at the bottleeck lik ad varyig packet sizes of BR ad TP cross-traffic. The plots of E[y ] as a fuctio of x for differet values of x = x are show i Fig. 1. As the two figures show, E[y ] coverges to x at 4 ad 1 ms, respectively, at which time the oise at the bottleeck router becomes zero-mea-additive. Note that similar results hold for multiple cogested routers, differet traffic patters, ad differet packet sizes. Also ote that the poit at which E[y ] coverges to x is ot ecessarily the value of the available badwidth as was suggested i prior work [8]. B. Recursive Model for Multi-Node Paths Assumig multiple cogested routers alog a path, the result i (25) o loger holds. To better uderstad multi-lik effects, we use the topology i Fig. 11, where the bottleeck lik has capacity 1.5 mb/s ad pre-bottleeck lik p =1.8 mb/s. Five FTP sources are attached to each of sd2 ad sd3 ad both liks are utilized by cross-traffic at 85%. Fig. 12 illustrates the covergece process of for the sigle-ode ad multi-ode cases. As Fig. 12(b) shows, estimates do ot coverge to the true value of =8ms, regardless of the umber of samples y i. Notice that it is possible to recursively exted the origial model i (26) to multiple cogested liks where the iput x of each lik is the output y of the previous lik. However, this model becomes itractable as we show ext. Add idex j to each process ad each radom variable to idicate that it is local to router j alog the path from the seder to the receiver. Further assume that Q (j) i is the queuig delay (due to cross-traffic ad other probe packets) i frot of packet i iside router j ad φ (j) i is some zero-mea oise process at router j at time i. The, we defie the followig recursive model: { j + ω (j) i, x (j) i <Q (j) i 1 y (j) i = j + φ (j) i, x (j) i Q (j) i 1, (38) where y (j) i is the departure spacig of the i-th packet-pair from router j, j = q/ j is the trasmissio delay of the probe packets over lik j, ad x (j) i is the arrival spacig betwee the probes at router j. Notice that if the arrival spacig betwee packets i 1 ad i (which is simply x (j) i = y (j 1) i ) is less tha the time packet i 1 speds i the buffer (i.e., Q (j) i 1 ), the two packets queue behid each other ad follow the model developed earlier i this paper. Whe the opposite holds, the packets do ot queue behid each other ad the router adds i.i.d. zero-mea oise φ (j) i to j.. ojecture of Impossibility Oe of the mai difficulties of this situatio is the stochastic mixture of the two types of oise i (38). While at this time we do ot offer a complete treatmet of this problem, we show that eve whe all liks follow the first model (i.e., all packets queue behid each other), the problem appears itractable. Uder these assumptios, (38) leads to: W (j) = j + 1 j r j (t i )x (j) i, (39) Proceedigs of the 12th IEEE Iteratioal oferece o Network Protocols (INP 4) /4 $ 2. IEEE

10 apacity estimates (kb/s) 15 1 bottleeck 5 =.5 mb/s =1 mb/s =1.2 mb/s Fig. 13. Evolutio of estimated badwidth for two cogested liks. where r j (t) is the cross-traffic process at router j. Foratwo cogested-router case, (39) becomes: W (2) = r 2 (t i )y (1) i 2 = r x 1 2 [ r 2 (t i ) 1 + r ] 1(t i )x 1 r 1 (u)r 2 (u)du. (4) Sice Poisso samples take at the receiver lead to obtaiig a time average of a mixed sigal itroduced by the two routers, it appears impossible to filter out the ukow cross-traffic statistics or the mai itegral i (4). We ext examie how the amout of cross-traffic i two cogested liks affects the estimatio accuracy of the earlier models developed i the paper. We set the average crosstraffic rates r 1 (t) ad r 2 (t) to be the same i both routers (i.e., r 1 = r 2 = µ) ad vary this value betwee 5 kb/s ad 1.2 mb/s. We agai use the topology i Fig. 11 ad plot i Fig. 13 the value of computed usig (27) whe crosstraffic is simultaeously ijected ito both liks ( p =2mb/s, =1.5 mb/s). As the figure illustrates, the estimates of the bottleeck badwidth coverge to a value that is sigificatly less tha the true capacity ad become progressively worse as µ icreases. VI. ONLUSION This paper examied the problem of estimatig the capacity/available badwidth of a sigle cogested lik ad showed a simple stochastic aalysis of the problem. Ulike previous approaches, our estimatio did ot rely o empirically-drive methods, but istead used a queuig model of the bottleeck router that specifically assumed o-egligible, o-fluid cross-traffic. It is also the first model to provide simultaeous asymptotically-accurate estimatio of both ad A i the presece of arbitrary cross-traffic. Our aalysis i the multi-ode cases suggests that the problem of obtaiig provably-coverget estimates of ad A i such eviromets does ot have a solutio. This isight possibly explais the lack of aalytical/stochastic justificatio for may of the proposed methods. Kowig that accurate estimatio of for the multi-ode case is impossible will potetially provide a importat foudatio for the methods that rely o empirically-optimized heuristics. REFERENES [1] B. Ahlgre, M. Björkma, ad B. Melader, Network Probig Usig Packet Trais, Swedish Istitute Techical Report, March [2] M. Allma ad V. Paxso, O Estimatig Ed-to-Ed Network Parameters, AM SIGOMM, August [3] J. Bolot, Ed-to-Ed Packet Delay ad Loss Behavior i the Iteret, AM SIGOMM, September [4] R.L. arter ad M.E. rovella, Measurig Bottleeck Lik Speed i Packet Switched Networks, Iteratioal Joural o Performace Evaluatio 27 & 28, [5]. Dovrolis, P. Ramaatha, D. Moore, What Do Packet Dispersio Techiques Measure? IEEE INFOOM, April 21. [6] A.B. Dowey, Usig PATHHAR to Estimate Iteret Lik haracteristics, AM SIGOMM, August [7] K. Harfoush, A. Bestavros, ad J. Byers, Measurig Bottleeck Badwidth of Targeted Path Segmets, IEEE INFOOM, March 23. [8] N. Hu ad P. Steekiste, Evaluatio ad haracterizatio of Available Badwidth Probig Techiques, IEEE Joural o Selected Areas i ommuicatios, vol. 21, No. 6, August 23. [9] V. Jacobso, ogestio Avoidace ad otrol, AM SIGOMM, [1] V. Jacobso, pathchar A Tool to Ifer haracteristics of Iteret Paths, ftp://ftp.ee.lbl.gov/pathchar/. [11] M. Jai ad. Dovrolis, Ed-to-Ed Available Badwidth: Measuremet Methodology, Dyamics, ad Relatio with TP Throughput, AM SIGOMM, 22. [12] M. Jai ad. Dovrolis, Pathload: A Measuremet Tool for Ed-to- Ed Available Badwidth, Passive ad Active Measuremets (PAM) Workshop, 22. [13] R. Kapoor, L. he, L. Lao, M. Saadidi, ad M. Gerla, approbe: a Simple ad Accurate apacity Estimatio Techique, AM SIGOMM, August 24. [14] S. Keshav, A otrol-theoretic Approach to Flow otrol, AM SIGOMM, [15] K. Lai ad M. Baker, Measurig Badwidth, IEEE INFOOM, [16] K. Lai ad M. Baker, Measurig Lik Badwidths Usig a Determiistic Model of Packet Delay, AM SIGOMM, August 2. [17] X. Liu, K. Ravidra, B. Liu, ad D. Loguiov, Sigle-Hop Probig Asymptotics i Available Badwidth Estimatio: A Sample-Path Aalysis, AM IM, October 24. [18] B.A. Mah, pchar: A Tool for Measurig Iteret Path haracteristics, bmah/ Software/ pchar/, Jue 21. [19] B. Melader, M. Björkma, P. Guigberg, A New Ed-to-Ed Probig ad Aalysis Method for Estimatig Badwidth Bottleecks, IEEE GLOBEOM, November 2. [2] A. Pasztor ad D. Veitch, Active Probig usig Packet Quartets, AM IMW, 22. [21] A. Pasztor ad D. Veitch, The Packet Size Depedece of Packet Pair Like Methods, IEEE/IFIP IWQoS, 22. [22] V. Paxso, Measuremets ad Aalysis of Ed-to-Ed Iteret Dyamics, Ph.D. dissertatio, omputer Sciece Departmet, Uiversity of aliforia at Berkeley, [23] S. Ratasamy ad S. Mcae, Iferece of Multicast Routig Trees ad Bottleeck Badwidths usig Ed-to-Ed Measuremets, IEEE INFOOM, March [24] V. Ribeiro, R. Riedi, R. Baraiuk, J. Navratil, ad L. ottrell, pathhirp: Efficiet Available Badwidth Estimatio for Network Paths, Passive ad Active Measuremets (PAM) Workshop, 23. [25] D. Sisalem ad H. Schulzrie, The Loss-delay Based Adjustmet Algorithm: A TP-friedly adaptatio scheme, IEEE Workshop o Network ad Operatig System Support for Digital Audio ad Video (NOSSDAV), [26] J. Strauss, D. Katabi, ad F. Kaashoek, A Measuremet Study of Available Badwidth Estimatio Tools, AM IM, 23. [27] R. Wade, M. Kara, ad P.M. Dew, Study of a Trasport Protocol Employig Bottleeck Probig ad Toke Bucket Flow otrol, IEEE IS, July 2. [28] R.W. Wolff, Stochastic Modelig ad the Theory of Queues, Pretice- Hall, Proceedigs of the 12th IEEE Iteratioal oferece o Network Protocols (INP 4) /4 $ 2. IEEE