TCP and Explicit Congestion Notification

Transcription

1 TCP and Eplicit Cngestin Ntificatin Sally Flyd Lawrence Berkeley Labratry One Cycltrn Rad, Berkeley, CA Abstract This paper discusses the use f Eplicit Cngestin Ntificatin (ECN) mechanisms in the TCP/IP prtcl. The first part prpses new guidelines fr TCP s respnse t ECN mechanisms (e.g., Surce Quench packets, ECN fields in packet headers). Net, using simulatins, we eplre the benefits and drawbacks f ECN in TCP/IP netwrks. Our simulatins use RED gateways mdified t set an ECN bit in the IP packet header as an indicatin f cngestin, with Ren-style TCP mdified t respnd t ECN as well as t packet drps as indicatins f cngestin. The simulatins shw that ne advantage f ECN mechanisms is in aviding unnecessary packet drps, and therefre aviding unnecessary delay fr packets frm lw-bandwidth delay-sensitive TCP cnnectins. A secnd advantage f ECN mechanisms is in netwrks (generally LANs) where the effectiveness f TCP retransmit timers is limited by the carse granularity f the TCP clck. The paper als discusses sme implementatin issues cncerning specific ECN mechanisms in TCP/IP netwrks. 1 Intrductin This paper prpses guidelines fr TCP s respnse t ECN (Eplicit Cngestin Ntificatin) mechanisms, and eplres the effect upn perfrmance f ECN mechanisms in TCP/IP netwrks. The paper discusses sme implementatin issues cncerning ECN mechanisms, but des nt make specific recmmendatins cncerning the use f ECN mechanisms in TCP/IP netwrks. In current TCP/IP netwrks, TCP relies n packet drps as the indicatin f cngestin. The TCP surce detects drpped packets either frm the receipt f three duplicate acknwledgements (ACKs) r after the timeut f a retransmit timer, and respnds t a drpped packet by reducing the cngestin windw [J88]. TCP implementatins als respnd t ICMP Surce Quench This wrk was supprted by the Directr, Office f Energy Research, Scientific Cmputing Staff, f the U.S. Department f Energy under Cntract N. DE-AC03-76SF messages, but Surce Quench messages are rarely used, in part because they can cnsume netwrk bandwidth in times f cngestin. The reliance n packet drps as the indicatin f cngestin is perfectly apprpriate fr a netwrk with ruters whse main functin is t rute packets t the apprpriate utput prt. Mst current ruters in TCP/IP netwrks have n prvisin fr the detectin f incipient cngestin. When a queue verflws, packets are drpped. When the TCP surce detects this packet drp, the TCP surce infers the presence f cngestin in the netwrk. Future ruters are likely t have mre develped mechanisms fr the detectin f incipient cngestin. With the DECbit scheme, fr eample, ruters detect incipient cngestin by cmputing the average queue size, and set the ECN bit in packet headers when the average queue size eceeds a certain threshld [RJ90]. Recentlyprpsed Randm Early Detectin (RED) gateways have a similar ability t detect incipient cngestin [FJ93]. Gateways with mechanisms fr detecting incipient cngestin befre the queue verflws are nt limited t packet drps as the methd f infrming surces f cngestin. Fr netwrks with mechanisms fr the detectin f incipient cngestin, the use f ECN mechanisms fr the ntificatin f cngestin t the end ndes prevents unnecessary packet drps. Fr bulk-data cnnectins, the user is cncerned nly with the arrival time f the last packet f data, and delays f individual packets are f n cncern. Fr sme interactive traffic, hwever, such as telnet traffic, the user is sensitive t the delay f individual packets. Fr such lw-bandwidth delay-sensitive TCP traffic, unnecessary packet drps and packet retransmissins can result in nticeable and unnecessary delays fr the user. Fr sme cnnectins, these delays can be eacerbated by a carse-granularity TCP timer that delays the surce s retransmissin f the packet. A secnd benefit f ECN mechanisms is that with ECN, surces can be infrmed f cngestin quickly and unambiguusly, withut the surce having t wait fr either a retransmit timer r three duplicate ACKs t infer a drpped packet. Fr bulk-data TCP cnnectins,

2 the delay fr the retransmissin f an individual packet is nt generally an issue. Fr bulk-data TCP cnnectins in wide-area envirnments, the cngestin windw is generally sufficiently large that the drpped packet is detected fairly prmptly by the Fast Retransmit prcedure. Nevertheless, fr thse cases where a drpped packet is nt detected by the Fast Retransmit prcedure, the use f ECN mechanisms can imprve a bulk-data cnnectin s respnse t cngestin. If the surce is delayed in detecting a drpped packet, perhaps due t a small cngestin cntrl windw and a carse-grained TCP timer, the surce can lie idle. This delay, when cmbined with the glbal synchrnizatin, can result in substantial link idle time. An additinal mtivatin fr the eplratin f ECN mechanisms in TCP/IP netwrks cncerns the pssibility f TCP/IP traffic traversing netwrks that have their wn cngestin cntrl mechanisms (e.g., ATM netwrks). With current implementatins f TCP, such netwrks are limited t packet drps as the nly viable mechanism t infrm TCP surces f cngestin. Such netwrks wuld benefit frm the additin t TCP f mre intelligent ECN-respnse methds. If this were the case, then fr TCP traffic that travels fr all r part f its path ver ATM netwrks, ECN mechanisms culd be invked at the edge f the ATM netwrk and used t infrm TCP surces f cngestin within the ATM netwrk. This use f ECN mechanisms t infrm TCP surces f cngestin wuld be independent f the cngestin cntrl mechanisms within the ATM netwrks. This paper eplres sme f the general advantages and disadvantages f ECN mechanisms in TCP/IP mechanisms, but des nt make recmmendatins fr r against specific ECN mechanisms (e.g., the additin f an ECN field t IP headers). The results in this paper are intended t be qualitative, nt quantitative. Fr eample, while it is clear that the use f ECN can reduce the packet delay fr lw-bandwidth delay-sensitive TCP traffic, the etent f this benefit depends n the eact netwrk tplgy, the traffic mi, and the details f the relevant gateway and transprt cngestin cntrl algrithms. The simulatins in this paper shw that the use f ECN can reduce packet delay, but they d nt quantify the epected reductin in packet delay in a particular netwrk. Sectin 2 discusses sme current ECN mechanisms, such as Surce Quench in TCP/IP netwrks. Sectin 3 briefly discusses the rle f the ruter in detecting incipient cngestin. Sectin 4 presents ur prpsal fr guidelines fr TCP s respnse t ECN; these guidelines differ frm thse f current netwrk mechanisms. Sectin 5 discusses the applicatin f these guidelines t the implementatin f Ren-style TCP in ur simulatr. Sectins 6 and 7 present ur simulatin results f LAN and WAN envirnments. Sectin 8.2 discusses sme f the implicatins, fr the evaluatin f ECN mechanisms, fr sme f the prpsed mdificatins t TCP r t gateway scheduling algrithms. Finally, Sectin 9 discusses varius implementatin issues cncerning ECN mechanisms in TCP/IP netwrks. These include cmparisns between Surce Quench and ECN fields in packet headers, pssibilities fr the incremental deplyment f ECN mechanisms in TCP/IP netwrks, a discussin f TCP clck granularity, and a discussin f IP-level ECN mechanisms fr TCP traffic ver ATM netwrks. Sectin 10 presents cnclusins and pen questins. 2 Current ECN mechanisms In this sectin we discuss briefly ECN mechanisms such as Surce Quench messages, DECbit s ECN bit, and FECN/BECN prpsals fr ATM netwrks. One purpse f this sectin is t describe the current ECN mechanisms in TCP/IP netwrks. This includes a detailed eplanatin f the inadequacies f current implementatins f Surce Quench. In this sectin we als describe the ECN mechanisms in the DECbit cngestin avidance scheme, partly t mtivate the smewhat-different guidelines that we prpse in the fllwing tw sectins fr ECN in TCP/IP netwrks. Finally, we prvide pinters t ther discussins f prpsed ECN mechanisms in the literature. Currently, ICMP Surce Quench messages are the nly ECN mechanisms in TCP/IP netwrks, but in fact Surce Quench is rarely used in the current Internet. A ruter r hst might send an ICMP Surce Quench message when it receives datagrams at a rate that is t fast t be prcessed [S94]. While RFC 1009 [BP87] required ruters t generate Surce Quenches when they ran ut f buffers, the current draft n Requirements fr IP Ruters [A94] specifies that a ruter shuld nt riginate Surce Quench messages, and that a ruter that des riginate Surce Quench messages must be able t limit the rate at which they are generated. In the draft, Surce Quench messages are criticized as cnsuming netwrk bandwidth, and as being bth ineffective and unfair. In cntrast, as discussed in the net sectin, the use f RED gateways with Surce Quench messages wuld cntrl the rate at which Surce Quench messages were generated, while increasing bth the effectiveness and the fairness f these messages. The guidelines fr TCP s respnse t Surce Quench predate such TCP cngestin cntrl mechanisms as Fast Retransmit and Fast Recvery [S94]. Frm the guidelines in [BP87], TCP implementatins shuld respnd t a Surce Quench by triggering a slw start, as

3 if a retransmissin timeut had ccurred. In BSD TCP implementatins, the TCP surce respnds t a Surce Quench by reducing the cngestin cntrl windw t ne, initiating Slw-Start. If the Surce Quench signals a drpped packet, and is therefre fllwed by either a retransmit timer timeut r by the Fast Retransmit prcedure, then the timeut recvery cde fllws the Surce Quench cde. In this case the slw start threshld ssthresh ends up being set t ne r tw segments, resulting in a very slw repening f the cngestin windw. This use f Slw-Start cmbined with a small slw start threshld makes the use f Surce Quench particularly unattractive fr large-windw TCP cnnectins in high-speed envirnments. This paper prpses alternate guidelines fr TCP s respnse t Surce Quench. In the DECbit cngestin avidance scheme [RJ90], the gateway uses a cngestin-ntificatin bit in packet headers t prvide feedback abut cngestin in the netwrk. When a packet arrives at the gateway, the gateway calculates the average queue length. When the average queue size at the gateway eceeds ne, the gateway sets the ECN bit in the packet header f arriving packets. The surce uses windw flw cntrl, and updates its windw nce every tw rundtrip times. If at least half f the packets in the last windw had the ECN bit set, then the cngestin windw is decreased multiplicatively. Otherwise, the cngestin windw is increased additively. In cntrast t the DECbit scheme, fr the ECN mechanism prpsed in this paper a single packet with the ECN bit set is t be interpreted by the transprtlevel surce as an indicatin f cngestin. [WRM91] reprts n eperiments in an OSI testbed mdified t include the cngestin-ntificatin bit prpsed in [RJ90]. A range f ECN-based rate-based cngestin cntrl schemes have been prpsed fr use within ATM netwrks. These include prpsals bth fr Frward ECN (FECN) and fr Backward ECN (BECN). An intrductin t sme f these prpsals can be fund in [N94]. 3 The rle f the ruter This sectin discusses the rle f the ruter in generating ECN messages. ECN messages culd be generated either by an IP ruter r by a bundary ruter fr an ATM netwrk that carries TCP/IP traffic. This sectin cnsiders IP netwrks with RED gateways, where the gateway mnitrs the average queue size and during cngestin uses a prbabilistic algrithm t chse which arriving packets t mark (e.g., t drp, r t set the ECN field in the packet header). In ur simulatins using RED gateways with ECN, the RED gateways set the ECN field in the packet header; in the simulatins using RED gateways withut ECN, the RED gateways drp the packet instead. In bth variants f RED gateways, the gateway drps a packet when a packet arrives t a full queue. It wuld be pssible t add ECN mechanisms t a traditinal Drp-Tail gateway, where the gateway simply drps arriving packets when the queue buffer is full. Fr eample, Drp-Tail gateways, after drpping a packet, culd send Surce Quench messages t the TCP surce. Hwever, we d nt advcate adding ECN mechanisms t Drp-Tail gateways, and we d nt investigate such schemes in this paper. If ECN mechanisms are added t a gateway, it makes sense t add at the same time mechanisms t mnitr the average queue size. We believe that the use f ECN mechanisms are f mst benefit in a gateway that ntifies cnnectins f incipient cngestin befre the queue actually verflws. 1 T a first apprimatin, RED gateways mark (e.g., drp) a percentage f arriving packets, where the eact percentage f arriving packets marked shuld be just enugh t cntrl the average queue size ver the lng run. Fr eample, in a LAN envirnment, where a TCP cnnectin increases its cngestin windw quite rapidly, a nn-ecn gateway might have t drp a significant fractin f arriving packets t cntrl cngestin. Fr RED gateways with a FIFO queue, if a certain fractin f bulk-data packets have t be drpped t cntrl cngestin, the RED gateway drps the same fractin f telnet packets. Current ruters generally have a single queue fr each utput prt. In the future, ruters culd have separate queues fr separate classes f traffic [BCS94]. In this case, the ECN mechanisms culd apply separately t each queue. As discussed in Sectin 8.3, this culd affect the mtivatins fr ECN mechanisms. 4 Guidelines fr TCP s respnse t ECN In this sectin we eplain ur guidelines fr TCP s respnse t ECN. These guidelines differ frm TCP s current respnse t Surce Quench messages, r frm the respnse f transprt prtcls t DECbit s cngestin ntificatin bit. These guidelines prvide that the receipt f a single ECN message serves as a ntificatin f cngestin t the TCP surce. At the same time, the guidelines ensure that the TCP surce des nt respnd t ECN messages mre frequently than necessary. Guidelines: TCP s respnse t ECN shuld be similar, ver 1 As an eample, [PP87] suggested that the gateway send Surce Quench messages when the queue size eceeds a certain threshld.

4 lnger time scales, t its respnse t a drpped packet as an indicatin f cngestin. Over smaller time scales (e.g., ne r tw rund trip times), TCP s respnse t ECN can be less cnservative than its respnse t a drpped packet as an indicatin f cngestin. In Tahe and Ren implementatins f TCP, after a packet has been drpped the TCP surce stps sending fr a time perid n the rder f a rund trip time (half a rund trip time fr Ren implementatins), allwing netwrk queues t dissipate smewhat. This delay is nt necessary as a respnse t an ECN, which des nt indicate a queue verflw. Fr TCP, the receipt f a single ECN (e.g., a single Surce Quench packet, r a single packet with the ECN bit set) shuld trigger a respnse t cngestin. This is unlike the DECbit cngestin cntrl scheme, where the surce respnds t cngestin nly if at least half f the packets in the last windw had the ECN bit set [RJ90]. The decisin t allw a single ECN message t trigger a respnse t cngestin requires a minimum f verhead. In additin, because the gateway des nt set the ECN field in every arriving packet when the average queue size is t high, the gateway can use prbabilistic algrithms t infrm particular surces f cngestin [FJ93]. Because the prbability that a cnnectin is ntified f cngestin is prprtinal t that cnnectin s share f the bandwidth at the cngested gateway, these prbabilistic algrithms reduce glbal synchrnizatin and imprve fairness. TCP shuld react t an ECN at mst nce per rundtrip time. The TCP surce shuld ignre succeeding ECNs if the surce has reacted t a previus ECN r t a drpped packet in the last rundtrip time. This als means that if, immediately after reacting t an ECN, the TCP surce receives three duplicate ACKs indicating a drpped packet, the TCP surce shuld nt repeat the reductin f the cngestin windw; the packet was prbably drpped befre the surce reduced its windw in respnse t the ECN. TCP shuld fllw the eisting algrithms fr sending data packets in respnse t incming ACKs. The respnse t an ECN des nt trigger the sending f any new (r retransmitted) data packets. TCP shuld fllw the nrmal prcedure after the timeut f a retransmit timer. That is, after a retransmit timer timeut the TCP surce shuld slw-start and retransmit the drpped packet. Hwever, the TCP surce shuld nt decrease the slw-start threshld ssthresh if it has been decreased within the last rundtrip time. 5 Implementing ECN in ur simulatr In this sectin we describe the implementatin f TCP s respnse t ECN in ur simulatr. This implementatin f ECN was made t a versin f TCP that incrprates the Fast Recvery cngestin cntrl algrithm in Ren TCP (4.3-ren BSD TCP), as well as the Slw- Start, Cngestin Avidance, and Fast Retransmit algrithms in the earlier Tahe TCP (4.3-tahe BSD TCP [J88, S94]). Fr the simulatins in this paper the RED gateways were given an ptin t set the ECN bit in the packet header, rather than drpping the packet, as an indicatin f cngestin when the buffer had nt yet verflwed. When the TCP receiver receives a data packet with the ECN bit set in the packet header, the receiver sets the ECN bit in the net utging ACK packet. First we briefly describe the Slw-Start, Cngestin Avidance, Fast Retransmit, and Fast Recvery algrithms in TCP. There are tw phases t the windwadjustment algrithm. The cnnectin begins in slwstart phase, and the current cngestin windw cwnd is dubled each rundtrip time until the cngestin windw reaches the slw-start threshld ssthresh. Then the cngestin-avidance phase is entered, and the cngestin windw is increased by rughly ne packet each rundtrip time. The cngestin windw is never allwed t increase t mre than the receiver s advertised windw, which we refer t as the maimum windw. In additin t using retransmit timers t detect lst packets, the surce uses the Fast Retransmit prcedure t discver a packet lss. If three duplicate ACK packets are received acknwledging a previusly-acknwledged data packet, the surce infers that a packet has been drpped. In the Tahe versin f TCP, the surce reacts t a packet lss by setting the slw start threshld t half the cngestin windw, decreasing the cngestin windw t ne packet, and entering the slw-start phase. In cntrast, with Ren s Fast Recvery algrithm the TCP surce des nt slw-start after inferring that a packet has been drpped. Instead, the TCP surce effectively waits half a rund trip time and halves the cngestin windw. The surce retransmits the drpped packet and uses incming duplicate ACKs t clck additinal utging packets. The Fast Recvery algrithm is described in mre detail in [S94]. 2 2 The descriptin f the Fast Recvery algrithm in [S94] is quite gd, and is the nly cmplete publicly-available descriptin f this algrithm that we are aware f. Hwever, we have a cautin t readers. First, the earlier printings (prir t the 4th) d nt distinguish between the Fast Retransmit and the Fast Recvery algrithms. Secnd, in the descriptin f the windw increase algrithm in the cngestin

5 Fllwing the guidelines in the previus sectin, upn receiving an ECN message (e.g., a Surce Quench message, r an ACK packet with the ECN bit set) at time when n respnses t cngestin have been made in rughly the last rundtrip time, the TCP surce halves bth the cngestin windw cwnd and the slw-start threshld ssthresh. Because there is n lss f incming ACKs t clck utging packets and n need fr a shrt pause t recver frm severe shrt-term cngestin, the TCP surce desn t slw-start. The TCP surce desn t respnd t succeeding ECNs until all packets utstanding at time have been acked. After receiving three duplicate ACKs at time when n respnses t cngestin have been made in rughly the last rundtrip time, the TCP surce fllws the Fast Retransmit and Fast Recvery prcedures described abve. The surce wn t respnd t an ECN r t anther set f three duplicate ACKs until all packets utstanding at time have been acked. ([F94] discusses sme f the prblems that result if the Fast Retransmit prcedure is invked mre than nce fr ne windw f data.) After receiving three duplicate ACKs sn after respnding t an ECN (e.g., when sme f the packets utstanding at the time f the respnse t the ECN have nt yet been ACKed), the surce desn t reduce ssthresh r cwnd, since that has recently been dne. The surce retransmits the drpped packet. After this, the surce fllws Ren s Fast Recvery prcedure, using incming duplicate ACKs t clck utging packets. 6 Simulatin results f ECN in LANs This sectin discusses the results f simulatins f TCP with ECN in lcal-area netwrks. The simulatin scenari cnsists f five bulk-data TCP cnnectins and ten telnet cnnectins in a LAN with ne cngested gateway. We cmpare several sets f simulatins. The first set uses Drp-Tail gateways, and the secnd set uses RED gateways that rely n packet drps fr the ntificatin f cngestin. The third set f simulatins uses ECN-capable RED gateways and TCP implementatins. The simulatins shw that fr the netwrks with ECN, the thrughput f the bulk-data cnnectins is high, and the telnet packet delay is lw, regardless f the buffer size, the TCP clck granularity, the TCP windw size, r the RED gateway variatin. Thus, the simulatins avidance phase, the earlier printings say that the cngestin windw is incremented by 1/cwnd plus a small fractin f the segment size each time an ACK is received. The inclusin f the small fractin f the segment size is an errr in the 4.3 Ren and 4.4 BSD implementatins, and shuld nt be emulated in future TCP implementatins. with ECN are rbust, delivering essentially ptimal perfrmance ver a wide range f cnditins. Fr the simulatins f RED gateways withut ECN, the results depend n the simulatin parameters. Fr mst f the simulatins, the telnet packet delay is less than ptimal (thugh nt disastrus). Fr a few f the simulatins with a carse TCP clck granularity, the aggregate thrughput is als lw. The telnet packet delay is wrst fr the simulatins f Drp-Tail gateways (withut ECN). In general, the simulatins shw that with ECN the perfrmance is less sensitive t the netwrk parameters, and that the use f ECN can imprve bth delay and thrughput. Hwever, while the simulatins shw the benefits f using ECN, the simulatins shw that under prper cnditins, it is pssible t get reasnable perfrmance withut ECN. Senders 10 usec 100 Mbps Gateway 10 usec 100 Mbps Figure 1: LAN simulatin scenari. 6.1 The LAN simulatin scenari Receiver The simulatin scenari in Figure 1 cnsists f five bulk-data TCP cnnectins and ten TELNET cnnectins frm five surces feeding int a single cngested link. The simulatins use a range f switch buffer sizes, windw sizes, gateway variatins, and TCP clck granularities. The simulatins are run fr buffer sizes f 60, 120, 180, and 240 kb. In ur simulatins with RED gateways, the minimum threshld fr the average queue size is set t 1/12-th f the buffer size, and the maimum threshld is set t three times the minimum threshld. The RED gateways drp all arriving packets when the average queue size eceeds the maimum threshld. The TCP in these simulatins uses Ren-style cngestin cntrl, with Slw-Start, Fast Retransmit, and Fast Recvery, mdified as described in the previus sectin t respnd t ECN. One set f simulatins uses a TCP clck granularity set t 0.1 msec, which fr this simulatin scenari reduces the wait fr the retransmit timer while at the same time aviding false timeuts.

6 Anther set f simulatins uses a TCP clck granularity set t 100 msec, which is clser t values f 500 msec used by many current TCP implementatins. The maimum TCP windw ranges frm 8 kb t 64 kb. The default parameters are set s that fr the first packet frm a TCP cnnectin, befre measurements have been made f the rundtrip time, the retransmit timer is set t three secnds, regardless f the TCP clck granularity. Fr the simulatins withut ECN, the wrstcase telnet delay is determined mainly by this initial value fr the retransmit timer. The telnet cnnectins send 40-byte packets at randm intervals drawn frm the tcplib distributin [DJ91]. The ten telnet cnnectins tgether send several hundred 40-byte data packets in ne 15-secnd simulatin. The bulk-data cnnectins send 1000-byte data packets. The start times fr the bulk-data cnnectins are staggered ver the first secnd f the 15-secnd simulatin. In Figures 2 and 3, the graphs illustrate the thrughput and delay perfrmance in the simulatins. The three clumns in Figures 2 and 3 shw simulatins with Drp- Tail gateways, RED gateways withut ECN, and RED gateways with ECN, respectively. The graphs in the tp rw shw the effective thrughput f the five bulk-data cnnectins, as a fractin f the ttal available bandwidth in bits per secnd. The secnd rw f graphs shw the bulk-data cnnectin that receives the smallest thrughput ver the 15-secnd simulatin. The third rw f graphs shws the telnet packet with the highest ne-way telnet delay, in secnds. This neway delay is the delay frm the first time that the packet is transmitted by the surce, until the packet is received by the receiver. The furth rw shws the average ne-way telnet delay. The fifth rw shws the fractin f the telnet packets that have a ne-way delay greater than 100 msec. (A rundtrip packet delay greater than 100 msec is likely t be nticeable t the telnet user.) Given the queue sizes and prpagatin delays in these simulatins, a ne-way packet delay greater than 100 msec is nly pssible fr a packet that is drpped at the gateway. Fr each graph the -ais shws the switch buffer size in kb. The fur lines in each graph crrespnd t fur different simulatin sets, with 8 kb and 64 kb maimum TCP windws, and with bth byte-based and packet-based gateways. Fr the (smewhat unrealistic) byte-based gateways, the queue size is measured in bytes rather than in packets, s that a small 40-byte TEL- NET packet is less likely t arrive t a full buffer than is a larger 1000-byte FTP packet. Fr the packet-based gateways, the queue size is measured in packets. With packet-based RED gateways the gateway s decisin t drp r mark a packet is independent f that packet s size in bytes, while with byte-based RED gateways the prbability that a packet is drpped (r marked) is prprtinal t that packet s size in bytes. We ran five simulatins fr each set f parameters. The result f each simulatins is marked n the graph, and the lines shw the averages f the five simulatins. 6.2 Results fr LAN simulatins As Figures 2 and 3 shw, fr all f the simulatins f RED with ECN the effective thrughput is high and the telnet packet delay is lw. Withut ECN the netwrk perfrmance can be significantly affected by the TCP clck granularity, by the level f cngestin (as a functin f the TCP maimum windw), and by whether the RED gateway is using a drpping plicy that is sensitive t packet size. Fr the simulatins in Figure 2 with the TCP clck granularity set t 0.1 msec, the thrughput is high fr all three sets f simulatins, Drp-Tail, RED withut ECN, and RED with ECN. Hwever, fr the simulatins f packet-based Drp-Tail and RED gateways withut ECN, telnet packets are ccasinally drpped, leading t ccasinal telnet packets with high delay. Fr the simulatins in Figure 3 f RED withut ECN with smaller switch buffers, a TCP clck granularity f 100 msec, and small TCP windws, the thrughput f the bulk-data cnnectins suffers. Because f the carse TCP clck granularity, a number f cnnectins culd be waiting fr a retransmit timer t epire, resulting in link idle time. Fr the simulatins with Drp-Tail gateways and small TCP windws, packets shuld never be drpped at the gateway. Fr the simulatins with Drp-Tail gateways, larger TCP windws, and a TCP clck granularity f 100 msec, the verall thrughput is high, but the graph f the Smallest Bulk-Data Thrughput shws that there is sme unfairness; with smaller buffer sizes, the thrughput f the smallest f the five bulk-data cnnectins is less than ptimal. Nte that a Drp-Tail gateway with a certain buffer size cannt be directly cmpared t a RED gateway with the same buffer size; the mre apprpriate cmparisn is between a Drp- Tail gateway and a RED gateway with a similar average queue size. These simulatins shw that the delay fr small telnet packets is much lwer with byte-based gateways. These byte-based gateways might be easy t implement in ur simulatr, but they are nt typical f current gateways. In additin, fr lw-thrughput delay-sensitive interactive traffic where the size f individual packets is similar t the size f bulk-data packets, byte-based gateways wuld nt imprve the perfrmance f the interactive traffic.

7 Bulk-Data Thrughput : byte-based DT, 8 kb windws : packet-based DT, 8 kb windws : byte-based DT, 64 kb windws : packet-based DT, 64 kb windws Drp Tail, TCP clck 0.1 msec Bulk-Data Thrughput RED withut ECN, TCP clck 0.1 msec Bulk-Data Thrughput RED with ECN, TCP clck 0.1 msec Smallest Bulk-Data Thrughput Drp Tail, TCP clck 0.1 msec Smallest Bulk-Data Thrughput RED withut ECN, TCP clck 0.1 msec Smallest Bulk-Data Thrughput RED with ECN, TCP clck 0.1 msec Highest Telnet Delay (in Secnds) Drp Tail, TCP clck 0.1 msec Highest Telnet Delay (in Secnds) RED withut ECN, TCP clck 0.1 msec Highest Telnet Delay (in Secnds) RED with ECN, TCP clck 0.1 msec Average Telnet Delay (in Secnds) Drp Tail, TCP clck 0.1 msec Average Telnet Delay (in Secnds) RED withut ECN, TCP clck 0.1 msec Average Telnet Delay (in Secnds) RED with ECN, TCP clck 0.1 msec Fractin f Packets with High Delay Drp Tail, TCP clck 0.1 msec Fractin f Packets with High Delay RED withut ECN, TCP clck 0.1 msec Fractin f Packets with High Delay RED with ECN, TCP clck 0.1 msec Figure 2: LAN simulatins with a 0.1 msec TCP clck.

8 Bulk-Data Thrughput : byte-based DT, 8 kb windws : packet-based DT, 8 kb windws : byte-based DT, 64 kb windws : packet-based DT, 64 kb windws Drp Tail, TCP clck 100 msec Bulk-Data Thrughput RED withut ECN, TCP clck 100 msec Bulk-Data Thrughput RED with ECN, TCP clck 100 msec Smallest Bulk-Data Thrughput Drp Tail, TCP clck 100 msec Smallest Bulk-Data Thrughput RED withut ECN, TCP clck 100 msec Smallest Bulk-Data Thrughput RED with ECN, TCP clck 100 msec Highest Telnet Delay (in Secnds) Drp Tail, TCP clck 100 msec Highest Telnet Delay (in Secnds) RED withut ECN, TCP clck 100 msec Highest Telnet Delay (in Secnds) RED with ECN, TCP clck 100 msec Average Telnet Delay (in Secnds) Drp Tail, TCP clck 100 msec Average Telnet Delay (in Secnds) RED withut ECN, TCP clck 100 msec Average Telnet Delay (in Secnds) RED with ECN, TCP clck 100 msec Fractin f Packets with High Delay Drp Tail, TCP clck 100 msec Fractin f Packets with High Delay RED withut ECN, TCP clck 100 msec Fractin f Packets with High Delay RED with ECN, TCP clck 100 msec Figure 3: LAN simulatins with a 100 msec TCP clck.

9 7 Simulatin results f ECN in WANS This sectin gives results frm simulatins f ECN and nn-ecn gateways in a wide-area envirnment. The thrughput fr the bulk-data traffic is similar in the simulatins with Drp-Tail gateways, RED gateways with ECN, r RED gateways withut ECN. Hwever, the packet delay fr lw-bandwidth telnet traffic is significantly lwer fr the simulatins with ECN. 0.5 msec 1 msec 3 msec 5 msec 100 Mbps A Gateway 10 msec 45 Mbps B Gateway 0.5 msec 2 msec 100 Mbps 1 msec 5 msec Figure 4: Simulatin scenari fr wide-area traffic. 7.1 WAN simulatin scenari The simulatin netwrk in Figure 4 has tw-way traffic cnsisting f siteen TELNET cnnectins in each directin, ccasinal FTP cnnectins with a limited amunt f data t transfer ( packets), and a number f bulk-data cnnectins with an unlimited amunt f data t transfer. Fr the simulatins in Figure 5 f a mderate traffic lad, there are fur bulk-data TCP cnnectins in each directin. Fr the simulatins in Figure 6 f a heavy traffic lad, there are twenty bulkdata TCP cnnectins in each directin. This is nt intended t be a realistic scenari; this is simply intended t illustrate that the perfrmance f nn-ecn gateways depends in part n the level f cngestin. The rundtrip prpagatin delays range frm 21 t 40 msec. Given 1000-byte packets, the bandwidth-delay prduct fr a single cnnectin ranges frm 118 t 225 packets. Each simulatin is run fr 10 secnds. The RED gateways in this simulatin have the same minimum and maimum threshlds fr the average queue size as described in the previus sectin. Hwever, as is apprpriate fr gateways intended fr widearea traffic, the time cnstant fr the lw-pass filter used t calculate the average queue size has been increased. 3 3 The weight used by the epnential weighted mving average The graphs in Figures 5 and 6 shw three sets f simulatins, as in the previus sectin. The first rw f graphs shws the aggregate thrughput f the bulk-data cnnectins that g frm a surce n the left t a receiver n the right, and the secnd rw f graphs shws the bulk-data cnnectin frm these that has the smallest thrughput. The third and furth rws f graphs shw the wrst-case and the average telnet packet delay fr telnet packets ging frm left t right. The fifth rw f graphs shws the average thrughput f the shrter data cnnectins with limited data t send. Nte that fr several f the graphs, the -ais in Figure 6 is different frm that in Figure 5. The simulatins use TCP cnnectins with a 100 msec TCP clck granularity. The simulatins with TCP cnnectins with a 0.1 msec TCP clck granularity give similar results, and are nt shwn here. 7.2 Results fr WAN simulatins The results f the WAN simulatins shw that, while thrughput is similar in all three sets f simulatins, the packet delay fr lw-bandwidth interactive traffic is much better fr the simulatins with ECN. The results are fairly similar fr the tw simulatin sets withut ECN, thse with Drp Tail gateways and thse with RED gateways. Hwever, it wuld be pssible t cnstruct WAN scenaris, like the LAN simulatins earlier in this paper, that illustrate sme f the advantages f RED gateways (with r withut ECN) ver Drp-Tail gateways [FJ93, VS94]. This culd prbably be the case, fr eample, fr scenaris that ehibit either glbal synchrnizatin and/r unfairness with Drp-Tail gateways. Nte that, fr bth Drp Tail gateways and RED gateways withut ECN, telnet packet delay is wrse with a larger number f bulk-data cnnectins r with TCP cnnectins with larger windws. In these simulatins with increased demand, an increased number f packet drps is required frm a nn-ecn gateway t cntrl cngestin. Fr the Drp-Tail gateways, the thrughput and delay perfrmance is particularly gd fr the simulatins with a smaller number f bulk-data cnnectins, smaller windws, and byte-based gateways. In this case the level f cngestin is fairly lw. Fr wide-area traffic, even fr thse cases where the surce has t wait fr a retransmit timer t detect a lst packet, a TCP clck granularity f 100 msec is nt a majr prblem, and the additinal delay t wait fr a retransmit timer has a less significant impact n perfrfilter t calculate the average queue size has been decreased frm t [FJ93].

10 Drp Tail, TCP clck 100 msec Bulk-Data Thrughput : byte-based DT, 64 kb windws : packet-based DT, 64 kb windws : byte-based DT, 128 kb windws : packet-based DT, 128 kb windws RED withut ECN, TCP clck 100 msec Bulk-Data Thrughput RED with ECN, TCP clck 100 msec Bulk-Data Thrughput Drp Tail, TCP clck 100 msec Smallest Bulk-Data Thrughput RED withut ECN, TCP clck 100 msec Smallest Bulk-Data Thrughput RED with ECN, TCP clck 100 msec Smallest Bulk-Data Thrughput Drp Tail, TCP clck 100 msec Highest Telnet Delay (in Secnds) RED withut ECN, TCP clck 100 msec Highest Telnet Delay (in Secnds) RED with ECN, TCP clck 100 msec Highest Telnet Delay (in Secnds) Drp Tail, TCP clck 100 msec Average Telnet Delay (in Secnds) RED withut ECN, TCP clck 100 msec Average Telnet Delay (in Secnds) RED with ECN, TCP clck 100 msec Average Telnet Delay (in Secnds) Drp Tail, TCP clck 100 msec Thrughput f Shrt Data Cnns. RED withut ECN, TCP clck 100 msec Thrughput f Shrt Data Cnns. RED with ECN, TCP clck 100 msec Thrughput f Shrt Data Cnns. Figure 5: WAN simulatins with a 100 msec TCP clck and a mderate number f bulk-data cnnectins.

11 Drp Tail, TCP clck 100 msec Bulk-data Thrughput : byte-based DT, 64 kb windws : packet-based DT, 64 kb windws : byte-based DT, 128 kb windws : packet-based DT, 128 kb windws RED withut ECN, TCP clck 100 msec Bulk-data Thrughput RED with ECN, TCP clck 100 msec Bulk-data Thrughput Drp Tail, TCP clck 100 msec Smallest Bulk-data Thrughput RED withut ECN, TCP clck 100 msec Smallest Bulk-data Thrughput RED with ECN, TCP clck 100 msec Smallest Bulk-data Thrughput Drp Tail, TCP clck 100 msec Highest Telnet Delay (in Secnds) RED withut ECN, TCP clck 100 msec Highest Telnet Delay (in Secnds) RED with ECN, TCP clck 100 msec Highest Telnet Delay (in Secnds) Drp Tail, TCP clck 100 msec Average Telnet Delay (in Secnds) RED withut ECN, TCP clck 100 msec Average Telnet Delay (in Secnds) RED with ECN, TCP clck 100 msec Average Telnet Delay (in Secnds) Drp Tail, TCP clck 100 msec Thrughput f Shrt Data Cnns. RED withut ECN, TCP clck 100 msec Thrughput f Shrt Data Cnns. RED with ECN, TCP clck 100 msec Thrughput f Shrt Data Cnns. Figure 6: WAN simulatins with a 100 msec TCP clck and a large number f bulk-data cnnectins.

12 mance. In this case, the simulatins with 100 msec TCP clcks are similar t thse with 0.1 msec TCP clcks. 8 Advantages and disadvantages f ECN In this sectin we discuss further sme f the advantages and disadvantages f ECN, bth fr the current Internet envirnment and fr varius prpsed mdificatins t that envirnment. As already discussed in the intrductin, the advantages f ECN include a reductin in packet drps and packet delay fr lw-bandwidth delay-sensitive TCP traffic and a prmpt ntificatin t end ndes f netwrk cngestin. 8.1 Disadvantages f ECN Tw disadvantages r ptential prblems with ECN cncern nn-cmpliant ECN cnnectins and the ptential lss f ECN messages in the netwrk. A nn-cmpliant TCP cnnectin culd set the ECN field t indicate that it was ECN-capable, and then ignre ECN ntificatins. Nn-cmpliant cnnectins culd als ignre Surce Quench messages. Hwever, fr a netwrk that uses nly packet drps fr cngestin ntificatin, a nn-cmpliant cnnectin culd als refrain frm making apprpriate windw decreases in respnse t packet drps. A nn-cmpliant cnnectin interested in reliable delivery cannt ignre packet drps cmpletely, but in the absence f mnitring and cntrls, a nn-cmpliant cnnectin culd cause cngestin prblems in either an ECN r a nn-ecn envirnment. A prblem with ECN messages that has n cunterpart with packet drps is that an ECN message (e.g., a Surce Quench message, r a TCP ACK packet with the ECN field set) culd be drpped by the netwrk, and the cngestin ntificatin culd fail t reach the end nde. Thus, neither Surce Quench messages nr the use f ECN fields in packet headers can guarantee that the TCP surce will receive each ntificatin f cngestin. Hwever, with RED gateways the gateway des nt rely n the surce t respnd t each cngestin ntificatin. The gateway will cntinue t set the ECN field in randmly-chsen packets as lng as cngestin persists at the gateway. In additin, a gateway implementing RED algrithms is particularly unlikely t drp a high fractin f packets. The ccasinal lss f an ECN message shuld nt be a serius prblem. 8.2 ECN with ther versins f TCP The simulatins in this paper use Ren-style TCP algrithms, mdified as suggested in this paper t respnd t ECN messages. In this sectin we discuss the implicatins f ECN fr sme prpsed mdificatins t TCP. One prpsed mdificatin t TCP, the additin f Selective Acknwledgements, wuld reduce TCP s reliance n the retransmit timer when multiple packets are drpped in ne rundtrip time. While this wuld imprve smewhat the rbustness f the respnse f bulkdata TCP cnnectins t packet drps as an indicatin f cngestin, this wuld nt reduce the ability f ECN mechanisms t reduce packet delay fr lw-bandwidth delay-sensitive TCP cnnectins. There are several prpsed mdificatins f TCP [BOP94, WC91, WC92] where the TCP surce wuld detect incipient cngestin frm the traffic dynamics f the cnnectin s traffic stream. Fr a netwrk such as the Internet withut admissins cntrl, a TCP surce cannt cmpletely prevent the netwrk frm drpping its packets. Nevertheless, with imprved detectin by TCP f incipient cngestin, the TCP surce culd reduce cngestin and cnsequently reduce the number f packets drpped. Hwever, the pssibility f imprved cngestin detectin by the end ndes des nt eliminate the need fr imprved cngestin detectin at the gateways. In particular, in the absence f prir infrmatin abut the fied prpagatin delay f a path, it is nt pssible fr end ndes t distinguish between prpagatin delay and persistent queueing delay (that is, queueing delay that eisted when the cnnectin was started, and that has persisted fr the duratin f the cnnectin). As the delay-bandwidth prduct f netwrk cnnectins increases, the buffer capacity as the gateways will als increase, and it will becme increasingly imprtant that these large buffers nt have persistent large queues. The detectin f persistent large queues is apprpriately dne in the gateway itself, and the ntificatin t the end ndes f this cngestin requires either packet drps r ECN. Thus, while prpsed mdificatins f TCP might decrease the number f packets drpped by the gateways as a result f buffer verflws, these mdificatins d nt eliminate the need fr sme frm f cngestin ntificatin frm the gateways t the end ndes. It seems unlikely t us that mdificatins f TCP with newer end-t-end cngestin avidance mechanisms wuld significantly reduce the usefulness f ECN. 8.3 ECN with prpsed mdificatins t ruter scheduling algrithms Other pssible changes in future netwrks such as the intrductin f pririty-based scheduling r f Fair- Queueing-based scheduling at the gateways culd cmplicate the traffic dynamics f TCP traffic in future net-

13 wrks. As an eample f pssible changes, the prpsed IPng header [H94] cntains a Flw Label with a 4-bit Traffic Class field identifying seven classes f flw-cntrlled (e.g., TCP) traffic, including classes fr unattended data transfer such as , attended data transfer such as FTP, interactive traffic such as telnet, and interactive cntrl traffic such as SNMP. These Flw Labels culd facilitate the use by gateways f separate queues r scheduling algrithms fr the different traffic classes [BCS94, F93]. The use f a separate traffic class fr interactive traffic wuld reduce packet drps fr this traffic during perids f lw demand frm the interactive traffic. Hwever, fr bth attended data transfer and interactive traffic, sme ntificatin f cngestin frm the gateways will cntinue t be required. Thus, the use f ECN mechanisms wuld still imprve the prmptness and rbustness f cngestin ntificatin fr data transfer cnnectins, and reduce unnecessary packet drps fr sme cnnectins in the interactive traffic class. While it is difficult t predict the future internet cngestin cntrl envirnment, and the precise advantages and disadvantages f ECN mechanisms in that envirnment, the basic advantages f ECN mechanisms f reducing unnecessary packet drps fr lw-bandwidth interactive traffic and f speeding the ntificatin f cngestin t bulk data cnnectins seem likely t remain. Further, in a hetergeneus internet sme ruters might find ECN mechanisms useful fr ECN-capable TCP cnnectins, withut ECN mechanisms being required fr all ruters. 9 Implementatin issues 9.1 Surce Quench messages vs. ECN fields in packet headers The guidelines in the paper fr the respnse f TCP surces t ECN messages are rthgnal t the questin f the mechanisms fr delivering this cngestin ntificatin t the surce. In this sectin we cmpare tw such mechanisms: Surce Quench messages, and ECN fields in packet headers. One advantage f using ECN fields in IP packet headers is that the ECN field places n additinal lad n the netwrk in times f cngestin. If a Drp-Tail ruter were t send a Surce Quench message fr every packet drpped at the ruter, the verhead f this Surce Quench traffic in times f cngestin culd be a significant prblem. Hwever, the use f intelligent gateway mechanisms such as thse in RED gateways wuld limit the verhead f the Surce Quench traffic. In additin, in particular envirnments such as LANs where the verhead f Surce Quench messages wuld be limited by the small number f hps n the return paths, the use f Surce Quench messages culd be quite justifiable. An advantage f Surce Quench messages (r ther frms f backward ECN ) ver frward ECN mechanisms such as ECN fields in packet headers is that with Surce Quench messages, the ntificatin f cngestin arrives at the surce as quickly as pssible. Althugh we have nt eplred the dynamics f backward ECN mechanisms such as Surce Quench in ur simulatins, this prmptness in the ntificatin f cngestin wuld be an advantage in cngestin cntrl schemes. Anther practical advantage f Surce Quench messages is that Surce Quench messages culd be used immediately in apprpriate netwrks, given mdified TCP implementatins with the respnse t Surce Quench messages prpsed in this paper. The use f an ECN field in TCP/IP netwrks wuld depend n the presence f the ECN field in IPng headers. Even if the evidence were sufficiently cmpelling t mtivate such an additin, it culd be years befre such headers were fully deplyed. As already mentined, bth Surce Quench messages and TCP acknwledgement packets with the ECN field set culd be drpped by a gateway, with the result that the TCP surce nde might never receive the cngestin ntificatin. If the netwrk drps a data packet that has the ECN bit set, the TCP surce will still infer cngestin when it detects the drpped data packet. Hwever, if the netwrk drps an ACK packet that has the ECN bit set, and the TCP surce later receives an ACK packet withut the ECN bit set that acknwledges a subsequent data packet, then the TCP surce will never receive the ntificatin f cngestin. Thus, neither Surce Quench messages nr ECN fields ensure reliable delivery f cngestin ntificatin. 9.2 Incremental deplyment f ECNcapable TCP One cncern regarding ECN fields in IP packet headers is with the incremental deplyment f ECN-capable TCP implementatins. If a gateway sets the ECN field in a packet header, hw des the gateway knw that the transprt-level prtcl is capable f respnding apprpriately? Fr an ECN field with tw bits, ne bit culd be used t indicate whether r nt the transprt prtcl culd respnd t ECN, and the secnd bit culd be used t indicate cngestin. Fr an ECN field with nly ne bit, these tw functins wuld have t be cmbined with a single bit. Fr an ECN field that cnsists f a single bit, ne value, say the OFF value, culd indicate ECNcapable Transprt Prtcl, and the ON value, culd

14 indicate either Nn-ECN Transprt Prtcl r Cngestin Ntificatin. Fr packets frm transprt-level surces that are nt capable f ECN respnse, the ECN field culd be set t ON (the default value). Fr packets frm transprt-level surces capable f ECN respnse, the ECN field culd be set t OFF. Nn-ECN-capable gateways wuld ignre the ECN field, simply drpping packets t indicate cngestin. ECN-capable gateways, seeing packets with the ECN field OFF, wuld knw that the crrespnding transprt prtcl was ECN-capable, and culd set the ECN field t ON fr apprpriate packets during times f cngestin. Fr arriving packets with the ECN field ON, the ECNcapable gateway wuld nt knw whether that packet came frm a nn-ecn-capable transprt prtcl, r whether the ECN field had been set by a previus gateway. In either case, if the gateway wanted t ntify a TCP surce abut cngestin, the gateway wuld drp the packet. This methd f incremental deplyment with a single-bit ECN field wuld mean that fr packets frm ECN-capable transprt prtcls, that packet wuld be drpped by a secnd ruter attempting t set the ECN bit. This can nly happen fr packets that pass thrugh multiple cngested gateways, where bth gateways chse that packet fr ntifying the surce f cngestin. Thus, ECN fields culd be deplyed in a hetergeneus envirnment where nly sme f the TCP implementatins were ECN-capable, and where nly sme f the ruters have prcedures fr setting the ECN field. Nte that this descriptin f a single-bit ECN field assumes a TCP cnnectin with ne-way traffic, where all f the data packets travel in ne directin and ACK packets travel in the ther. Fr a TCP cnnectin with tw-way data transfer, a secnd bit wuld be needed in the ECN field, r sme additinal mechanism wuld be needed t return an indicatin f cngestin frm the receiver t the surce. A cncern with incremental deplyment als eists fr Surce Quench messages. If a gateway wants t use Surce Quench messages, the gateway wuld nt knw whether the TCP implementatin was a ld implementatin with a fairly drastic respnse t Surce Quench messages, r a newer implementatin with the respnses recmmended in this paper. Hwever, in this case the prblem with lder implementatins wuld nt be that they wuld ignre Surce Quench messages entirely, but that they wuld back ff fr t lng in respnse t a Surce Quench message. This wuld be an incentive fr users t upgrade t newer TCP implementatins, given an envirnment with ruters using Surce Quench messages. 9.3 Imprving the TCP clck granularity? In the simulatins in this sectin, the advantages f ECN mechanisms in TCP/IP netwrks are mst prnunced fr LAN traffic with TCP implementatins limited by a carse-granularity TCP clck. This carse-grained TCP clck limits the granularity f TCP s measurements f current rundtrip times, used t determine the value fr the retransmit timer. The simulatin results in this paper, alng with ther results [RF94], argue in favr f imprving the granularity f TCP clcks. Unfrtunately, even if there were n hardware cnsideratins, and TCP designers culd set the TCP clck granularity t the ptimal value, it is nt bvius what that ptimal value shuld be. It seems clear (t us) that a TCP clck granularity f 100 msec (r slightly smaller) wuld be mre apprpriate fr current netwrks than the TCP clck granularity f 500 msec in many current TCP implementatins. Hwever, in the current algrithms fr setting the TCP retransmit timer, the carse granularity fr the TCP clck is deliberately used as a lw-pass filter t filter ut cmmn traffic variatins [J88]. This filtering implicitly accunts fr cmmn traffic dynamics such as interactins between lcal and lng haul traffic. Changing t an arbitrarily-fine-grained TCP clck (e.g., cnsiderably smaller than 100 msec) wuld remve this filtering, resulting in false retransmits in many scenaris. If a finegrained TCP clck were used, this filtering wuld have t be replaced by a substantially mre sphisticated estimatin prcess. The additin f ECN mechanisms t TCP/IP netwrks has the advantage f reducing the imprtance f the TCP clck granularity, thereby increasing the general rbustness f the netwrk. 9.4 TCP ver ATM The investigatin f ECN in this paper cncerns nly TCP/IP netwrks; we are nt cnsidering the varius prpsals fr ECN in ATM netwrks. In particular, we are nt cnsidering the cngestin cntrl strategies that might be used inside the ATM netwrks. We d, hwever, cnsider a scenari f TCP/IP traffic where part r all f the path might cnsist f ATM netwrks. The ATM netwrk needs mechanisms t infrm TCP cnnectins f cngestin. At the mment, the nly viable mechanism is fr the ATM netwrk t drp TCP/IP packets, either inside r at the bundaries f the ATM netwrk. If IP-level ECN mechanisms (e.g., Surce Quench, ECN fields in IP packet headers) were available fr ATM netwrks t infrm TCP surces abut cngestin, the ATM netwrks culd invke these mechanisms at the bundary f the ATM netwrks where frame segmenta-

15 tin and reassembly ccur. Fr eample, fr a TCP/IP netwrk where sme f the TCP surces were ECNcapable, the ATM bundary ruter culd drp TCP/IP packets t indicate cngestin t nn-ecn-capable TCP surces, and invke ECN mechanisms fr packets frm ECN-capable TCP surces. 10 Cnclusins and Future Wrk We have prpsed specific guidelines fr TCP s respnse t Surce Quench messages r t ther ECN mechanisms. We wuld prpse that these guidelines be used t mdify TCP s respnse t Surce Quench messages. If TCP implementatins had a mre clearlydefined respnse t Surce Quench messages, then netwrks such as ATM LANs culd cnsider whether r nt t use Surce Quench messages as a cntrlled ntificatin f cngestin t TCP/IP cnnectins traversing that netwrk. Fr a wide-area netwrk the verhead f Surce Quench messages makes their use prblematic. Hwever, the lgistical difficulties f adding ECN fields t IP packet headers makes the use f ECN fields prblematic as well. The prpsed IPng packet header [H94] has n space allcated fr an ECN field, and it is nt clear if IPng ptins, with a minimum length f 8 ctets, wuld be an apprpriate place fr an ECN field. The simulatins in this paper suggest the ECN mechanisms wuld give a clear, if mdest, benefit in TCP/IP netwrks. Hwever, we see this research as a preliminary investigatin f the advantages and disadvantages f ECN mechanisms in TCP/IP netwrks. A main advantage f ECN mechanisms is in aviding unnecessary packet drps, and therefre aviding unnecessary delay fr packets frm lw-bandwidth delaysensitive TCP cnnectins. This advantage will be mst prnunced in a highly-cngested netwrk where a high frequency f packet drps is required t cntrl cngestin. A secnd advantage f ECN mechanisms is in netwrks (generally LANs) where the effectiveness f TCP retransmit timers is limited by the carse granularity f the TCP clck. With ECN, the cngestin ntificatin is prmptly received by the TCP surce, and the cnnectin des nt remain idle, waiting fr a TCP retransmit timer t epire, after a packet has been drpped. While t sme etent the ver-carse granularity f the TCP clck culd be crrected, and the TCP retransmit timer algrithms suitably mdified, the use f ECN mechanisms, by reducing the number f packet drps, reduces the dependence n the retransmit timer. One disadvantage f ECN mechanisms discussed earlier in the paper is that ECN messages (e.g., Surce Quench messages, r TCP ACK packets with the ECN field set) culd be drpped by the netwrk befre reaching the TCP surce. Fr a TCP cnnectin, packet drps are a reliable (if smetimes slw) indicatin f cngestin. Preliminary simulatins f a wide-area scenari with tw-way traffic and multiple cngested gateways, sme with Drp-Tail gateways and sme with ECNcapable RED gateways, d nt shw perfrmance prblems frm drpped ECN messages. In additin, the number f drpped ECN messages shuld be small in a netwrk with ECN mechanisms and RED-style gateways. 11 Acknwledgements This wrk benefited frm nging discussins with Van Jacbsn. We als thank Vern Pasn, Allyn Rmanw, Curtis Villamizar, and tw perceptive annymus reviewers fr much helpful feedback. This wrk was mtivated in part by discussins with members f the Traffic Management Wrking Grup f the ATM Frum. References [A94] Almquist, P., Requirements fr IP Ruters, Wrking Draft, IETF, draft-ietf-rreq-iprutersrequire-01.tt, April [BCS94] Braden, B., Clark, D., and Shenker, S., Integrated Services in the Internet Architecture: an Overview, Request fr Cmments (RFC) 1633, IETF, June [BP87] Braden, R., and Pstel, J., Requirements fr internet gateways, Request fr Cmments (Standard) RFC 1009, IETF, June [BOP94] Brakm, L.S., O Malley, S.W., and Petersn, L.L., TCP Vegas: New Techniques fr Cngestin Detectin and Avidance, Prceedings f SIGCOMM 1994, pp [DJ91] Danzig, P., and Jamin, S., tcplib: A Library f TCP Internetwrk Traffic Characteristics, Reprt CS-SYS-91-01, Cmputer Science Department, University f Suthern Califrnia, Available via FTP t ecalibur.usc.edu as pub/jamin/tcplib/tcplib.tar.z. [F93] Flyd, S., Link-sharing and Resurce Management Mdels fr Packet Netwrks, t be submitted t IEEE/ACM Transactins n Netwrking. Available by annymus ftp frm ftp.ee.lbl.gv in papers/link.ps.z, September 1993.

16 [F94] [FJ93] [H94] [J88] [N94] Flyd, S., TCP and Successive Fast Retransmits, technical reprt, Octber Available by annymus ftp frm ftp.ee.lbl.gv in papers/fastretrans.ps. Flyd, S., and Jacbsn, V., Randm Early Detectin Gateways fr Cngestin Avidance, IEEE/ACM Transactins n Netwrking, V.1 N.4, August 1993, p Hinden, R., IP Net Generatin Overview, IETF internet draft, wrk in prgress, Octber Jacbsn, V., Cngestin Avidance and Cntrl, Prceedings f SIGCOMM 1988, August 1988, pp (An updated versin f this paper is available by annymus ftp frm ftp.ee.lbl.gv in cngavid.ps.z.) Newman, P., Traffic Management fr ATM Lcal Area Netwrks, IEEE Cmmunicatins Magazine, Vl. 32 N. 8, August 1994, pp [WRM91] Wilder, R., Ramakrishnan, K.K., and Mankin, A., Dynamics f Cngestin Cntrl and Avidance in Tw-Way Traffic in an OSI Testbed, ACM Cmputer Cmmunicatins Review, V.21, N.2, April 1991, pp [PP87] Prue, W., and Pstel, J., Smething a Hst Culd D with Surce Quench, Request fr Cmments (RFC) 1016, July [RF94] Rmanw, A., and Flyd, S., Dynamics f TCP Traffic ver ATM Netwrks, Prceedings f SIGCOMM 1994, August 1994, pp [RJ90] Ramakrishnan, K.K., and Jain, R., A Binary Feedback Scheme fr Cngestin Avidance in Cmputer Netwrks, ACM Transactins n Cmputer Systems, Vl. 8 N. 2, pp , [S94] W. Stevens, TCP/IP Illustrated, Vlume 1, Addisn-Wesley, Reading, MA, [VS94] Villamizar, C., and Sng, C., High Perfrmance TCP in ANSNET, submitted t Cmputer Cmmunicatins Review, September [WC91] Wang, Z., and Crwcrft, J., A New Cngestin Cntrl Scheme: Slw Start and Search (Tri-S), ACM Cmputer Cmmunicatin Review, V.21 N.1, January 1991, pp [WC92] Wang, Z., and Crwcrft, J., Eliminating Peridic Packet Lsses in the 4.3-Tahe BSD TCP Cngestin Cntrl Algrithm, ACM Cmputer Cmmunicatin Review, V. 22 N. 2, April 1992, pp