A NEW ACTIVE QUEUE MANAGEMENT ALGORITHM BASED ON NEURAL NETWORKS PI. M. Yaghoubi Waskasi M. J. Yazdanpanah

Transcription

1 A NEW ACTIVE QUEUE MANAGEMENT ALGORITHM BASED ON NEURAL NETWORKS M. Yaghoub Waskas M. J. Yazdananah N. Yazdan Control and Intellgent Processng Center of Excellence Electrcal and Comuter Engneerng Deartment Unversty of Tehran P.O. Box: 14395/515, Tehran, IRAN Abstract: Actve Queue Management (AQM) ales a sutable control olcy uon detectng congeston n networks. In ths aer, an adatve Proortonal-Integral () controller based on Artfcal Neural Networks (ANN) s aled to AQM for the objectve of congeston avodance and control n mddle nodes. The roosed controller s smle and can be easly mlemented n hgh-seed routers. Neural Network (NN) dynamcally adats ts arameters wth resect to changes n the system. It s antcated that ths results n better resonse comared to lnear controllers due to the nonlnear nature of NN. We smulated our method n ns2 and comared ts erformance wth the conventonal controller. The smulaton results show NN yelds better erformance. Coyrght 25 IFAC Keywords: Neural Networks, Queung Network Models, Nonlnear Systems, Comuter Networks, Controllers. 1. INTRODUCTION Congeston s a major roblem n data networks and t s consdered an actve and a hot research area for researchers (Low, et al., 22). Intally, ths roblem ntroduced by Jacobson (1988), n the last 8 s, t caused roosng dfferent mroved versons of TCP n order to have more control on congeston. Most of the roosed methods for congeston are mlemented on end nodes whle there s no secal control olcy n mddle nodes and ackets are smly droed when the queues are full, known as DroTal (Low, et al., 22). In 1993, RED (Random Early Detecton) was ntroduced to regulate queues lengths n mddle nodes based on a desred olcy (Floyd, Jacobson, 1993). Ths aroach, named Actve Queue Management (AQM), manages the queue length by drong ackets randomly before the queue buffer overflows. Researchers have found that the basc RED does not meet ther requrements and t causes some nstablty n the network when the number of sources ncreases or the arameters of the network change (Hollot, et al. 21a). Even n some cases ts erformance s worse than the smle DroTal olcy (Chrstansen, et al., 2). Some new aroaches based on the rmary RED have been ntroduced to mrove the erformance of RED and solve ts challenges (Ott, et al., 1999; Feng, et al., 1999a). Some new methods also have been roosed such as BLUE (Feng, et al., 1999b) and GREEN (Feng, et al., 22) to acheve better erformance and rovde more sohstcated control olcy. Defntely, usng a mathematcal model can gve a more general vew of a dynamcal model. We can

2 also better analyze the behavor of the system and ts transent resonse to an nut based on a mathematcal model. Due to the nonlnear and tme varant dynamcs of the global comuter network, some researcher attemted to descrbe ts behavor by some mathematcal formula (Floyd, 21; Msra, et al., 2). One of the best models of ths tye about TCP s a model that was develoed by Msra, et al. (2), whch s based on the flud flow model. In ths model, the robablty of acket dro s calculated accordng to the control law. Therefore, one s rmary goal s to desgn a controller that roduces arorate control sgnals. The frst controller of ths model was a smle controller roosed by Hollot, et al. (21a) whch gves a better erformance than RED. Unfortunately, studes roved that AQM wth controller s not robust n resonse to uncertantes n the network and ncreasng the number of sources (Fengyuan, et al., 22). Therefore, desgnng a more robust algorthm became a hot research toc (Fengyuan, et al., 22). Some researchers tred to ntroduce non-model-based controllers because of ther robustness n arameters changng (Fengyuan and Xumng, 22; Fatta, et al., 22). Certanly, any new AQM algorthm should have better erformance and easy to mlement. In ths aer, we aly an adatve controller based on ANN to AQM n order to control traffc n mddle nodes and avod congeston n the network. Our algorthm s smle to mlement and have a more satsfactory erformance n comarson to the exstng mechansms. NN dynamcally adats ts arameters wth changes n the system. Due to the nonlnear nature of NN, t s antcated that our controller results n better resonse than those of lnear controllers desgned before. The rest of the aer s organzed as follows: In Secton 2, we ntroduce the actve queue management scheme. Secton 3 resents the mathematcal modelng of congeston n comuter networks. In Secton 4, the structure of NN s dscussed. Smulaton result s resented n Secton 5. Fnally, Secton 6 concludes the aer. 2. ACTIVE QUEUE MANAGEMENT In the algorthms revously roosed, the router was drong ackets when ts queue buffer was full. Ths olcy s known as DroTal. One of the most mortant weaknesses of DroTal s that the router dros any arrvng ackets even a short-lved flow, known as mce, when long-lved flows, known as elehants, fll the buffer. Ths conflcts wth the well resected farness rncle of comuter networks. Meanwhle, the varaton of the queue length n mddle nodes causes jtter, varaton n the delay, n acket delvery, whch s not desred for real-tme alcatons. The amount of delay exerenced n the network deends on the amount of round tr tme of ackets. Equaton 1 descrbes ths relevance. q( R ( a + (1) C Index ndcates flow number and a s a constant ndcatng the roagaton delay of the lnk. C and q ( are lne caacty and the queue length resectvely. The second art of equaton 1 reresents the tme wasted to rocess ncomng ackets. Wth fxng the queue length, snce a and C are constants, we can guarantee a constant round tr tme delay. Therefore, AQM can hel to rovde Qualty of Servce (QoS) n networks. RED was the frst roosed AQM olcy whch dros all arrvng ackets f the average queue length exceeds the maxmum threshold. In ths model routers rocesses and sends all ackets f the average queue length was smaller than the mnmum threshold. When the average queue length les between the maxmum and mnmum thresholds, ackets are droed wth a robablty of, whch s roortonal to the average queue length. The RED behavor s llustrated n fgure 2. Dro 1 max Probablty Data Packets Congested Router Routed Packets Lost Packets tmn tmax Queue Length Acknowledgement Fg. 1. A mddle node router Fgure 1 shows a mddle node router that ales the actve queue management olcy for ackets arrvng to ts queue. Fg. 2. RED drong olcy Indeed, AQM acts as a regulator for the queue length regardng the control theory ont of vew. The nut of ths controller s the degree of congeston measured by the average and the current queue lengths. The outut s the robablty of acket drong or markng. The control block dagram of comuter networks s llustrated n fgure 3.

3 Desred Queue Length halves the congeston wndow sze n acket loss due to the multlcatve decrease n AIMD. Equaton 2 models an end to end system. We should also model the mddle node s behavor or the queue length varaton dynamc. So we have: Fg. 3. AQM as a controller There are many control theoretc-based aroaches for AQM such as, F, REM, AVQ and etc. In the next secton, we ntroduce a mathematcal model for comuter networks whch hels us to desgn a controller for AQM. 3. MATHEMATICAL MODEL OF COMPUTER NETWORKS A ractcal model of actve queue management based on flud flow was roosed n 2 and verfed by some comuter smulatons, whch has ntroduced an accetable accuracy for the roosed model (Msra, et al., 2). The behavor of acket loss n ths model was consdered as a Posson rocess. Then, a stochastc dfferental equaton s derved based on some assumtons. Consder a router wth N sources. As exlaned revously, the round tr tme of ackets n each flow s as below: dq( W C + (4) dt N 1 R ( q) The decreasng art of ths equaton corresonds to the lne caacty, whle the ncreasng art s the number of ackets arrvng to the queue. Hollot, et al., (21), assume a constant round tr tme and suose same sources and rewrte equaton 2 and 4 as the followng (Hollot, et al. 21b): 2 W t W& 1 ( ) ( ( t R ) R 2R W ( q& ( N C R (5) Indeed, they gnored some delays n the model arameters and clearly stated that these assumtons are accetable. Equaton 5 s nonlnear wth tme delay n ts nut. The AQM control law should roduce a sutable robablty, (, whch eventually yelds a good erformance. Fgure 4 reresents an overall vew of the AQM controller oston. q( R ( a + C Now, we can determne the congeston wndow sze by: dw dt W dn (2) R ( 2 Where dn s equal to 1 when the acket loss occurs. Equaton 2 conssts of two ncreasng and decreasng arts. The ncreasng art ndcates the addtve ncrease of the AIMD algorthm whle the decreasng art ndcates the multlcatve decrease. Chu and Jan (1989) try to rove that AIMD acheves far and effcent network utlzaton. AIMD ncreases congeston wndow sze by one n each round tr tme. Ths occurs n the congeston avodance hase of TCP. Equaton 2 only consders the congeston avodance hase of TCP. Ths s true for elehant flows such as FTP flows, snce the tme of slow start n long-lved flows s small. To model ths hase, we have: Wdt dw (3) R ( In ths equaton, we gnore the effect of the acket loss n the slow start stage. The decrease secton Fg. 4: AQM block dagram 4. AND NN CONTROLLERS The roosed controller n (Hollot, et al. 21a) calculates the robablty of dro usng the error and ntegral of the error between the current queue sze and the desred queue length. A formal controller s as follows (Frankln, et al., 1995): ( K + K τ ) dτ (6) Where K and K are the roortonal and ntegral coeffcents and the equaton of error s: q( q desred (7) t

4 controller yelds a zero steady state error because of the ntegral term. The most challengng roblem n desgnng a controller s choosng the arorate K and K. In (Hollot, et al. 21a), the authors lnearzed the nonlnear equatons and desgned a robust and stable controller usng ths lnear model. They assumed a mnmum number of sources and a maxmum amount of round tr tme for ther uncertanty boundares. However, t can be shown that the controller dose not oerate well when the number of flows ncreases or the arameters of network changes. We ntroduce a neural network-based (Matsukuma, et al., 1997) controller that adats ts arameters wth resect to the dfference between the current queue length and the desred queue length. Consder the one layer feedforward neural network that s shown n fgure 5. t τ) dτ Fg. 5. A smle neural network-based (NN) e ( s the error functon and f (x) s the actvaton functon of neuron. We have: ( Whle x K f ( x) K K + K t τ ) dτ f (x) (8) We choose the actvaton functon as a hyerbolc tangent. The maxmum value of the hyerbolc tangent s 1, so t s sutable for robablty functon that ts maxmum s 1. 2kx 1 e f ( x) tanh( kx) (9) 2kx 1+ e In ths equaton, k nstances for the sloe of hyerbolc tangent functon. S ( A lower k yelds a smoother actvaton functon and a hgher k yelds a sharer actvaton functon. The hyerbolc tangent functon for k 1 s shown n fgure 6. To udate the weght of neural networks, we use the error back roagaton rule. Usng the steeest decent, we have: K K η new old K K new η old (1) Where E s the cost functon. The cost functon can be defned as the sum of square errors, and so we have: Therefore, 1 2 E Σ( ) (11) 2 ( q qdesred ) f ( x) t ( q qdesred ) f ( x) τ ) dτ (12) We choose 1 for convenence. For the ntal weght of the neural network, we use the roosed coeffcents mentoned by Hollot, et al., (21). Neural network wll change ther weght that acts as coeffcent accordng to system roertes. Our exerments show that the learnng rates of order of 1-12 are sutable for ths roblem. 5. SIMULATION RESULTS We evaluated the effectveness and erformance of the roosed method va smulatons usng ns2 smulator. The network toology for our smulatons s shown n fgure 7. In these smulatons, we consder a network wth a lnk caacty of 15Mbs. The only bottleneck lnk les between node 1 and node 2. Our smulaton duraton was 1 seconds. controller s gans were chosen as the coeffcent derved n (Hollot, et al. 21a). The learnng rates of the neural network were equal to 2e-1. Fg. 7. Network toology of our smulatons Fg. 6. The actvaton functon of neural network Smulaton 1: For the frst exerment, we set the round tr tme to 246 mllseconds and consdered

5 4 ft sources connected to node 1. The mean ackets sze of TCP ackets was 5 bytes. The desred queue length was 2 ackets whle the hyscal lmt of our queue was 8 ackets. The queue length resonses for and NN are dected n fgure 8. Consderng ths fgure, t s clear that the roosed method should results n a better erformance and queue regulaton than that of the controller. Our roosed method guarantees the maxmum queue length of 2 ackets whle the AQM wth oscllates around 2 ackets. Varaton of the queue length n the algorthm causes jtterng n delay that s not desred n multmeda alcatons. 8 7 ackets. Whle controller oscllates heavly, the NN regulates the queue length to the desred value. Smulaton 5: In ths exerment we decrease the tme delay of the lnk to 7 mllseconds. It s obvous from fgure 12 that NN controller acts better n ths test bed n comarson to controller NN NN Tme (seconds) Fg. 1. N2, RTT246ms, Q Refrence 2 kts Tme (seconds) 7 Fg 8. N6, RTT246ms, Q Refrence 2 kts Smulaton 2: Now, we ncrease the number of sources to 4. Whle has a sluggsh resonse, our method acheves a more accetable and faster resonse (fgure 9) NN NN Tme (seconds) Fg. 9. N4, RTT246ms, Q Refrence 2 kts Smulaton 3: We tested the controllers at one of the end of stablty sectrum by reducng the number of sources to 2. As llustrated n fgure 1, controller exhbts oscllatons whle the NN controller oerates n a relatvely stable mode. Smulaton 4: Fgure 11 shows the queue length lots when we ncrease our desred queue length to Tme (seconds) Fg. 11. N6, RTT246ms, Q Refrence 4 kts NN Tme (seconds) Fg 12: N4, RTT7ms, Q Refrence 2 kts

6 6. CONCLUSION We roose a new method for actve queue management based on Neural Network controller. Our algorthm adats tself n resonse to uncertantes n network arameters. Our method s smle to mlement and deloy n hgh seed routers. To nvestgate the erformance of the roosed controller, some comuter smulatons have been done usng ns2 smulator. The smulatons results show that NN controller has a sueror erformance than that of the classcal controller n dfferent networks. Our next ste wll be studyng the effect of unresonsve flows lke UDP on the AQM. REFERENCES Chu D. and R. Jan (1989). Analyss of the Increase/Decrease Algorthms for Congeston Avodance n Comuter Networks, Journal of Comuter Networks and ISDN, Vol. 17, No. 1, Chrstansen M., K. Jeffay, D. Ott, and F.D. Smth (2). Tunng RED for web traffc, Proceedngs of ACM/SIGCOMM. Fatta G. D, G. Lo Re, and A. Urso (22). A Fuzzy Aroach for the Network Congeston Problem, Comuter Communcatons, vol. 25, , Elsever Scence. Feng W., Dl D. Kandlur, Debanjan Saha, and Kang G. Shn (1999a). A Self-Confgurng RED Gateway, Proceedngs of IEEE/INFOCOM. Feng W., et al. (1999b). Blue: A New Class of Actve Queue Management Algorthms, Techncal Reort CSE-TR , Unversty of Mchgan. Feng W., A. Kaada, and S. Thulasdasan (22). GREEN: Proactve Queue Management over a Best-Effort Network, Proc. of IEEE Globecom 22. Fengyuan R., L. Chuang, Y. Xunhe, S. Xumng, and W. Fubao (22). A Robust Actve Queue Management Algorthm Based on Sldng Mode Varable Structure Control, Proc. Of IEEE INFOCOM. New York US: IEEE, Fengyuan R. Y. R. and S. Xumng (22). Desgn of a Fuzzy Controller for Actve Queue Management, Comuter Communcatons, vol. 25, , Elsever Scence. Floyd S. and V. Jacobson (1993). Random Early Detecton Gateways for Congeston Avodance, IEEE/ACM Trans. Net., vol. 1, no. 4, Floyd S. (21). A Reort on Recent Develoments n TCP Congeston Control, IEEE Commun. Mag., vol. 39, no. 4, Frankln G. F., J. Davd Powell, and Abbas Emam- Naen (1995). Feedback Control of Dynamc Systems, Addson-Wesley. Hollot C. V., Vshal Msra, Don Towsley, and We- Bo Gong (21a). On Desgnng Imroved Controllers for AQM Routers Suortng TCP Flows, Proc. of INFOCOM 21. Hollot C. V., Vshal Msra, Don Towsley, and We- Bo Gong (21b). A Control Theoretc Analyss of RED, Proceedngs of IEEE/INFOCOM. Jacobson V. (1988). Congeston Avodance and Control, Proc. ACM SIGCOMM 88, Low S., F. Pagann and J. Doyl (22). Internet Congeston Control, IEEE Control System Magazne, vol. 22, no. 6, Matsukuma T., A. Fujwara, M. Namba, and Y. Ishda (1997). Non-Lnear D Controller Usng Neural Networks, IEEE Msra V., W. Gong, and D. Towsley (2). Flud- Based Analyss of a Network of AQM Routers Suortng TCP Flows wth an Alcaton to RED, Proc. of ACM SIGCOMM 2, avalable at ft://gaa.cs.umass.edu/ub/ Ott T. J., T. V. Lakshman, and L. H.Wong (1999). SRED: Stablzed RED, Proceedngs of IEEE/INFOCOM.