Proceedigs of the World Cogress o Egieerig ad Computer Sciece 0 Vol II WCECS 0, October 4-6, 0, Sa Fracisco, USA Performace Aalysis over Software Router vs. Hardware Router: A Practical Approach Edward Guille, Aa María Sossa ad Edith Paola Estupiñá Abstract For implemetatio of layer 3 coectivity, it is possible to use multiple vedor equipmets with hardware ad software based solutios. Performace results betwee close source ad ope source routig solutios are importat parameters for etwork desigers. Performace aalysis i covergece time, throughput ad delay betwee routig solutio based o virtual software router (VSR) ad routig solutio based o proprietary hardware router (PHR) is preseted. Results show that a VSR have a better covergece time i compariso with hardware router ad the throughput performace is better o PHR. Idex Terms Covergece Time, Delay, Performace Measuremet, Routig Solutios, Throughput. I. INTRODUCTION Beefits such as flexibility, scalability, affordability, security, admiistratio ad relatioship betwee cost ad profit are importat cosideratios whe choosig a routig solutio. This is because the better routig solutio performace, the more reliable will be the implemetatio ad later use. The etwork admiistrator s decisios are supported by performace aalyses. The aalyses help to seek ad evaluate differet choices of hardware ad software routig solutios to achieve ad optimal etwork performace. These aalyses have bee made i order to determie how to evaluate specific routig solutios through performace metrics ad the comparisos betwee routig solutios were made with multiple etworks scearios. Evaluatios made i this paper are idepedet ad it caot be see as a positive or egative opiio agaist ay vedor or provider, it is just a techical result of etwork tests. Over the last few years, several research ad developmet efforts have bee dedicated to aalyze performace over routig solutios. I 999, IETF itroduces RFC 544 Bechmarkig Methodology for Network Itercoect Devices. This documet preseted a methodology to measure the performace of data etwork devices []. I 007, Tolly Group elaborated differet test to compare the E. Guille is with Telecommuicatios Egieerig Departmet, Military Uiversity Nueva Graada. Bogotá, Colombia. (e-mail: edward.guille@uimilitar.edu.co). A. Sossa is with Telecommuicatios Egieerig Departmet, Military Uiversity Nueva Graada. Bogotá, Colombia (e-mail: gissic@uimilitar.edu.co) E. Estupiñá is with Telecommuicatios Egieerig Departmet, Military Uiversity Nueva Graada. Bogotá, Colombia. (e-mail: edith.estupia@uimilitar.edu.co). ISBN: 978-988-95-4-4 ISSN: 078-0958 (Prit); ISSN: 078-0966 (Olie) stadard hardware betwee routig solutios versus. Throughput ad delay were measured o routig solutios [], [3]. I march 009, IXIA a testig performace s provider made a descriptio to measure the covergece time o data etwork. Although IXIA described how to measure the performace of routig protocols [4]. I this paper a geeral overview of etworkig routig solutios ad its features is preseted. I additio a comparative aalysis of routig performace measuremets such as covergece time, throughput ad delay betwee a closed source routig solutio ad ope source is show. This paper is orgaized as follows. I sectio II basics cocepts of routig solutios are preseted, sectio III shows the routig performace measuremet, sectio IV presets the test scearios. Sectio V explais the simulatio results ad its aalysis ad fially sectio VI presets coclusios. II. NETWORK ROUTING SOLUTIONS: BASICS AND FOUNDATIONS A routig solutio fids a route for data packets from a source to a destiatio poits. These solutios are desiged to satisfy the eeds the user s data etwork. The users might be compaies or service providers [5]. I data etworks, routig solutios are classified i ope source or closed source, the classificatio is show i the Fig.. Fig. Routig solutio classificatio. Closed source solutios ruig o hardware ad ope source solutios ruig o software ad hardware. A. Networkig Routig Solutio: A Ope Source Overview Ope software solutios might be viable alteratives to expesive proprietary etwork devices [3]-[6]. Vedor s ope system ca scale better i eterprise or service provider edge deploymets. The ecoomic advatages of ope solutios provide users a cost-effective way to icrease performace that is uattaiable with proprietary routig solutios. A beefit of ope solutios is that it ucouple software from hardware. This advatage permit to achieve software features ad service extesibility with a truly itegratio eviromet [7]. WCECS 0
Proceedigs of the World Cogress o Egieerig ad Computer Sciece 0 Vol II WCECS 0, October 4-6, 0, Sa Fracisco, USA Software Router (SR) is low-cost optio ad moderate performace [8]. A SR with stadard hardware x86 parameters permits to have routig equipmets with high performace for the eterprises [9]. A SR ruig i a virtual eviromet is the called a Virtual Software Router (VSR). A host workstatio may be able to allocate a sigle or multiple VSR. The mai differece betwee SR ad a VSR is most of the VSR fuctios rus o software while SR iterfaces rus o hardware. Therefore, the performace of a VSR is expected to be lower tha a SR. For this reaso, sigificat research efforts are focused i optimizig the iteral architecture of Ope Software Router (OSR), its researches are cocetrated o developig strategies to build more powerful routig devices [6]-[8]. Ope source based hardware appliace is composed by routig software ad stadard hardware platforms. The ope etworkig solutio removes ay reliace o proprietary hardware [0]. B. Networkig Routig Solutio: A Closed Source Overview Hardware-Appliace closed source routig solutios are sold with proprietary software ad hardware. Market offers several routig solutios such as,, Juiper, Alcatel, amog others. The costraits of hardware ad software ad the applicatios cotrol limit users to have scalable etwork []. Istead of buyig a closed source hardware-appliace a Iteret Services Provider (ISP) or a etwork admiistrator could use logical elemets ruig o SR; this choice may deped of the eeds of etwork [6]. The aim of this research is to perform a compariso betwee a closed routig solutio i hardware i.e. ad a routig solutio ope source i.e ruig as VSR. Table I shows the features comparatives of this routig equipmet []. TABLE I COMPARATIVE FEATURES CISCO VS. VYATTA CHARACTERISTIC VYATTA CISCO System Operatig Liux IOS Ope Source No Static Routig Routig Protocols (RIPv-RIPNG, OSPFv-OSPFv3,BGP) NAT, VPN IPSec, VPN SSL No FTP Cliet, TFTP Cliet, Telet. SSH Server /No HTTP Server, DHCP Server, DHCP Cliet SNMP C. Network Routig: Protocols Routig protocols are better kow as dyamic routig protocols ad these are classified i Iterior Gateway Protocols (IGP) ad Exterior Gateway Protocols (EGP) [3], [4]. The classificatio of routig protocol is show i Fig.. Fig.. Routig protocols classificatio accordig to routig domai. This Figure shows a classificatio accordig to algorithm used for each routig protocol. III. ROUTING PERFORMANCE MEASUREMENT I data etworks performace measuremets are ecessary for uderstadig curret data etwork behavior. The measuremet aalysis help to idetify weakesses o data etwork ad it will allow evaluatig how to fix it [5]. There are may performace metrics used to evaluate etwork performace, such as: throughput, delay, jitter, covergece time, badwidth ad packet loss [6], [7]. A. Covergece Time Accordig to The etwork has coverged whe all routers have complete ad accurate iformatio about the etwork [8]. I the route covergece process, all compoets of router are updated, as well as the Routig Iformatio Base (RIB) ad Forwardig Iformatio Base (FIB) icludig the most recet route chages of etwork topology or iformatio about available or uavailable of etwork s liks [9]. The measure of how fast the routers reach the state of covergece is called covergece time ad it is give i secods [0]. I the updatig route each etwork s router rus a routig algorithm to recalculate metrics ad build a ew routig table based o this iformatio. Oce all routig tables have bee updated, covergece is completed. A time diagram is a useful tool to uderstad how to measure the covergece time. Time diagram is show i Fig. 3. Fig. 3. Time Diagram for measurig covergece time betwee two routers. These figure shows the update process over routig table whe occur a failure i a router s etwork. ISBN: 978-988-95-4-4 ISSN: 078-0958 (Prit); ISSN: 078-0966 (Olie) WCECS 0
Proceedigs of the World Cogress o Egieerig ad Computer Sciece 0 Vol II WCECS 0, October 4-6, 0, Sa Fracisco, USA Covergece time C T is give by equatio () ad it is described by two parameters: Reactivatio Time T (Reactivatio), Error Time T (Error) [9]. CT = T T () C = T T B. Delay T reactivati o error Delay D of a packet i data etworks is time take up by the packet to reach the destiatio after it leaves the source []. Delay is usually affected by etwork ifrastructure due to the queuig at switches ad routers []. Network delay is composed by trasmissio, queuig ad propagatio. Delay icludes playback delay, delay for buffer-jitter ad processig delay at destiatio termial [3], [4]. Figure 4 shows a possible sceario to measure delay o cliet. The relatio time is defied by two parameters: T A is time whe the ACK is received o server to idicate cofirmatio of the packet receptio ad T B - is the time whe a ew packet is received i the cliet side. The time relatioship is give by equatio () relatio betwee the user-data load L ad the trasmissio time t [5]. Throughput is represeted by equatio (5) ad it is obtaied by dividig the file size give i bits betwee the time take by the data i reachig its destiatio i secods. The throughput is usually give i Mbps. L T = (5) t IV. NETWORK SCENARIOS Performace ad quality of service (QoS) aalysis was performed over two differet scearios. The scearios are based o both routig architectures ad. Routig protocols RIPV ad OSPFV were cofigured. Throughput, covergece time ad delay measuremets were aalyzed. These routig performace measuremets were made with a etwork aalyzer i cliet ad the etwork server. Fially, the aim is to evaluate the etwork performace betwee both routig solutios. A. Base Sceario : Base sceario represets a traditioal Local Area Network (LAN) ad it is show i Fig. 6. The etwork sceario is composed by three routers, a FTP server ad a cliet. Fig.4. Time diagram to measure delay i cliet. This figure shows data trasmissio o data etwork, it process is useful for measuremet delay o cliet. T cliet = T T () B A time diagram to measure the server delay is show i fig 5. The time relatioship o server is give by equatio (3) ad it is described by two parameters: T C idicate the time whe a ACK is received o server side, this ACK idicate the cofirmatio of receptio packet. T D - is the time whe a packet is received i the cliet side [3]. A Fig. 6. Base Sceario B. Base Sceario : Base sceario is show i Fig 7. This etwork sceario has the same equipmets that were used i base sceario. Fig.5. Time diagram i server side. This figure shows the time elapsed betwee the receptio of a ACK i FTP server ad the sedig a ew packet. TServer = TD T (3) C Fially equatio (4) shows total Delay of a sedig file: Delay = T Cliete T Server (4) C. Throughput Throughput is the maximum rate at which oe of the offered frames are dropped by device ad it s give by the Fig.7. Base Sceario Some specific tests were desiged to perform a comparative performace aalysis. The test descriptios are show i table II. TABLE II SCENARIOS DESCRIPTION Base Sceario Test N Protocol Routig Solutio Test RIPV CISCO Test RIPV VYATTA Test 3 OSPFV CISCO Test 4 OSPFV VYATTA Test 5 RIPV CISCO Test 6 RIPV VYATTA Test 7 OSPFV CISCO Test 8 OSPFV VYATTA ISBN: 978-988-95-4-4 ISSN: 078-0958 (Prit); ISSN: 078-0966 (Olie) WCECS 0
Proceedigs of the World Cogress o Egieerig ad Computer Sciece 0 Vol II WCECS 0, October 4-6, 0, Sa Fracisco, USA C. Equipmet Characteristic CISCO: Techical specificatio of selected router is show table III. TABLE III CISCO ROUTER -TECHNICAL SPECIFICATIONS Characteristics RAM Memory Capacity Etheret Port Serials Port Router 8 768MB Max. 56 MB compact flash 3 Fast Etheret (0/00Mbps) VYATTA: router was cofigured over virtual machie. Table IV presets the techical specificatio. TABLE IV VYATTA ROUTER -TECHNICAL SPECIFICATION Characteristics Virtual Machie VYATTA RAM Memory 5 MB Capacity Hard Disk 8 GB Etheret Port 3 Fast Etheret (0/00Mbps) Serials Port 0 Operatig Systems Widows XP Virtualizatio Software Oracle VM Virtual Box 3..8 V. RESULTS I order to study the performace of each routig solutio, the tests defied i table II was performed. I these tests the performace metrics metioed i sectio III was measured. Also routig performace metrics were studied with statistical parameters such as: Arithmetic mea () ad Stadard deviatio (). The aim is to fid average ad dispersio about covergece time, throughput ad delay. Arithmetic mea is give by equatio (6), where β represets data measure i each metric. Stadard deviatio is give by equatio (7) ad was calculated i each case. = = βi i = i = ( βi ) Fig.8. Histogram of covergece time measured o sceario usig IPV4. With RIPV OSPFV. I Fig. 9 the best time of covergece is betwee the iterval 0- s for, ad Fig.9 the iterval time is betwee -4 s for. (6) (7) A. Base Sceario ad : Covergece Time Measuremets of covergece time are summarized i a histogram for base scearios ad. The results are show i Fig 8 ad Fig 9. The histograms show covergece time itervals of the tests performed. I each test was calculated with the equatio 6 ad with the equatio 7. This measuremet results show that i the base-scearios ad, covergece time is smallest o. I Fig 8 the best time of covergece is ito the iterval 6-8 s for ad Fig 8 the iterval time is betwee 4-6 s. ISBN: 978-988-95-4-4 ISSN: 078-0958 (Prit); ISSN: 078-0966 (Olie) Fig.9. Histogram of covergece time measured o sceario usig IPV4 i ad with RIPV OSPFV. B. Base Sceario ad : Delay Delay i base sceario is show i Fig. 0 ad delay i base sceario is show i Fig. The tests show that the delay is affected by the itercoectios routers ad also there are ot a relevat differece betwee ad Fig 0 shows that most represetative time of delay is betwee the iterval 0.60-0.69 ms for ad Fig 0 the iterval time is betwee 0.40-0.60 ms for. WCECS 0
Proceedigs of the World Cogress o Egieerig ad Computer Sciece 0 Vol II WCECS 0, October 4-6, 0, Sa Fracisco, USA Fig.0. Histogram of delay measured o sceario usig IPV4. With RIPV OSPFV Fig shows that there is ot a represetative time iterval ad it behavior is the same for ad. Although delay presets a bigger spread of data i all measuremet i the Fig is show that the most represetative iterval time is betwee 0.40-0.60 ms for. Fig.. Histogram of throughput measured o sceario usig IPV4. With RIPV OSPFV I Fig 3 ad Fig 3, the most represetative iterval is the same ad its value is 34-36 Mbps. Throughput stability is better o but shows a better throughput value. Fig.. Histogram of delay measured o sceario usig IPV4. With RIPV OSPFV C. Base Scearios: Throughput Throughput i base sceario is show i Fig ad Fig. 3 shows throughput results o base sceario. I Fig the most represetative value of throughput is i the iterval 3-34 Mbps ad Fig is the 30-3 Mbps. Fially it is possible to coclude that have a smaller spread of data. This behavior shows its stability. Fig.3. Histogram of throughput measured o sceario usig IPV4. With RIPV OSPFV Table V shows the performace results ad icludes statistical parameters results such as: ad. Routig Protocol N RIPV RIPV OSPFV OSPFV RIPV RIPV OSPFV OSPFV TABLE V ROUTING PERFORMANCE MEASUREMENTS Routig Covergece Delay(ms) Solutio Time(s) 44,5 0,68 0,85 0,45 4,87 0,69 0,4 0, 68,75 0,48,69 0,7 7,08 0,6 5,54 0, 07,45,0,05,83 6,4,8 5,49 0,4 04,97 0,57,53 0,44 6,99,3 6,6 0,55 Throughput (Mbps) 69,96 4,30 3,0,93 7,76,6 30,35,96 35,96 6, 34,87 0,80 47,6,99 34,84 0,80 ISBN: 978-988-95-4-4 ISSN: 078-0958 (Prit); ISSN: 078-0966 (Olie) WCECS 0
Proceedigs of the World Cogress o Egieerig ad Computer Sciece 0 Vol II WCECS 0, October 4-6, 0, Sa Fracisco, USA Fially, Fig. 4 shows the behavior of Arithmetic mea () for each data measured, covergece time, delay ad throughput. (c) Fig.4. Metrics measured o scearios usig IPV4. covergece time measured i secods delay measured i ms. (c) throughput measured i Mbps. Fially, this Figure allows to summarize the data s aalize. Covergece time is shorter i tha. Delay does t have chages i the measuremets, except i sceario ad (c) throughput shows that has a better performace tha. VI. CONCLUSION Geeral performace measuremets showed high stability i stadard deviatio for routig solutios, with a predictable behavior for covergece time, delay ad throughput desig parameters. A covergece time i is low. These results show the software speed to fix a faulty commuicatio lik. Covergece time over i compariso with is 70% better while has a better throughput. The Delay results were similar i both routig solutio (see Fig ). The possibility of modifyig the physical features VSR, such as: RAM memory, hard disk or etwork iterfaces is easier. The modificatio over physical features allows to have better performace o VSR i compariso with dedicated routig solutios. ACKNOWLEDGMENT We would like to thak to Security ad Commuicatio System Research Group GISSIC ad to Military Uiversity Nueva Graada. Bogotá, Colombia. REFERENCES [] S. Brader, J. Mc Quaid. RFC 544: Bechmarkig Methodology for Network Itercoect Devices. March 999.. [Olie] Available : http://www.ietf.org/rfc/rfc544.txt [] VYATTA Ic. White Paper: Why is better tha, March 007. [Olie] Available: http://www.vyatta.com/dowloads/whitepapers/_better_tha_c isco.pdf [3] Tolly Group. Tolly Report:.. Competitive Gigabit Etheret LAN RoutigThroughput Evaluatio versus 8 Itegrated Service Router. March 007. [Olie] Available: http://www.cs.ait.ac.th/~o//tolly0790 March007.pdf [4] IXIACOM. Measurig Network Covergece Time. March 009. [Olie] Available: http://www.ixiacom.com/library/white_papers/dowload.php?mkt_paper= covergece [5] Deepakar Medhi Karthikeya Ramasamy. Network Routig Algorithms, Protocols, ad Architectures. ISBN 3: 978-0--088588-6. [6] Adrea Biaco, Robert Birke, Luca Giraudo ad Nafag Li, Multistage Software Routers i a Virtual Eviromet ISBN: 978--444-5638-3 [7] Tolly Group. Tolly Report:,Ic. BGP Performace Evaluatio versus 704VXR Router. March 008. Page [Olie] Available: http://www.cs.ait.ac.th/~o//tolly0889bgpperfmar0 08.pdf [8] Roberto Rojas-Cessa, Khodaker M. Salehi, ad Komla Egoh Experimetal Performace Evaluatio of a Virtual Software Router New Jersey Istitute of Techology, Newark, NJ 070, USA. [9] Solutios. i Box. [Olie] Available: http://www.vyatta.com/solutios/eterprise/o-virtualized [0] Solutios. Hardware Appliaces[Olie] Available: http://www.vyatta.com/solutios/physical/appliaces [] ad Juiper, a compay threateed by ope source routig CIO España. y Juiper, ameazados por ua compañía de routig de código abierto. 006 [Olie] Available: http://www.idg.es/cio/estructura/imprimir.asp?id=468&cat=ot [] Héctor Egea Chueca ad Daiel Morató Osés. Adaptatio of routers use of laboratory practices routers Ope Source, Adaptació del uso de routers e prácticas del laboratorio a routers Ope Source. Pamploa. 0 [3] Net workig Academy. CCNA Exploratio 4.0 Routig Protocols ad Cocepts. Itroductio to Dyamic Routig Protocols Chapter 3. Itroductio ad Advatages. [4] Net workig Academy. CCNA Exploratio 4.0 Routig Protocols ad Cocepts. Itroductio to Dyamic Routig Protocols Chapter 3. Classifyig Dyamic Routig Protocols. 3.. IGP ad EGP. [5] Shufe, L. Lu, H. Xijia, Z. Study of Network Performace Measuremet Based o SNMP. The 8th Iteratioal Coferece o Computer Supported Cooperative Work i Desig Proceedigs.003. [6] Y. T, Dog, Y. Zhag L. Network Performace Measuremet Ad Aalysis. Part : A server-based Measuremet Ifrastructure. [7] Mohammad, N. Md. Ahsa Simulatio-Based Comparative Study of EIGRP ad OSPF for Real-Time Applicatios. September 00 [8] Net workig Academy. CCNA Exploratio 4.0 Routig Protocols ad Cocepts. Itroductio to Dyamic Routig Protocols Chapter 3. Classifyig Dyamic Routig Protocols. 3..5 Covergece. [9] Iteret Egieerig Task Force (IETF). RFC 64. Termiology for Bechmarkig Lik-State IGP Data-Plae Route Covergece pp.5. ISSN 070-7. [0] CCNP. Advaced Routig Compaio Guide. Press. 004. pp. 93f. ISBN -5873-35-8 [] Ahhas El Gamal, James Mamme. Balaji Rahhakar, Devavrat Shah Throughput-Delay Trade-off i Wireless Networks Departmets of EE ad CS Stalord Uiversity, 004. ISSN: 0-7803-8355-9. [] I. Baroak, M. Halas, Mathematical Represetatio of VoIP Coectio Delay, Departmet of Telecommuicatios, Slovak Uiversity of Techology, Prague, Slovak Republic, April 008. [3] E. Guille, D. Padilla, J. Alvarez, Throughput Evaluatio o Network Firewall Architectures. New Advaced Techologies, Aleksadar Laziica (Ed.), ISBN: 978-953- 307-067-4, INTECH, March 00. [4] E. Guille, D. Chaco, VoIP Networks Performace Aalysis with Ecryptio Systems, World Academy of Sciece, Egieerig ad Techology, Issue 58, October 009, ISSN: 070-374, pp 688-695. [5] S. Brader, J. McQuaid. RFC 4: Bechmarkig Termiology for Network Itercoectio Devices. Julio 99. [Olie] Available: http://www.ietf.org/rfc/rfc4.txt ISBN: 978-988-95-4-4 ISSN: 078-0958 (Prit); ISSN: 078-0966 (Olie) WCECS 0