1 Quality of Service on the Internet: Evaluation of the IntServ Architecture on the Linux Operative System 1 Elisabete Reis Polytechnic Institute of Coimbra Fernando Melo Polytechnic Institute of Guarda Marília Oliveira Gonçalo Quadros Edmundo Monteiro Communication and Telematics Laboratory Center for Informatics and Systems of the University of Coimbra Pólo II, Pinhal de Marrocos, 33 Coimbra Abstract 1 Recent Linux kernels offer a wide variety of traffic control functions 2. These cover the mechanisms required to support the Integrated Services architecture developed in the IETF. This document analyses the ability of Traffic Control enabled routers, under the Linux operating system, to provide the desired service grade. I. INTRODUCTION Today s Internet provides only a best-effort service, where all traffic is equally treated and there are no delivery guarantees. However, emerging applications have very diverse Quality of Service (QoS) expectations and thus, the current Internet model becomes inadequate and limiting. One proposal to support QoS on the Internet is the IETF Integrated Services 3 (IntServ) approach. This proposal specifies the Guaranteed Service  and the Controlled-Load Service . Its implementation requires that network elements contain the following modules: Admission Control, Classifier and Scheduler of packets. These modules constitute the Traffic Control. Under the Controlled-Load Service, applications are treated similarly to the case when it is deployed a best-effort service with light loads. This service is designed for adaptive realtime applications that can tolerate variance in packet delays. The Guaranteed Service guarantees an assured level of bandwidth, firm bounds on end-to-end queuing delays, and that packets will not be discarded due to queue overflows. This service is intended for applications highly sensitive to delays, as real time video. The specification of IntServ needs a signalling mechanism such that applications can require the appropriate QoS level. The Resource Reservation Protocol (RSVP)  is the standard signalling protocol for the IntServ architecture. This paper is organized as follows: in section II is presented a brief description of QoS support in Linux; the testbed is described in section III; experiments and discussion of results are presented in section IV; section V contains the major conclusions and topics that will be treated in future work. II. QoS SUPPORT in LINUX Linux kernel 4 supports a number of advanced networking features, including QoS. The QoS support provides a framework for the implementation of various IP QoS models like Integrated Services and Differentiated Services 5, in a module generically denominated Traffic Control (TC). The Traffic Control module consists of four building blocks: queuing disciplines, classes, filters and policing. Currently there are many queuing disciplines supported in Linux , including Class Based Queuing (CBQ), Clark-Shenker-Zhang (CSZ), Priority, Token Bucket Flow (TBF), Stochastic Fair Queuing (SFQ), Random Early Detection (RED) and First In First Out (FIFO). In the Linux implementation tested, CBQ is the only discipline that is able to handle RSVP and Integrated Services, and thus it is the discipline used in this work. This implementation closely follows the research work done by Sally Floyd [5, 6, 7 and 8]. In brief terms, the CBQ discipline assumes that a flow that has been accepted by the admission control module for guaranteed service will be assigned its own class with the highest priority [Floyd1998]. Since all flows belong to classes that have the same priority, they will be served in weighted round robin (WRR). 1 This work was partially supported by the Portuguese Ministry of Science and Technology (MCT), under program PRAXIS XX I-PROJECT /P/EEI/1168/1998 (Project QoS II Quality of Service in Computer Communication Systems). 2 4 <ftp://linux.wauug.org/pub/net/ip-routing/ip-route2-current.tar.gz.> 5 <http://www.ietf.org/html.charters/diffserv-charter.html> 3 <
2 III. DESCRIPTION OF EXPERIMENTAL TESTBED In order to test the capability of the Linux Traffic Control implementation to support the Integrated Services architecture, we used the testbed shown in Figure 1. This testbed employs two end-hosts connected through three serially interconnected routers. All computers, except Console, run the Linux operating system, RedHat 5.2, kernel 2.2.8, and have RSVP installed. The Traffic Control modules were installed and configured in the kernel of the routers to accommodate RSVP operation. The Console machine runs the MS Windows 98 operating system and accommodates the Chariot Console application 6. supported). The reclassified traffic class was created to accommodate non-conformant reclassified guaranteed traffic. Besteffort Root 45% 5% 5 % Prio=4 Control Messag. Prio=2 IntServ Guarant. 1 Guarant. 15 Reclass. Traffic Prio=1 Prio=1 Prio=6 Router A 1 Mbits Router B 1 Mbits Router C Fig. 2 Link-sharing structure for the interface. Path InterServ Routers 1 Mbits 1 Mbits Resv In the tests, we generated several combinations of data flows, belonging to the QoS classes considered (guaranteed and best-effort). The traffic best-effort was generated by Mgen 7 tool and the traffic of guaranteed flows by the Chariot tool. Sender Chariot Console Receiver Fig. 1 - Testbed utilised to test the capability of the Linux traffic control implementation. The Sender and Router A have Intel Celeron processors at 366Mhz. Routers B and C have Intel Celerons processors at 45Mhz. All of them have 64 Mbyte of RAM. The Receiver has a Pentium II processor at 35Mhz and 32Mbyte of RAM. The Console machine is a Pentium III with 45Mhz CPUs and 128M byte of memory. The link between the Router A and Router B was configured to 1Mbps. All the others links are 1Mbps. The discipline Class-Based Queuing was activated and configured on the output interfaces of all routers. The linksharing structure configured in the interfaces is shown in Figure 2. The total capacity of the link was distributed as follows: 45% of bandwidth was allocated to best-effort traffic; 5% to RSVP control messages traffic; and 5% to IntServ traffic (a maximum of 15 guaranteed flows are 6 Chariot (Demo Version) <http://www.ganymede.com/products/chariot/index.phtml> During these experiments, a number of parameters were captured and examined to gain insight on the service provided by routers. These parameters included maximum queuing delay, loss rate and bandwidth (QoS parameters that reflect the characteristics of the Guaranteed Service), number of packets put with success in the queue, number of packets reclassified and number of packets waiting in the queue. To capture these parameters, a number of tools were employed including ttt 8 and an improved version of tc 9. The Linux kernel was modified in order to allow for the measurement of maximum and medium packet queuing delays. IV. EXPERIMENTS This document analyses the ability of Traffic Control enabled routers, under the Linux operating system, to provide the desired service to applications whose flows have been accepted for guaranteed service. The results presented in this paper concern the evaluation of the characteristics of the router where exists the bottleneck (router A). Since the other routers of the testbed are connected at 1Mbps, all traffic, including best-effort traffic, is forwarded without delay and losses in the queues. 7 Mgen - Multi-generator - <www.iihe.ac.be/mice-nsc/ftp/util/mgen> 8 ttt - Tele Traffic Tapper - <http://www.csl.sony.co.jp/person/kjc/software.html> 9 tc user-level program written by Alexy and possibilita a communication and configuration of the kernel code or modules.
3 We started the evaluation of this implementation of the guaranteed service by doing validation tests concerning the Admission Control and Packet Classifier mechanisms. The Admission Control module showed a satisfactory behaviour. However, we verified that the bandwidth provided by the CBQ discipline to the IntServ class was not completely used by the flows with reservation. The evaluation of the Classifier was successful, because we verified that the packets were put on the correct queues. Although not presented here, we verified in preliminary tests, that consecutive RSVP control packets were dropped, leading to false soft timeout. It is thus questionable whether control messages should be sent as best-effort packets. In the light of these results, we decided to create a separate class with high priority for these packets. After the validation tests, we conducted two groups of experiments. In the first set, we did performance tests, evaluating the service provided to flows under different traffic loads. In the second set, we evaluated the ability of routers to provide the required level of service for a variable number of flows, doing a scalability study. The evaluation was done according to the following parameters: queuing delay, loss percentage and throughput. A. Performance tests The main objective of these tests is to evaluate the performance of Traffic Control enabled routers under different traffic loads. Table I - Traffic mix. Guaranteed Service Load (Mbps) Best-Effort Traffic (Mbps) Total Load (Mbps) 4 X,5 2 4 X, X, X, X, X, X, X, X, X, X, X, The total load generated in each test is composed of traffic belonging to guaranteed flows and best-effort traffic. In all tests, were generated four guaranteed flows, with a reserved bandwidth of,5 Mbps each. The remaining of the total load is composed of best-effort traffic according to the values presented in table 1. With these test conditions we generated a total load of 2%, 4%, 6%, 8%, 9%, 1%, 11%, 12%, 14%, 16%, 18% and 2% of the capacity of the output interface of router A, that is, the router where it is located the bottleneck. Since the load of the guaranteed flows remains constant, best-effort traffic is responsible for achieving the loads described above. 6% 5% 4% 3% 2% 1% % Losses - RouterA Fig. 3 - Losses rate to different loads in Router A. The analysis of the results of these tests showed that the flows with reservation did not suffer any loss, even with high loads (Figure 3). However, best-effort traffic shows a considerably different behaviour. This type of traffic does not suffer losses until congestion occurs, that is, when 9% of the interface capacity is attained. After this moment, the percentage of dropped packets belonging to best-effort traffic rises significantly, with the level of congestion in the router TotalLoad [% ] Maximum Queing Delay - RouterA TotalLoad [% ] Fig. 4 - Maximum queuing delay to different loads in RouterA. Figure 4 shows that when the router becomes congested (9% to 1%), there is a noticeable increase in the maximum queuing delay experienced by guaranteed flows (although far below the maximum delay allowed to guaranteed traffic). After this, the maximum queuing delay remains approximately the same as the load in the router
4 increases, until the maximum load considered. These results show that the router is able to provide the desired delay bound to guaranteed flows. Since the same amount of resources was allocated to all guaranteed flows, we expected that the maximum queuing delays suffered by their packets would be equivalent. However, Figure 4 shows that the maximum queuing delay of flow 4 is slightly higher than the values obtained by the other flows with reservation, particularly for a total load of 1 Mbps (1% of the interface capacity). This is due to the fact that this is the last flow to be processed and at this point the router does not have the ability to serve all flows with the same performance. 5,% 4,5% 4,% 3,5% 3,% 2,5% 2,% 1,5% 1,%,5%,% Losses - RouterA N º G uaranteed F lo w s Best-effort ControlM. Guranteed1 Guranteed2 Guranteed3 Guranteed4 Guranteed5 Guranteed6 Guranteed7 Guranteed8 Guranteed9 Guranteed1 Guranteed11 Guranteed12 Guranteed13 Guranteed14 When the router becomes congested, the best-effort flow shows, like the guaranteed flows, an increase in the maximum queuing delay. The best-effort flow has two major differences relatively to guaranteed flows: the maximum queuing delay increases more significantly and keeps increasing for higher loads, even though, not as sharply. This result is due to the fact that for situations of extreme congestion, the queues are full, the maximum queuing delay is achieved and performance degradation is reflected in the loss of packets. The analysis of the tests presented in this section showed that, under all traffic loads introduce in the network, the commitment that the guaranteed flows should receive the reserved bandwidth was respected. The remaining bandwidth was occupied by best-effort traffic. B. Scalability tests The deployment of Quality of Service in the Internet requires solutions that are scalable. This is thus a major characteristic upon which a solution like the one evaluated in this paper must be tested. For this purpose, we have injected a constant load of 1 Mbps into the network, but with a different number of flows. We conducted test with 4, 6, 8, 1, 12 and 14 guaranteed flows. The evaluation was achieved comparing the service given to the first four flows, when there is besteffort traffic, and in situations where there are 2, 4, 6, 8 and 1 additional flows with guaranteed service requirements. The analysis of the tests described above showed that guaranteed flows did not experience any losses, even with the larger number of guaranteed flows (Figure 5). These results prove that the implementation of guaranteed service evaluated satisfies the commitment of providing a service without losses. Fig. 5- Losses rate to different number of guaranteed flows in Router A. As we stated before, in this set of tests, the total load is kept constant. However, Figure 5 shows that, as the number of guaranteed flows increases, the percentage of best-effort packets that are dropped rises significantly. This is due to the fact that lower priority flows are only served after the higher priority flows. In this case, the scheduler will spend more time processing the high priority flows, degrading the performance of best-effort traffic Maximum Queuing Delay - RouterA N º G uaranteed Flo w s Guaranteed5 Guaranteed6 Guaranteed7 Guaranteed8 Guaranteed Fig. 6- Maximum queuing delay to different number of guaranteed flows in Router A. Figure 6 shows that the maximum queuing delay experienced by the guaranteed flows increases with the number of guaranteed flows (left axis), contributing to service degradation. Best-effort traffic (right axis) performance is also damaged by the existence of a higher number of guaranteed flows. This fact is even more significant, since the degradation occurs for smaller loads of best-effort traffic. The reason for this observation is that since there are more high priority flows, the scheduler spends less time processing packets from the best-effort queue, and this traffic is delayed even for lower loads.
5 V. CONCLUSIONS and FUTURE WORK The guarantees provided relatively to bandwidth were also evaluated according to the test conditions described in this section. We verified that all flows with guaranteed service received the bandwidth that was previously reserved C. Discussion of results Several important conclusions can be drawn from this performance study of Traffic Control capable routers to support Integrated Services and provide the Guaranteed Service. The algorithms implemented to provide guaranteed service are efficient in the control of packet loss rate. The flows that have been accepted to the guaranteed service can expect a null loss rate, independently of the state of load and of the number of guaranteed flows through the router. We can verify, in a similar manner, that the maximum queuing delay suffered by guaranteed flows is controlled and kept at a satisfactory level, independently of the state of load at the router. However, the maximum queuing delay, achieved by guaranteed flows, increases significantly when the number of guaranteed flows that are active is higher, for the same load. These results lead to the conclusion that the guarantees given by routers to flows when they have been accepted can be jeopardized in the presence of a higher number of flows. The measurements reported in this paper were performed on a relatively small testbed and at low line speeds. So, we believe that this implementation can introduce scalability problems when used in wide area networks. The guarantees provided relatively to bandwidth were also evaluated. We verified that all flows with guaranteed service received the bandwidth that was previously reserved, independently of traffic load and of the number of guaranteed flows. The analysis of the evaluation parameters of best-effort traffic shows that the number of guaranteed flows contributes to the degradation of best-effort traffic performance. During the tests we verified that it is important to send RSVP control messages at high priority. This is due to the fact that if they are handled as best-effort traffic, during congestion consecutive control packets are dropped or delayed leading to false soft timeout. In this case the guarantees given to guaranteed flows might be questioned. In this paper we presented the analysis of the performance of the IntServ architecture on Linux. We presented and discussed results that show the ability to control packet loss rate even when the router is in a congestion state. However, we verified that the service provided suffers degradation, relatively to the maximum queuing delay, when there is a higher number of guaranteed flows. Future work will include tests using different traffic patterns, including bursty traffic, TCP and UDP flows. We will also do tests where the amount of total bandwidth of the interface dedicated to IntServ is 25% and 75% of its capacity. The evaluation of the Integrated Services architecture when the Controlled-Load Service is used with the Guaranteed Service will also be done. VI. REFERENCES  S. Shenker, C. Partridge, R. Guérin, Specification of Guaranteed Quality of Service, Internet Engineering Task Force, Network Working Group, Request For Comments 2212, September  J. Wroclawski, Specification of the Controlled-load Network Element Service, Internet Engineering Task Force, Network Working Group, Request For Comments 2211, September  R. Braden, L. Zhang, S. Berson, S. Herzog, S. Jamin, Resource ReServation Protocol (RSVP) Version 1 Functional Specification, Request For Comments 225, September  S. Radhakrishnan, Linux Advanced Networking Overview Version1, August  S. Floyd, V. Jacobson, Link-sharing and Resource Management Models for Packet Networks, IEEE/ACM Transactions on Networking, Vol. 3 No. 4, pp , August  S. Floyd, Notes on CBQ and Guaranteed Service Draft document, July  S. Floyd, Notes of Class-Based Queueing: Setting Parameters, Informal notes, September  S. Floyd, M. Speer, Experimental Results for Class- Based Queueing, September 1998, not published.