Performance Measurement of IP Networks using the Two-Way Active Measurement Protocol INGMAR BÄCKSTRÖM

Performance Measurement of IP Networks using the Two-Way Active Measurement Protocol INGMAR BÄCKSTRÖM Master of Science Thesis Stockholm, Sweden 2009

Performance Measurement of IP Networks using the Two-Way Active Measurement Protocol INGMAR BÄCKSTRÖM Master s Thesis in Computer Science (30 ECTS credits) at the School of Computer Science and Engineering Royal Institute of Technology year 2009 Supervisor at CSC was Olof Hagsand Examiner was Karl Meinke TRITA-CSC-E 2009:038 ISRN-KTH/CSC/E--09/038--SE ISSN-1653-5715 Royal Institute of Technology School of Computer Science and Communication KTH CSC SE-100 44 Stockholm, Sweden URL: www.csc.kth.se

Abstract Computer networking is becoming a new standard for exchanging information. Many new services have been introduced to IP networks, and real time applications such as streaming media and Voice over IP are becoming more popular. These types of services are sensitive to disturbances; therefore it is important that the providing networks meet their requirements. To verify the quality of a network and to gather knowledge of its characteristics, several methods of network performance measurements have been developed. This master s thesis investigates active end-to-end measurements, and presents notions, metrics and methods for measuring the quality of IP networks. An implementation of the Two-way Active Measurement Protocol, TWAMP, has been made, and evaluated by comparing the results with three other active measurement tools. TWAMP is the first standardized protocol of its kind as it collects two-way performance data in one measurement. Metrics such as latency, packet loss, packet duplication and packet reorder have been measured, and the results show that TWAMP is a competitive protocol for network performance measurements.

Sammanfattning Tjänstekvalitetsmätning av IP-nätverk med TWAMP Datornätverk spelar en allt större roll vid utbyte av information. Många nya tjänster har gjorts tillgängliga för IP-nätverk, och realtidsapplikationer som mediaströmmar och IP-telefoni blir alltmer populära. Dessa tillämpningar är känsliga för störningar, och det är viktigt att det underliggande nätverket uppfyller tjänsternas krav. För att verifiera nätverksanslutningens prestanda och för att undersöka dess egenskaper har flera metoder för tjänstekvalitetsmätningar utvecklats. Det här examensarbetet undersöker aktiva mätningar mellan två ändpunkter. Det beskriver begrepp, mätvärden och metoder som används för att bestämma kvaliteten av IP-nätverk. Ett nätverksprotokoll med detta syfte, TWAMP, the Two-way Active Measurement Protocol, har implementerats och utvärderats gentemot tre andra aktiva mätverktyg. TWAMP är det första protokoll av sitt slag som har standardiserats, då det samlar tvåvägsinformation i en mätning. Mätvärden som latens, förlorade paket, dubbletter och paket som levereras i oordning har undersökts, och resultaten visar att TWAMP är ett bra alternativ för att genomföra tjänstekvalitetsmätningar av IP-nätverk.

Acknowledgements This master s thesis was conducted at Prosilient Technologies, and I would like to thank my colleagues for their input and for their encouragement. I would especially like to thank my supervisor Olof Hagsand for his support and for the time that he has invested in this project, as well as for giving me the opportunity to gain a better understanding of the field of network performance measurement. Ingmar Bäckström, Stockholm, April 2009.

Contents 1 Introduction 1 1.1 Background................................ 2 1.2 Purpose.................................. 3 1.3 Outline.................................. 3 2 Network Performance 5 2.1 Notions.................................. 5 2.1.1 Singletons, Samples and Statistics............... 5 2.1.2 Sampling............................. 5 2.1.3 Timing Considerations...................... 6 2.2 Metrics.................................. 8 2.2.1 Connectivity........................... 8 2.2.2 Packet Loss............................ 9 2.2.3 Loss Pattern........................... 9 2.2.4 Round-trip Delay......................... 10 2.2.5 One-way Delay.......................... 10 2.2.6 Delay Variation.......................... 10 2.2.7 Reordering............................ 12 2.2.8 Duplicates............................. 13 2.2.9 Further metrics.......................... 14 3 Methods 15 3.1 The One-way Active Measurement Protocol.............. 15 3.1.1 OWAMP-Control......................... 17 3.1.2 OWAMP-Test........................... 18 3.1.3 Current Implementations.................... 18 3.2 The Two-way Active Measurement Protocol.............. 19 3.2.1 TWAMP-Control......................... 20 3.2.2 TWAMP-Test........................... 20 3.2.3 TWAMP Extensions....................... 20 3.2.4 Current Implementations.................... 21 3.3 Reference methods............................ 21 3.3.1 Ping................................ 21

3.3.2 PTAnalyzer............................ 21 4 Implementation and Results 23 4.1 TWAMP Light.............................. 23 4.2 Evaluation................................. 24 4.3 Results................................... 27 4.3.1 Netem Test Sessions....................... 27 4.3.2 Internet Test Sessions...................... 28 5 Summary and Conclusion 31 5.1 Discussion................................. 31 5.2 Improvements and Recommendations.................. 33 5.3 Conclusion................................ 33 Bibliography 35 Appendices 38 A Format of Test Packets 39 B Internet Test Sessions 41

Chapter 1 Introduction The usage of IP based networks has increased enormously during the last decade and the trend is evident; the Internet continues to grow rapidly. According to a forecast, Internet traffic increased with 46 percent in 2007 and is estimated to grow by another 51 percent in 2008[1]. The annual growth rate will then slightly decrease, and by 2012 the total annual IP traffic is estimated to reach about 500 exabytes per year. Internet video, excluding peer to peer file sharing, accounts today for one quarter of all consumer Internet traffic. The forecast points out that high definition video will be responsible for a majority of the increase in bandwidth usage. Along with the quick growth of media streams and video conferences, Voice over IP, VoIP, is also becoming more popular. Although it is not as bandwidth consuming as visual media, this service still requires available bandwidth to operate, and as all interactive real time applications, it is sensitive to quality disturbances in the network. To assure the quality of these services, it is important that the providing Internet connection fulfils the requirements of capacity and available bandwidth. Network performance measurement is not a new phenomenon, but lately, this field has come more in focus. This science, also known as communication network metrology, is very wide, and several research projects have been launched with different purposes[2][3][4][5]. Some studies investigate traffic characterization and traffic modelling, whereas others focus on network topology or path symmetry. The results of the projects are useful for several parties: researchers can adopt network characteristics in simulation models to develop new network protocols; Internet Service Providers, ISPs, can determine network trends for resource capacity planning and discover performance bottlenecks; network equipment designers might find the data useful when designing routers, and the end-users are interested in measuring the quality of the Internet connection they are paying for. Consumers using real time services with Quality of Service constraints, QoS, often negotiate QoS levels with their ISP through a Service Level Agreement, SLA. Network performance measurement is an efficient approach for customers to verify the SLA, as well as a marketing strategy for ISPs to demonstrate the capacity and reliability of their networks. 1

CHAPTER 1. INTRODUCTION 1.1 Background Network performance measurements can be divided into two different categories: active and passive monitoring[6][7]. When performed passively, the network is only observed, and packet headers are captured and analyzed. The passive mode can yet again be divided into two sub types. At a microscopic level, the passive measurement analyzes every packet passing through the measurement node and notes its characteristics. When measuring at a macroscopic level, packets passing through the node are aggregated into flows and then analyzed. Passive network performance measuring techniques are especially useful for traffic engineering, as they describe flow dynamics and distribution. A drawback with passive monitoring is that it collects and stores a great amount of data and consumes a lot of memory. This type of measurement typically presents information from only one node in the network. It is difficult to collect data from a packet travelling between measurement points, and even though it can be done, matching packet information from different nodes is a difficult task. Active monitoring techniques are on the other hand well suited for performing end-to-end measurements between nodes in a network. Measurement data, or probe packets, are injected into the network. The packets are transmitted between the test end points, and measurement data is collected upon the packets arrival. This method can also be divided into different sub categories. Cooperative measurement tools consist of a client and a dedicated server for collecting measurement metrics. When applying non cooperative tools, only one client program is used. Measurement data is sent to an existing, independent server, such as a web server, and its response is used for computing the result. Other tools rely on generated replies, such as ICMP messages. A problem with active measurement techniques is that they generate traffic, and may influence the behaviour of the network. This problem is known as intrusiveness or invasiveness. When designing active measurement tools, it is therefore important that the amount of data generated by the probe packets does not affect the network performance. Approaches for Active Network Measurements When performing active measurements, probe packets are typically timestamped at the source host, transmitted by the network to the destination host where another timestamp is recorded. When a sufficiently large number of test packets have been transferred between the end points, statistics such as packet loss, latency, packet reorder and packet duplication may be computed. For a long time, the Ping command[8] has been used for measuring the roundtrip time, RTT, between two hosts. The one-way delay between the nodes has then been calculated as half the RTT[9]. However, several studies[10][11] show that path asymmetry often prevail between end points in a large network, and this approximation is therefore incorrect. 2

1.2. PURPOSE Several sophisticated measurement tools[12] have been developed in order to give a more accurate representation of communication networks. Measurements distinguish the forward path from the reverse path, probing packets can be shaped to resemble actual traffic and tools can be used to monitor the network performance from an end-user s point of view. Active measurement techniques also have the advantage of considering the underlying network as a black box. No knowledge is needed about the network topology or how individual routers treat the traffic. These methods generate input packets and analyze the packets output by the network. This makes the procedure simple but yet powerful as test endpoints, either software or hardware nodes, can easily be moved to different locations in the network to isolate a malfunctioning link. Several different tools for active network measurement exist, but these techniques lack interoperability. The Internet Engineering Task Force, IETF[13], is an open international community developing Internet standards. It consists of many work groups, and the IP Performance Metrics work group, IPPM[14], is currently releasing standards for metrics and methods used for network performance measurement. 1.2 Purpose The purpose of this master s thesis is to investigate the active end-to-end network measurement area. The problem within this field is the lack of interoperable tools, and in this project we have implemented and evaluated the upcoming TWAMP standard. The thesis is focused on the progress of the IETF IPPM group, and describes metrics and methods used for network performance measurement. An implementation of the TWAMP protocol has been made and compared to existing measurement tools. 1.3 Outline The report is divided into five chapters. After introducing the reader to the field, we explain the terminology used throughout the thesis, and describe different network performance metrics. Within the second chapter, Network Performance, the reader will achieve comprehension of what we actually measure. The third chapter, Methods, explains how the measurements are carried out. We elaborate on four different active measurement techniques, with focus on OWAMP and TWAMP. In the following chapter, Implementation and Results, we clarify our TWAMP implementation, present how we evaluated the method and give account of our results. The last chapter, Summary and Conclusion, constitutes the completion of the report; different aspects of the protocols and implementations are discussed as well as improvements and the competitiveness of the TWAMP protocol. 3

Chapter 2 Network Performance To get an accurate representation of a network s characteristics, the IETF IPPM work group has defined a framework for performance metrics[15]. This chapter describes notions and metrics that are useful when performing measurements in an IP network. 2.1 Notions 2.1.1 Singletons, Samples and Statistics The metrics can be separated into three distinct categories. A singleton metric is a single, atomic measurement of a packet characteristic, for example the loss or the delay of a packet. Sample metrics are derived from singleton metrics by collecting a sequence of distinct instances together. A sample metric of packet delay between two hosts can be defined as a collection of singleton delay metrics between the two nodes during one hour. The singleton metrics are collected at intervals defined by a statistical distribution. Statistical metrics are derived from sample metrics by computing statistics of the collected samples. By computing the mean latency value of packets from a sample metric, a statistical metric has been produced. 2.1.2 Sampling There are different methods for collecting sample metrics. When periodic sampling is used, measurements are collected with fixed intervals. Samples can also be collected at moments random in time, and the sampling intervals are then determined by a statistical distribution. Periodic Sampling With periodic sampling, samples are derived from measurements separated by a fixed amount of time, a method which is commonly used for its simplicity. Transmitting uniform packets over a network at fixed intervals simulates a constant bit- 5

CHAPTER 2. NETWORK PERFORMANCE rate stream, also known as periodic streams, which resembles to encoded audio and video traffic[16]. This method does however have drawbacks since regularly spaced singleton measurements may be synchronized with other periodic behaviours in the network. There is therefore a risk that the measurement only observes parts of the periodic pattern, and that only a part of the performance spectrum is sampled. If the measurements are made with fixed intervals, they are also easier to predict and may be manipulated. Components in the network can anticipate when measurements are being made and can then temporarily change their behaviour, which will result in false measurements. As previously mentioned, the periodic streams can be configured to have resembling characteristics of media flows. These kinds of services are often time critical, and measuring their quality is of great interest. Although the periodic sampling only analyses parts of the performance spectrum, this may well match the spectrum used for media flows, which simplifies the estimate of the application performance. The analysis of network performance is also simplified in other ways. Some characteristics of a network, such as delay variation and reordering, are sensitive to the sampling frequency. When the impairments are time-varying themselves, the analysis can be facilitated by a constant sampling frequency. Random sampling To avoid the potential problems with periodic sampling, another method called random additive sampling can be used. The samples are collected at randomly generated intervals with a common statistical distribution. The randomness of the samples depends on the quality of the distribution. If the function for example generates a constant value with probability one, then the sampling is equivalent to periodic sampling. With random additive sampling, measurements are less predictable and they will not cause synchronization problems to the same extent. However, the sampling still remains predictable to a certain degree unless the exponential distribution is used. Poisson sampling uses exponential distribution with rate lambda to compute the arrival of new samples. With Poisson sampling it is impossible to predict arrival time of the next sample, a feature which sometimes is desired. There are other similar techniques based on statistical distributions for collecting samples. The geometric sampling is closely related to the Poisson sampling, and its arrival times for sampling cannot be predicted either. These methods do however not model media streams with the same accuracy as periodic sampling does. 2.1.3 Timing Considerations When performing latency measurements in a network, timing between nodes is crucial for obtaining consistent results. This paragraph explains notions related to clock uncertainty, and presents different approaches for clock synchronization. 6

2.1. NOTIONS Offset, Synchronization, Accuracy, Skew, Drift A clock s offset defines the difference between the time reported by the clock and the true time defined by Coordinated Universal Time, UTC. The difference in time between two clocks is called the relative offset or synchronization. It is not always critical that the offset is zero as long as all measurement nodes agree on what time it is. If the relative offset of a pair of clocks is zero, they are described as synchronized. The accuracy of a clock is defined as the absolute value of the offset. If the clock s offset is zero, it is described as accurate. Skew measures the change of a clock s accuracy in time, a clock might for example gain or lose a couple of milliseconds per hour. A clock s skew may be affected over time by for example changes in temperature. Drift measures the variation of skew in time. Wire time Active delay measurement techniques usually include timestamping packets at end hosts upon transmission and reception. However, timestamps do not always indicate the true delay since the packet has to be processed by the system before it is timestamped. The notion wire time defines the time taken for a packet to travel between hosts. Let us introduce an Internet host observing the link in question in order to make these concepts clearer. The wire exit time for a packet on that link is the time when the observation host has spotted all bits of the packet. The wire arrival time for a packet is when any bit of the packet has appeared at the observation host. An alternative definition[17] of wire time is the packet s transmission time at the MAC interface. In practice, timestamps are set and read by measurement software just prior to sending or after receiving a packet, these are known as host times. The difference between wire times and host times depends on the latency between receiving the packet at kernel space and transfer it to user space where it can be timestamped. This error can be reduced by using a network interface card specially designed for timestamping and directly synchronized to a GPS receiver[18], or by timestamping the packets at kernel level. Synchronization methods Clock synchronization is difficult to achieve. The Global Positioning System, GPS, can be used, which offers nanosecond accuracy[19]. GPS uses positioning information from different satellites to compute the current time[20]. This method is however complicated and expensive to deploy in larger scale. Long wave radio transmissions can also be used to propagate the correct time. DCF77 is a German project for transmitting a long wave time signal. The radio transmissions are not affected by obstacles and do not require an antenna for reception. DCF77 allows time signals to be registered under a millisecond range[21]. The Precision Time Protocol, PTP, defined in the IEEE 1588-2002 standard[22] is a new emerging standard which uses the link level for clock synchronization. The 7

CHAPTER 2. NETWORK PERFORMANCE timestamps are generated by hardware and provides accuracy within the microsecond range[23]. The Network Time Protocol, NTP, is a widely used IP based method for synchronizing computers connected to networks[24]. It has a hierarchical design where the top of the tree, stratum 1, consists of computers synchronized to an external source, typically an atomic clock or a GPS receiver. Computers synchronized to stratum 1 hosts become stratum 2 clocks, and act as NTP servers for higher strata. The NTP protocol is semi-self-organizing. An NTP server needs to be manually configured with a list of synchronization peers, but then automatically chooses which peers to communicate with in order to get a good synchronization. The NTP daemon in a host adjusts the clock tick instead of directly setting the new time received by a server. In that manner, discontinuity is prevented and the transition to the correct time is made smoother. The accuracy of NTP depends on the quality of the network connection between the server and the client, but with a reasonably good link, the accuracy of the protocol is typically around 1-2 milliseconds[25]. 2.2 Metrics 2.2.1 Connectivity One of the most basic metrics is connectivity[26]. There are several types of this metric, and the parameters differ slightly, but they all use booleans for metric units. The most fundamental type is instantaneous unidirectional connectivity, which describes if a packet transmitted from the source host at a specific time will arrive to its destination or not. This metric is mostly used as building blocks for further connectivity metrics, but one-way connectivity can also be of interest when for example testing firewalls. This definition can be widened to apply in both directions between two hosts. If both hosts comply with this criterion to one another, they fulfil the metric instantaneous bidirectional connectivity. In similar ways, unidirectional and bidirectional interval connectivity are defined. The connectivity is now measured within a time interval and is not as momentary as the previous metric. This metric might however not be useful when defining full connectivity. The bidirectional interval connectivity is defined over a short interval, and it is not certain that two hosts can reply to the packet they receive from one another. Another metric called interval temporal connectivity is defined as a source host having instantaneous connectivity to its destination within a time interval, and the destination host has instantaneous connectivity back to the source host at a later time in the same interval. This interval is specified to be long enough for transmitting a packet and its response between two hosts. The packets sent in the two directions may be of different types, since some applications use different types of packets for forward and reverse traffic. 8

2.2. METRICS 2.2.2 Packet Loss There is another metric which defines whether a packet was successfully transmitted over the network or not. The singleton metric definition for one-way packet loss from a source host to its destination at a specific time is described with a boolean[27]. Its value is zero if the destination host received the packet, and one otherwise. With this metric it is also important to separate lost packets from packets received with a large delay. For many applications, treating packets with long delay as lost is not considered a big problem. For example, a TCP packet which is delayed with several multiples of the round-trip time may as well have been declared lost. Packets from media streams can be treated the same way if they arrive after the playback point. The two hosts must be somewhat synchronized in time and should share another communication channel, possibly a control protocol, by which the destination host knows the type of the packet and when it was sent. 2.2.3 Loss Pattern One important factor when measuring performance, for both real-time and nonreal-time applications, is the loss pattern[28]. Two transmissions with identical loss rates but different loss distributions can present two widely different performance experiences. Loss distance describes the difference in sequence numbers between two lost packets, defined as zero for the first packet lost. A loss period begins if the previous packet was received as expected, and the current packet is lost. With this definition the sequence numbers of the test packets must be increased monotonically. The loss period is an integer with an initial value of zero, and is incremented each time a new loss period begins. The loss period for a successfully received packet is zero. An example of loss distance and loss period is shown in Figure 2.1. Loss period: 1 2 3 s1 s2 s3 s4 s5 s6 s7 s8 s9 Loss distance: 0 1 2 3 1 Figure 2.1. Crossed out packets are lost. Loss distance describes the separation between packet losses, and the loss period analyses the consecutive packet loss for each occurrence. 9

CHAPTER 2. NETWORK PERFORMANCE 2.2.4 Round-trip Delay The round-trip delay metric measures the time it takes for a packet to be transmitted from the source host to its destination, and then back again[29]. The source host timestamps the packet upon transmission. When the destination host receives the packet, it immediately sends another packet as response back to the source. When the response packet is received, the source host records another timestamp. The value of the metric is then computed by subtracting the two timestamps and any known uncertainties. If the packet is lost, the round-trip delay is undefined. No distinction is made between in what direction the packet was lost. The definition itself is ambiguous in that matter: it is not specified which node acts as source, and which one acts as destination. There is no strict need for synchronization between the two hosts, since only the source host performs and collects the timestamps. If its clock is synchronized with a time service, erroneous results may be presented if the clock is adjusted between the timestamps. 2.2.5 One-way Delay There are several reasons for performing one-way measurements: asymmetric routing and queuing, Quality of Service agreements etcetera. Round-trip measurements examine the union of two distinct paths, and do not treat the forward and reverse path separately. One-way delay is a singleton metric measuring the time taken for a packet to travel from the source host to its destination[30]. An upper bound for the metric needs to be specified to determine whether a packet is delayed but still on its way, or whether it is lost. The metric is undefined or infinite if the packet never is received. If multiple copies of the same packet are received, the arrival time of the first packet establishes the one-way delay. The methodology for determining the delay varies depending on the type of the packet, but the approach is similar for all types. To achieve fruitful measurements, it is important for the sending and receiving hosts to have synchronized clocks. It is also important that the padding of the test packet is filled with randomized bits to avoid that the packet is compressed on its path. 2.2.6 Delay Variation There are two different notions describing delay variation. Inter packet delay variation, IPDV, and packet delay variation, PDV. Delay variation is important for measuring the performance of a network, in most cases it is the maximum delay variation within a high percentile that is of interest. This is particularly true for media streams which require a delivery of packets at regular intervals, and the measurement is meaningful when sizing buffers for these sorts of applications. 10

2.2. METRICS Inter Packet Delay Variation IPDV analyses the difference between one-way delay metrics of two selected packets within a stream[31]. The singleton metric one-way ipdv defines a single instance of an ipdv measurement, but the measurement is only statistically usable when combined in pairs. The measurement is performed by sending a stream of equally sized packets between the measuring nodes, and two packets are selected from the stream. The two packets are timestamped at the source and at the destination hosts, and the one-way delay metric is computed. The one-way delays of the two packets are subtracted, and the inter packet delay variation can be measured. Using this technique, the clock synchronization between the two hosts is not as crucial as in one-way delay measurements, the relative offset is not noticeable after the subtraction. If one or both of the chosen packets are lost or do not arrive within the specified time limit, the IPDV metric is undefined. Packet Delay Variation Instead of comparing two following packets in the stream, PDV uses a reference packet from the interval, typically the packet with the lowest delay. The PDV of a packet is then computed by comparing its one-way delay with the reference value. This way, PDV will be normalized to the minimum delay and can only take values that are positive or zero[32]. This metric has not yet been standardized by the IETF IPPM group, but a draft documenting the difference between IPDV and PDV has been released. Both metrics are affected by the packet loss ratio, but only IPDV is affected by the loss distribution. In the worst case where every other packet is lost, IPDV is unusable. IPDV can however measure the network s ability to retain the original spacing between packets, it does also reduce the demands on the system s clock in the aspect of skew and stability. IPDV is memory less and only describes the difference in delay between two adjoined packets whereas PDV is defined over a longer interval. PDV is a more usable metric, and especially PDV percentiles are used when sizing play out buffers and describing long-term trends. A comparison of values between IPDV and PDV is shown in Table 2.1. Table 2.1. Comparison between IPDV and PDV. This PDV measurement uses the packet with the lowest delay (10ms) as reference value. Packet number 1 2 3 4 5 6 7 8 9 10 Delay (ms) 20 15 30 10 15 20 25 10 10 15 IPDV n/a -5 15-20 5 5 5-15 0 5 PDV 10 5 20 0 5 10 15 0 0 5 11

CHAPTER 2. NETWORK PERFORMANCE 2.2.7 Reordering Packets sent over a network might be transmitted over different paths, and may therefore not be received in the intended order. The Internet Protocol does not account for lost or misplaced packets, and this must be taken care of by higher-level protocols. To accomplish correct packet order, each packet must contain a unique identifier, a sequence number, which identifies the order in which the packets were sent. The sequence number is then used at the destination to detect rearranged packets. As an example, eight packets, sent with consecutive sequence numbers, arrive to their destination as described in Figure 2.2. Reordered s1 s2 s3 s7 s4 s5 s6 s8 e = 3 e = 2 e = 1 New sequence discontinuity of size 3. Figure 2.2. Packets arriving out of order. Packet s7 starts a new sequence discontinuity and packets s4, s5 and s6 are considered as reordered with extent e. There are two alternatives when distinguishing the packet disorder: either the packet with sequence number s7 is reordered, or packets with identifiers s4, s5 and s6 are labelled as misordered. The IETF IPPM group defines packets with lower packet identifiers than their predecessors, or late packets, in this case packets with sequence number s4 to s6, as out of order[33]. Using the alternative definition and declaring packet s7 as reordered is not recommended, since it is upon the arrival of packets s4 to s6 that separates this occurrence from packet loss. The goals of reordering metrics are to determine whether or not a packet has been reordered and to what degree. To determine the extent of reorder, a number of different metrics have been defined to fulfil the need of different applications. The first notion is a function that generates a series of unique increasing numbers, this could simply be an increase of a consecutive integer. The second basic concept is an algorithm at the receiving side, a counter, which determines the next expected sequence number. Normally, this number equals the sequence number of the previously received packet increased by one. The metric reordered defines if a packet is misordered or not. The metric is false if the sequence number of the received packet is equal to or larger than the next expected sequence number, and the counter is then increased. If the packet identifier is smaller than the expected sequence number, the metric is true and the counter is not increased. If duplicated packets are received, only the first packet is considered reordered. If a packet arrives with a larger sequence number than expected, this indicates a sequence discontinuity caused by either packet loss or reordering. The size of the discontinuity is the number of missing packets and can be computed by subtracting the expected sequence number form the arrived packet s identifier. Source code for calculating the named metrics is shown in Figure 2.3. 12

2.2. METRICS Reordered ratio stream is a sample metric to compute the ratio of reordered packets. Under a time interval, it simply divides the number of reordered packets with the number of total packets received, excluding duplicates. This metric gives an estimate of the percentage of reordered packets in the network. If the value is zero there is no need to apply further reordering metrics to the stream. The sample metric reordering extent stream defines to what extent the packets in a stream have been reordered. The reordering extent is the distance in packets between a reordered packet, and the earliest packet received with a larger sequence number. Late time stream defines the lateness of a reordered packet. By calculating to what extent the packet has been reordered, its expected arrival time can be compared to its real destination time, and the lateness can be computed. The sample metric byte offset stream indicates the storage in bytes needed to accommodate the reordered packets. The byte offset value for a reordered packet is the total payload size of packets with a larger sequence number that arrived prior to the reordered packet. These packets can then be buffered until the previous sequence of packets is complete. Reordering gaps describe the distance between reordering discontinuities and reordering free runs define the number of consecutive ordered packets between misordered packets. The reordering metrics are particularly useful when designing general receiver buffer systems. if s >= NextExp, then /* s is in-order */ if s > NextExp then SequenceDiscontinuty = True; SeqDiscontinutySize = s - NextExp; else SequenceDiscontinuty = False; NextExp = s + 1; Type-P-Reordered = False; else /* when s < NextExp */ Type-P-Reordered = True; SequenceDiscontinuty = False; Figure 2.3. Reordering metrics calculation for packet with sequence number s. NextExp corresponds to the next expected sequence number. 2.2.8 Duplicates When data is sent over a network connection, the receiving host usually expects exactly one copy of each sent packet. We have already discussed packet loss, but this passage treats the opposite occurrence: when a packet is sent, multiple copies arrive to the receiving node. This case has not yet been standardized by the IETF IPPM group, but an Internet draft has been published[34]. The metric one-way packet arrival count is an integer that measures the number of identical and uncorrupted copies of a packet which are received by a measurement node during a specified time interval. Packets are considered identical if they contain 13

CHAPTER 2. NETWORK PERFORMANCE identical information fields and if they were sent between the same hosts. The IP headers do not need to match, since for example the time to live, TTL, field may vary if a duplicated packet has been routed differently. If no packet is received the metric is undefined. The metric one-way packet duplication is similar to one-way packet arrival count, but instead of counting the number of occurrences of a packet, it counts the number of additional identical and uncorrupted copies received. If a single copy of a packet is received, the metric is zero, and it is undefined if no packet has arrived. This metric equals to one-way packet arrival count subtracted by one. There are two proposed statistical metrics for one-way duplication. One-way packet duplication fraction and rate gives the fraction and rate of additional packets that arrived in a stream. The fraction is calculated by removing all packets with undefined values of one-way packet duplication, and the difference is then divided by the total number of packets sent and not lost. The rate is calculated similarly, but the numerator consists of packets with one-way packet arrival count larger than one. 2.2.9 Further metrics The number of different network measurement tools[12] is very large, and the list of performance metrics can of course be elongated. The IETF IPPM work group has released standards and drafts which treat several new concepts. Spatial composition[35] computes the performance of a complete path based on measurements from separate path segments, and multi metrics[36] describe how measurements can be performed in a multicast scenario with one source and multiple destinations. Metrics for measuring network capacity[37] have also been released. There are several other organizations[38][39] that publish standardization documents. The International Telecommunication Union, ITU[40], which creates standards for telecommunication, has released a recommendation known as the E-model[41]. The E-model is a computational approach for transmission planning, and can be used to assess conversational quality by evaluating parameters such as latency, packet delay variation, packet loss and encoding technique. This model can be mapped to an estimated mean opinion score, MOS[42]. The MOS is a subjective metric indicating the perceived quality of received media, and is well suited for evaluating the quality of VoIP applications. 14

Chapter 3 Methods Four different measurement methods have been compared in this thesis, and the techniques differ somewhat from each other. The evaluation has been based on results collected from the TWAMP prototype implemented in this project, Internet2 s One-way ping, Ping and PTAnalyzer developed by Prosilient Technologies[43]. The well known Ping utility uses the ICMP protocol to transmit its packets, and does only calculate the round-trip time. Ping is therefore not affected by unsynchronized clocks. PTAnalyzer collects two-way measurement data, but uses its own synchronization plane, which allows it to operate independently from the system clocks. This technique sends its traffic as UDP packets. The UDP protocol is also used by OWAMP and TWAMP, but since these two methods rely on the system time, it is of great importance that the clocks in the measurement nodes are synchronized. 3.1 The One-way Active Measurement Protocol In order to analyze the different metrics, test traffic is sent between measurement end points, and the results may then be compiled. There are several existing measurement platforms for collecting metrics in IP networks, but due to the lack of interoperability the IETF IPPM group has published a new standard for one-way measurements. The One-way Active Measurement Protocol, OWAMP, analyzes unidirectional network characteristics, and can calculate metrics such as one-way loss and one-way delay[44]. The authors vision is to make one-way measurements as common as the usage of today s ICMP based round-trip measurements. This can be achieved by a widespread deployment of open OWAMP servers and the fact that more and more hosts on the Internet have clocks synchronized to GPS or NTP. The design goals of the protocol, besides giving an accurate understanding of the network, are to provide flexible measurement nodes and that the protocol traffic will be difficult to detect and manipulate. The traffic is difficult to detect since it consists of a stream of UDP packets to and from negotiated port numbers. The protocol also implements modes of authentication and encryption to make the traffic impossible to alter and indistinguishable from other data in the network. 15

CHAPTER 3. METHODS The OWAMP protocol consists of two different protocols: OWAMP-Control and OWAMP-Test. The control protocol is used for setting up and terminating test sessions, collect results etcetera, whereas the test protocol constitutes the test packets exchanged between the measurement nodes. The two protocols are closely related, but the test protocol can be used with some other control protocol than OWAMP- Control. The IETF IPPM group does however recommend implementing the two protocols together to ensure interoperability. OWAMP has been divided into different logical roles in order to provide maximum flexibility. The following roles are defined: session-sender, session-receiver, server, control-client and fetch-client. The different roles and their relationship are described in Figure 3.1. Session- Sender Control- Client Server Proprietary protocol OWAMP- Control OWAMP- Test Proprietary protocol OWAMP- Control Session- Receiver Fetch- Client Figure 3.1. Possible relationship between OWAMP s logical roles. The control-client is an end system which contacts the server to initiate or terminate OWAMP-Test sessions. The server manages OWAMP-Test sessions, and configures the measurement end points with data from the control-client. The server is also able to return the measurement results from a session. The session-receiver and session-sender are two end points of the test session between which test packets are transmitted. The role of the fetch-client is to initiate requests for fetching the results of a completed test session. One host may play several different logical roles, and a simple setup can be achieved with only two hosts. One host acts control-client, fetch-client and session-sender, whereas the other host acts server and session-receiver, as shown in Figure 3.2. Control- Client Fetch- Client Session- Sender OWAMP- Control OWAMP- Test Server Session- Receiver Figure 3.2. OWAMP setup between two hosts. 16

3.1. THE ONE-WAY ACTIVE MEASUREMENT PROTOCOL 3.1.1 OWAMP-Control As previously mentioned, OWAMP is built up by two interrelated protocols: OWAMP- Control and OWAMP-Test. The control protocol establishes a TCP connection to a well-known port, and this connection remains open throughout the entire OWAMP session. OWAMP-Control data is transmitted only before and after the test, and not during the test session itself, with the exception of a request for terminating the test session earlier than previously negotiated. Both the control and test protocol support three different modes: unauthenticated, authenticated and encrypted, where the two latter modes require a shared secret. The authenticated and encrypted modes provide increased security; authenticated traffic prevents packets from being injected by unknown users, and encryption prevents routing equipment from distinguishing the stream of test packets and alter its priority. The control connection is initiated by the control-client. The server responds with a greeting, and presents supported modes, a challenge and other parameters used to setup the test session. These parameters are used in authenticated and encrypted modes and will not be thoroughly discussed here. If the server does not want to communicate, e.g. due to the limit of maximum concurrent tests has been reached, this information is sent as response and the server may close the connection. Upon receiving the response packet, the control-client may close the connection if the desired mode is not supported, otherwise it must transmit a packet containing the mode of the current session. The next packet exchanged is sent from the server to the control-client and contains an accept field and a timestamp indicating the time when the instance of the current test server was created. The connection setup is now complete. The control-client can now issue one of the following commands: request-session, start-session, stop-session and fetch-session. The packets exchanged during a request-session contains information about addresses, ports, packet size, number of test slots, number of packets, start time, timeout and a unique session identifier, SID. The server responds with an accept-session message to confirm the upcoming test session. The control-client and the server have now agreed on a send schedule; in case of lost packets, the session-receiver still knows when they were supposed to be sent. The test session is started when the client issues a start command. The server transmits a start acknowledgement, and if the session was accepted, the test streams are started immediately. When test sessions have ended, or in order to cancel a test session before it is completed, either of the two hosts can issue a stop command. This command includes the number of sessions to stop, the session identifier and a range of sequence numbers indicating which packets that will be excluded from the test session. When a test session is completed, the client issues a fetch command to collect the measurement data containing the SID and the range of sequence numbers to collect. The server responds with a fetch acknowledgement and then transmits the timestamped packet records back to the control-client, which computes one-way metrics on the received data. 17

CHAPTER 3. METHODS 3.1.2 OWAMP-Test The OWAMP-Test protocol is layered over UDP and transmits singleton measurement packets between the two end points negotiated during the request-session. The test packets contain a sequence number, a timestamp indicating when the packet was sent, an error estimate representing the sending host s offset to UTC and an optional packet padding. The timestamp consists of 64 bits where the first 32 bits are an unsigned integer indicating the number of seconds elapsed since January 1900. The last 32 bits constitute the fractions of the last second that have passed. The test packets contain further parameters if encrypted or authenticated mode is used. A detailed description of the packet format is shown in Appendix A. The session-sender starts transmitting the packets according to the send schedule, and upon arrival, the receiver timestamps the packets and stores the sequence number, send time, receive time and TTL from the IP header. This data will later be transferred back to the control-client for computation of performance metrics. 3.1.3 Current Implementations The first draft of OWAMP was released to public in November 2000 and about six years later, in September 2006, the final standardizing document was released. Three different implementations of OWAMP have been distinguished, of which one project was discontinued in 2005. Possibly a larger number of implementations have been done, but not publicly released with source code. Internet2 One-way Ping The network consortium Internet2 has developed an application, One-way ping[45], which implements the OWAMP protocol using the programming language C. The current stable implementation, version 3.0c, was released in 2007. It supports the complete standard and is available for Unix-like systems. One-way ping is a typical client-server application, where the client program, Owping, contacts the server running a daemon known as Owampd. When the send schedule is negotiated, each side spawns a child process which constitute the test end points. The two end points are identical, and by default, two measurement sessions, one in each direction, are started and the results are returned back to the client. J-OWAMP J-OWAMP[46], a Java implementation of the OWAMP protocol has been made available by H. Verga et al. It is platform independent and does also include a web interface where sessions can be configured and results presented visually. Another feature of J-OWAMP is its built in NTP client, which offers direct synchronization with an NTP server. This eliminates the need of a synchronized system clock. The program offers single one-way test sessions and confidence test interval sessions, where the test session is split into several intervals conducted over a longer period 18

3.2. THE TWO-WAY ACTIVE MEASUREMENT PROTOCOL of time, to achieve better reliability. Conformance tests between J-OWAMP version 1.2 and One-way ping version 1.6f were successfully performed[47], but the development of J-OWAMP ceased in 2005. Java Client for OWAMP Since the J-OWAMP project has been discontinued, the only current OWAMP implementation is available for Unix systems. To facilitate for end users, a Java implementation featuring a manageable applet is desirable. M. Hofstra is currently developing the client part of OWAMP in Java to make it interoperable with Internet2 s One-way ping. This project is carried out as a part of Google Summer Code 2008[48]. 3.2 The Two-way Active Measurement Protocol The Two-way Active Measurement Protocol, TWAMP, is a protocol for collecting two-way metrics[49]. The protocol is still in the development phase, it is currently being reviewed and will probably be standardized in the near future. TWAMP is based on the OWAMP protocol and their architectures are very similar, there are however important changes in the logical model. The session-receiver from OWAMP is called session-reflector. Unlike the session-receiver it does not collect any measurement data, but instead upon arrival of a measurement packet, it sends an immediate response packet back to the session-sender. In the TWAMP architecture, the session-sender is able to receive measurement data and to communicate the results back to the control-client. Since the session-reflector does not collect any packet information, the role of fetch-client is superfluous, and this entity has been removed from the TWAMP architecture. The role of the server is similar to OWAMP, it configures the test end points. However, if the server is implemented on the same host as a session-reflector, it is not able to return the measurement results, as it was capable of in OWAMP. A model of the TWAMP architecture is shown in Figure 3.3. Session- Sender TWAMP- Test Session- Reflector Proprietary protocol Proprietary protocol Control- Client TWAMP- Control Server Figure 3.3. Possible relationship between TWAMP s logical roles. 19