Wide Area Network Latencies for a DIS/HLA Exercise

Wide Area Network Latencies for a DIS/HLA Exercise Lucien Zalcman and Peter Ryan Air Operations Division Aeronautical & Maritime Research Laboratory Defence Science & Technology Organisation (DSTO) 506 Lorimer St, Fishermens Bend MELBOURNE, VIC 3207 Keywords: Latency; Wide Area Networks; Latency; Distributed Interactive Simulation; High Level Architecture ABSTRACT: Many issues are involved in the successful conduct of an Advanced Distributed Simulation (ADS) exercise undertaken over a large geographic area. One key issue is the extent to which the geographic separation between the simulation nodes creates unacceptable latencies between transmission and reception of data. Latency problems can lead to lack of fidelity in the simulated exercise which ideally requires real time response. This paper reports experiments which estimate these transmission latencies over large geographic areas. The effect of the protocol used (eg. Distributed Interactive Simulation (DIS) or Higher Level Architecture (HLA)) on the latency is also discussed. 1. Introduction Many issues are involved in the successful conduct of Advanced Distributed Simulation (ADS) exercises that are undertaken across a large geographic area. One of the most important is the extent to which the geographic separation creates unacceptable latencies across the network. Many, if not all, ADS applications use the IEEE 1278 standards [1] for communicating data. This paper reports the use of standard network tools to estimate the latency expected over a Wide Area Network (WAN). The effect of the data packet size, and thus the ADS communication protocol used (eg. DIS or HLA), on the WAN latency is also examined. Typical latencies over Australia and world wide are reported. 2. TTCP and Project SEA 1412 As part of Australia s multinational simulation research with the United States, Canada, United Kingdom and New Zealand through The Technical Cooperation Program (TTCP) countries, DSTO s Air Operation Division is producing a document describing how to set up an ADS experiment to run over an international WAN. This document will discuss the issues involved and will include hardware, software and network configurations and costs. Some main areas of interest to be reported in this work are: Integrated Services Digital Network (ISDN) services available, equipment required, initial and recurring costs bandwidth estimates WAN latencies effect of communications protocol used ISDN channel aggregation and router compression effect of insufficient WAN bandwidth This document will be available through TTCP to any participating countries. The first customer of this work will most likely be the RAN s SEA 1412 project which (in part) is required to establish a WAN which will link various fleet bases in Perth and Sydney with shore-based simulation facilities at HMAS Watson [2]. 3. Tools for Measurement of Network Latencies Two main tools were used to measure network latency - the Ping and Traceroute utilities. The following sections describe the application of these tools. 3.1 The Ping Utility The ping utility is a system administrator's tool used to determine if a computer is operating and its network connections are intact. Ping places a timestamp in a packet which is sent through the network to a particular Internet Protocol (IP) address. The computer that sent the packet then waits (or 'listens') for a return packet. If the connections are good the target computer transmits a return packet which is used to compute how long each packet exchange took - the Round Trip Time (RTT). Two useful common ping options are: -c count - Send count packets and then stop. The other way to stop is type CNTL-C. -s packetsize - Change the size of the test packets. Large packets that are not a power of two bytes must be fragmented. The Internet ping measurements between host computers were repeated several thousand times (using the c parameter mentioned above). The minimum of these measured ping times (latency) can be assumed to approximate the ISDN latency using the maximum Internet bandwidth available with the same communications equipment latency present in the Internet connection. Ping can also be used to send packets directly to World Web (WWW) sites without the need to know their IP address. Therefore the latency of transmissions

to remote WWW servers in locations of interest can be studied. Such WWW servers are easily found on the Internet. 3.2 The Traceroute Utility Traceroute is a network debugging utility that attempts to trace the path a packet takes through the network. Traceroute transmits packets with small TTL (Time To Live) values. TTL is an IP header field that is designed to prevent packets from running in loops. Every router that handles a packet subtracts one from the packet's TTL. If the TTL reaches zero, the packet has expired and is discarded. Traceroute relies on the common router practice of sending a Time Exceeded message back to the sender when this occurs. By using small TTL values which quickly expire, traceroute causes routers along a packet's normal delivery path to generate these messages which identify the router. A TTL value of one should produce a message from the first router; a TTL value of two generates a message from the second; etc. Round trip times are reported for each packet. Traceroute also reports any additional messages such as destination or host unreachable. Traceroute is used to check that the communications path used by the ping utility is the shortest path to the required destination and that the destination computer is really where it is supposed to be. 4. Wide Area Network Latency The latency present in a WAN can be described as: WAN Latency = L d + L n + L p (1) where L d is the latency due to the time required for the packet to physically travel the required distance on the cable, L n is the latency incurred by the packet being processed by any communications equipment in the route travelled, and L p is the latency incurred due to the time required for the fixed size packet to pass through the finite size communications pipe available. The objective of this work is to develop a simple method to predict the required ISDN WAN latency. It is assumed that equation (1) applies to both ISDN and Internet WAN connections. The difference between these is that an ISDN WAN is a guaranteed bandwidth service whereas Internet bandwidth is not guaranteed. An Internet connection takes whatever bandwidth is available and shares the connection with any other traffic using it at the time. For an Internet connection, the required communications equipment is already in place and the L d, L n and L p parameters in equation (1) can easily be estimated and/or measured. For an ISDN connection there will be no additional (time delaying) communications equipment other than that provided at each end of the communications link provided by the user(s). This assumes that any repeaters etc. provided by the ISDN service provider(s) do not add any considerable delays. 4.1 Physical Distance Latency The physical distance latency L d can be simply estimated as: L d (seconds) = cable distance (kms) / 300,000 (2) The physical cable distance can be obtained from Telstra Australian Broadband Bearer Network maps [3]. Distances measured from Sydney, Melbourne and Adelaide to other Australian destinations are shown below in Table 1 together with the physical latency predicted by equation (1). Return distances are given since the latency will be compared to round trip times measured using the Ping utility described in Section 3. Sydney Melbourne Adelaide Sydney - 1850 (6) 3000 (10) Melbourne 1850 (6) - 1300 (4) Adelaide 3000 (10) 1300 (4) - Brisbane 1600 (10) 3750 (13) 4150 (14) Rockhampton 2800 (9) 4950 (17) 5350 (18) Cairns 4800 (16) 6950 (23) 7350 (18) Darwin 9000 (30) 7300 (24) 6000 (20) Perth 8250 (28) 6550 (22) 5250 (18) Table 1. Approximate round trip distances in km and latencies (L d s) in ms (in italics) from Sydney, Melbourne and Adelaide to Australia wide destinations. 4.2 Packet Size Latency The packet size or pipe latency (L p ) can be simply estimated as: L p (seconds) = packet size / pipe size (3) The packet size latency will indicate what effect the protocol (DIS or HLA) will have on the total WAN latency. Ping allows the packet size to be varied (see Section 3). Figure 1 shows the minimum ping time versus ping packet size in increments of 100 bytes between an RMIT computer in Melbourne, Australia and a University of NSW computer system in Sydney, Australia. 25 23 21 19 17 15 0 200 400 600 800 1000 Packet Size (bytes) Figure 1. Minimum ping time versus packet size for communication between RMIT and the server with address http://www.unsw.edu.au. The differences in latencies are attributed to the differences in the packet sizes. Thus the pipe size is determined from the slope of Figure 1 to be roughly 1 Mbps. This is the effective maximum bandwidth between the two computer systems and thus at least this

bandwidth is available at each end. The Visual Ping tool [4] which can do this measurement over Internet predicts minimum bandwidths of 965 kbps and 650 kbps, respectively, for the Sydney and Melbourne computer systems. This infers that it is most likely that both systems will have a 1 Mbps pipe size. Ryan et al. [5] predict that a bandwidth of approximately 800 kbps is required (ie. a 1 Mbps pipe) for a typical naval warfare scenario using the DIS protocol. An Entity State DIS Protocol Data Unit (PDU), the most commonly issued PDU [6], is slightly more than two kilobits long [7]. A two kilobit packet would require four ms to complete a return trip through a 1Mbps pipe. If HLA were used to reduce this packet size to one kilobit this may save on the bandwidth required (eg a 0.5 Mbps pipe) but would only reduce the WAN latency by a few ms. Further, increasing the WAN bandwidth (pipe size) would only reduce the DIS/HLA WAN latency by a few ms as most of the latency would be due to the time taken for the packet to travel along the cable and the delay caused by the communication devices within the WAN connection. 4.3 Communications Equipment Latency The communications equipment latency (L n ) is the sum of all the latencies incurred whilst passing the packet through any communications equipment (such as routers etc.) along the route travelled. The network ping utility was used to determine the round trip total latency for a predetermined size network packet. Re-arranging equation (1) gives L n = WAN Latency - L d - L p (4) where WAN Latency can be the time returned by ping, L d is determined as described in Section 4.1, and L p is determined as described in Section 4.2 The communications equipment latency incurred in an ISDN system would come from the communication devices installed at each end of the Wide Area Network connection by the user(s). This latency can be approximated by measuring ping time for a small packet to a nearby computer system with a large pipe so that L d and L p can be considered negligible. The ping time of three ms between RMIT and both Melbourne and Monash Universities can be used as an upper estimate of L n. The effective bandwidth between RMIT and these two institutions will be approximately 1 Mbps as this appears to be the maximum bandwidth from the RMIT computer and it is unlikely to be less than 1 Mbps from either Melbourne or Monash Universities (see Section 4.2). Since Melbourne University is less than one km away from RMIT and Monash less than 30 km from RMIT, L d and L p will be negligible compared to the measured interstate ping times. 5. Measurements of Latencies 5.1 Wide Area Network Latency across Australia Equation (1) can be used to predict the latency for both an ISDN and Internet WAN. The time required for the packet to physically travel on the cable (L d ) is identical for both networks and is reported in Section 4.1 for Australia wide connections. In an ISDN network the communications equipment latency (L n ) is the sum of the latencies in the only important communication devices present, those installed at each end of the WAN connection by the user(s). In Section 4.3 this is estimated to have a maximum value of three ms. In an Internet network the communications equipment latency (L n ) is the sum of the latencies from all the communications devices present along the network which have been installed by the Internet service provider. This is different in every new situation. For a typical Australian naval exercise [5] a pipe size of at least 800 kbps is required close to the 1 Mbps pipe sizes measured on some of the university WWW servers. The default (RMIT computer) ping packet size of 64 bytes (512 bits) would add approximately one ms latency for a 1 Mbps pipe size to a round trip ping measurement on both an Internet and a dedicated ISDN WAN. To determine the pipe latency (L p ) for a 64 byte packet the ping measurement was repeated for packet sizes of 64, 128, 192 and 256 bytes. Figure 2 shows the measured ping round trip times in ms for 64 byte packets between RMIT and various locations in Australia. The predicted ISDN latencies, as described above, are shown in brackets where the three numbers correspond to L d, L n and L p. The measured (minimum) ping times are always greater than the predicted ISDN latencies (for the same size 64 byte packet). This is most likely due to the communications equipment latency being always greater in an Internet WAN than in an ISDN WAN and the Internet bandwidth not being guaranteed to be optimal. A typical World Wide Web address used is also shown in Figure 2. 5.2 Predicted ISDN Latency for a DIS Exercise from Sydney Equation (1) was used to predict the ISDN latency for a DIS exercise with participation from Sydney ie. the Maritime Warfare Training Centre (MWTC) at HMAS Watson. Values for cable distance latency (L d ) from Sydney to other locations were obtained from equation (2), and the communications equipment latency (L n ) was assumed to be 3 ms (Section 4.3). For a 1 Mbps pipe a 2 kilobit packet (the size of a DIS Entity State PDU packet) would add slightly more than 4 ms for a round trip. Assuming some overhead for the 2 kilobit packet to be disassembled and reconstructed by the communications equipment at each end of the ISDN network (ie. aggregation) a value of 5 ms was used for the pipe latency (L p ). The predicted ISDN (round trip) latencies from

Sydney to other locations of interest in Australia are shown below in Table 2. City Latency (ms) Brisbane 18 Cairns 24 Darwin 38 Perth 36 Adelaide 18 Melbourne 14 Table 2. Predicted ISDN (round trip) latencies from Sydney to other locations in Australia in ms. These predicted round trip latencies are well below the acceptable value of 100 ms for a tightly coupled DIS exercise [8]. Therefore DIS exercises involving HMAS Watson in Sydney and other most likely places of interest in Australia should not suffer detrimental affects due to ISDN WAN latency. 5.3 World Wide Latencies The data presented in Figure 2 indicate that the ping utility can be used to obtain an upper estimate of the required ISDN latency. Figure 3 shows some measured minimum (from a trial of 2000 pings) ping values between Melbourne, Australia and some international (TTCP country) locations. In some recent international DIS experiments [9] the ISDN ping time between DSTO Melbourne and a server at Communications Research Centre (CRC) in Ottawa, Canada (address: http://buzz.dgrc.doc.ca) was found to be 350 ms. This can be compared to an Internet ping time between the RMIT computer system and the CRC server of 440 ms. 6. Conclusions A simple methodology has been developed to estimate ISDN latency. The ping utility can be used to measure an upper estimate of the required ISDN latency and traceroute can be applied to check the route used. Both the measured upper estimate of, and the predicted, ISDN latencies indicate that DIS WAN experiments can be carried out between main population centres in Australia with latencies of less than 100 ms. International DIS exercises to nearby countries, such as New Zealand, may be able to be carried out with latencies less than 100 ms. However for countries further away, such as the United Kingdom and the Unites States, the latencies are likely to exceed 100 ms. Switching from DIS to HLA data packets would not significantly reduce the ISDN Wide Area Network latency for a typical exercise. The smaller size of the HLA packets will only account for a negligible reduction in latency across a WAN. 7. References 1. IEEE 1278.1a-1998 (1998). IEEE Standard for Information Technology - Protocols for Distributed Interactive Simulation Applications (DIS 2.14) 2. Marshall, S. LCDR RAN, Maritime Warfare Training Centre Project Director. (March, 1998). Maritime Warfare Training Centre Project Management Issues, Industry Day, SimTecT 98, Adelaide, Australia 3. Telstra Broadband Bearer Network Australia, National IDN Region, Capacity Planning, 12/242 Exhibition Street, Melbourne, phone - (03) 9634-7552 4. Visual Ping web site: http://sitka.triumf.ca/cgibin/visual-ping 5. Ryan, P. and J. Morton. (March 1997). Network Traffic Prediction for Distributed Interactive Simulation Exercises, Proc. SimTecT 97, Canberra, Australia 6. Perry, N. and P. Ryan. (March 1998). Analysis of Logged DIS PDU Traffic Generated by the NavySAF Synthetic Force System, Proc. SimTecT 98, Adelaide, Australia 7. Doris, K. and M. Loper. (1993). DIS Network Traffic Analysis Estimation Techniques, Proc. 15 th I/ITSEC Conference, Orlando FL, US 8. The DIS Vision - A Map to the Future of Distributed Simulation, DIS Steering Committee, Version 1 - May, 1994, IST-SP-94-01 9. Zalcman, L. et al. (March 1998). ModSAF Experiment on an International ISDN Wide Area Network, Proc. SimTecT 98, Adelaide, Australia 8. Author Biographies Dr Lucien Zalcman graduated from Melbourne University with a BSc (Hons.) in 1973. He was awarded a PhD in Physics from Melbourne University in 1980. In 1984, he joined DSTO as an Information Technology Officer. Since 1992 he has been employed as a Senior Professional Officer in Air Operations Division specialising in the field of Distributed Interactive Simulation. Dr Peter Ryan is a Senior Research Scientist with the Air Operations Division of the Defence Science & Technology Organisation. He graduated from Melbourne University with a BSc (Hons) (1975) and PhD (1981). After postdoctoral work at the University of Massachusetts, USA he joined DSTO in 1985. His research interests include the modelling and simulation of military operations with particular interest in applications of Advanced Distributed Simulation.

Figure 2. Measured minimum round trip Internet ping values (latencies) and predicted (L d +L n +L p ) ISDN latencies from Melbourne to some other Australian locations in ms. The address of the World Wide Web server pinged is also included. Figure 3. Measured minimum round trip Internet ping values (latencies) between Melbourne, Australia and some international (TTCP country) locations in ms. The address of the World Wide Web server pinged is also included.