Enterprise Network Control and Management: Traffic Flow Models

Enterprise Network Control and Management: Traffic Flow Models William Maruyama, Mark George, Eileen Hernandez, Keith LoPresto and Yea Uang Interactive Technology Center Lockheed Martin Mission Systems Crossman Avenue Sunnyvale, CA 989 (8) 73-73, Fax (8) 73-3 E-mail: bill.maruyama@lmco.com Abstract - The exponential growth and dramatic increase in demand for network bandwidth is expanding the market for broadband satellite networks. It is critical to rapidly deliver ubiquitous satellite communication networks that are differentiated by lower cost and increased Quality of Service (QoS). There is a need to develop new network architectures, control and management systems to meet the future commercial and military traffic requirements, services and applications. The next generation communication networks must support legacy and emerging network traffic while providing user negotiated levels of QoS. Network resources control algorithms must be designed to provide the guaranteed performance levels for voice, video and data having different service requirements. To evaluate network architectures and performance, it is essential to understand the network traffic characteristics. I. INTRODUCTION This paper provides a baseline enterprise network traffic model characterized in terms of application, protocol, packet and byte distributions. The metrics are defined by protocol, percentage compositions of traffic and arrival distributions. A Hybrid Network Testbed (HyNeT) and a network management tool suite, the Integrated Network Monitoring System (INMS), were developed by the Interactive Technology Center (ITC), Lockheed Martin Mission Systems to automate the complex process of performing network evaluations. This unique hardware based network simulation capability is utilized to generate, monitor and record network performance metrics. These characteristics are important for evaluating design decisions to specify the placement of network services, component algorithms and resource allocations to operate an efficient communication network. corporation requiring high bandwidth network connectivity across geographically dispersed locations. The data set includes Simple Network Management Protocol (SNMP), Remote Monitoring (RMON) and Packet Trace metrics. The INMS data archive is continually expanded to include additional time and topological sampling points for further analysis and trend studies. Samples were selected to develop traffic flow characteristics for network protocols to be used to specify flows for performance, accounting and bundling. The HyNeT provides "hardware in the loop" simulation for high fidelity analysis of communication network architectures utilizing the traffic models. Currently, the traffic data is being used to characterize the future traffic demands for an advanced military network system. II. NETWORK TOPOLOGY The data samples were collected at the uplink interface for the local segment consisting of approximately 5 subnets supporting 7 workstations. This point of interest, depicted in Fig., reference point (A), is a candidate for satellite replacement, augmentation or secondary backup. The Local Area Network (LAN) segment is currently connected into the corporate Wide Area Network (WAN) via a -megabit/sec Permanent Virtual Circuit (PVC) allocated from a (Digital Service, level-3 () Asynchronous Transfer Mode (ATM) service. Local Campus mb pvc mb pvc mb pvc Corporate ATM Backbone mb pvc mb pvc mb pvc mb pvc mb pvc mb pvc Internet mb pvc The traffic models and methodologies described in this paper are based on the data captured by the INMS from a Lockheed Martin network segment. The traffic sampling is representative of a network segment within a multinational A Fig.. Corporate Network Topology Internet -783-5538-5/99/$. (c) 999 IEEE

Distributed network monitoring equipment was deployed at key reference points. The corporate backbone uplink and interfaces into the local campus backbone were instrumented. The majority of the local network utilizes Ethernet technology. The campus backbone employs -megabit switched Ethernet technology before connecting into the corporate ATM cloud. The data utilized in this report was collected from a -megabit Ethernet SPAN (Switched Port Analyzer) port enabling monitoring traffic of the ATM uplink, depicted in Fig.. VLAN Corporate ATM Backbone ATM Local Campus Backbone Swtiches and Routers Fast Ethernet Fast Ethernet VLAN SPANS A B Network Sniffer RMON Probe packet analysis software extracts the packet arrival time, length, source and destination IP addresses, transport type, source and destination ports and data length. The analysis module tracks flows defined by a unique <source IP address, source port, destination IP address, destination port, transport protocol> quintuple. Flow related statistics are reported every -second (tunable) interval. that have not seen a packet for a -second (tunable) period are considered expired and removed from the list of active connections. IV. NETWORK TRAFFIC UTILIZATION The INMS SNMP agents collected data on March, 999 providing a -hour view of traffic sampled at 5-minute intervals shown in Fig. 3. As expected the network activity is higher during working hours. The network traffic is highest during normal working hours between 7am to 8pm with network utilization running at 5 megabits/second, (8% utilization) peeking around am-pm at -megabits/second (7% utilization). The percentage utilization is reflective of the directly connected -megabit Ethernet link. However the uplink capacity is limited to the 5 megabit interface connecting the local campus to the corporate backbone. Fig.. Network Monitoring Points Byte and Packet Utilization on March, 999 III. MEASUREMENT METHODOLOGY The selected network statistics consist of traffic volume categorized by bytes, packets and flows. Detailed information includes distributions of traffic composition categorized by network protocols and flows. The mechanism utilized to collect data consists of several monitoring techniques, Simple Network Management Protocol (SNMP), Remote Monitoring version (RMON-) and Packet Analyzer polls. These techniques were used to provide a complete picture of the traffic consisting of statistical data for a -hour time period, with high-resolution traces for smaller sample periods. Interface packet and byte counts were extracted by the INMS SNMP agent queries of the routers and switches. The INMS also queried packet and protocol statistics captured by passive RMON monitoring probes providing higher-level application information categorized by TCP and UDP port assignments. The SNMPv and RMON- statistics provide network utilization data providing -hour by 7-day coverage. An Ethernet Packet Analyzer tool was used to capture the first bytes of every packet. Saving this detailed protocol information created extremely large data sets for even short periods of time. Detailed packet categorization and flow information was performed by custom ITC software. The Bytes and Packets :: ::3 :3: :5:9 ::5 :8:3 ::7 :: :3:3 Input Octets 7835 955 3577 73 7 99 E+5 E+5 E+5 E+5 3E+5 E+5 E+5 E+5 O utput Octets 3537 8 795 599 E+5 3E+5 3E+5 3E+5 E+5 E+5 3E+5 3E+5 E+5 Input Uni-Cast 58.5.3 87.8 9 7. 98.8 7 5 33 9.9 7. 5. Output Uni-Cast. 7.7 87.5. 39.7 87.7 8 38 8 9 89 7. 7. 39.8 Input Multi-Cast.35 8.3 8.73..33.98.98 5.8 5.38 5.53..5 8.8.38 Output Multi-Cast.8.5.7.7 3.9 3.38.7.8..83.98.8.3. Input Errors O utput Errors Date and Time (5-sec. intervals) Fig. 3 SNMP Link Utilization Data The INMS RMON agents collected data for January 9, 999 providing a -hour view of traffic sampled at 5-minute intervals. The pre-configured protocols and applications were captured on the local campus uplink. As expected, IP is the dominant network protocol comprising 95% of the total network traffic. The non-ip traffic maintained a relatively stable utilization while the IP traffic utilization fluctuated with the working hours. A utilization spike of roughly 5- megabits/sec, roughly double normal traffic was recorded for January 9, 999 as depicted in Fig.. :5: :7:9 :8:5 ::3 ::7-783-5538-5/99/$. (c) 999 IEEE

Network Protocol Utilization on January 9, 999 TCP Port Utilization on January 9, 999 7 5 Utilization Percentage 8 Percent Utilization 3 9::: 9::35 9:3:5 9::5 9::5 9:8:: 9:9: 9:: 9::5 9:: 9::5 9:7: 9:9: 9::5 9::3 55 55 5 5 5 9:9: 9:3: 9:: 5 9:9: 5 9:3:5 9::5 9:5:5 9:9: 9::3 9:5:5 IP.7 7.953.93.98.97. 3.55.75 8.9.88...85.589.538.9 IPX.7..73.53.8...88.7.9.73..8...58 ATALK.3.73.3..8..8.3.7..3.5.5..3.9 DECNET.3....3...3.3..5.33.... Date / Time (5-min. Intervals) 9:9: 5 Fig.. Network Protocol Utilization The INMS RMON agents also collected IP protocol counters providing a -hour view of TCP/IP and UDP/IP utilization for January 9, 999. The traffic is depicted in Fig. 5. IP traffic comprised 95% of the network protocol bytes flowing on the uplink. TCP was the dominant IP protocol comprising 9% of the IP traffic. Similar protocol distributions were observed in 997 []. The ratio of TCP to UDP bytes was roughly :. The TCP traffic was user generated fluctuating widely, rising and falling with the working hours. UDP traffic was relatively constant generated autonomously. The increased utilization of the network links was due to TCP traffic. 9::3 5 IP Protocol Utilization on January 9, 999 /3/9 9: /3/9 9:5:5 /3/9 9:8: /3/9 9:: FTP-Data.5.9.5.8..39.5.39.89.93.8.57. E-. Telnet.5.....79.5.7...5.5.7.. SMTP..9.5.3.9.35.8.87..83.38.93.8.3.5 HTTP.7..7.39.73 3.9.889.9 5.9.87 3..9.75.39.58 Port #55 E-....3.7... X-Server..5.5.5.5.5.....8.5.5.5.5 HTTPS..3. 3E- 8E- E- Date & Time (5-min. Intervals) Fig.. TCP Port Utilization Early Internet studies [] summarizing traffic growth trends on the NSFNET backbone, showed exponential growth in 99. Web traffic began to overtake the dominant file transfer and mail applications. In April 995, the traffic distribution by packet count was: Other (7%), HTTP (%), FTP-data (%), NNTP (8%), Telnet (8%), SMTP (%), and Domain (%5). A 998 MCI/vBNS Internet study [3] reported the predominant traffic was HTTP comprising (7%) of the packets. Other applications contributing significant percentages of traffic had reduced contributions, FTP-data (3%), NNTP (%), Telnet (%), SMTP (5 %) and Domain (3%). HTTP is the application driving bandwidth utilization on the Internet and on corporate networks. Percent Utilization 8 9::: 9::35 9:3:5 9::5 9::5 9:8:: 9:9: 9:: 9::5 9:: 9::5 9:7: 9:9: 9::5 9::3 55 55 5 5 5.3 ICMP (IP-)...3.5...5....5... TCP (IP-).58.75..9 3.3 5.3 7.38. 8.53 7.99.98 3.3.9.59.9 UDP (IP-7)..7.7.8.95..3.5.95..7.8..8. IP~....3..53.8.3..7.78.9.33..9 Date & Time (5-min. Intervals) Fig. 5. IP Protocol Utilization V. NETWORK TRAFFIC FLOWS A 7-minute packet trace was taken from the local campus uplink on February, 999. The packet trace data verified similar bandwidth utilization as the previous measurement techniques. This section of the report utilizes the flow concept to provide additional traffic information. The general definition of a flow [] is a sequence of packets traveling from a source to destination, without assuming any particular type of service. The particular definition used for this section defines a flow as uni-directional, distinguished by the source, destination and application. The flow reporting and timeout parameters selected for the graphs and analysis of this report were set to -second timing intervals. The INMS RMON agents captured common IP applications. Fig. depicts seven application utilization percentages. HTTP traffic contributed 78% of the total traffic for the -hour period. FTP data comprised 7% of the total traffic and SMTP comprised % of the total traffic. The fluctuation in HTTP traffic drove the utilization of the link. FTP utilization occasionally peeked utilizing up to 5- megabits/sec of the network bandwidth, comprising close to 5% of the network traffic. The combined HTTP and FTP data closely reflect the aggregate traffic curves. The ITC Packet Analysis software was used to produce bandwidth utilization graphs for the aggregate traffic transmitted on the campus uplink as depicted in Fig. 7. The TCP and UDP bytes reflected the same utilization as reported by the RMON agents. Within the 7-minute sample period the aggregate traffic peaked at -Mbit/sec and averaged - Mbit/sec. The TCP/UDP byte ratio was approximately 5:. The average link utilization was 75Kbytes/sec with a standard deviation of Kbytes/sec. The link averaged 5 packets/sec with a standard deviation of 35 packets/sec. 3-783-5538-5/99/$. (c) 999 IEEE

8, flows were detected averaging 9 new flows per second. For the -second reporting period, the software maintained the state for roughly 3 active flows. Traffic on February, 999 flow information that could be utilized to manage router queues to provide some level of QoS. Fig. 8. depicts the flow size distribution. % of the flows are less than 99 bytes, 99% of the flows are smaller than K bytes. Bytes, Packets and TCP 8 9 3 38 33 78 5 38 3 85 3 33 3 33 3 3 95 99 333 3 99 TCPPackets 3 5 5 5 3 97 9 3 7 77 75 87 7 8 3 TCPBytes E+5E+5E+7E+7E+E+E+8E+5E+E+E+E+5E+5E+8E+E+9E+9E+E+E+8E+E+9E+ UDP 5 8 3 3 35 5 7 5 38 3 75 38 5 UDPPackets 73 33 9 3 57 75 9 3 3 55 9 57 5 8 7 88 58 5 5 UDPBytes 8 9 39 7 95 89 7 7 33 738 7 8 798 99 5 95 95 98 5 AGG 3 73 78 3 37 39 9 373 3 59 39 7 38 357 37 38 3 37 37 37 338 3 AGGPackets 3 8 57 7 3 7 5 3 9 7 79 8 8 9 95 9 39 7 AGGBytes 9 38 5 75 93 3 8 7 85 59 78 9 3 333 35 37 388 7 E+5E+5E+7E+7E+7E+E+8E+5E+E+E+E+5E+5E+8E+E+9E+9E+E+E+8E+E+E+ Capture Period (-sec. Intervals) Fig. 7. Traffic - Bytes, Packets and The definition of the flow timeout parameter and reporting period had significant effects on the number of active flows. The larger the timeout period, the larger the flow tables, and with that the processing overhead to maintain the active flow states. The flow definition is designed specifically not to utilize protocol state information, it utilizes an expiration timeout to determine the end of a flow. This report utilized a timeout expiration of -second in an attempt to accurately map flows to application level connections while minimizing the number of active flow maintained for processing. Number of 8 8 Under 99 to 99 to 99 3to 399 Flow Byte Distribution on February, 999 to 99 5to 599 to 99 Aggregate 75 553 558 33 388 59 3 79 55 35 7 3 DNS 3 5 89 95 8 7 7 99 FTP 3 5 5 7 HTTP 93 55 98 77 37 57 575 39 35 3 87 SMTP 3 3 5 3 9 5 SNMP 9 39 85 33 37 5 3 Telnet 9 375 97 39 9 9 7 9 87 7 5 7to 799 9to 999 Flow Sizes in Byte kto9 999 Fig. 8. Flow Byte Distributions k k M M M G Fig. 9 depicts, 3% of the packets are transferred within a millisecond. The majority of these transfers are single packet flows. The packet analyzer code uses a / millisecond constant to pad the last packet transfer time for each flow. Single packet flow duration default to this pad. % of the flows are between / to 5 seconds in duration. Flow Rate Distribution on February, 999 The following table lists the number of flows, active flows and new flows per second maintain at the defined expiration timeout period. 8 Flow Timeout.5.5 5 Value (Secs.) Number of 3 95 8 55 9 Avr. Number of 7* 3* 3 7 Active New per 3 5 9 3 5 Second * The average number of active flows is reported for a -second interval, if the expiration timer is less than second the interval is equal to the flow timeout value. This report utilized a -second timeout and a -second reporting period to map flows application connections. Utilizing smaller timeouts and reporting periods may provide more accurate flow mappings for most applications, although some application connections would be mapped into multiple flows. Small timeouts and reporting periods enable real-time.5 5 7.5 5 5 75 5 5 75 5 5+ ALL 83 5 983 79 3 5787 9 3 33 5 85 9883 737 DNS 8 8 7 5 5 3 3 79 FTP 8 8 3 HTTP 8 885 87 5 37 9 38 89 5 57 9 9383 55 595 SMTP 8 5 9 5 5 3 3 SNMP 5 5 8 3 35 33 78 Telnet 8 9 7 3 5 7 8 8 9 5 3 5 7 Data Rate KB Fig. 9. Flow Rate Distributions The calculated data rates show a surprising 8% of the transfers exceed -Mbit/second data rate, Fig.. -783-5538-5/99/$. (c) 999 IEEE

Flow Duration Distribution on February, 999 3 5 5 5 5 5 5 5 5 5 ALL 537 78 39 889 887 99 59 75 8 39 DNS 3 7 3 3 7 38 33 FTP 9 HTTP 7 85 85 335 3737 785 93 75 9 9 SMTP 3 3 58 87 58 3 SNMP 55 7 9 37 5 9 Telnet 5 3 9 5 9 7 7 5 8 7 8 Duration ms Fig.. Flow Duration Distributions Fig.. depicts the number of packet per flow. 3% of the flows contain a single packet, 9% of the flows consist of or less packets. 5 3 Flow Packet Count Distribution on February, 999 + VI. SUMMARY The corporate network traffic at the WAN uplink displays characteristics similar to Internet traffic models. The dominant network traffic is HTTP running over TCP/IP. These models can provide detailed characteristics for the data type packets for multimedia networks. The flow can also be directly imported to simulation tools to provide a higher degree of accuracy for network design and planning. These data models are used on the HyNeT to generate traffic to analyze gateway designs, flow multiplexing and bandwidth allocation prototypes. The ITC utilized this data to analyze efficiency of a TCP gateway for high latency networks. The benefit and sizing of the TCP gateway design is dependent on the TCP connection characteristics. These characteristics were extracted from the traffic flow models. The ITC report [] measured noticeable benefits after the data set exceeds -kilobytes. The data from the traffic models support the need for TCP gateways. The data from a 7-minute sampling found 9 flows were larger than -kilobytes, 3 were at least a megabyte. These large flows were generally associated with FTP, HTP and SMTP data transfers, suggesting that classifying network flows and routing only HTTP, FTP and SMTP flows to the gateway would also improve the gateway efficiency. 5 5 5 5+ ALL 85 77 77 8 3 5 DNS 85 8 8 3 FTP 3 HTTP 9 3789 87 39 8 SMTP 3 39 5 7 3 SNMP 9 3 8 8 8 Telnet 55 788 39 3 8 5 Packets per Flow Fig.. Packet Count Distributions Fig. depicts the number of data bytes per packet. 5% of the packets contain no user data. These packets can be accounted for by TCP acknowledgments and other network control messages. Some TCP optimizations cluster these packets conserve bandwidth. 35 3 Flow Bytes per Packet VII. REFERENCES [] B. Hine, P. Hontalas, T. Fong, "Lockheed Martin Corporate Traffic Estimate", Fourth Planet, Inc. Los Altos, CA, Sept 997, unpublished. [] K.D. Frazer, NSFNET: A partnership for High-Speed Networking, Final Report 987-995, Merit Network Inc., 995. [3] Greg Miller and Kevin Thompson, "The Nature of the Beast: Recent Traffic Measurements from an Internet Backbone", paper 73, INET'98 Conference, Geneva, Switzerland, July 98. Bytes per Packet 5 5 5 [] V. Bharadwaj, "Optimizing TCP/IP for Satellite Environments, Phase Implementation Report", Center for Satellite and Hybrid Communication Networks, University of Maryland, College Park, 998. 8 5 5 + ALL 99 858 595 7978 555 DNS 57 359 7 FTP 3 3 85 HTTP 3 957 55 8 57 753 SMTP 87 3 39 7 SNMP 83 Telnet 98 835 7 8 85 7 Packets Fig. Bytes per Packet Distributions 5-783-5538-5/99/$. (c) 999 IEEE