Issues in the Passive Approach of Network Traffic Monitoring

Transcription

1 Issues in the Passive Approach of Network Traffic Monitoring Adrián Pekár, Eva Chovancová, Peter Fanfara and Jana Trelová Department of Computers and Informatics, Faculty of Electrical Engineering and Informatics Technical University of Košice, Košice, Slovak Republic {adrian.pekar, eva.chovancova, peter.fanfara, Abstract This paper deals with the issues arising during network traffic monitoring performed by the passive approach of measurement. It describes the most important characteristics of network traffic, which are usually measured during their monitoring, as well as different approaches for the evaluation of network traffic properties. Current research and development in the field of network monitoring is focused on the abilities to measure various network traffic characteristics. The greatest effort is placed on the maximization of their efficiency and minimization of their impacts on the overall functionality of the network. Since there are many approaches for measuring network traffic properties, only the most commonly used ones will be discussed, i.e. active and passive measurement. The paper is concluded with some notes aimed at the future direction by which should research work proceed. I. INTRODUCTION Nowadays, vast majority of modern electronic devices is connected to a network. These devices, among others, include computers, supercomputers, mobile and wireless devices, satellites, tablets, and other systems. They are usually called by one common name end node or end system. Computer networks describe a set of interconnected devices for communication and information exchange. Data transfer between end systems is provided by network devices which serve the computer networks. Therefore, behind the full functionality of today s Internet is a wide range of network devices such as repeater, hub, switch, bridge, router, gateway, etc. Equally important network components are the links enabling physical transmission of data, whether between network devices, end systems or a combination thereof. Over the last decade, computer networks have become an integral part of everyday life. They are used for personal communication, data sharing, public dissemination of news, transfer of private data, collaboration, telephony, business, education, entertainment, socializing, etc. In the vast majority, the source of these activities are the users and applications they use. The result is a huge volume of data that form the traffic of computer networks. With such a wide and diverse characteristics, an important aspect during the design, management and optimization of today s complex and high-speed computer networks is to measure and monitor their various properties. Network measurements and monitoring provide important information to the users, Internet service providers (ISP), researchers or other members of the public and scientific community. On the basis of this information is possible to ensure the quality, performance and full functionality of computer networks, their services and applications. In the following sections an analytical background of network monitoring and their properties will be presented; starting with a brief description of the most important traffic characteristics, whose measurement is in terms of network monitoring useful; continuing with the definition of the identified open issues occurring during monitoring of traffic characteristics; up to the verification of these issues. The last section of this paper draws a conclusion and some future directions. II. MOTIVATION In the terminology of computer networks along with the term monitoring often appears measurement. In addition, in some cases is about measurement spoken as if it was an inseparable part of monitoring. The fact is, that measurement and monitoring represent a wide area of computer networks, besides both of them form an indispensable pillar of a mechanism, which stands behind today s high-speed computer networks. The relationship between monitoring and measurement is quite simple. Without measurement of network traffic and subsequent evaluation of the measured data would be impossible to perform network monitoring. Without monitoring would be then impossible to control, manage, secure and optimize networks. While in case of network monitoring and measurements the most important entities are traffic characteristics, in case of evaluation the most important role play the apparatus selected from the area of mathematical sciences. Therefore, an exceptional attention should be payed to them during network monitoring and other related tasks. Network traffic measurement is important for a wide range of different activities. For example, performance analysis to solve problems concerning bandwidth, packet loss or delay requires an accurate measurement of network traffic. Network traffic characteristics can be also useful for network-based applications to adapt their requirements to the properties and limits of the network. User experiences with network-based applications is given by the frequency, duration and severity of events (e.g. failures, changes in routing or miss-configuration), that affect the quality of the used network services. Tools for

2 the detection, isolation and determination of the root causes of these events are very important [20]. Not only to timely react and minimize the effects of these events, but also to maintain the competitiveness and reputation of ISPs. From the measured data providers can obtain important information, which are usually used for performance characteristics evaluation, anomaly detection and customer invoicing. Although computer networks belong to the ever-evolving fields of information technology, the number of new connections whether in form of devices or users and rapid increase in the volume and content of the traffic makes monitoring more and more difficult. Individual tasks related to monitoring and evaluation of measured values is surrounded by a couple of issues. Over the past decade adequate part of these issues had been successfully resolved. Various techniques [12], methods and tools were created for collecting and evaluating [1] the characteristics of converged networks. However, some problems still remain unresolved. III. PROPERTIES OF COMPUTER NETWORKS Computer Networks are characterized by many different properties, that from the perspective of network traffic monitoring is important to measure. Some of these properties are related to physical components, others to the traffic itself. An important group of properties is represented by those characteristics, which arise from the interaction of physical components and the traffic. Time is one of the basic entities, which also plays an important role in almost every operation associated with network monitoring. A. Properties of Physical Components Physical components constitute the basic elements of networks, which among others, include [7]: Links interesting parameters to measure is their propagation delay and capacity. Routers interesting features to measure is the IP address, type of supported protocols, time required to deliver the packet, etc. B. Traffic Properties Today s computer networks are capable of transmitting large amount of data. These data can be considered as a collection of packets or bytes, which define the traffic of networks. Traffic usually covers all types of data transmission (video, voice, data, control messages, etc.) in a given unit of time, but in some cases it can by limited to a specific session, messages, records or group of users. So an interesting property to measure in case of network monitoring are the packets, packet trains [7] or octets (bytes) of the traffic. Another important property of network traffic is the IP flow. According to the IPFIX standard [6], IP flow is defined as a set of IP packets passing an observation point during a certain period of time. All the packets belonging to the same flow have common features. More detailed information about IP flows and the IPFIX protocol is provided in [6], [16]. C. Characteristics Resulting from the Interaction of the Network Infrastructure and the Traffic There are many properties of traffic, that are affected by the state of the network. These properties can be thought as the result of the interaction between the traffic and network infrastructure [7]. Characteristics resulting from this interaction are often referred to as the parameters related to quality of services (QoS) [11]. The most important parameters are depicted in Tab. I. TABLE I MOST IMPORTANT PARAMETERS OF QUALITY OF SERVICES Parameter Bandwidth Packet loss One-way delay (OWD) Description Parameter defined as the effective amount of data transfered per a unit of time. Number of undelivered or corrupted packets. Time required to send a packet from a source to a destination. Jitter A parameter defined for two packets (P i, P i+1 ) as the difference between the value of one-way delay of packet P i and the value of one-way delay of packet P i+1. Round-trip time (RTT) Throughput D. Time Characteristics Time required to a packet to travel from a source to a destination and back. Number of packets transmitted per a unit of time. A frequent requirement during network traffic monitoring is capturing time characteristics. The accuracy of the measured properties like round-trip time (RTT) or one-way delay (OWD) are highly dependent on the measurement of time characteristics. Since the individual network devices are often distributed in a significant distance, obtaining accurate time information can be a demanding task. Many problems are related to the accuracy of clocks, by which time characteristics are determined [9]. IV. NETWORK TRAFFIC MONITORING Network monitoring describes the activity of a system, which continuously monitors the entire network and its traffic. The main feature of this system is, that in case of any anomaly, failure or unusual event, it immediately alerts the system administrator. Except continuous surveillance, monitoring is also used in case of the management of network traffic, various accesses, components, etc. These already complex tasks further complicates the topological (physical connection of all components) and computational (routing and management activities of network components) complexity of the networks. Network monitoring is highly conditioned by data collection. This activity is known as network properties measurement and among with the evaluation of the measured data, it forms the most important parts of the monitoring process. There are many approaches for network traffic parameters measurements. The most common are active and passive measurement approaches. Both provide different types of results, while they are typically used in various measuring setups and for different purposes.

3 A. Active Measurement Active measurement of network properties is based on the ability to insert probe packets into the monitored network. By monitoring and subsequent measurement of probe packets is possible to obtain the operational visibility of the network and other important properties. An example of active measurement is shown in Fig. 1. Fig. 2. The architecture of passive measurement. Fig. 1. The architecture of active measurement. In some cases, probe packets are directly sent to a server or applications. This way, it is possible to get an overview of the state of network services. The following two properties can be derived from active measurements: active measurement requires additional traffic generation, the traffic and its parameters are artificial. One unsolicited side effect of active measurements can be the increase of network load, which can lead to the affection or complete degradation of the measurement results. Another disadvantage of active measurement is, that it provides only limited information about the observation point. Instead, it provides various kind of characteristics about the connection between two nodes. Usually they are performance properties, such as round-trip time (RTT), average packet loss, bandwidth of the connection or packet throughput. In some cases, active measurements can provide information about asymmetric delay or routing changes between two nodes. Existing tools based on active measurement are for example Ping or Traceroute. B. Passive Measurement The process of passive measurement does not require additional generation or modification of the existing traffic. The measured traffic is generated only by the connected users and network applications. An example of passive measurement is depicted in Fig. 1. Passive measurement is usually performed by the means of: Network components (switches, routers, end devices, etc.) with built in traffic collection mechanism. These mechanisms,for example, are Netflow [5], SNMP [3], etc. Software tools (BEEM [10], Wireshark [24], etc.) designed for collecting and processing network data. Collecting measured data from these resources is performed periodically. The evaluation of the collected data determines the required network traffic characteristics (performance, state, etc.). The advantage of passive measurements is, that it is performed on real traffic. Unlike active measurement, passive measurement does not increase the load of the network traffic. Unfortunately, pulling and collecting the measured data can cause a certain increase of traffic (especially in case of capturing every single packet). So this advantage is only relative. A solution describes assigning an individual path for the measured traffic. This way, information related to passive measurement will not be interfered with the real traffic, which will consequently lead to unaffected results. Considering, that in some cases each packet is measured, passive measurement may face problems related to privacy or information security [21], [22]. In contrast to active measurements, passive measurements provide detailed set of information about the observation point. These information are for example types of data that make up the traffic mix (data, services, protocols, audio, video, etc.), packet intensity, packet timings or packet delay. V. ISSUES RELATED TO THE MEASUREMENT OF NETWORK TRAFFIC CHARACTERISTICS Network traffic monitoring and properties measurement describe a challenging task. Especially in large and complex computer networks. Measurements can be virtually realized at any point (location) of the network. Since all of these points has some specific features which are valuable to measure during network monitoring the location of observation points belongs to one of the most common issues [7]. Another common problem describes the notion of time. Time is one of the basic entities, which plays a key role in almost every operation related to network traffic monitoring and has a strong effect on the accuracy of the results. This section provides a brief description of the most critic open issues. A. Data volume A common problem of the infrastructural entities (router, links, etc.) is to determine the appropriate volume of the measured data. The main task of network entities is to ensure a smooth and fast data transfer. In case of today s converged networks it means a huge amount of data.

4 The deployment of traffic monitoring can in some cases instead of quality and manageability increase bring an opposite effect, i.e. bring load to the entities or traffic. If for instance the individual network devices are affected by control messages and measured data transfered between the monitoring system and observation points, it can easily lead to the decrease of traffic performance, delays or even data loss. Depending on higher level protocols [19], this data loss may or may not be handled. In such case, network monitoring loses its benefits and in certain cases it becomes undesirable. In order to cope with increasingly demanding requirements of high-speed links monitoring mechanisms have to be regularly optimized. B. Asymmetry of System Resources Another issue is the asymmetry between system resources of measuring (monitoring) system and the network, which is measured (monitored) [13]. Although only a small part of traffic is measured, the network has still a lot more devices than the monitoring system. This results in an incomparable difference between their computational capabilities. In addition, capturing and storing information about traffic for later analysis is further restricted by parameters such as bandwidth, speed and capacity of the memory or disk. Since traffic bursts can also arise, when ensuring the execution of the most common tasks such as filtering or routing, it is difficult to estimate the system requirements of the router. If a router has to allocate a fixed portion of system resources for measurements, most will do at the expense of other functionalities (like limiting the ability to cope with sudden traffic changes). Even if a hardware capable of monitoring the appropriate part of traffic flows exists, collecting simpler metrics such as counts (packet or octet counts) can consume a significant portion of available system resources. This asymmetry requires the development of efficient techniques and approaches for network traffic monitoring. C. Creation and Export of Flow Records Routers besides capturing packets are often involved in creating aggregated information about traffic flows. This kind of aggregation is ensured by a flow record, which consists of information about key characteristics of the network traffic. These records are periodically exported for various purposes, such as monitoring traffic properties, accounting or network management [5], [6]. Routers from Cisco Systems [4], by their proprietary Netflow protocol [5] are capable of creating such flow records, which contain important statistics about network traffic. The most used fields of these flow records are presented in Tab. II. Netflow records are created: after the expiration of an optionally adjustable time period, while the end points are inactive (passive timeout), if one party terminates the connection, after exceeding an optionally adjustable time period, while the end points are still active (active timeout), if the router needs to empty its buffer. Despite the fact that flow records represent an effective form of aggregated meta-information, simple monitoring of a large number of flows and subsequent generation of records may significantly load the router [7]. Limitations of the memory and computation functionalities in combination with the main task of the router which is routing have led to a variety of sophisticated methods for reducing the amount of processed data. Such techniques are packet sampling [2] and summarization [8]. Like routers, tools for collecting flow records which are often some external software of hardware components of the measurement may also have their own bandwidth, memory size or computational capacity. It follows, that without optimization of these tools or measurement techniques itself, packet dropping or data loss can occur [7], [23]. D. The Error Rate of Monitoring Tools Most of the existing monitoring mechanisms, during the analysis of the measured values take into account only one property (attribute) of the network traffic [18]. If the evaluated value of this property fluctuates around a predefined limit, the monitoring tool may incorrectly judge the situation and report false alarms. Such a situation can happen, if the permissible value of, e.g. bandwidth, fluctuates around a standard value, which may consequently cause, that the monitoring mechanism limit some functionalities of the system or report too many warnings even if threads or problems are not present [18]. In this case, the accuracy and quality of the monitoring mechanisms can be easily queried. E. Real-Time Data Evaluation The ability of processing adequate portion of measured data in real-time describes also a difficult task. Storing the data in a database is in terms of real-time data evaluation absolutely inapt [23]. Too frequent export of flow records can cause the reduction of network traffic performance, delays or data loss. Therefore, data exchange between the monitoring system and the observation point(s) is normally performed in larger time scales. Although by this approach is possible to reduce the unwanted traffic load, the evaluation of data in real-time becomes hardly performable. Therefore, it is necessary to determine the right balance between the time intervals of flow record exports and efficient use of available (system) resources. However, this task in today s networks with frequently changing characteristics is without automated processes almost impossible to achieve. VI. EXAMINING THE IDENTIFIED ISSUES To verify the identified issues occurring during network traffic monitoring, a group of experiments was performed. The experiments were focused: on the estimation of the average volume of data, that must be processed during monitoring; on the determination of the average usage of system resources, which network monitoring brings in itself.

5 TABLE II THE MOST USED NETFLOW V9 FIELD TYPE DEFINITIONS [5] Field Type Value Length (bytes) Description. SRC_TOS 5 1 Type of Service byte setting when entering incoming interface. L4_SRC_PORT 7 2 TCP/UDP source port number i.e.: FTP, Telnet, or equivalent. IPV4_SRC_ADDR 8 4 IPv4 source address. L4_DST_PORT 11 2 TCP/UDP destination port number i.e.: FTP, Telnet, or equivalent. IPV4_DST_ADDR 12 4 IPv4 destination address. SRC_AS 16 N (default is 2) Source BGP autonomous system number where N could be 2 or 4. DST_AS 17 N (default is 2) Destination BGP autonomous system number where N could be 2 or 4. DST_TOS 55 1 Type of Service byte setting when exiting outgoing interface. IP_PROTOCOL_VERSION 60 1 Internet Protocol Version Set to 4 for IPv4, set to 6 for IPv6. If not present in the template, then version 4 is assumed. flowstartmicroseconds 154 datetimemicroseconds The absolute timestamp of the first packet of this Flow. flowendmicroseconds 155 datetimemicroseconds The absolute timestamp of the last packet of this Flow. Fields flowstartmicroseconds and flowendmicroseconds are defined by IANA registry of IPFIX information elements. To determine the average volume of data which is necessary to process during network monitoring and the approximate load of system resources, some measurements in two existing networks were performed. Each of them had different number of connected devices and various speed of links. The experiments were realized repeatedly. From the obtained results the average values of the data volume collected within 1 hour were determined. During measurements, the average load of the router (Cisco 2811, IOS 12.4T) which except standard routing tasks also participated in the generation end export of Netflow [5] flow records was also recorded. The results are summarized in Tab. III. TABLE III SUMMARIZATION OF THE PERFORMED EXPERIMENTS Network I Network II Number of end devices Speed of interface/link 100 Mb/s Mb/s Volume of transferred data 210 MB MB Count of transferred packets Average usage of CPU without Netflow 3 % 31 % Average usage of CPU with Netflow 6 % 48 % The results indicate that although neither of network monitoring load the link or the system resources to their maximum limits, yet, it can be concluded, that a relatively large amount of data was transferred through the networks. Moreover, with the increase of the speed of lines the system requirements would proportionally increase too. For example, when monitoring traffic on links with speed 1 GB/s, it would be necessary to process and evaluate approximately 112 TB of data per a day. However, at present, there are much more faster approaches to transfer data [17]. It can be thus concluded, that the problems related to large volume of data, that have to be processed during monitoring and the resulting utilization of system resources are relevant and present in today s computer networks. In addition, the analysis in [15] confirms, that existing techniques and methods addressed to solve these problems are not quite sufficient. VII. DIRECTIONS OF NETWORK TRAFFIC MONITORING OPTIMIZATION By the notion optimization of network traffic monitoring is meant the design and implementation of such mechanisms and methods, which are addressed to solve the issues surrounding network monitoring. Due to the open problems mentioned in previous sections and the results of the performed experiments, the optimization of network traffic monitoring should mainly focus on: the minimization of the overall network congestion and load caused by monitoring, the increase of the efficiency of the use and minimization of the load of system resources by the monitoring mechanisms, the increase of the accuracy of the evaluation processes, the maximization of the ability of real-time data evaluation. These goals should be achieved by an adaptive export method of information about the network traffic flows. The load of system resources by the monitoring mechanisms is due to the changing nature of network traffic often ineffective. A significant lack of existing solutions is the absence of adaptive exporting methods, which take into account the actual state and properties of network traffic. A conceptual design of such an adaptive export method is the following: As problems mentioned in previous section mainly occur during passive measurement of network properties, future direction should be aimed at the adaption of some particular properties of flow records export to the actual state of network traffic. Such properties include, for example, the volume of the data, that is used for the generation of flow records or the time interval of their export. With this approach is possible to significantly contribute to the above mentioned objectives of optimization. In case of adaptive export, it is also necessary to determine the appropriate balance between the export intervals and the efficient utilization of the available network and system resources. Export intervals are important for realtime monitoring, which favor the smallest period of time intervals.

6 To achieve the efficient evaluation of data a multidimensional analysis of various traffic properties will be used. In such analysis, during the evaluation of the actual state of network traffic, instead of a single property, more parameters will be taken into account. Using this method is possible to significantly reduce the number of false judgments of the monitoring mechanism. Such an error is for example the one, when the monitoring system raise an alarm even in the case of absence of a threat or problem. A key factor is the definition of properties, that will be one of the input parameters of the adaptive export method. However, this task requires further experiments, which should clearly identify those properties, which by changing their values significantly affect the performance of the network and the efficiency of the utilization of system resources by the monitoring systems. VIII. CONCLUSION AND FUTURE WORK An important aspect of developing, managing and optimizing wide and complex computer networks is the measurement of their burden and behavior. An indispensable tool for the accomplishment of this task is network traffic monitoring. By network monitoring can be possible to detect internal or external attacks or ensure the smooth functionality of delay-sensitive application (VoIP, VoD). It also allows Internet Service Providers (ISP) to ensure the conditions specified in the Service Level Agreement (SLA). Individual tasks related to the network monitoring and the evaluation of the measured properties are surrounded with many problems. Reasonable portion of these problems has been for the last decade successfully solved. However, some problems remained unsolved. This paper gives a brief overview of those properties of networks, which are during their monitoring and measurement the most frequently followed, as well as different approaches for determining the characteristics of network traffic. It also lists some open issues, that occur during monitoring. The output of this paper is a conceptual design of an adaptive method for the export of records about IP flows. This method is addressed to optimize the monitoring of network traffic. Future work will be aimed at the preparation of a more detailed design of this method as well as its implementation in the SLAmeter [14] network traffic metering and monitoring tool s architecture. ACKNOWLEDGMENT This work was supported by the Slovak Research and Development Agency under the contract No. APVV (70%). This work is also the result of the project implementation Development of the Center of Information and Communication Technologies for Knowledge Systems (project number: ) supported by the Research & Development Operational Program funded by the ERDF (30%). REFERENCES [1] N. Ádám, Single input operators of the df kpi system, Acta Polytechnica Hungarica, vol. 7, no. 1, pp , [2] M. Canini, D. Fay, D. Miller, A. Moore, and R. Bolla, Per flow packet sampling for high-speed network monitoring, in Communication Systems and Networks and Workshops, COMSNETS First International, jan. 2009, pp [3] J. Case, M. Fedor, M. Schoffstall, and J. Davin, Simple Network Management Protocol (SNMP), RFC 1098, Internet Engineering Task Force, Apr [4] I. Cisco Systems, Understanding and designing sundials, [Online]. Available: [5] B. Claise, Cisco Systems NetFlow Services Export Version 9, RFC 3954 (Informational), Internet Engineering Task Force, [6] B. Claise, Specification of the IP Flow Information Export (IPFIX) Protocol for the Exchange of IP Traffic Flow Information, RFC 5101 (Proposed Standard), Internet Engineering Task Force, [7] M. Crovella and B. Krishnamurthy, Internet Measurement: Infrastructure, Traffic and Applications. John Wiley and Sons, Inc, [8] B. Donnet, T. Friedman, and M. Crovella, Improved algorithms for network topology discovery, Passive and Active Network Measurement, pp , [9] J. Giertl, L. Husivarga, M. Révés, A. Pekár, and P. Fecil ak, Measurement of network traffic time parameters, in In Proceedings of the Eleventh International Conference on Informatics, (Informatics 11), Rožňava, Slovakia, 2011, pp [10] F. Jakab, L. Koščo, M. Potocký, and J. Giertl, Contribution to qos parameters measurement: The basicmeter project, in Proc. International Conference on Emerging elearning Technologies and Applications (ICETA 05), Košice, Slovakia, 2005, pp [11] H. J. Lee, M. S. Kim, J. W. Hong, and G. H. Lee, QoS Parameters to Network Performance Metrics Mapping for SLA Monitoring, [12] M. Michalko, Video streaming in wireless networks using avismo concept, Journal of Information, Control and Management Systems, vol. 9, no. 2, pp , [13] R. Pang, Towards understanding application semantics of network traffic, Ph.D. dissertation, Princeton University, Princeton, NJ, USA, [14] A. Pekár, J. Giertl, M. Révés, P. Fecil ak, and M. Antl, The sla meter tool, in In Proceedings of Electrical Engineering and Informatics 3, (EEI III), 2012, pp [15] A. Pekár, Modelovanie a návrh systémov pre monitorovanie siet ovej prevádzky, [16] J. Quittek, T. Zseby, B. Claise, and S. Zander, Requirements for IP Flow Information Export (IPFIX), RFC 3917 (Informational), Internet Engineering Task Force, [17] N. Schlepple, M. Nishigaki, H. Uemura, K. Obara, H. Furuyama, Y. Sugizaki, H. Shibata, and Y. Koike, 40 gbps high-speed link over thin gi 50/125 plastic optical fibers and compact optical sub-assembly, [18] I. Shimokawa and T. Tarui, Network monitoring method based on self-learning and multi-dimensional analysis, in In Proceedings of The Second International Conference on Advances in Information Mining and Management, (IMMM 12), 2012, pp [19] A. Tanenbaum and D. Wetherall, Computer Networks: International Version. Prentice Hall, [20] L. Vokoroko, A. Baláž, and B. Madoš, Intrusion detection architecture utilizing graphics processors, Acta Informatica Pragensia, vol. 1, no. 1, pp , [21] L. Vokorokos, N. Ádám, and A. Baláž, Application of intrusion detection systems in distributed computer systems and dynamic networks, in In Proceedings of Computer Science and Technology Research Survey, (CST 08), 2008, pp [22] L. Vokorokos, A. Kleinová, and O. Látka, Network security on the intrusion detection system level, in In Proceedings of the 10th IEEE International Conference on Intelligent Engineering Systems, (INES 06), 2006, pp [23] L. Vokorokos, A. Pekár, and N. Ádám, Data preprocessing for efficient evaluation of network traffic parameters, in IEEE 16th International Conference on Intelligent Engineering Systems, (INES 12), 2012, pp [24] Wireshark, Network protocol analyzer, [Online]. Available: