Three Levels Network Analysis for Anomaly Detection

Transcription

1 Three Levels Network Analysis for Anomaly Detection Bruno B. Zarpelão 1, Leonardo S. Mendes 1, Mario L. Proença Jr. 2 and Joel J. P. C. Rodrigues 3 1 School of Electrical and Computer Engineering, University of Campinas (UNICAMP), Campinas, SP, Brazil 2 Computer Science Department, State University of Londrina (UEL), Londrina, PR, Brazil 3 Instituto de Telecomunicações, University of Beira Interior, Covilhã, Portugal s: {bzarpe,lmendes}@decom.fee.unicamp.br, proenca@uel.br, joel@ubi.pt Abstract - Anomaly detection is fundamental to ensure reliability and security in computer networks. In this work, it is proposed an anomaly detection system that monitors the network in three different levels. In the first one, data is collected from Simple Network Management Protocol (SNMP) objects and compared to profiles of normal traffic, in order to detect behavior changes. Second level of analysis includes a dependency graph that represents the relationships between SNMP objects. It is used to analyze first level alerts, confirming the occurrence of anomalies in device level. In the third level of analysis, second level alerts are grouped according to network topology information, and network administrators are informed about the context where the anomaly occurred. Tests were performed in a real network environment and good results were obtained. 1. INTRODUCTION Evolution of networking technologies has encouraged the creation of new and sophisticated services for government, academy and industry. Therefore, computer networks have become essential in a diverse array of environments, where reliability and security have been highlighted as key issues [1]. As computer networks present these advances, managing them efficiently becomes more important and harder. Today s networks are composed by heterogeneous software and hardware elements, characterizing complex scenarios where it is impossible to work without tools like anomaly detection systems. Anomalies are unexpected changes in traffic levels that can cause from small performance degradation to disruption of network operations. They can be caused by flash crowds, malfunctioning, network devices failures, vendor implementation bugs, misconfigurations, transfer of very large files, outages and malicious attacks such as DoS (Denial of Service), DDoS (Distributed Denial of Service) and worms [2]. This work proposes an anomaly detection system that organizes the network analysis in three levels. In the first one, a comparison is performed between data collected from SNMP objects and profiles of normal traffic. A hysteresisbased mechanism is used and first level alerts are generated when behavior deviations are detected. In the second level, first level alerts are analyzed together, taking into account the characteristics of the SNMP objects. Each SNMP object brings a different view of the network device, and system efficiency is improved by combining information from these different perspectives. When anomalies are confirmed for a device, second level alerts are generated. Finally, in the third level, second level alerts are grouped in order to provide to the network administrator a wide view of the problem in the network. Besides detecting anomalies, the proposed system informs the network administrator about the context where they are inserted, helping to find the cause and solution of the problem. The remainder of this paper is organized as follows. Section 2 presents the related work. Section 3 depicts the proposed solution for anomaly detection, based in a three levels analysis. This section is divided in three parts, one for each analysis level. Section 4 brings evaluation results obtained in a real network environment. Finally, conclusion and future work are presented in the section RELATED WORK Denning [3] presented the first work that used the characterization of network normal behavior to detect anomalies. Since this proposal, many authors have presented solutions using the same model, which has been combined with different techniques such as generalized likelihood ratio (GLR) [2], general wavelet filters [4], principal component analysis (PCA) [5], discrete wavelet transform [6], and intelligent flow sampling [7]. Lakhina et al. [5] presented an important work concerning detection and characterization of network-wide anomalies. By using the PCA technique, data collected from the network was separated into two disjoint subspaces, enabling anomaly behavior identification. Besides detecting anomalies, this work searched for causes of anomalies, showing the problem scenario to the network administrator. Ringberg et al. [8] addressed a work that also applies PCA. They evaluated how difficult is the adoption of proposed solution, concluding that it is hard to configure and deploy in production environments. An anomaly diagnosis system is proposed in [9]. With similar objectives than [4], they proposed to detect and classify the anomalies, showing to the network administrator the traffic flow that is responsible for the problem. The

2 solution is organized in three parts. At first, two different data windows are compared, in order to find changes in some measurements such as number of packets, bytes and flows. In the second step, if any change has been detected previously, the network is scanned to find which flows are responsible for the anomaly. Finally, in the last step, anomalies are classified and signatures are created and stored to be used in future analyses. In this work, it is proposed an anomaly detection system that analyzes the network in three different levels, using heuristics, simple algorithms and the Simple Network Management Protocol (SNMP) [10]. Our anomaly detection system is a lightweight solution, which does not use flow or packet instrumentation. Besides, it is able to offer useful reports containing additional information about the problem to network administrators. with these characteristics of network traffic behavior. In our work, BLGBA model is used to generate different profiles of normal behavior for each day of the week, meeting this requirement. 3. ANOMALY DETECTION SYSTEM Architecture of the proposed anomaly detection system is presented in Figure 1. At first, there is a module that is responsible for collecting the network information from Management Information Bases (MIB) [11]. For each SNMP object analyzed in each network device, there is an instance of the module for object level analysis. This module compares the profiles of normal behavior to the data collected from SNMP objects. For each monitored device, there is an instance of the module for device level analysis, which is responsible for analyzing all the object level alerts and generating a device level alarm if the anomaly is confirmed. Finally, the module for network level analysis gathers all device level alarms and analyzes them to verify if they are related, using network topology information. If different device level alarms are related to the same problem, a network level alarm is generated and a report is sent to the network administrator. 3.1 Object level analysis Object level analysis is defined as a step where each SNMP object data is analyzed to detect unexpected changes of behavior. Therefore, it is necessary to characterize the normal behavior of each SNMP object, what represents a significant challenge due to the non-stationary nature of network traffic. In this work, SNMP object data is characterized by using the Baseline for Automatic Backbone Management (BLGBA) model [12], which is applied in historical data of a given SNMP object to construct the Digital Signature of Network Segment (DSNS) [12]. Traffic behavior is composed by daily cycles, where traffic levels are usually higher in working hours. Traffic levels are also distinct for different days, as can be verified when workdays traffic is compared to weekends traffic [4], [12]. The traffic characterization model should be able to deal Figure 1 - Architecture of proposed model. The BLGBA algorithm was developed based on a variation in the calculation of statistical mode. In order to determine an expected value to a given second of the day, the model analyzes the values for the same second in previous weeks. These values are distributed in frequencies, based on the difference between the greatest G aj and the smallest S aj element of the sample, using 5 classes. This difference, divided by five, forms the amplitude h between the classes, h = (G aj S aj )/5. Then, the limits of each L Ck class are obtained. They are calculated by L Ck = S aj + h*k, where C k represents the k class (k = 1...5). The value that is the greatest element inserted in the class with accumulated frequency equal or greater than 80% is included in DSNS. This process is performed for all seconds of a day, building the DSNS, which has an expected value for each second of any day of the week. After characterizing the traffic, object level analysis is performed by comparing the DSNS to real data. The comparison algorithm is based in a hysteresis mechanism, which includes a parameter named delta (δ) to decrease the probability of generating false positive alarms. Three different events drive the hysteresis mechanism operation. At first, DSNS value is used as a threshold. When a real value collected from the SNMP object is greater than the DSNS value, a type 1 event is identified and the hysteresis interval is initiated. A new threshold is established with the real value. During the hysteresis interval, every time a real value overcomes the current threshold, a type 2 event is identified and a new threshold is established. When the amount of type 2 event occurrences is greater than the value defined in δ, a type 3 event is identified. Then, a first level alert is generated. Figure 2 presents a flow chart that illustrates this algorithm.

3 compose a path in the graph that begins in an initial point of monitoring and ends in a final one, an anomaly is detected. Figure 3 - Dependency graph used in device level analysis. 3.3 Network level analysis Figure 2 - Flow chart of hysteresis algorithm. 3.2 Device level analysis In the device level analysis, first level alerts are analyzed to confirm if an anomaly is occurring or not. By monitoring different SNMP objects, diverse views of the device are obtained, enabling the system to generate a more reliable second level alert. First level alerts are analyzed according to a dependency graph that represents the relationships between the SNMP objects. Each vertex represents a SNMP object and each edge represents a possible path of anomaly propagation between two objects. Since the objects ifinoctets and ifoutoctets have a different instance for each interface of the device, there is a vertex for each instance. The dependency graph includes objects from groups interface, ip and tcp, covering main operations of network devices such as servers, switches and routers. Figure 3 shows the dependency graph. Aiming to perform the analysis, initial and final points of monitoring are defined for each device. For instance, in a proxy server where the objects ipinreceives, ipindelivers and tcpinsegs are monitored, the object ipinreceives is the initial point and the object tcpinsegs is the final point of monitoring. These objects are selected according to their positions in relation to device data streams. In this proxy server example, ipinreceives is the first monitored object in input data stream and tcpinsegs is the last object monitored in the same stream. First level alerts generated in the same five-minute time frame are analyzed together. If the objects with alerts In the network level analysis, second level alerts are analyzed to show a network-wide view of the problem to the network administrator. All devices affected by an anomaly present changes in their traffic levels, and various alerts are generated in different points of the network, signaling the same problem. Therefore, third level of analysis is used to group alerts from different devices that belong to the same problem, aiming to improve the reports quality. This module of the system uses network topology information to group the second level alerts. If they are generated in the same five-minute time frame and belong to devices that are connected, a single third level alert is generated, gathering all related second level alerts. 4. PERFORMANCE EVALUATION AND RESULTS Tests were performed in two important devices related to the security of State University of Londrina (Brazil) network: the firewall (IP address: ) and the proxy server (IP address: ). In order to complete the scenario, a switch that interconnects the both devices was also monitored (IP address: ). Data used for tests was collected in April Proxy server is connected in the interface 3011 of the switch and the firewall is connected in the interface Figure 4 illustrates the test scenario.

4 generated only one report, containing all alarms and showing the context where the problem was occurring. Figure 4 - Test scenario. Table I presents the selected values for parameters of object level analysis module. Values for δ were defined after analyzing previous data. The polling interval was defined according to SNMP agents constraints, always using the smallest possible value for this parameter. It is possible to observe that δ values depend on polling intervals. When the polling interval is greater, fewer samples are analyzed during a hysteresis interval and a smaller delta is needed. In each device, the same values were applied for all objects. Table 1 - Parameter values used in tests. hysteresis δ value polling interval s 2 10 s s 1 10 s s 20 1 s Two metrics of performance were calculated: detection rate and false positive rate. The first one shows how many anomalies were detected in relation to the total of anomalies. The second metric calculates how many alarms were generated for situations that were not considered as anomalies in relation to the total amount of alarms. The results were good. The detection rate was 83.56% and the false positive rate was 12.62%. Figures 5, 6, 7, and 8 present a case of a real anomaly that occurred on April 27. They show plots containing real traffic, DSNS and first level alerts for each object in each device. It is possible to observe a great difference between real and expected traffic. All monitored objects had alerts generated in the same time frame, between 22h50 and 22h55. Second level alerts were generated in all devices. An anomaly was occurring and the security devices of the network should have been checked. All second level alerts were grouped in a third level alert. Instead of sending to the network administrator a lot of alerts reporting problems device by device, the system Figure 5 - Anomaly and respective alarms in the proxy server. 5. CONCLUSION AND FUTURE WORK This paper proposed an anomaly detection system that combines analysis from three different levels, in order to produce useful reports to network administrators. The solution is able to detect anomalies in device level, grouping these alerts to show to network administrators which network devices are affected and how the anomaly is propagating through the network. Results were obtained from tests in the network environment of State University of Londrina (Brazil). Collected false positive and detection rates were great. The behavior of the system during the occurrence of a real anomaly was presented. It was demonstrated that the report was useful, since it provided a wide view of the problem. It would not be possible if the analysis was performed device by device, without taking into account the connections between them. Future work includes the improvement of the system, aiming to classify and characterize the anomalies, showing to the network administrator the exact root-cause of the problem.

5 Figure 6 - Anomaly and respective alarms in the switch, interface Figure 7 - Anomaly and respective alarms in the switch, interface Figure 8 - Anomaly and respective alarms in the firewall. ACKNOWLEDGMENTS This work has been supported by The State of São Paulo Research Foundation (FAPESP), Brazil, and by Instituto de Telecomunicações, Next Generation Networks and Applications Group (NetGNA), Portugal. [3] D. E. Denning: An Intrusion-Detection model, IEEE Transactions on Software Engineering, v. 13, no. 2, pp , [4] P. Barford, J. Kline, D. Plonka and A. Ron, A signal analysis of network traffic anomalies, Proceedings of the 2nd ACM SIGCOMM Workshop on Internet measurement, 2002, pp [5] A. Lakhina, M. Crovella, C. Diot: Diagnosing Network- Wide Traffic Anomalies. ACM SIGCOMM Computer Communication Review, Proc. of the 2004 conference on Applications, technologies, architectures, and protocols for computer communications, V. 34, pp , [6] S. S. Kim and A. L. N. Reddy, Statistical techniques for detecting traffic anomalies through packet header data, IEEE/ACM Transactions on Networking, V. 16, n. 3, [7] G. Androulidakis, V. Chatziqiannakis and S. Papavassiliou: Network anomaly detection and classification via opportunistic sampling, IEEE Network, V. 23, n. 1, pp. 6-12, [8] H. Ringberg, A. Soule, J. Rexford and C. Diot: Sensitivity of PCA for traffic anomaly detection Proceedings of the 2007 ACM SIGMETRICS international conference on Measurement and modeling of computer systems, pp , [9] S. Farraposo, P. Owezarski, e E. Monteiro: A Multi- Scale Tomographic Algorithm for Detecting and Classifying Traffic Anomalies. Proceedings of IEEE International Conference on Communications 2007, pp , [10] W. Stallings: SNMP, SNMPv2, SNMPv3, and RMON 1, 2 and 3. Addison-Wesley, [11] K. McCloghrie, M. Rose: Management Information Base for Network Management of TCP/IP-based internet: MIB-II. RFC 1213, mar [12] M. L. Proença Jr., C. Coppelmans, M. Bottoli, L. S. Mendes: The Hurst Parameter for Digital Signature of Network Segment. 11th International Conference on Telecommunications (ICT 2004), pp , REFERENCES [1] A. Patcha e J. M. Park, "An overview of anomaly detection techniques: Existing solutions and latest technological trends," Computer Networks, v. 51, no. 12, pp , [2] M. Thottan, C. Ji: Anomaly Detection in IP Networks IEEE Transactions in Signal Processing, v. 51, n. 8, pp , 2003.