APPLYING C4.5 CLASSIFICATION TO THE MONITORING OF NETWORK DEVICE CONNECTIVITY

Transcription

1 APPLYING C. CLASSIFICATI TO THE MITORING OF NETWORK DEVICE CNECTIVITY Rangsipan Marukatat *, Watcharapong Suebsiri, Supapong Putthayangkura, Sorarath Ngantawee, and Siriprapa Srigrajang Department of Computer Engineering, Faculty of Engineering, Mahidol University, Thailand. egrmr@mahidol.ac.th ABSTRACT We developed a Network Device Connectivity Monitor (NeDCoM) which retrieves data such as latency, bandwidth, and connectivity from network devices. Using such data, it constructs a C. classification model to predict possible failures of these devices. Thereafter, NeDCom monitors the devices and raises an alarm should a condition indicated by C. s prediction arise. Index Terms Network monitoring, simple network management protocol, SNMP, classification, C.. INTRODUCTI As network becomes one of the most critical infrastructures in many organizations, it is vital to insure its availability and reliability. Besides malicious attacks, user behaviors can contribute to network vulnerability. An evidence is the flood of internet traffic that led to Google s crash moment after the death of Michael Jackson [, ]. In organization scale, heavy traffic tends to have recurring pattern. Fowler and Leland [] observed R&D department s network and found congestion during.00.00,.00.00, and of each day. Kotz and Essien [] observed campus wireless network and found traffic bursts during of each day and on Monday of each week, coinciding with class activities. Power outage and cable cut are also causes of disruption that sometimes possess patterns. For instance, they might occur frequently in stormy season. A number of research have sought to detect early signs of breakdown or performance degradation in the network. Data mining fits perfectly with such detective role. Bivens et al [] used neural network to predict souces of network congestion. Their input was packet measures (average count, variance, and third moment) of each network device, and output was a bit vector encoding suspecious devices. The flow rate of these devices can then be restricted, manually or automatically, to avoid the foreseen problem. In Yamanishi and Maruyama [6], dynamic syslog mining was employed to detect symptoms of failure and identify new emerging failure patterns. Correlation between devices in the event of any failure was also discovered. Other works on predicting network fault or congestion are, for example, [8,9] (they are reviewed in Section.). Our Network Device Connectivity Monitor (NeDCoM) employs data mining techniques to analyze traffic behavior in local area networks and detect possible breakdown of a certain device. It consists of three main modules:. Data Retriever. This module probes network devices by using Simple Network Management Protocol (SNMP) and ping commands. It stores the data in a log file.. Data Analyzer. This module builds a C. classification model by using the logged data. The model describes conditions in which a device connectivity will be or OFF.. Device Monitor. This module periodically feeds the latest data from each device to the classification model. It raises an alarm once an OFF status is predicted. The rest of this paper is organized as follows. Section briefly reviews SNMP and C. classification which are the central ideas of our research. Section presents NeDCoM framework, followed by system testing results in Section. Section finally concludes the paper.. RELATED BACKGROUND.. Simple network management protocol (SNMP) SNMP is set of standard, simple operations that remotely monitor and manage network devices. Manageable devices include any IP device running SNMP agent. The agent may be a separate program or a builtin component of the device. The protocols are based on requests and responses between SNMP manager and managed devices. The former queries certain information from the latter, or requests the latter to perform certain operations. Responses are conveyed in MIB (Management Information Base) variables [] which can be categorized into ten groups, as shown in Table. There are MIB variables in total. Many of them convey redundant information. Thottan and Ji [8] chose five 9

2 MIB variables, from IF (ifinoctets and ifoutoctets) and IP (ipinreceives, ipindelivers, and ipoutrequests) groups, to predict network fault by using statistical techniques. On the other hand, Kulkarni et al [9] predicted network congestion by feeding three variables from IP group (ipinreceives, ipforwdatagrams, and ipoutdiscards) and router s queue statistics to CART and C.0 classification models. Based on their findings, IF and IP variables are the most useful ones for detecting network fault and congestion. These variables, however, convey overlapped information as the traffic flows between the interface and the IP layers. For example, ipinreceives accumulates incoming traffic from all network interfaces, with the measure for each interface being ifinoctets. In the same layer, ifinucastpkts, ifinnucastpkts, ifoutucastpkts, and ifoutnucastpkts measure the amount of incoming unicast, incoming nonunicast, outgoing unicast, and outgoing nonunicast traffic, respectively. The first two variables are added to ifinoctets while the other two variables are added to ifoutoctets. Following the above rationale, we initially focus on IF variables, ifinoctets and ifoutoctets. These variables count the number of octets received and transmitted through each network interface. The total number of interfaces can be checked from ifnumber variable. These information allow us to compute the total volume of incoming and outgoing octets for each device. The SNMP manager was implemented by using Java SNMP API, version. []... C. Classification C. builds a classification model from training data. The model describes the condition of independent attributes that leads to each class prediction. It is typically in the form of a decision tree with the following properties: leaf nodes are the predicted classes; internal nodes are the independent attributes; and arcs leaving an internal node are decisions according to the attribute values of that node. The C. algorithm is roughly as follows: the whole set of data is first regarded as a single tree node. Within each iteration, an independent attribute that best split the data in the current node is selected. Then, child nodes are generated each of them contains the data in each split. The algorithm stops once all the data are successfully classified or certain criteria are met. Otherwise, another independent attribute is selected to split a wavering node. An attribute is selected on the basis of its yielding information gain ratio. That is, entropy before and after the split are measured. The most reduction of entropy indicates the most stability incurred by the classification, hence the most information gain. This measure is normalized to weight out the effect of multiway split (i.e. high information gain, but too many trivial nodes). Details of C. can be found in [, ]. We implemented C. classification by using Javabased Weka API, version.. []. Table. Groups of MIB variables Group System IF AT IP ICMP TCP UDP EGP Transmission SNMP Description Device s name, location, description Network interface (adapter) and traffic Address translation Statistics about IP packets Statistics about ICMP messages TCP algorithms, parameters, statistics Statistics about UDP traffic Statistics about exterior gateway protocol Mediaspecific MIBs Statistics about SNMP traffic Fig. NeDCoM s main window. NEDCOM FRAMEWORK The Network Device Connectivity Monitor (NeDCoM) was written in Java, requiring Java Runtime Environment (JRE) version.6 or higher. Fig. shows its main window. In this research, it ran on a personal computer (NeDCoM host) connected to wired and wireless local area networks (LAN and WLAN, respectively). The maximum speed of LAN is 0 Mbps and that of WLAN is Mbps. Managed devices or hosts in the LANWLAN are Router, FTP Server, File Server, and two other personal computers. Each of them ran Windows XP or Vista with SNMP service, and received or transmitted data via a single network interface... Data retriever module C. model Latest status Log file NeDCoM probes the hosts by using ping commands and SNMP. It stores the following data in a log file: IP address Utilization (%) Average latency (ms) 0

3 Timerelated data (six attributes: day of week, date, month, year, hour, and minute) Connectivity Latency and connectivity are obtained from ping results. Ping communicates with a host via Internet Control Message Protocol (ICMP). It sends ICMP Echo Request to a host and waits for ICMP Echo Reply. The connectivity is set to OFF if the percentage of lost packets exceeds a certain threshold; otherwise, it is set to. Lost packets, indicated by timedout echo replies, occur by a couple of reasons. For example, the host may be broken, not configured properly, disconnected from LANWLAN, or sit behind a firewall. These cases typically incur 0% loss rate. Furthermore, the network may become congested that some replies cannot be delivered in time (the term OFF is slightly misleading, but it is an indication that the host is potentially vulnerable). class J8 { public static void main(string[] args) throws Exception { Object[] i = new Object[9]; int status = 0; i[0] = args[0]; i[] = Double.valueOf(args[]).doubleValue(); i[] = Double.valueOf(args[]).doubleValue(); i[] = args[]; i[] = Double.valueOf(args[]).doubleValue(); i[] = args[]; i[6] = Double.valueOf(args[6]).doubleValue(); i[] = Double.valueOf(args[]).doubleValue(); i[8] = Double.valueOf(args[8]).doubleValue(); System.out.println(J8.classify(i)); public static double classify(object[] i) throws Exception { double p = Double.NaN; p = J8.N660(i); return p; static double N660(Object[] i) { double p = Double.Nan; if (i[] == null) { p = 0; else if (((Double)i[]).doubleValue() <=.0) { p = 0; else if (((Double)i[]).doubleValue() >.0) { p = J8.Neff8(i); return p;... Fig. Classification model embedded in J8 class Fig. Classification model rendered in a decision tree Bandwidth utilization is calculated from SNMP octet measures [] as follows: ( ifinoctets ifoutoctets) 8 Utilizatio n = 0. () Speed Time ifinoctets and ifoutoctets are the number of incoming and outgoing octets within an observed time interval. Timerelated data are taken from NeDCoM host s clock. This log file is used as training data for classification models, with the first nine attributes being independent attributes and the last one being the target attribute or class... Data analyzer module NeDCoM lets user construct a C. (also known as J8 in Weka) classification model from the log file. Parameters required for the model construction includes Binary or multiway split Minimum size of leaf node Pruning strategy The model, embedded in a Java class (see Fig. ), reads the values of independent attributes and predicts whether a host would be or OFF. It can be rendered in a decision tree, as shown in Fig.... Device monitor module NeDCoM periodically feeds the latest data from each host to the classification model. These latest data are also displayed in its main window (see Fig. ). Summarized in Table, the responses to prediction and actual events can be categorized into five cases as follows:. If the host s previous, predicted, and current status are s, NeDCoM moves the current status to the previous one and continues checking the latest status against the prediction.. When the host is OFF unexpectedly, NeDCoM adds this case to the log file, for it to be used in training a new classification model. Furthermore, misprediction count is increased. Once this count reaches a certain threshold, users will be warned that a new, improved classification model should be constructed.. If the previous status is but the predicted status is OFF, NeDCoM raises an alarm. Because our goal is to prevent such problem, wrong prediction (i.e. the host is still ) in this case is desirable and expected.. If, despite the alarm, the foreseen problem still occurs (i.e. the host is eventually OFF), we regard this case as a critical one. Therefore, it will be used to train the next classification model, making the model better aware of this problem in the future.. If the host s previous status is OFF, NeDCoM simply waits until the host is reconnected.

4 Table. Prediction and responses Correct Logged Previous Predicted Current?? OFF OFF OFF OFF OFF Recall that the OFF status does not necessarily mean total host failure. In many cases, it is an indication of heavy congestion at that particular host, which results in too many ping s timedout replies. Once the host is reconfigured, either automatically or manually, and the amount of traffic is dropped, the connectivity status will return to. For flexibility, NeDCoM allows offsets to be added to the current utilization and time. For example, suppose that the utilization offset is %. An alarm will be raised if the host s current utilization is within % of that predicted to cause disconnectivity, given that the other attribute values also match the predicted ones. Likewise, if the time offset is minutes, an alarm will be raised minutes earlier than the predicted time.. SYSTEM TESTS Two synthetic data sets were used to train first classification models. Each data set contained 800 records. The utilization ranged from 0 to 0% and the latency ranged from 0 to 60 ms. The connectivity was either or OFF according to the following conditions: Data set A If (utilization ), then OFF If (utilization AND latency ), then Otherwise, Data set B If (utilization AND latency 0), then OFF Otherwise, Functional tests on NeDCoM s alarms (cases and in Table ) and responses to unexpected misprediction (case ) were conducted. The following were set up for the tests:. L and L: The personal computers within the LAN continuously downloaded files of various sizes from the File Server. Initial classification models were built from data set A in L and data set B in L.. W and W: Likewise, the personal computers within the WLAN continuously downloaded files from the FTP Server. Initial classification models were built from data set A in W and data set B in W.. L: After a period of smooth traffic in the LAN, cable connected to a host was unplugged.. W: After a period of smooth traffic in the WLAN, a host s wireless signal receiver was switched off. Table. Alarming tests in LAN, observed on File Server L L Util Latency Alarm Util Latency Alarm Table. Alarming tests in WLAN, observed on Router W W Util Latency Alarm Util Latency Alarm 6 Table. of FTP Server and NeDCoM s warning about misprediction (and new model construction) Previous L Predicted Misprediction Warning Previous W Predicted Misprediction Warning L, L, W, and W were simulated ten times each. Average utilization, average latency, and alarming results on the monitored hosts are displayed in Tables and. In summary, everytime the current latency and utilization of the monitored host matched any OFF condition, an alarm was duly raised by NeDCoM. L and W were simulated five times each. of the monitored hosts and warning about misprediction are

5 shown in Table. Because the misprediction threshold was set to, the warning was raised every time the host was unexpectedly disconnected. Unfortunately, due to the rarity of the events, they had no impact on the new classification model despite being included in the training data set. These tests only affirm that NeDCoM is able to monitor network devices and response to classes of events as it was designed to. But the testing environments are only LAN and WLAN connecting a few devices, and the simulated traffic may not reflect the real one. To further evaluate NeDCoM s performance, strengths, and weaknesses, true pictures of the traffic in various conditions are first needed. It is indeed one of our ongoing research directions.. CCLUSI This paper has presented a Network Device Connectivity Monitor (NeDCoM). It retrieves data from network devices via SNMP and ping, and builds a C. classification model from these data. The model is used to monitor each device thereafter. Once a sign of device disconnectivity is detected, NeDCoM raises an alarm. It also keeps track of the model s accuracy and when the misprediction count reaches a certain threshold, users will be warned that a new model is needed. As mentioned earlier, one of our research directions is to evaluate NeDCoM in conditions where traffic behavior is close to the real one. We have yet to fully analyze the model construction policy, e.g. when and upon which data set a new model should be constructed. Other issues such as false alarms and the effect of NeDCoM s probing on the overall network performance should also be investigated. In another direction, NeDCoM itself can be enhanced. For example, more data can be extracted from other MIB variables, and alternative classification techniques can also be incorporated. Finally, although ping is a simple tool for checking the status of network devices, there are a few limitations in real practice. A common problem is that ping s ICMP packets are blocked as a precaution against pingsweeping or pingflooding attacks. An alternative strategy is to check whether a device is connected to some particular services by using connectionbased protocols such as TCP (Transmission Control Protocol). This will be included in our future work. 6. REFERENCES [] M. Shiels, Web slows after Jackson s death, BBC News, 6 June 09. Retrieved from 80.stm. [] H. J. Fowler and W. E. Leland, Local area network characteristics, with implication for broadband network congestion management, IEEE Journal on Selected Areas in Communications, 9(), pp. 99, 9. [] D. Kotz and K. Essien, Analysis of a campuswide wireless network, Wireless Networks, (), pp., 0. [] J. A. Bivens, B. K. Szymanski, and M. J. Embrechts, Network congestion arbitration and source problem predition using neural networks, Smart Engineering System Design,, pp., 0. [6] K. Yamanishi and Y. Maruyama, Dynamic syslog mining for network failure monitoring, Proc. of the th ACM SIGKDD Intl. Conf. on Knowledge Discovery and Data Mining, Chicago, IL, pp. 9908, 0. [] K. McCloghrie and M. Rose, Management information base for network management of TCPIPbased internets: MIBII, RFC, 9. [8] M. Thottan and C. Ji, Statistical detection of enterprise network problems, Journal of Network and Systems Management, (), 99. [9] P. G. Kulkarni, S. I. McClean, G. P. Parr, and M. M. Black, Deploying MIB data mining for proactive network management, Proc. of the rd Intl. IEEE Conf. on Intelligent Systems, pp. 06, 06. [] Java SNMP package (opensource), [] J. R. Quinlan, C.: programs for machine learning, Morgan Kaufmann, 9. [] I. H. Witten and E. Frank, Data mining: practical machine learning tools and techniques ( nd ed), Elsevier Inc., 0. [] Weka: data mining software in Java (opensource), [] D. Mauro and K. Schmidt, Essential SNMP ( nd ed), O Reilly Media Inc., 0. [] O. Garnham, Michael Jackson fan traffic crashes Google, Twitter and other sites, IT World Canada Newsletter, June 09. Retrieved from ac68ba96e0.html.