What intrusion detection approaches work well if only TCP/IP packet header information is available?

Transcription

1 What intrusion detection approaches work well if only TCP/IP packet header information is available? by ORYSPAYEV DOSSAY ORYSPAYULI Master Thesis Supervisor: Dr.ir. Aiko Pras (INF/DACS) Dr.ir. Remco van de Meent (INF/DACS) Design and Analysis of Communication Systems Faculty of Electrical Engineering, Mathematics and Computer Science University of Twente August 2006, Enschede (The Netherlands)

2 Abstract Proliferation of heterogeneous network systems and increasing usage of Internet makes network security issue to be more and more important. Various services offered on the Internet are having problems of being unavailable for authorized users because of denialof-service attacks. There are also different kinds of attacks like scans and various exploits. These days there are a lot of network intrusion detection systems (NIDSs) which try to detect these attacks. Some of these systems detect attacks according to full packet analysis, and some of them detect attacks by just analyzing TCP/IP packet header. This thesis investigates the NIDSs which work well if they are given only the TCP/IP packet header information. In the first part, we investigate the state-of-the-art NIDSs and then describe them according to common criteria. Which helped us to select two of them for further analysis. In the second part of investigation we analyze each of these NIDSs given 1999 DARPA data set, which contains full packet information. In the third part of investigation we analyze each of these NIDSs given only TCP/IP packet header information of 1999 DARPA data set. Additionally, we investigated each of these NIDSs given University of Twente s traffic repository data. We mainly focus on the number of detected atacks, false alarms generated, and different types of alarms generated by each of these NIDSs. Generated graphs are investigated in order to get an idea of the NIDSs given the set of data. Our results show that the performance of the SNORT NIDS (actually the rules/policies) was much better than the performance of the Bro NIDS (actually the rules/policies). 2

3 Preface This thesis work is the result of 6 months (February August, 2006) master assignment in the chair Design and Analysis of Communication Systems (DACS), Faculty of Electrical Engineering, Mathematics and Computer Science (EEMCS) in the University of Twente (Enschede/The Netherlands), under the supervision of Dr.ir. Aiko Pras. Chapter 1 presents the brief introductory information on intrusion detection systems, research questions, and approach to be followed. Chapter 2 presents information on the state-of-the-art network intrusion detection systems (NIDSs) read from books, papers, and Internet. Chapter 3 and Chapter 4 present main results of the thesis work. Chapter 3 analyzes selected NIDSs given the full packet information (i.e., 1999 DARPA data set). Chapter 4 analyzes selected NIDSs given only the TCP/IP packet header information (i.e., 1999 DARPA data set (with truncated information) and M2C Repository data). Chapter 5 completes the thesis work. It contains the conclusions and interesting suggestions for future work. 3

4 Acknowledgements I would like to thank my supervisor, Dr.ir. Aiko Pras for his many suggestions, constant support, and teaching me how to do a research in an independent manner. I thank my beloved wife Zhanar, for being patient and encouraging me all the way through the start and end of my study at University of Twente. I am thankful to my parents for their moral support. I also want to thank my friends Muzaffar Igamberdiev and Sain Saginbekov for just chatting with me when there was a need. Last but not least, I want to express my gratitude to Jan Schut for all the help I have received while my study at University of Twente. 4

5 Contents ABSTRACT... 2 PREFACE... 3 ACKNOWLEDGEMENTS INTRODUCTION Introduction to intrusion detection systems Motivation Research question Approach Structure of thesis Intended audience OVERVIEW OF NETWORK INTRUSION DETECTION SYSTEMS Bro EMERALD GrIDS MINDS NetSTAT NSM PHAD SNORT Conclusion FULL PACKET ANALYSIS Introduction to 1999 DARPA data set Results of NIDS given 1999 DARPA data set Results of SNORT Results of Bro Conclusion PACKET HEADER ANALYSIS Introduction to University of Twente s traffic repository Results of NIDSs given 1999 DARPA data set Results of SNORT Results of Bro Results of NIDSs given University of Twente data set Results of SNORT Results of Bro Conclusion CONCLUSIONS AND FUTURE WORK Conclusions Future work APPENDIX A: SNORT Usage APPENDIX B: Bro Usage REFERENCES

6 Chapter 1 Introduction This chapter describes intrusion detection systems (IDSs) and some of their characteristics. Additionally gives information on motivation, research questions, and approach that was used in order to perform the thesis. 1.1 Introduction to intrusion detection systems Proliferation of heterogeneous network systems and increasing usage of Internet makes network security issue to be more and more important. The packets traveling in the network are vulnerable to any kind of attacks, be it denial of service (DoS) attacks, disclosure of payload, or any other. Intrusion detection systems mainly in real-time sniff the packets passing through the network and by carefully examining these packets generate alerts to the system security officers whenever the packets are thought to be suspicious. A typical intrusion detection system [32] is shown below: Figure 1.1 Very simple intrusion detection system [32]. In the Figure 1.1 the detector performs the actions of the intrusion detection system. Detector gathers information from the system on which it was deployed. It uses three kinds of information [32]: long-term information to be used (e.g., is the techniques to detect the intrusions) is saved in the database, configuration information about the present 6

7 state of the system, and audit information that describes the actions/events happening on the system. Efficiency of the intrusion detection systems consists of the following [32]: 1) Accuracy: Accuracy deals with the proper detection of the intrusions and absence of false alarms. Inaccuracy occurs when the intrusion detection system reports non intrusive actions as intrusive. 2) Performance: The performance of the intrusion detection system depends on the rate at which it processes the information. If this rate is too low then the real time sniffing is likely to be not possible. 3) Completeness: The capability of the intrusion detection system to detect all the attacks is called completeness of the system. 4) Fault tolerance: Intrusion detection systems should be resistant to any kind of attacks from the intruders. 5) Timeliness: Intrusion detection systems should react analyze and report the systems security officers as quick as possible, in order to let them give time to react before the attack is completely performed. Characterization of intrusion detection systems according to their detection methods [32, 33, 18] and according to their data source [32, 18] is shown in Figure 1.2. Detection method Misuse based Anomaly based Intrusion detection system Application log files Data source Host log files Network packets Multi network/ infrastructure Figure 1.2 Characteristics of intrusion detection systems. 7

8 Intrusion detection systems according to their detection methods can be misuse based (also called knowledge based) or anomaly based (also called behavior based). The [32] uses the terms knowledge based and behavior based instead of misuse based and anomaly based, from the point that they feel that these terms more accurately describe the techniques being used by these two detection methods. But, we prefer the latter ones because of their widespread usage in intrusion detection research area. Both terms are explained hereafter. Misuse based: In misuse based intrusion detection systems the knowledge about the attacks are collected. This may be the previous successful attacks performed to other systems, may be the knowledge of experts about the vulnerabilities of the system, or any other source of information which may predict the possible attack. All these information are written with a system specific language and then stored as a set of rules/policies inside the intrusion detection system. And when the intrusion detection system starts to work it just checks the stream of information for any signs of attacks, which are obtained from the set of rules/policies. Below we see the Figure 1.3 [33], which depicts the typical misuse detection system. Figure 1.3 A typical misuse detection system [33]. Advantage of misuse based intrusion detection systems is that, in theory, they have low false alarm rate [32]. And the analysis process of alarms (because of the written set of rules/policies) makes it easier for the security system officers to understand and react quickly. Disadvantage of this detection method is that keeping the knowledge base of such intrusion detection system up to date is not easy. Even after gathering information about the attacks it is time consuming to analyze them and update the knowledge base of the intrusion detection system. The other drawback is that misuse based intrusion detection systems face the generalization issue [32], because most of the attacks are dependent on the operating system, version, platform, and application. Anomaly based: Anomaly based intrusion detection systems observe the behavior of the system or user for a particular time. And then they assume it as normal or expected behavior of the system or user. After building this normal or expected behavior of the system or user intrusion detection systems assume all the deviations from the normal or expected behavior of the system or user as an intrusion. Below we see the Figure 1.4 [33] which depicts the typical anomaly detection system. 8

9 Figure 1.4 A typical anomaly deteciton system [33]. Advantage of the anomaly based intrusion detection systems is that they can detect new (unknown to the system) attacks. They are less dependent on the operating system specific information. One of the major disadvantages of the anomaly based intrusion detection systems is the generation of high number of false alarms. Over the time the normal behavior of the system or user may change, which will require the system to be not available for some period of time. In which time attacks may be performed. Also at the learning phase of normal behavior of system or user the intrusion detection system may be introduced some attacks. Which it will assume as normal behavior through all its operation till the next update of the normal behavior. Intrusion detection systems according to the data source are divided into four: application log files, host log files, network packets, and multi network/ infrastructure. The most common (i.e., mentioned in most of the intrusion detection research papers as the data source) of these nowadays are host log files (also called host based) and network packets (also called network based). We prefer to use both terms of these two data sources of intrusion detection systems interchangeably. All four data sources of intrusion detection systems are expalined hereafter. Application log files: An application based intrusion detection system examines the behavior of the application, generally in the form of log file [18]. Host log file: A host based intrusion detection system examines log files of the host such as process accounting information, user behavior, output of the application based intrusion detection systems log files operating on the host [18]. Network packets: Network based intrusion detection systems monitor and analyze the network traffic. It may have access to the outputs generated by the application and host based intrusion detection systems operating within the monitored network environment [18]. Multi network/infrastructure: A multi-network intrusion detection system generally takes the form of an incident response team (IRT), where the input of the system comes from sites within their constituency [18]. The data source for this 9

10 specific category may be outputs of all the listed categories of data source and also may be of its kind. 1.2 Motivation There are lots of intrusion detection systems [13-17] developed and in many of these references authors also show how each of them differ from others by classifying them into detection techniques used, origin of data, etc. Most of these intrusion detection systems check both the payload and header information of the packets. For example, SNORT network intrusion detection system. It does not mean that all the intrusion detection systems have to check both header and payload of each packet in order to find the attacks. One example of finding suspicious packets by just looking at header information is TTL of 0, which might be unusual. And example for finding suspicious packets by looking at payload information may be the viruses. It is also possible to find the suspicious packets by looking at both, i.e. header and payload, e.g. if the packet with the same data originating from one host destined to hundreds or millions of other hosts. In [20] it is said that in the future there will be no need for the payload checkers, because of the data encryption. However in [1, 19] authors state that the header information alone will leak information on many attacks, since attacks are mainly made on the payload. But on the other hand it is reasonable to do a research on finding those intrusion detection systems that perform well than others when additionally to other characteristics they will also be performing well given only the TCP/IP packet header information. Since attackers may want to: - prevent that others access/use of systems and services (denial-of-service (DoS) attacks), - learn which systems exist and could potentially be compromised (scans), - compromise those systems which they found (various exploits). Most of these attacks can be detected only by looking at TCP/IP packet header data [31]. Additionally these are other reasons to do just TCP/IP packet header analysis: - full packet analysis is CPU intensive and since networks are getting faster this would slow down the analyzes of packets with proportion to the passing number of packets, - in the future packet payload is expected to be encrypted, most of the denial-ofservice (DoS) attacks are at the TCP/IP header level. 1.3 Research questions After the brief motivation following questions arise: 1) What are the main network intrusion detection systems (NIDSs) described in literature? Do they operate on only TCP/IP packet header, only payload, or both? 10

11 2) Which NIDS are openly available and can be evaluated free of charge? 3) How do these openly available NIDSs perform if they operate on the entire packet data? 4) How do these openly available NIDSs perform if they operate only on TCP/IP packet header data? Note: In answering questions 3 and 4, the performance will be expressed in number and type of reports and false alarms. 1.4 Approach To answer the Question 1, existing literature was investigated. We used Internet, in particular Google scholar. The University of Twente s library (especially e-journals) was used. To answer the Question 2, references of the literature searched for answering Question 1 were used. To answer the Question 3, the 1999 DARPA data set [25] was used. We used trace files of four weeks (1, 2, 4, and 5). The weeks 1 and 2 contain training data, and weeks 4 and 5 are testing data DARPA data set [25] contains a number of well-known intrusions. For each openly available NIDS package we investigated how many of these known intrusions were detected, and how many false alarms were generated. To answer the Question 4, again the 1999 DARPA data set [25] was used. This time we used the tcpdpriv [30] to truncate the 1999 DARPA data set file to contain only TCP/IP packet header data. Additionally, to compare the NIDS we used University of Twente-EWI repository which stem from M2C Measurement Data Repository Project [26] and which contain only TCP/IP packet header data. Note that for this dataset intrusions have not been analyzed before and are therefore unknown. Whenever needed to count false alarms and different types of alarms we wrote some small programs in Java. Also we used Ethereal [10] to analyze the trace files. 1.5 Structure of thesis Introductory information on thesis work and intrusion detection systems is given in Chapter 1. We present an overview of the existing NIDSs, i.e. the state-of-the-art NIDSs, and present the openly available NIDS for evaluation in Chapter 2 which answers Questions 1 and 2. 11

12 As an answer for Question 3 we investigate the proposed NIDSs of Chapter 2 by analyzing 1999 DARPA data set [25]. As an answer for Question 4 we investigate the proposed NIDSs of Chapter 2 by analyzing again 1999 DARPA data set [25] (but this time with packets of this dataset do not contain payload information). Additionally we test the NIDSs given University of Twente-EWI traffic repository data. Which differs from 1999 DARPA data set [25] in that there is no information available on the possible intrusions of these traces and it contains only TCP/IP packet header data only. Lastly we show the structure of the thesis in Figure 1.5. Chapter 3 Chapter 1 Chapter 2 Chapter 5 Chapter 4 Figure 1.5 Structure of thesis. 1.6 Intended audience The intended audience of this thesis is anyone using NIDS. This thesis will not explain everything about NIDS. The thesis will provide information on the state-of-the-art in intrusion detection systems. It will also give brief information on usage of SNORT [21] and Bro[7] NIDSs. 12

13 Chapter 2 Overview of network intrusion detection systems In this chapter we present state-of-the-art NIDS. The presentation of the NIDS follows in alphabetical (ascending) order. For each of the NIDS presented there will be given the following information: where was it developed?, when was it developed?, has development stoppped?, is software openly available?( if yes: what OS?, have I been able to install it?), does it inspect complete packet?/does it inspect only TCP/IP packet headers?, is it anomaly or misuse based?, and lastly description. 2.1 Bro Where was it developed?: Lawrence Berkeley National Laboratory, Berkeley, CA, USA. When was it developed?: Vern Paxson started implementing it in 1995 [28]. Paper (describing the Bro system) appeared and first version appeared in 1998 [7, 28]. Has development stopped?: No. It is actively maintained by Vern Paxson and other group and worldwide members/users. Is software openly available?: Yes. -What OS?: Bro works on FreeBSD, Linux, and Solaris. But the performance is best under FreeBSD. -Have I been able to install it?: Yes, on FreeBSD 6.1. Does it inspect complete packet?: Yes. Is it anomaly or misuse based?: Bro utilizes both anomaly and misuse detection [28]. Description: Bro [7] is UNIX based NIDS. In [7] Bro is described as follows:, a stand-alone system for detecting network intruders in real-time by passively monitoring a network link over which the intruder s traffic transits. Structure of the Bro system is shown in the Figure 2.1, below. 13

14 Figure 2.1 Structure of the Bro system. As can be seen from the Figure 2.1 Bro consists mainly of three parts; a libpcap [8], an event engine, and a policy script interpreter. Libpcap [8] is a packet capture library used by tcpdump [9]. In this system the tcpdump snapshot is set so that the entire packet is captured. Packets captured by tcpdump are passed to the event engine, which reduces them to the high-level network events. At this level the packet header information is checked. If there are any problems with the header information the system generates an event and discards the packet. If there is nothing to worry about then the event engine looks for the connection state associated with the two IP addresses and two port numbers. It creates a new one if there is no match. Then it sends the packet to a handler for the corresponding connection. Bro maintains tcpdump trace file of the traffic it sees on the network. Upon return the connection handler indicates whether the engine has to record the entire packet, only the header, or nothing at all. After processing the packet event engine looks for the events generated if there are any. If there are, then it processes each event in FIFO manner until the queue is empty. Lastly, policy script interpreter generates scripts written in the Bro language. For every event passed to the interpreter, it generates a script (a semi-compiled code for the corresponding handler). These codes in turn can execute different Bro scripting commands like recording data to a disk, generating new events, etc. 14

15 2.2 EMERALD Where was it developed?: Computer Science Laboratory, SRI International, CA, USA. When was it developed?: Has development stopped?: n/a. Is software openly available?: No. Does it inspect complete packet?: Yes. Is it anomaly or misuse based?: Uses both misuse and anomaly based detection [12]. Description: The EMERALD (Event Monitoring Enabling Responses to Anomalous Live Disturbances) environment is a distributed scalable tool suite for tracking malicious activity through and across large networks [12]. The objective of EMERALD is to bring anomaly detection tools into the practical world. The architecture of the EMERALD introduces hierarchically layered approach to network surveillance, which is composed of service analysis, domain-wide analysis, and enterprise-wide analysis. Service, domainwide, and enterprise-wide analysis covers misuse of individual components and network services within the single domain, misuse across multiple services and components, and misused across multiple domains, respectively. The service analysis is used to simplify and decentralize the surveillance of the network activities for any misuse or intrusion in operation. For that purpose EMERALD introduces the concept of service monitors, which are dynamically deployable, highly distributed, and independently tunable. Dynamically deployed service monitors provide information on the infrastructure and services of the underlying domain. Information gathered by one service monitor can be disseminated to other service monitors through EMERALD s subscription-based communication scheme. In this manner it also provides domain monitors, and enterprise layer monitors. The general EMERALD monitor architecture is illustrated in the Figure 2.2 below. 15

16 Figure 2.2 The Generic EMERALD Monitor Architecture. As seen from Figure 2.2, EMERALD consists of the following parts: resource objects, scalable profile-based anomaly detection, scalable signature analysis, a universal resolver, and message system. We will not describe scalable profile-based anomaly detection and scalable signature analysis since they are self explanatory. However, EMERALD deploys specific detection and analysis approaches, the curious reader is referred to [12]. By shortly describing all the other parts composing the EMERALD system we end the description of EMERALD system. A pluggable resource object is the library of the target specified data which allows the system to remain independent of the analysis target to which it is deployed. A universal resolver is an expert system which receives intrusion and suspicion reports from the profiler and signature engines. Then it invokes the appropriate response handlers within the resource engine. Resolver supports much functionality such as subscription interface which helps other IDSs to participate in the EMERALD layered scheme, extensive intermonitor sharing of analysis results, and many more. Details of resolver can be found in [12]. The duplex messaging system allows EMERALD monitors to exchange information. 2.3 GrIDS Where was it developed?: University of California at Davis, USA. When was it developed?: Has development stopped?: n/a.. 16

17 Is software openly available?: No. Does it inspect complete packet?: Yes. Is it anomaly or misuse based?: Anomaly based. Description: GrIDS [11] was started by the idea of the need for the intrusion detection system which is decentralized and capable of aggregating information on a large scale. GrIDS is a graph-based intrusion detection system, which means that it represents the hosts and network activities in a network as graphs. These nodes (hosts) and edges (activities) may come from other IDSs or network sniffers that are equipped with filter to be able to send their outputs to GrIDS. Since single graph is hard and not efficient to analyze GrIDS uses graph spaces where each graph space contains the graphs of the same type. Type of graphs in the graph space is constructed by the rule set. So each graph space has its own rule set. A rule set modifies graphs of only its own graph space and has no effect on other graph sets. Rule specified at each node only valid for the descendants of that node, which reduces the ambiguity between similar rules in the different parts of the network. 2.4 MINDS Where was it developed?: Minnesota University, USA. When was it developed?: n/a. Has development stopped?: n/a. Is software openly available?: No. Does it inspect complete packet?: No. Only packet header information. Is it anomaly or misuse based?: Anomaly based. Description: A MINDS [3, 4] (Minnesota Intrusion Detection System) is an unsupervised anomaly based intrusion detection system. It uses data mining to detect the network intrusions. Figure 2.3 shows the MINDS System. 17

18 Figure 2.3 MINDS System. Here we shortly describe the functionality of the modules that are used by the MINDS system to detect network intrusions. Input of the MINDS system is the data collected by the Netflow flow-tools [5]. The reason for this was that the flow-tools only capture the packet header information. Then the first thing is extraction of features from the input data which is performed in the feature extraction module. Basic features include IP address of the source and destination, port numbers, etc. Then the output from feature extraction module is entering the known attack detection module, where it is divided into known attacks which are directly sent to the analyst, and unknown ones are entering the anomaly detection module (known attacks are removed from further analysis). Anomaly detection module assigns an anomaly score to each of the network connections using the outlier detection algorithm. The last steps are anomaly scores and association pattern analysis modules. After the anomaly detection s assignment of the scores for each network connection the anomaly scores are shown to the analyst who decides whether the highly anomalous attacks are real ones or not. Lastly association pattern analysis summarizes the highly anomalous network connections according to the information gathered from the anomaly detection module. This summary is used by the analyst to see if these intrusions may be put in the known attack list. 2.5 NetSTAT Where was it developed?: University of California at Santa Barbara, USA. When was it developed?: n/a. Has development stopped?: n/a.. Is software openly available?: Yes. -What OS?: RedHat 7.3/i

19 -Have I been able to install it?: Yes, but could not run. Does it inspect complete packet?: No. Only packet header information. Is it anomaly or misuse based?: Anomaly based. Description: NetSTAT (A network-based intrusion detection system) was developed at University of California at Santa Barbara. NetSTAT system uses State Transition Analysis Technique (STAT, ) to detect the network intrusion. Meaning, that it models attacks detected on the networked environment as state transition diagrams. Figure 2.4 shows the architecture of the NetSTAT. Figure 2.4 NetSTAT architecture. As it is shown in the Figure 2.4 NetSTAT is a distributed application formed by the following four components: the network fact base, the state transition scenario database, a collection of general purpose probes, and the analyzer. The network fact base contains the information about the underlying network topology and the network deployed. The system security officer interacts with network fact base by constructing, inserting, and browsing data about the network. State transition representations of the intrusions to be detected are managed in the state transition scenario database. The system security officer can browse and edit the state transition 19

20 representations in the state transition scenario database. Active intrusion detection components of the NetSTAT are probes, which monitor the network traffic at specific places in the network. Their startup configuration is received from the analyzer. Last, but not the least, is analyzer which is used by the system security officer as a means of analyzing and controlling the detection of the selected attacks. 2.6 NSM Where was it developed?: University of California at Davis, USA. When was it developed?: Has development stopped?: Yes, on Is software openly available?: No. Does it inspect complete packet?: Only IP and TCP protocols. Is it anomaly or misuse based?: Both anomaly and misuse based. Description: The NSM [2] (Network Security Monitor) was developed by the University of California at Davis researchers. The start and end date of the system is 1989 and 1995, respectively. NSM is used with a four dimensional matrix. The matrix axes are: source (a host generating the traffic), destination (a host expecting the traffic sent by source), service (mail, login, etc), and connection ID (a unique identifier for the connection). The mask of normal traffic is generated. Detection process is looking for anything outside the mask of the normal traffic. NSM was built on Sun-3/50 workstation. NSM consists of the following components (linked in a pipeline fashion): (1) A packet catcher, (2) a parser, (3) a matrix generator, (4) a matrix analyzer, (5) and a matrix archiver. Below we give a short description on each of the components of the NSM: a) A packet catcher is getting all the traffic of the network and passes it to the parser. It is the only component which is platform dependent. b) Parser, having detailed information on the protocol of each layer gets the detailed information on each of the protocols and passes this information to the matrix generator. c) A matrix generator takes the information gathered from parser, and finds a cell in the traffic matrix to which it belongs and increments the counter in that cell. d) The matrix analyzer examines the matrix representing the current matrix. It also alarms the system security officer of any unusual behavior. 20

21 e) Finally, the matrix archiver writes the current matrix to a disk. 2.7 PHAD Where was it developed?: Department of Computer Sciences, Florida Institute of Technology, Melbourne, FL/USA. When was it developed?: Has development stopped?: n/a. Is software openly available?: Yes. -What OS?: Both on UNIX based machines and Windows OS family. -Have I been able to install it?: Yes, but usage needs some changes in the code of the program (which was written in C++ language). Does it inspect complete packet?: Only packet header fields at the data link (Ethernet), network (IP), and transport/control layers (TCP, UDP, and ICMP). Is it anomaly or misuse based?: Anomaly based. Description: PHAD [1] (Packet Header Anomaly Detection for Identifying Hostile Network Traffic) is developed by Matt Mahoney. In [1] the author presents his system and presents the outputs of his system given the 1999 DARPA data set [25]. [1] was published on PHAD is a anomaly-based intrusion detection system that learns the normal ranges of values for each packet header fields at the data link (Ethernet), network (IP), and transport/control layers (TCP, UDP, ICMP). Normally, anomaly-based intrusion detection systems detect the anomalous actions if the performed action deviates from the accepted normal action. Other way of defining the anomalous actions is to rank the anomalous actions, by how unusual they are. The ranking concept sets as a result that the rare the action occurs much it has chances of being anomalous. It will also be possible for user to specify a threshold for the detection of rare events. 2.8 SNORT Where was it developed?: Stanford Telecommunications, Inc. When was it developed?: Has development stopped?: No. Is software openly available?: Yes. -What OS?: Runs approximately on 23 different platforms including Linux, Solaris, BSD, IRIX, HP-UX, Msc OS X, and Win32. 21

22 -Have I been able to install it?: Yes, both on Linux based machine and Windows XP. Does it inspect complete packet?: Yes, but can be configured to inspect only packet header data only. Is it anomaly or misuse based?: Misuse based. Description: SNORT [21, 27] is a freely available network intrusion detection system. It is a cross-platform and rule-based analysis tool. Snort has a set of rules which it uses to decide whether the data is suspicious or not. SNORT [27] has an advantage in that it is easy to use, easy to get installed, easy to configure, easy to capture network packets, easy to analyze the alerts generated. Also it can be installed on many platforms, and you can use it free of charge. It consists of many rules, and uses Boyer-Moore algorithm to perform quick pattern matching. Additionally Snort has real-time alerting capability, with alerts being sent to syslog, Server Message Block (SMB) ``WinPopup'' messages, or a separate ``alert'' file [27]. 2.9 Conclusion After the brief description of each NIDSs presented, we show the Table 2.1 which answers the first and second research questions formulated in subchapter 1.3. NIDS When Detection method Input Openly Status name Misuse Anomaly Packet header Full packet available Bro 1998 Yes Yes Yes Yes Yes Active EMERALD 1996 Yes Yes Yes Yes No Stopped GrIDS 1996 No Yes Yes Yes No Stopped MINDS n/a No Yes Yes No No n/a NetSTAT n/a No Yes Yes No Yes Stopped NSM Yes Yes Yes Yes No Stopped, 1995 PHAD 2001 No Yes Yes No Yes n/a SNORT 1999 Yes No Yes Yes Yes Active Table 2.1 Characteristics of NIDS. We have selected SNORT and Bro for further analysis. First of all, because they are openly available and secondly because they are actively used. Both NIDSs have lots of users worldwide who write signatures/policies and provide beginners/experts information on various aspects of the NIDS used. Additionally, there are several books written on the usage of SNORT NIDS. 22

23 Chapter 3 Full packet analysis This chapter presents results of the SNORT and Bro NIDS given 1999 DARPA data set. One important thing to be mentioned before continuing is that we took SNORT with its default signatures and Bro with policies provided. So, the answers in this chapter and next one mainly depend on the signatures/policies provided by the NIDSs. We use week 1, 2, 4, and 5 s inside and outside (tcpdump formatted) data only for full packet analysis. Subchapter 3.1 starts with short introduction to the 1999 DARPA Intrusion Detection Evaluation data set. 3.1 Introduction to 1999 DARPA data set There were lots of intrusion detection systems developed by researchers from all over the world. These systems somehow had to be evaluated for their false and true alarm rate (detection capability). In 1998 Information Systems Technology group of MIT Lincoln Laboratory, under the sponsorship of Defense Advanced Research Projects Agency (DARPA ITO) and Air Force Research Laboratory (AFRL/SNHS), started this evaluation process [25]. The 1999 DARPA Intrusion Detection Evaluation data set contains three training week data and two testing week data. First and third week contain no attacks, this was given as training set for anomaly based systems. Second week several attacks. And lastly, last two weeks contain 201 instances of about 56 different types of attacks. 3.2 Results of NIDSs given 1999 DARPA data set In the following subchapters and we present the results of the SNORT and Bro NIDSs given 1999 DARPA data set. We use data of weeks 1, 2, 4, and Results of SNORT Weeks 1 and 2 contain training data. Week 1 contains no attacks, but week 2 contains several attacks. We show the result of the SNORT given the week 1 data in Table 3.1. In total it generated alarms, which are all false alarms. Since this week is attack free, i.e. contains no attack. On the other hand, week 2 contains several attacks. SNORT results for this week data are shown in Table 3.1. None of the alarms is an indication of a listed attacks for week 2. So they are all false alarms. The Table 3.1 also shows the number of alarms generated by the SNORT NIDS given the week 4 and 5 data. 23

24 Week Monday Tuesday Wednesday Thursday Friday Total Table 3.1 Number of alarms generated by SNORT for 1999 DARPA data set. For week 4 SNORT generated alarms in total,and in total for week 5. Only 3 of the week 4 s alarms are real attacks detected by SNORT. We show these detected attacks below in Table /31-16:45: [**] [1:402:7] ICMP Destination Unreachable Port Unreachable [**] [Classification: Misc activity] [Priority: 3] {ICMP} > /31-19:21: [**] [1:384:5] ICMP PING [**] [Classification: Misc activity] [Priority: 3] {ICMP} > /31-16:45: [**] [1:402:7] ICMP Destination Unreachable Port Unreachable [**] [Classification: Misc activity] [Priority: 3] {ICMP} > Table 3.2 Attacks of week 4 detected by SNORT. And only one of the week 5 s alarms generated by SNORT is a real attack, which is shown in Table /06-14:36: [**] [1:553:7] POLICY FTP anonymous login attempt [**] [Classification: Misc activity] [Priority: 3] {TCP} :3267 -> :21 Table 3.3 An attack of week 5 detected by SNORT. In Table 3.4 we show the different types of alarms generated for each of the weeks of 1999 DARPA data set. Week Number of alarm types Total 132 Table 3.4 Number of different alarm types generated by SNORT for 1999 DARPA data set. 24

25 3.2.2 Results of Bro In Table 3.5 we show the results of the Bro NIDS given week 1, 2, 4, and 5 data. Week Monday Tuesday Wednesday Thursday Friday Total Table 3.5 Number of alarms generated by Bro for 1999 DARPA data set. Since week 1 had no attack. All the 179 alarms generated by Bro are false alarms. Bro also could not detect the attacks of week 2, so all the 217 alarms generated by Bro are false alarms, too. And similarly, all the alarms generated by Bro for weeks 4 and 5 are false alarms, totally. In Table 3.6 we list the different types of alarms generated for each week by Bro NIDS. Number of alarm types Week 1 3 Week 2 10 Week 4 5 Week 5 13 Total 13 Table 3.6 Number of different alarm types generated by Bro for DARPA data set. At the end of this subchapter we show the graph representing the number of false alarms by each of the NIDSs given the four week of the DARPA 1999 dataset. 3.3 Conclusion In this chapter we answered the research question 3: How do these openly available NIDS perform if they operate on the entire packet data? (the performance is expressed in number and type of reports and false alarms). This measure of the NIDS particularly depends on the signatures/policies they provide by default. 25

26 Snort Bro week1 week2 week4 week5 Figure 3.1 Difference between SNORT and Bro false alarm generation given the 1999 DARPA data set. As we can see from the Figure 3.1, SNORT generates too many false alarms in comparison to Bro. Also SNORT generated 132 different types of alarms, whereas Bro generated only 13 different types. But on the other hand, SNORT detected 4 attacks in total. From these analysis we can conclude that signatures/policies of NIDSs play major role in the number of false alarm generation and capability to fetch an attack. Both systems performed not good enough. 26

27 Chapter 4 Packet header analysis We again use the 1999 DARPA data set which contain weeks 1, 2, 4, and 5 input and output tcpdump files. But this case we use only the packet header data of the 1999 DARPA data set. For which we use the tcpdpriv [30] with the following options: tcpdpriv -A50 -P99 -r <full tcpdump file> -w <header tcpdump file>. Traces are anonymized: transport layer payload data has been removed and source and destination IP addresses have been remapped randomly, however per trace file consistent, i.e. for every trace file it is guaranteed two identical IP addresses are mapped to the same random IP address [31]. Actually, there was no need for the anonymization of the 1999 DARPA data set traces, since they are openly available for the intrusion detection evaluation. Instead we could use tcpdump [9] -s <snaplen> option or editcap(one of command line tools of Ethereal [10]) with -s <snaplen> option. But we preferred tcpdpriv [30] because the same method was used for the University of Twente traffic repository traces. Subchapters and present the results of the SNORT and Bro given 1999 DARPA data set (with only packet header data). First we present results of SNORT in subchapter and then results of Bro in subchapter Subchapter and present the results of the SNORT and Bro given the University of Twente traffic repository data. Which is different from 1999 DARPA data set by having only header information of the packets, and having no information on the attacks. We use all the data of Location 1, and first five data of each of Locations 2, 3, and 4. In subchapter we present the results of the SNORT, and in subchapter we present the results of the Bro system. 4.1 Introduction to University of Twente s traffic repository The traces of University of Twente were collected and used as part of the M2C Measurement Data Repository Project [29]. These traces are collected from four locations [26, 29]: On location #1 the 300 Mbit/s (a trunk of 3 x 100 Mbit/s) ethernet link has been measured, which connects a residential network of a university to the core network of this university. On the residential network, about 2000 students are connected, each having a 100 Mbit/s ethernet access link. The residential network itself consists of 100 and 300 Mbit/s links to the various switches, depending on the aggregation level. The measured link has an average load of about 60%. Measurements have taken place in July

28 On location #2, the 1 Gbit/s ethernet link connecting a research institute to the Dutch academic and research network has been measured. There are about 200 researchers and support staff working at this institute. They all have a 100 Mbit/s access link, and the core network of the institute consists of 1 Gbit/s links. The measured link is only mildly loaded, usualy around 1%. The measurements are from May - August Location 3: Location #3 is a large college. Their 1 Gbit/s link (i.e., the link that has been measured) to the Dutch academic and research network carries traf c for over 1000 students and staff concurrently, during busy hours. The access link speed on this network is, in general, 100 Mbit/s. The average load on the 1 Gbit/s link usually is around 10-15%. These measurements have been done from September - December Location 4: On location #4, the 1 Gbit/s aggregated uplink of an ADSL access network has been monitored. A couple of hundred ADSL customers, mostly student dorms, are connected to this access network. Access link speeds vary from 256 kbit/s (down and up) to 8 Mbit/s (down) and 1 Mbit/s (up). The average load on the aggregated uplink is around 150 Mbit/s. These measurements are from February - July Results of NIDSs given 1999 DARPA data set In subchapters and we present the results of the SNORT and Bro systems given only header information of the 1999 DARPA data set s week 1, 2, 4, and 5 data Results of SNORT Below in Table 4.1 are the results of the SNORT given week 1, 2, 4, and 5. Week Monday Tuesday Wednesday Thursday Friday Total Table 4.1 Number of alarms generated by SNORT for packet header only of 1999 DARPA data set. All the generated alarms are false alarms. SNORT could not detect even the previous four attacks of weeks 4 and 5. Below Table 4.2 shows the different types of alarms generated by SNORT for these four weeks. Week Number of alarm types

29 5 16 Total 16 Table 4.2 Number of different alarm types generated by SNORT for packet header only 1999 DARPA data set Results of Bro Below Table 4.3 shows the results of the Bro given only the packet header information of the four weeks of 1999 DARPA data set. Bro also could not detect any of the attacks listed for weeks 2, 4, and 5. Week Monday Tuesday Wednesday Thursday Friday Total Table 4.3 Number of alarms generated by Bro for packet header only of 1999 DARPA data set. And in Table 4.4 we show the different number of alarms generated by Bro for these four weeks of 1999 DARPA data set. Week Number of alarm types Total 6 Table 4.4 Number of different alarm types generated by Bro for packet header only of 1999 DARPA data set. We see from Figure 4.1 that Bro generated to many false alarms on the same data but this time only with header information. Even though both of the systems payload checkers were enabled, SNORT did not generate too many false alarms in respect to Bro. 29

30 week1 week2 week4 week5 Snort Bro Figure 4.1 Difference between SNORT and Bro false alarm generation given the packet header only of 1999 DARPA data set. 4.3 Results of NIDSs given University of Twente data set In the following subchapters and we present the results of the SNORT and Bro NIDSs given University of Twente s traffic repository data Results of SNORT Table 4.5 shows the number of alarms generated by SNORT for each of the locations and total number of generated alarms for the University of Twente traffic repository data (UT data set) that we selected. UT data set Location1 Location2 Location3 Location4 Total Number of alarms Table 4.5 Number of alarms generated by SNORT for UT data set. In Table 4.6 we can see the number of different types of alarms generated for each of the locations of UT data set. Location Number of different types Total 26 Table 4.6 Number of different alarm types generated by SNORT for UT data set. 30

31 Below in Figure 4.2 we show all the 26 types of alarms generated by the SNORT system for the UT data set. Figure 4.2 All the types of alarms generated by SNORT for UT data set Results of Bro Table 4.7 shows the total number of alarms generated by the Bro system given the UT data set. UT dataset Location1 Location2 Location3 Location4 Total Number of alarms Table 4.7 Number of alarms generated by Bro for UT data set. In Table 4.8 we show the different types of alarms generated by the Bro system. Location Number of different types Total 9 Table 4.8 Number of different alarm types generated by Bro for UT data set. 31

32 All these different 9 types of alarms generated by Bro for UT data set are listed below in Figure 4.3. Figure 4.3 All the types of alarms generated by Bro for UT data set. Figure 4.4 shows the alarms generated by both systems for the UT data set. And Figure 4.5 shows the different types of alarms generated by both systems for the UT data set loc1 loc2 loc3 loc4 Snort Bro Figure 4.4 Difference between SNORT and Bro alarm generation given the UT data set. 32

33 SNORT Bro 5 0 loc1 loc2 loc3 loc4 Figure 4.5 Difference between number of different types of alarms generated by SNORT and Bro given the UT data set. 4.4 Conclusion Figure 4.1 shows that Bro generates too many false alarms with respect to SNORT. If we compare Figures 3.1 and 4.1 we see that Bro generated too many false alarms. This difference for both NIDSs SNORT and Bro is depicted in Figures 4.6 and 4.7, respectively. SNORT NIDS before after week1 week2 week4 week5 Figure 4.6 Difference between number of false alarms generated by SNORT before removing and after removing the payload information of 1999 DARPA data set. 33

34 Bro NIDS week1 week2 week4 week5 before after Figure 4.7 Difference between number of false alarms generated by Bro before removing and after removing the payload information of 1999 DARPA data set. From these two figures (i.e., 4.6 and 4.7) we can say that SNORT s payload checking rules seem to be disabling (act like they undestand that the packets were truncated) themselves automatically whenever it is presented only header information of the packet. On the other hand Bro acts as if it had to see full packet information. SNORT NIDS before after week1 week2 week4 week5 Figure 4.8 Difference between number of different types of false alarms generated by SNORT before removing and after removing the payload information of 1999 DARPA data set. 34