Anomaly based Network Intrusion Detection System

Anomaly based Network Intrusion Detection System Dinakara K

Anomaly based Network Intrusion Detection System Thesis Submitted in Partial fulfillment of the requirements for the Degree Of Master of Technology In Computer Science and Engineering By Dinakara K (06CS6026) Under the supervision of Prof. Jayanta Mukhopadhyay Prof. S.K. Ghosh Computer Science and Engineering Indian Institute of Technology Kharagpur -721302, India (May 2008)

Computer Science and Engineering INDIAN INSTITUTE OF TECHNOLOGY KHARAGPUR Certificate This is to certify that the thesis entitled Anomaly based Network Intrusion Detection System which is being submitted to the Indian Institute of Technology, Kharagpur, for the award of the degree of Master of Technology in Computer Science and Engineering by Dinakara K., Roll No. 06CS6026 has been carried out by him under our guidance. This thesis, in our opinion, is worthy of consideration for the award of degree of Master of Technology in accordance with the regulations of this institute. (Dr. Jayanta Mukhopadhyay) Professor, Dept. Computer Science and Engineering Indian Institute of Technology Kharagpur 721 302, India (Dr. S. K. Ghosh) Asst. Professor, School of Information Technology Indian Institute of Technology Kharagpur 721 302, India

ACKNOWLEDGEMENTS Many people deserve to be acknowledged for their contribution to this work and even more need to be mentioned for their enthusiasm and support in the last one year. This page is for them all. I want to start by thanking my project guides Dr. Jayanta Mukhopadhyay and Dr. S. K. Ghosh. Thanks for their invaluable guidance, incessant inspiration, prolific encouragement and for just being there whenever I needed you the most. Their untiring help and constructive suggestions during the course of the project have helped me in learning a lot and without which it would have been difficult to complete the thesis work. I express my sincere thanks to Dr. D. K Nanda, Chief Systems Manager, Computer and Informatics Centre, IIT kharagpur for providing the facility for sniffing the IIT network. I am deeply indebted to Dr. G Athithan, Head, Intelligence Systems Division, Centre for Artificial Intelligence and Robotics, Bangalore for his precious guidance and support given for my thesis work. Sincere thanks to my friends, Biswajit Paul, Girish Gokuldasan and Dinesh Singh Kutiyal for their support and constructive suggestions throughout this project as well as the whole course. I would love to dedicate this thesis to my parents whose cooperation, support, affection and well wishes enabled me to complete this endeavour successfully. Above all I humbly acknowledge the grace and blessings of thy supreme power that capacitates me to fulfill this well nurtured dream. Dinakara K (06CS6026)

CONTENTS ACRONYMS AND ABBREVIATIONS... 3 LIST OF FIGURES... 4 LIST OF TABLES... 6 1. CHAPTER 1...7 1.1. INTRODUCTION... 7 1.2. BRIEF HISTORY OF IDS... 7 1.3. TYPES OF IDS... 8 1.4. DETECTION TECHNIQUES... 9 1.5. DEPLOYMENT SCENARIOS OF IDS... 11 1.6. SNIFFING THE NETWORK TRAFFIC WITH IDS... 13 1.7. IDS RESPONSES AGAINST ATTACK... 15 1.8. SNORT, A OPEN SOURCE SIGNATURE BASED IDS... 16 1.9. RELATED WORK... 19 1.10. MOTIVATION AND OBJECTIVE... 20 1.11. OBJECTIVE... 21 1.12. ORGANIZATION OF THESIS... 22 2. CHAPTER 2...23 2.1. SYSTEM ARCHITECTURE... 23 2.2. SENSOR/DECODER... 23 2.3. PREPROCESSOR... 24 2.4. ANOMALY DETECTION PRE-PROCESSOR... 25 2.5. DETECTION ENGINE... 26 2.6. ALERT MODULE... 27 2.7. BASIC ANALYSIS AND SECURITY ENGINE (BASE)... 28 2.8. OPERATING ENVIRONMENT... 30 3. CHAPTER 3...31 3.1. RESEARCH APPROACH... 31 3.2. STATISTICAL MOMENTS OR MEAN AND STANDARD DEVIATION MODEL... 36 3.3. HOTELLING S T 2 HYPOTHESIS, A MULTIVARIATE STATISTICAL TECHNIQUE... 37 3.4. BAYESIAN CLASSIFICATION, A PROBABILISTIC TECHNIQUE... 38

4. CHAPTER 4...40 4.1. EXPERIMENTAL RESULTS AND DISCUSSION... 40 4.2. EVALUATION SCHEME... 40 4.3. COMPARATIVE RESULTS... 43 4.4. DISCUSSION... 44 5. CHAPTER 5...46 5.1. CONCLUSION... 46 6. APPENDIX A...50 6.1. CHARTS OF DIFFERENT NETWORK PARAMETERS OBTAINED WHILE EXPERIMENTATION... 50 6.2. SCREENSHOTS OF BASE CONSOLE... 58 6.3. TYPICAL VALUESOF NETWORK PARAMETERS FOR NORMAL TRAFFIC IN THE TARGET NETWORK... 60 6.4. TYPICAL VALUES OF NETWORK PARAMETERS FOR ANOMALOUS TRAFFIC IN THE TARGET NETWORK.. 61 7. APPENDIX B...62 7.1. GLOSSARY OF TECHNICAL TERMS... 62 8. APPENDIX C...65 8.1. ATTACK DESCRIPTION... 65 9. APPENDIX C...68 9.1. THE TCP/IP PROTOCOL STACK... 68 9.2. IP HEADER... 69 9.3. TCP HEADER... 70 9.4. UDP HEADER... 71 9.5. ICMP HEADER... 71 9.6. TCP CONNECTION ESTABLISHMENT... 72 9.7. TCP CONNECTION TERMINATION... 73 Department of Computer Science and Engineering, Indian Institute of Technology, Kharagpur 2

ACRONYMS AND ABBREVIATIONS ACL : Access Control List ARP : Address Resolution Protocol BASE : Basic Analysis and Security Engine DDOS : Distributed Denial of Service DMZ : Demilitarized Zone DNS : Domain Name Server DOS : Denial of Service HTTP : Hyper Text Transfer Protocol ICMP : Internet Control Message Protocol IP : Internet Protocol NIC : Network Interface Card NIDS : Network Intrusion Detection System PCRE : Perl Compatible Regular expression RPC : Remote Procedure Call SPAN : Switched Port Analyzer TAP : Test Access Point TCP : Transmission Control Protocol TTL : Time to Live UDP : User Datagram Protocol Department of Computer Science and Engineering, Indian Institute of Technology, Kharagpur 3

LIST OF FIGURES FIGURE 1: NETWORK IDS PLACED BEFORE THE GATEWAY FIREWALL... 11 FIGURE 2: NETWORK IDS IN THE DMZ... 12 FIGURE 3: NETWORK IDS WITHIN THE PRIVATE NETWORK... 12 FIGURE 4: NETWORK IDS SNIFFING THE NETWORK IN A HUB ENVIRONMENT... 13 FIGURE 5: NETWORK IDS SNIFFING THE NETWORK USING TAP DEVICE... 14 FIGURE 6: DEPLOYMENT SCENARIO OF NIDS WITH SENSORS IN STRATEGIC POINTS... 15 FIGURE 7: SNIFFED PACKET (SNORT V)... 17 FIGURE 8: SNIFFED PACKET ( SNORT DEV)... 17 FIGURE 9: ALERTS GENERATED IN INTRUSION DETECTION MODE... 18 FIGURE 10: OVERALL SYSTEM ARCHITECTURE... 23 FIGURE 11: NETWORK IDS SENSOR... 23 FIGURE 12: NETWORK IDS PRE-PROCESSOR... 24 FIGURE 13: ANOMALY DETECTION PRE-PROCESSOR... 25 FIGURE 14: SCREENSHOT OF BASE CONSOLE SHOWING THE GENERATED ALERTS... 27 FIGURE 15: BASE CONSOLE SHOWING THE ALERT STATISTICS... 28 FIGURE 16: BASE CONSOLE SHOWING THE DETAILS OF SNIFFED PACKET... 29 FIGURE 17: TIME SLOTS USED IN GENERATING THE NETWORK PROFILE... 33 FIGURE 18: ALGORITHM FOR GENERATING THE PROFILE... 33 FIGURE 19: ALGORITHM FOR DETECTION... 34 FIGURE 20: FLOW CHART DEPICTING THE OVERALL WORKING OF ANOMALY DETECTION TECHNIQUE 35 FIGURE 21: NORMAL DISTRIBUTION CURVE WITH DIFFERENT CONFIDENCE INTERVALS... 36 FIGURE 22: MULTIVARIATE GAUSSIAN DISTRIBUTION CURVE... 39 FIGURE 23: TRAFFIC PATTERN IN THE COURSE OF A DAY (MONDAY)... 50 FIGURE 24: TCP PACKET COUNT IN THE COURSE OF A DAY (MONDAY)... 50 FIGURE 25: TCP STATISTICS IN THE COURSE OF A DAY ( MONDAY )... 51 FIGURE 26: UDP PACKET COUNT IN THE COURSE OF A DAY (MONDAY)... 51 Department of Computer Science and Engineering, Indian Institute of Technology, Kharagpur 4

FIGURE 27: UDP STATISTICS IN THE COURSE OF A DAY ( MONDAY )... 52 FIGURE 28: ICMP PACKET COUNT IN THE COURSE OF A DAY ( MONDAY )... 52 FIGURE 29: ICMP PACKET COUNT IN THE COURSE OF A DAY ( MONDAY )... 53 FIGURE 30: NUMBER OF CONNECTIONS IN THE COURSE OF A DAY (MONDAY)... 53 FIGURE 31: CONNECTION STATISTICS IN THE COURSE OF A DAY (MONDAY)... 54 FIGURE 32: TRAFFIC STATISTICS IN THE COURSE OF A DAY (SATURDAY )... 54 FIGURE 33: TRAFFIC STATISTICS IN THE COURSE OF A DAY ( SUNDAY )... 55 FIGURE 34: TRAFFIC STATISTICS IN THE COURSE OF A WEEK... 55 FIGURE 36: INTRUSIVE TRAFFIC STATISTICS IN THE COURSE OF A DAY (MONDAY)... 56 FIGURE 37: INTRUSIVE TRAFFIC STATISTICS IN THE COURSE OF A WEEK... 57 FIGURE 38: AVERAGE TRAFFIC STATISTICS IN THE COURSE OF A DAY (MONDAY )... 57 FIGURE 39: BASE CONSOLE DISPLAYING THE TRAFFIC STATISTICS BY PROTOCOL... 58 FIGURE 40: BASE CONSOLE DISPLAYING THE ALERTS STATISTICS... 58 FIGURE 41: BASE CONSOLE DISPLAYING UNIQUE ALERTS... 59 Department of Computer Science and Engineering, Indian Institute of Technology, Kharagpur 5

LIST OF TABLES TABLE 1: TYPICAL VALUES OBTAINED FOR THE NORMAL AND INTRUSIVE NETWORK TRAFFIC WITH HOTELLING S AND BAYESIAN DISCRIMINATOR FUNCTIONS 42 TABLE 2: CHART SHOWING THE COMPARATIVE RESULTS OF THE EXPERIMENTS 43 TABLE 3. EXPERIMENTAL RESULTS ON MIT_LL DARPA 1999 DATA SET 44 Department of Computer Science and Engineering, Indian Institute of Technology, Kharagpur 6

1. CHAPTER 1 1.1. INTRODUCTION Internet is forcing organizations into an era of open and trusted communications. This openness at the same time brings its share of vulnerabilities and problems such as financial losses, damage to reputation, maintaining availability of services, protecting the personal and customer data and many more, pushing both enterprises and service providers to take steps to guard their valuable data from intruders, hackers and insiders. Intrusion Detection System has become the fundamental need for the successful content networking. IDS provide two primary benefits: Visibility and Control [1]. It is the combination of these two benefits that makes it possible to create and enforce an enterprise security policy to make the private computer network secure. Visibility is the ability to see and understand the nature of the traffic on the network while Control is the ability to affect network traffic including access to the network or parts thereof. Visibility is paramount to decision making and makes it possible to create a security policy based on quantifiable, real world data. Control is key to enforcement and makes it possible to enforce compliance with security policy. 1.2. BRIEF HISTORY OF IDS The idea of detecting the intrusions or system misuses by looking at some kind malicious patterns in the network or user activity was initially conceived by James Anderson in his report titled Computer Security Threat Monitoring and Surveillance [2] to US Air Force in the year 1980. In the year 1984, the first prototype of Intrusion Detection System which monitors the user activities, named Intrusion Detection Expert System (IDES) was developed. In the year 1988, Haystack became the first IDS to use patterns and statistical analysis for detecting malicious activities, but it lacked the capabilities of real time analysis. Department of Computer Science and Engineering, Indian Institute of Technology, Kharagpur 7

Meanwhile, there were other significant advances occurring at University of California Davis' Lawrence Livermore Laboratories. In the year 1989, they built a IDS called Network System Monitor (NSM) for analyzing the network traffic. This project was subsequently developed into IDS named Distributed Intrusion Detection System (DIDS). Stalker based on DIDS became the first commercially available IDS and influenced the growth and trends of future IDS. In the Mid 90 s, SAIC developed Computer Misuse Detection System (CMDS), a host based IDS. US Air Force s Cryptographic support centre developed Automated Security Incident Measurement (ASIM), which addressed the issues like scalability and portability. The intrusion detection market began to gain in popularity and truly generate revenues around 1997. In that year, the security market leader, ISS, developed a network intrusion detection system called Real Secure. A year later, Cisco recognized the importance of network intrusion detection and purchased the Wheel Group, attaining a security solution they could provide to their customers. Similarly, the first visible host-based intrusion detection company, Centrax Corporation, emerged as a result of a merger of the development staff from Haystack Labs and the departure of the CMDS team from SAIC. From there, the commercial IDS world expanded its market-base and a roller coaster ride of start-up companies, mergers, and acquisitions ensued. Martin Roesch, in the year 1998 launched a light weight open source Network IDS named SNORT [3], which has since then gained much popularity. In year 1999 Okena Systems worked out the first Intrusion Prevention System (IPS) under the name Storm Watch. IPS are the systems which not only detect the intrusions but also are able to react on alarming situation. These systems can co-operate with firewall without any intermediary applications. 1.3. TYPES OF IDS Depending upon the level of analysis IDS is classified into two major types: Department of Computer Science and Engineering, Indian Institute of Technology, Kharagpur 8

Network based IDS (NIDS): Monitors and analyzes the individual packets passing around a network for detecting attacks or malicious activities happening in a network that are designed to be overlooked by a firewall s simplistic filtering rules. Host based IDS (HIDS): Examines the activity on individual computer or host on which the IDS is installed. The activities include login attempts, process schedules, system files integrity checking system call tracing etc. Sometimes two kinds of IDS are combined together to form a Hybrid IDS. Generally IDS has two components Central Administration (Management) Module: Provides centralized facility for managing and monitoring of all the installations of Intrusion Detection System and hence centralized way of analyzing and detecting the intrusions. It has the complete view of the various activities and events occurring in different segments of the organizational network. Moreover the policy settings, actions to be triggered, patches/signature updation, fine tuning of sensors can be achieved with this module. IDS Sensors (Agents): Analyses the network traffic and identifies attacks and security breaches, which take place by exploiting the technology of network implementation, reports the alerts to the Management module and performs the preset actions. IDS Agents are more autonomous in their functions as compared to the Sensors. 1.4. DETECTION TECHNIQUES Various techniques are in place for intrusion detection which can be broadly classified as follows. Department of Computer Science and Engineering, Indian Institute of Technology, Kharagpur 9

Signature/pattern based Detection: In this technique, the sensors which are placed in different LAN segments filter and analyse network packets in real time and compares them against a database of known attack signatures. Attack signatures are known methods that intruders have employed in the past to penetrate a network. If the packet contents match an attack signature, the IDS can take appropriate countermeasure steps as enabled by the network security administrator. These countermeasures can take the form of a wide range of responses. They can include notifications through simple network management protocol (SNMP) traps or issuance of alerts to an administrator s email or phone, shutting down the connection or shutting down the system under threat etc. An advantage of misuse detection IDS is that it is not only useful to detect intrusions, but it will also detect intrusion attempts; a partial signature may indicate an intrusion attempt. Furthermore, the misuse detection IDS could detect port scans and other events that possibly precede an intrusion. Unauthorised Access Detection: In unauthorised access detection, the IDS detects attempts of any access violations. It maintains an access control list (ACL) where access control policies for different users based on IP addresses are stored. User requests are verified against the ACL to check any violations Behavioural Anomaly (Heuristic based) Detection: In behavioural anomaly detection method, the IDS is trained to learn the normal behavioural pattern of traffic flow in the network over an appropriate period of time. Then it sets a baseline or normal state of the network s traffic, protocols used and typical packet sizes and other relevant parameters of network traffic. The anomaly detector monitors different network segments to compare their state to the normal baselines and look for significant deviations. Department of Computer Science and Engineering, Indian Institute of Technology, Kharagpur 10

Protocol Anomaly Detection: With this technique, anomaly detector alerts administrator of traffic that does not conform to known protocol standards. As the protocol anomaly detection analyzes network traffic for deviation from standards rather than searching for known exploits there is a potential for protocol anomaly to serve as an early detector for undocumented exploits. 1.5. DEPLOYMENT SCENARIOS OF IDS There exist three strategic locations where NIDS can be installed in the network for effective monitoring of the network, as depicted in the diagrams below. Before the Gateway firewall: In this point, the NIDS can keep track of all network events of interests, even those attacks which subsequently may fail. As it has to handle large traffic, NIDS ought to be installed on a faster machine so that analysis is done in real time. Also it has to be configured correctly so that number of false alarms can be reduced. Figure 1 shows such a configuration. Internet Router Firewall Firewall Private Network DMZ Network IDS Public Servers Figure 1: Network IDS placed before the Gateway Firewall Department of Computer Science and Engineering, Indian Institute of Technology, Kharagpur 11

In the DMZ (De-Militarized Zone): Placing IDS within the DMZ enables it to monitor the traffic which is already partly filtered off through the gateway firewall as depicted in figure 2. This reduces the burden on the IDS but also limits its visibility Internet Router Firewall Firewall Private Network Network IDS DMZ Public Server Figure 2: Network IDS in the DMZ Inside the private corporative network: The last possibility where NIDS can be stationed is within the corporate network as shown in figure 3. Such a location aims at monitoring the attacks emerging from the local networks and also those which are transmitted via firewall. As the number of attacks possible in this place is lesser than the preceding cases, this makes the application demands smaller. In this case IDS generates few false alarms. The scope of visibility is limited to within the corporate network, thus will not be able to detect the failed attacks as in the previous cases. Internet Router Firewall Firewall Private Network Public Servers DMZ Network IDS Figure 3: Network IDS within the private network Department of Computer Science and Engineering, Indian Institute of Technology, Kharagpur 12

It is always advisable to install NIDS on systems other than firewall so that attacker using the fact that firewall together with the IDS on a single computer can pump in malicious traffic to generate too many false alerts, and at the same time consuming system resources affecting the operations of firewall. 1.6. SNIFFING THE NETWORK TRAFFIC WITH IDS In order to monitor the network, the traffic in that segment of the network has to be made available to the Network IDS. There exists several ways to eavesdrop the network packets without obstructing their normal flow across the network as mentioned below. Sniffing the network packets in a Hub environment Hub Hub Network IDS Figure 4: Network IDS sniffing the network in a Hub environment A network Hub is a physical layer device, hence whenever data frames arrive, it simply broadcasts them to all other ports. Only the destination system processes the data while other machines discard. In such an environment, IDS can be connected to one of the Hub ports with its NIC in promiscuous or general mode which enables it to get all the network packets moving around the network. Such a configuration is depicted in figure 4. Eavesdropping via port mirroring or SPAN (Switched Port ANalyser) port in a switched environment: In a switched network, the packets from a source machine are forwarded only to the respective destination machine as specified by the IP address unlike in Department of Computer Science and Engineering, Indian Institute of Technology, Kharagpur 13

the case of a network connected via Hub where packets are broadcasted to every other machine in the network. In such an environment, sniffing is made possible by a technique called Port Mirroring or Switched Port Analyzer where the mirrored port gets a copy of packet from all other ports. Machine with IDS is connected to the mirrored port or SPAN port in promiscuous mode so that it can process all the packets irrespective of their destination. Because of the aggregation of traffic on a single SPAN port, there are chances of packet drop. Sniffing the traffic using Network TAP (Test Access port): Network IDS Internet Router TAP Switch Private Network Figure 5: Network IDS sniffing the network using TAP device Network TAPs [4] are the hardware devices having three interfaces, entry, exit and test port. IDS is connected to the test port where it can see the entire network traffic as shown in figure 5. TAPs does not introduce any delay or affect the data movement in the network and operates transparently as it doesn t possess IP and hardware address. Stealth mode operation The Network IDS has to operate transparently to avoid the intruders from targeting the IDS itself. So generally the IDS is configured to work in a special mode called Stealth mode. In this arrangement, the IDS sniffing interface is put in promiscuous mode without assigning the IP address, thus only listening to the packets flowing across the network keeping its presence transparent from network users. Usually the IDS has two Network interfaces, one to monitor the network and the second one for administrative purposes, like configuring IDS, updating Department of Computer Science and Engineering, Indian Institute of Technology, Kharagpur 14

signatures, communication with IDS sensors/manager,dispatching alerts etc. Attacker can easily detect the configuration and location of IDS by analyzing these messages in the network. It is possible therefore to guard the IDS by encoding its messages or to create a separate network for management as shown in the diagram. The advantage of having a separate network between IDS Manager and IDS Sensors is not only to provide security but also to ensure out of band communication, meaning no bandwidth of the existing network is utilized for its communication. Router Firewall Firewall Internet Switch Subnet DMZ IDS Sensor IDS Sensor IDS Sensor Public Server Subnet IDS Sensor IDS Admin Console Figure 6: Deployment scenario of NIDS with sensors in strategic points It is generally recommended to use IDS sensors inside and outside the firewall or between each firewall in a multi-layered environment and host based IDS on all critical or key hosts. IDS Management Module and its sensors communicate via zero bandwidth LAN segment in a transparent or stealth operation mode. This kind arrangement enables the IDS to have complete view of the organizational network and can even detect the failed attempts of attacks while reducing the chances of being compromised. Figure 6 depicts a complete deployment scenario of Network IDS. 1.7. IDS RESPONSES AGAINST ATTACK Whenever IDS detects any intrusions or attacks, it reacts as per the preconfigured settings. The responses can range from mere alert notifications to blocking of the attacks based on the severity. The appropriate reactions on the Department of Computer Science and Engineering, Indian Institute of Technology, Kharagpur 15

threats are a key issue for safety and efficacy. Generally the responses can be of three types [2] Active response: IDS by itself cannot block attacks, however can take such actions which can lead to stopping of attacks. Such actions can be for example, sending TCP reset packets to the machine(s) which is being the target of attack, reconfiguring router/firewall as to block the malicious connection. In extreme cases, IDS can even block all the network traffic to avoid potential damage to the firm. Passive response: Passive solutions deliver information to IDS administrator on the current situation and leave the decision to take appropriate steps to his discretion. Many commercial systems rely on this kind of reactions. Examples for this kind of actions can be simple alarm messages and notifications. Notifications can be sent on email, cellular phone or via SNMP messages. Mixed response: Mixed responses combine both active as well as the passive responses appropriately as per the needs of situation. 1.8. SNORT, A OPEN SOURCE SIGNATURE BASED IDS SNORT is a libpcap based lightweight network intrusion detection system, capable of performing real-time traffic analysis and packet logging on IP networks [5]. It can perform protocol analysis, content searching/matching and can be used to detect a variety of attacks and probes, such as buffer overflows, stealth port scans, CGI attacks, OS fingerprinting attempts, and much more. Snort uses a flexible rules language to describe traffic that it should collect or pass, as well as a detection engine that utilizes a modular plug-in architecture. Snort has a real-time alerting capability as well, with alerts being sent to syslog, a separate alert file or even Windows computer via Samba. Department of Computer Science and Engineering, Indian Institute of Technology, Kharagpur 16

The first version of SNORT was released in 1998 by Martin Roesch under GPL license. Currently version 2.8 is running. Snort has three primary modes of operation [3]. They are Sniffer In this mode, SNORT simply eavesdrop the packets and displays them like tcpdump program. Depending on the flags used with SNORT, we can determine how detailed information we want to avail. Figure 7 shows the minimal details of a packet captured by SNORT. Figure 7: Sniffed Packet (snort v) Packet logger: Whenever the SNORT user wants to record the packets captured by the IDS, SNORT has to be run in the Packet logger mode, specifying the directory name where the packets are to be logged. It logs packets either in tcpdump format (binary) or in decoded ASCII format. Figure 8 shows descriptions of packets sniffed by the SNORT program. Figure 8: Sniffed Packet ( snort dev) Department of Computer Science and Engineering, Indian Institute of Technology, Kharagpur 17

Intrusion Detection mode: In this mode, SNORT will not record every packet that it sniffs but logs only those events which triggered its rules as shown in figure 9. Figure 9: Alerts generated in intrusion detection mode SNORT Rule structure: SNORT rules are written in PCRE format which are straight forward and quite powerful. These rules are editable as per the need. Generally the rule structure has two logical parts Rule header contains The type of action SNORT has to take on matching of a rule (e.g. alert, log) Protocols (IP, ICMP, TCP, UDP) Sender IP address and the port number Flow direction (incoming, outgoing or both) Receiver IP address and the port number Source port and destination. Rule options contains Alert messages and information on which parts of the packet should be inspected to determine if the rule action should be taken. Rule Header Rule Option Department of Computer Science and Engineering, Indian Institute of Technology, Kharagpur 18

The sample SNORT rule given above says that if the payload of a TCP packet matches with the content 00 01 86 a5 originated from any source address and any port number to the destination address 192.168.1.0/24 with port number 111 generate alert message mountd access. 1.9. RELATED WORK Network intrusion detection systems like snort [3] or Bro [11] typically use signature based detection, matching patterns in network traffic to the patterns of known attacks. This works well, but has the obvious disadvantage of being vulnerable to novel attacks. An alternative approach is anomaly detection, which models normal traffic and signals any deviation from this model as suspicious. The idea is based on work by Forrest et al. (1996), who found that most UNIX processes make highly predictable sequences of system calls in normal use. Network anomaly detectors look for unusual traffic rather than unusual system calls. ADAM (Audit Data and Mining) [12] is an anomaly detector trained on both attack-free traffic and traffic with labelled attacks. It monitors port numbers, IP addresses and subnets, and TCP state. ADAM uses a naive Bayes classifier which means that the probability that a packet belongs to some class (normal, known attack, or unknown) depends on the a-priori probability of the class, and the combined probabilities of a large collection of rules under the assumption that they are independent. In the IDES/NIDES systems [9], [10], a statistical based anomaly detection technique is used to represent the expected normal behaviour of a subject and variance due to noises. The statistical-based anomaly detection technique overcomes the problems with rule-based anomaly detection technique in handling noises and variances. However, the statistical technique in IDES/NIDES is a univariate technique that is applied to only one behaviour measure, where as many intrusions involve multiple subjects and multiple actions having impact on multiple behaviour measures. Hence, a multivariate anomaly detection technique is needed for intrusion detection. Department of Computer Science and Engineering, Indian Institute of Technology, Kharagpur 19

Matthew V. Mahoney and Philip K. Chan developed Packet Header Anomaly detection for identifying Hostile Network (PHAD) [16],[17] that learns the normal ranges of values for each packet header field at the data link (Ethernet), network (IP), and transport/control layers (TCP, UDP, ICMP). PHAD detects some of the attacks in the DARPA data set that involve exploits at the transport layer and below. The paper, Detecting Novel Network Intrusions Using Bayes Estimators [18] authored by Daniel Barbara and et al suggests a method called pseudo-bayes estimators as a means to estimate the prior and posterior probabilities of new attacks. Then a Naive Bayes classifier is used to classify the instances into normal instances, known attacks and new attacks. 1.10. MOTIVATION AND OBJECTIVE Despite the fact that intrusion detection systems are commercially developed and used for more than a decade, there still exist many issues around IDS. Some of the shortcomings of the current IDS which handicap its effectiveness are discussed below. a) Only the known attacks are detected in signature based techniques which simply means no protection is offered against novel attacks or new variants of existing intrusions. A small variation in the attack pattern can invalidate a signature. By the time the new signatures/patches come up the intrusions might have done the intended damages. b) How well a signature captures the attacks in its string is again a matter of concern. There are quite a few such poorly written signature codes. So the actual attack pattern may stretch across multiple packets, easily evading the detection system. c) In order to perform an exhaustive signature based search, the processing and memory needs are very high and in the real time scenario, there is quite likely hood of missing genuine attacks. Also, there is the problem of ever increasing attack signature databases. Department of Computer Science and Engineering, Indian Institute of Technology, Kharagpur 20

d) Also the attackers can frame such malicious packets that are likely to have many attack signatures to keep the detection engine busy and in the course of action some packets with real attack patterns will find their way into the internal network, thus evading the detection system e) There is another class of attacks which targets the detection algorithms as elucidated below. String matching algorithms are the core component of any signature detection mechanism and there is not a single string matching algorithm which can be efficient in any given situation. So the sly intruders can fabricate and send the packets which cause the algorithms to run in the worst case complexities. f) And what if the attacker sends packets with signatures spread across multiple packets, use techniques like stealth scanning. g) In anomaly approach, though new kinds of intrusions are detected, this benefit is paralyzed by high number of false alarms. More over improper/ insufficient training to anomaly module results in showing the genuine changes in the network traffic pattern as suspicious activities only to raise the number of false positives and false negatives. 1.11. OBJECTIVE The aim of the present work was to design and develop of a Anomaly or behavioural based Network Intrusion Detection System which can detect intrusions based on behavioural patterns (i.e. without the use of signatures) and can also detect novel attacks which are anomalous in nature. The work also aimed at reducing number of false alarms by characterizing the target network with appropriate network parameters and analyzing them with mathematical models. Literature survey reveals that, the Bayesian Analysis is successfully used in the SPAM filters but in the area of IDS it is still not explored to great extent. So in this work, Bayesian classification technique is used for discriminating the anomalous attacks from that of normal activities. Hotelling s Multivariate Department of Computer Science and Engineering, Indian Institute of Technology, Kharagpur 21

statistical hypothesis technique and statistical mean- variances model are also being used. The project is integrated with a open source signature based IDS called SNORT so that it forms a complete package having both signature and anomaly techniques for effective defence against the Network attacks 1.12. ORGANIZATION OF THESIS This report is organized as follows. Section 1 gives brief introduction to the project topic, Types and techniques for IDS, deployment scenarios of IDS etc. Then related work in the field of IDS is covered. It also talks about the motivation for taking up the project and objectives set for the project. Chapter 2 deals with the system architecture, explains the individual components of the IDS. Next section i.e. Chapter 3 explains the techniques used in the research. Chapter 4 deals with the results and discussions. Finally chapter 5 covers the conclusion and the future directions for enhancing the capabilities of the present IDS. Department of Computer Science and Engineering, Indian Institute of Technology, Kharagpur 22

2. CHAPTER 2 2.1. SYSTEM ARCHITECTURE The proposed architecture of Network IDS has various components as depicted in the figure 10. This architecture is based on SNORT, which is a open source Network IDS [19]. The components execute different functionalities which are discussed below. Figure 10: Overall System architecture 2.2. SENSOR/DECODER Figure 11: Network IDS sensor Department of Computer Science and Engineering, Indian Institute of Technology, Kharagpur 23

The NIC is put in promiscuous mode to sniff all the packets in the network irrespective of their target. The decoder receives the packets from the libpcap packet capturing library and processes them. Formal checker evaluates the packet structure for truncated packet headers and proper checksum, depending on whether it is an Ethernet, ARP, IP, TCP, UDP or ICMP packets. When Formal checker detects an error in the packet structure, it informs the decoder and the packet is discarded from further processing. Figure 11 shows the block diagram of the sensor/decoder. This module executes following functionalities. - Sniffs all the network packets visible to it in real time. - Extract the header and payload information from the Ethernet frame. - Updates the Ethernet, ARP, RARP, IP, TCP, UDP and ICMP counter as and when the respective packets are received - Perform necessary checks on header and payload information. - Sniffed packets sent to the Pre-processor 2.3. PREPROCESSOR This module takes the packets from the decoder and performs the functions like IP de-fragmentation, building the sessions for reassembly of packets etc. Several pre-processors are available with SNORT to execute the necessary tasks as depicted in Figure 12. This module also hosts the Anomaly learning and detection pre-processor used for detecting the intrusions leading to anomalies. Figure 12: Network IDS pre-processor Department of Computer Science and Engineering, Indian Institute of Technology, Kharagpur 24

The pre-processor has following responsibilities: - De-fragments the fragmented IP packets - Reassembles the TCP packets into streams - Normalizes Application Layer protocols like Telnet/HTTP - Detects Port scans/evasion Attacks - Pre-processed packets sent to Detection Engine - Anomaly Detection pre-processor detects the intrusive activities in the network 2.4. ANOMALY DETECTION PRE-PROCESSOR This module helps to detect network based intrusions which manifests in abnormal network behaviour. It runs in two phases, learning (Training) mode and detection mode. In the learning mode, the module learns the traffic pattern of the entire network and records the corresponding network parameters. Once the learning is over, the network profile is generated using the profiler program. This profile is used to detect the anomalies when the module runs in the detection mode. Figure 13 shows the structure of Anomaly detection pre-processor. Figure 13: Anomaly Detection pre-processor Department of Computer Science and Engineering, Indian Institute of Technology, Kharagpur 25

It performs following functionalities: In the Learning mode - Measures the network parameters at regular intervals as configured by user - Stores these values into a log file at regular interval In the Detection mode - Measures the network parameters at regular intervals - Reads baselined values from the file - Finds statistical deviations (Mean and Variance) - Computes values for Hotelling s expression and Bayesian discrimination function - Triggers the alerts on detecting any abnormalities in the traffic pattern 2.5. DETECTION ENGINE It is the main part of the entire system which is responsible for detecting the attack signatures in the pre-processed packets. The overall system performance directly depends on this module. Some of the main functions handled by this module are listed below. - Parses the rules and build an internal data structure that holds the rules in a customized tree structure. Once the tree is built, loads it into memory. - Passes traffic through this rule tree for comparing the packet header and data against the rules. (Uses strings matching algorithms) - Report to Alert module on packets that have found to be carrying malicious data. - If any new rules have been added or if existing rules are modified or deleted then updates the same to the detection engine tree structure. - When the application is exited this will clean up all memory allocated for building the detection engine. Department of Computer Science and Engineering, Indian Institute of Technology, Kharagpur 26

2.6. ALERT MODULE - Sends the alerts triggered by the Detection Engine to Alert Console in real time. - Stores the alerts into a alert file (/var/log/snort) and/or into a Database such as MySQL as per the configuration Open source php based console, called Basic Analysis and Security Engine (BASE) is integrated with the Alert Module to enhance the user friendliness. The figure 14 shows screenshot of the BASE console. Figure 14: Screenshot of BASE console showing the generated alerts Department of Computer Science and Engineering, Indian Institute of Technology, Kharagpur 27

2.7. BASIC ANALYSIS AND SECURITY ENGINE (BASE) BASE is a open source code written in the PHP programming language which displays information from a database in a user friendly web front end [6],[7]. It is based on the code from the Analysis Console for Intrusion Databases (ACID) project. Apache web server has to be setup for running BASE. Figures 15 and 16 shows the screenshots on BASE console Figure 15: BASE console showing the alert statistics When used with Snort, BASE reads both tcpdump binary log formats and Snort alert formats [7]. Snort must be configured to log alerts to the database used by BASE (for example. MySQL). The alerts from Anomaly detection pre-processor can also be viewed on BASE console. Once data is logged and processed, BASE has the ability to graphically display both layer-3 and layer-4 packet information. Department of Computer Science and Engineering, Indian Institute of Technology, Kharagpur 28

It also generates graphs and statistics based on time, sensor, signature, protocol, IP address, TCP/UDP port, or classification. The BASE search interface can query based on alert Meta information such as sensor, alert group, signature, classification, and detection time, as well as packet data such as source/destination addresses, ports, packet payload, or packet flags. Thus BASE allows for the easy management of alert data. The administrator can categorize data into alert groups, delete false positives or previously handled alerts, and archive and export alert data to an email address for administrative notification or further processing. Support for user logins and roles, allowing an administrator to control what is seen through the web interface. Figure 16: BASE console showing the details of sniffed Packet Department of Computer Science and Engineering, Indian Institute of Technology, Kharagpur 29

2.8. OPERATING ENVIRONMENT The development work is carried out in C language on Linux platform to comply with the SNORT program. The following software/tools are used for the development and execution of the project ANJUTA - Open source IDE BASE - Basic Analysis and Security Engine GCC - GNU C Compiler to compile the components. Libpcap - Linux Packet capturing library MYSQL - Centralized database storage. RHEL4 - Redhat Enterprise Linux 4 SNORT - Open Source Network Intrusion Detection System The IDS works efficiently on a system with the following configuration: Pentium IV 2.0 GHz 512MBRAM 40 GB Hard Disk or higher 10/100 Mbps Ethernet Interface Card. Department of Computer Science and Engineering, Indian Institute of Technology, Kharagpur 30

3. CHAPTER 3 3.1. RESEARCH APPROACH The primary task was to characterize the target network in terms of suitable network parameters. The parameters are chosen such that their values will change perceivably in normal and intrusive conditions. The features considered are the commonly seen protocols in the network traffic, the traffic data rate and the flow direction. In essence, the Anomaly model tries to capture the network behaviour in terms of two quantities intensity and heterogeneity. Intensity refers to the number of occurrences of a given network parameter over a period of time (for example number of TCP connections or number of outgoing HTTP packets etc) while heterogeneity refers to the observed pattern of the nature of network activities over time (for example the data rate of HTTP packets in different time segments of the day or observations like web traffic is more during the beginning of office hours and then drops. It rises again during the closing hours etc). These two quantities closely relate to activities occurring in any given network and thus can represent the behaviour of network under the assumption that network behaviour has certain degree of repeatability. Once the network behaviour is quantified with these parameters, the next step would be to observe how they vary with time. The observation has to be made on different days of a week because the network behaviour changes over working days and non working days of a week and also on general holidays. The Anomaly based IDS has two operational modes. Learning (or training) mode: In this mode, the IDS learns the normal traffic behaviour in terms of representative feature set characterizing the target network. It collects the statistics of the selected network parameters for different types of days (Week days from Department of Computer Science and Engineering, Indian Institute of Technology, Kharagpur 31

Monday to Friday, Saturdays and Sundays) and then stores them into a specified file for subsequent processing. The frequency of statistics collection is set as per requirement; it is set by default to 10 minutes. IDS is put in this mode for sufficient period to learn the normal network behaviour. Sufficient training period is the key factor in reducing the false alarms. When IDS is learning the normal behaviour, the target network is assumed to be free from attacks and intrusions Following attributes are considered for characterizing the network: TCP Packet count (incoming, outgoing and within LAN) UDP Packet count (----------------- --------------------) ICMP Traffic (----------------- --------------------) The number of TCP connections Web Traffic (incoming, outgoing) DNS Traffic (--------- ------------) Data rates TCP traffic in kb/s (--------- ------------) Data rates UDP traffic in kb/s (--------- ------------) Data rates HTTP traffic in kb/s (--------- ------------) Data rates DNS traffic in kb/s (--------- ------------) Once the learning is over, profile for the target network is generated with the gathered data using a profiler. If statistics collections is done at every 10 minutes and the learning period is say 1 month, total 24 sample values are available for each network parameter corresponding to each hour of the week day. Hence the profile is generated for each hour of the day over entire week. This implies that total 168 baseline vectors are established for the entire week, each vector containing 25 network parameters. The profile also contains 168 inverse matrices each of the order 25 x 25, accounting for number of parameters in consideration. This profile is used by Anomaly detection module during the detection phase. The IDS is also trained to learn the network behaviour in the presence of network intrusions. Intrusions are simulated using the MIT-DARPA training data set. Network profile is also generated for this condition. Figure 17 shows the time slots used for generating the profile. Department of Computer Science and Engineering, Indian Institute of Technology, Kharagpur 32

Figure 17: Time slots used in generating the network profile When the network environment changes for genuine reasons, it may result into a number of false positives. In such situations the Anomaly model can be updated by rerunning the training phase on the changed traffic and rebuilding the profile using profiler program. The logic for profile generation is given in figure 18. Input : The file containing the features values logged during the learning phase Output : files containing the mean, standard deviations and inverse matrices of feature set begin for i =1 to Num.of week days do for j =1 to Num. of hours in a day do Read the feature values logged during learning phase; for k =1 to Num. of network features do find sum of the values corresponding to the same hour and day of the week; Compute Average values and standard deviation for each feature; end n Compute l, m= 1 T ( x l )( x m μ) μ where n is the total number of features Compute the Determinant of above covariance matrices if Determinant 0 Consider the neighbouring covariance matrix having positive Determinant Compute inverse matrix corresponding to each Covariance matrix Figure 18: Algorithm for generating the profile Department of Computer Science and Engineering, Indian Institute of Technology, Kharagpur 33

Detection mode: In this mode, IDS detects in real time, the network based attacks leading to abnormal traffic pattern. The abnormality is decided on the basis of the network profile constructed earlier. The profile contains 168 vectors corresponding to each hour of the day over entire week, each vector containing as set of 25 features which describes the network. The Anomaly detection module samples the selected network parameters at regular intervals, as in the case of learning mode, checks whether they comply with already established network profile for that particular hour and day of the week. If it detects significant deviations, then it triggers alerts. The logic for detection is given in figure 19 Input : The file containing the network profile Output : Sends alert in case a event is detected as intrusion begin for i =1 to Num.of week days do for j =1 to Num. of hours in a day do for k =1 to Num. of network features do Read Average values and standard deviation for each feature; Read the inverse matrices Read the determinant matrix corresponding to each inverse matrix Compute (μ ± σ) for each parameter if ( μ σ > x > μ + σ) Compute x is intrusive 2 1 T = (X μ )S ( X μ ) T end 2 if T exceeds the threshold flag alerts 1 1 T 1 Compute gi ( X ) = ln S ( X μ ) S ( X μ) + ln p( I ) 2 2 if g i ( X ) exceeds the threshold flag alerts Figure 19: Algorithm for Anomaly Detection Department of Computer Science and Engineering, Indian Institute of Technology, Kharagpur 34

The flow chart in figure 20 shows the overall working of Anomaly Detection technique. Figure 20: Flow chart depicting the overall working of Anomaly Detection Technique Department of Computer Science and Engineering, Indian Institute of Technology, Kharagpur 35

3.2. STATISTICAL MOMENTS OR MEAN AND S TANDARD DEVIATION MODEL Statistical based anomaly detection techniques use statistical properties (mean and variance) of normal activities to build a statistical normal profile and employ statistical tests to determine whether observed activities deviate significantly from the normal profile [20]. Figure 21: normal distribution curve with different confidence intervals The arithmetic average, or the mean, is a statistic that measures the central tendency of a set of data. It is given by, μ n i = n x i = 1 Where μ = mean x i = value of i th observation of a given parameter, i =1 n n = total number of observation in a sample The Standard Deviation is a measure of the amount of data dispersion around the mean. It is given by, n σ = i i = 1 n 1 ( x 2 μ ) Where σ =standard deviation x i = value of i th observation of a given parameter, i =1 n μ = mean n = total number of observation in a sample The values of μ and σ are established for each of the network parameter x i. Department of Computer Science and Engineering, Indian Institute of Technology, Kharagpur 36

If the value of x i goes beyond ( μ ± n* σ ), it simply indicates an anomalous situation and can be flagged as alert. It is difficult to determine thresholds above which an anomaly should be considered intrusive. Setting threshold too low results in false positives and setting it too high results in false negatives. So the confidence interval is chosen suitably based on the experimentation [21]. Figure 21 shows different confidence intervals for a Gaussian distribution. 3.3. HOTELLING S T 2 HYPOTHESIS, A MULTIVARIATE STATISTICAL TECHNIQUE When there are enough computational resources and the security level is also high then "multivariate models" are a good choice since they produce better results with less false alarm rate as compared to mean and standard deviation model. Hence these are recommended for the IDS. Hotelling s T 2 test is a multivariate statistical process control technique that detects anomalies in the activities of a network. It can be assumed as the multivariate extension of mean/standard deviation model, employing an n dimensional mean vector and the corresponding covariance matrix. 2 Hotelling s Τ statistic for an observation X i is determined by [13],[14] Τ 2 = ( X i μ ) T S 1 ( X i μ ) Where X i = ( X i, X i2, X i3... X 1 ip ), denotes an observations of p variables at time t μ = μ, μ, μ,... μ ), denotes a vector of mean values of p variables at time t ( 1 2 3 p and S is the covariance matrix given by, 1 S = ( n 1) n 1 T ( X i μ )( X i μ), where n is the data sample size The computed 2 Τ value is small if the data point conforms to the norm profile. If the value of the Τ 2 statistic is greater than a threshold value, then the null hypothesis that the event is normal is rejected and signals anomalous Department of Computer Science and Engineering, Indian Institute of Technology, Kharagpur 37

behaviour. The threshold value is set based on the observed values of Τ 2 for normal and intrusions during the learning phase. Hotellings Τ 2 test provides a complete data model of multivariate data. Since it uses the covariance matrix S of p variables, it detects both mean shifts and their interrelationship in a multivariate manner which is important in finding the network anomalies. The test detects three kinds of events. They are normal, suspicious and Intrusive. Normal corresponds to the events which comply with previous normal traffic pattern. Suspicious means the events which are deviated to some extent from their normal behaviour and Attack indicates there is a large variation in the observed and expected traffic pattern. 3.4. BAYESIAN CLASSIFICATION, A PROBABILISTIC TECHNIQUE In probabilistic classification method, a pattern is assigned to the class that is most probable, given the observed features, i.e., point x of a feature space is assigned to the class that maximize ( C / x) p j The classification problem is formulated in terms of estimating the posterior probability that pattern x belongs to one of the m data classes Posterior probability depends on - The prior probability p C ) i.e. the likelihood that a random selected pattern belong to class C i ( i - The class conditional probability density function p x / C ) i.e. the distribution of patterns of class ci in the selected space. Baye s Theorem: ( i Bayesian statistics, in the most general form, provides a framework for combining observed data with prior assumptions in order to model stochastic systems [23], [24]. p( C i p( x / Ci ) / x) = p( Ci ) = p( x) M i= 1 p( x / Ci ) p( Ci ) p( x / C ). p( C ) i i Department of Computer Science and Engineering, Indian Institute of Technology, Kharagpur 38

Any function that computes the conditional probabilities p( Ci / x) is referred to as discriminant function. Given an observation x, the Bayes theorem provides a method to compute p( Ci / x). p(x) can be ignored, since it is the same for all the classes and thus does not help in discriminating the classes. The likelihood function p x / C ) denotes a probability density function of ( i the vector samples x given a particular estimate C i of the underlying probability distribution generating that data. A multivariate normal distribution is assumed for p x / C ). Figure 22 shows the multivariate Gaussian distribution curve. ( i Figure 22: Multivariate Gaussian distribution curve A Gaussian or multivariate normal distribution is characterized by its mean value vector μ and its covariance matrix S and has the distribution function, f ( μ, Σ) = 1 1 T 1 exp{ 2 ( X μ) S ( X μ)} ---------- (2) p 1/ 2 (2π ) S Here X is a p dimensional pattern vector of real valued attributes The discriminant function g i (X ) can be derived by using the equations (1) and (2). 1 1 T 1 gi ( X ) = ln S ( X μ ) S ( X μ) + 2 2 ln p( I ) The values of g i (X ) can distinguish the intrusions from the normal events. Department of Computer Science and Engineering, Indian Institute of Technology, Kharagpur 39

4. CHAPTER 4 4.1. EXPERIMENTAL RESULTS AND DISCUSSION To evaluate the system, two major indicators of performances are chosen. - Detection rate - False positive rate Detection rate is defined as the number of intrusion instances detected by the system divided by the total number of intrusion instances present in the test set. The false positive rate is defined as the total number of instances that were wrongly detected as intrusions divided by the total number of normal instances. These are good measures of performances since they measure what percentage of intrusions the system is able to detect and how many incorrect classifications it makes in the process. The following sub sections give the details of evaluation scheme and the results obtained. 4.2. EVALUATION SCHEME The Anomaly IDS is trained for five weeks to learn the normal network traffic of the IIT, Kharagpur. The model considers a vector of 25 network attributes to describe the target network. The IDS is also trained for more than three weeks to learn the network behaviour under intrusions. The intrusions are simulated in the network using MIT-DARPA 1999 data set. The training data contains a total of 4396 vector data points for normal traffic and 2120 vector data points for intrusive traffic. The training period covers different types week days (working, Saturday and non working days). The network profile is generated using the training data which contains a total of 168 vector data points corresponding to each hour of the day over the entire week. The same training data and the test data is used with all the three techniques discussed earlier. Department of Computer Science and Engineering, Indian Institute of Technology, Kharagpur 40

About MIT-DARPA IDS Evaluation In 1998, the Information Systems Technology Group of Lincoln Laboratory at MIT, in conjunction with the Air Force Research Laboratory (AFRL) and the Defence Advanced Research Projects Agency (DARPA), began work to develop a standard for the evaluation of Network IDS. Developing this evaluation meant the creation of consistent and repeatable network traffic. The traffic was created through the study of 4 months of data from Hanscom Air Force Base and approximately 50 other bases. Using that data, they were able to generate and simulate network traffic, while introducing attacks, probes and intrusions into the data. Both training and testing data were simulated and two types of traffic were published. Training data is traffic in which the attacks were known from the start. A second set of data contains traffic in which the attacks were not described explicitly. Data sets of Week 1 and Week 3 contain attack free traffic while Week 2 contains training data with attacks. Week 4 and Week 5 are the testing data containing network attacks in the midst of normal background data. Test Data sets contains four categories of simulated attacks DoS Denial of service (e.g. SYN flood) R2L -- unauthorized access from remote machine (password guessing) U2R unauthorized access to super user or root functions (buffer overflow attacks) Probing --surveillance and other probing vulnerabilities (port scanning) A more complete discussion on this is available at the Lincoln Laboratory/ MIT site [22]. Department of Computer Science and Engineering, Indian Institute of Technology, Kharagpur 41

The table 1 gives the values obtained for the Hotelling s multivariate expression and Bayesian Classifier for normal and intrusive network traffic. Values for Hotelling s Statistic Values for Bayesian Classifier Normal Intrusive Normal Intrusive 1 7.74E+09 1.32E+17 3.07E+08 6.59E+16 2 7.60E+08 9.07E+16 1.48E+07 4.54E+16 3 5.60E+08 6.26E+16 1.32E+07 3.13E+16 4 4.49E+08 6.05E+16 1.07E+07 3.02E+16 5 1.59E+08 4.35E+16 1.04E+07 2.18E+16 6 8.84E+07 2.97E+16 1.03E+07 1.48E+16 7 5.10E+07 2.60E+16 6.70E+06 1.30E+16 8 4.50E+07 2.37E+16 6.52E+06 1.19E+16 9 2.95E+07 1.95E+16 2.88E+06 9.77E+15 10 2.46E+07 1.57E+16 2.74E+06 7.85E+15 11 2.09E+07 1.09E+16 1.71E+06 5.44E+15 12 1.93E+07 9.58E+15 2.16E+05 4.79E+15 13 1.36E+07 9.34E+15 2.60E+05 4.67E+15 14 1.34E+07 6.34E+15 7.19E+05 3.17E+15 15 1.17E+07 5.19E+15 1.29E+06 2.59E+15 16 8.36E+06 5.12E+15 1.40E+06 2.56E+15 17 7.88E+06 3.79E+15 1.41E+06 1.89E+15 18 6.27E+06 2.64E+15 1.59E+06 1.32E+15 19 5.67E+06 2.29E+15 1.63E+06 1.15E+15 20 4.85E+06 2.28E+15 2.42E+06 1.14E+15 21 3.26E+06 3.32E+14 2.84E+06 1.66E+14 22 3.18E+06 2.67E+14 3.13E+06 1.34E+14 23 2.82E+06 2.67E+14 3.94E+06 1.33E+14 24 2.80E+06 2.12E+14 4.18E+06 1.06E+14 25 2.59E+06 1.65E+14 5.85E+06 8.25E+13 26 1.44E+06 1.08E+14 6.70E+06 5.39E+13 27 5.20E+05 7.73E+13 6.82E+06 3.87E+13 Table 1: Typical values obtained for the normal and intrusive network traffic with Hotelling s and Bayesian discriminator functions By manually analysing a large set of values obtained for Hotelling s and Bayesian discriminators, it is found that following values more closely discriminate the normal activities from the intrusive ones. Hotellings Technique: On an average, the values for normal activities lie between 1.00E+06 to 5.00E+07 while for intrusive the values are above.90e+08. Bayesian Technique: On an average, the values for normal activities lie between 2.00E+05 to 9.00E+07 while for intrusive the values are above 1.50E+08 Department of Computer Science and Engineering, Indian Institute of Technology, Kharagpur 42

4.3. COMPARATIVE RESULTS Attack Name Tools/Data set used Count Detection using different Techniques Probabilistic (Bayesian Classifier) Statistical (Hotelliing's Hypothesis) Statistical (Mean ± 2*SD) ping flood ping tool 15 15 15 15 DoS attack ddos open source tool 5 5 5 5 TCP RST attack neti open source code 5 5 5 5 TCP Syn flood attack neti open source code 7 7 7 6 UDP attack neti open source code 10 10 10 10 X mas scan nmap tool 5 5 4 4 MIT_ DARPA NTinfoscan 1999 Data set 1 0 0 0 pod " " 2 2 2 2 back '' " 2 0 0 0 httptunnel " " 2 0 0 0 land " " 2 2 2 2 secret " " 3 0 0 0 portsweep " " 3 3 3 2 eject " " 3 0 0 0 mailbomb " " 2 2 2 2 ipsweep " " 3 3 2 2 satan " " 2 1 1 1 neptune " " 2 2 2 2 Total 74 62 60 58 Detection Accuracy (%) 83.78 81.08 78.38 Total Alerts generated 65 64 67 No. of Attacks missed 12 16 20 False Positive 4.62 6.25 13.43 Department of Computer Science and Engineering, Indian Institute of Technology, Kharagpur 43

rate (%) False Negative rate (%) 16.22 21.62 27.03 Positive Prediction rate (%) 95.40 90.63 78.30 Table 2: Chart showing the comparative results of the experiments Table 2. given below shows the results obtained by Daniel Barbara et al using pseudo-bayes estimators [6] Table 3. Experimental results on MIT_LL DARPA 1999 Data set. Source: http://www.cs.ubc.ca/local/reading/proceedings/siam_datamining2001/pdf/sdm01_29.pdf 4.4. DISCUSSION The experiment clearly revealed that the Bayesian classification method gives better detection rate and less false positives in detecting the intrusions among the three techniques discussed in the project. The detection accuracy of 84 % is achieved using the Bayesian method with the false positive rate of 4.6%. Hotelling s statistical method gave a hit rate of 81% at 6.2% false positive rate. The performance metrics for statistical Moments (mean and standard deviation) model yielded hit rate of 78% while the false positive rate was 13%. The Department of Computer Science and Engineering, Indian Institute of Technology, Kharagpur 44

comparative analysis with the previous works also reveals that the Bayesian approach is a superior technique. In summary, the results show that the approach followed in this thesis is quite effective and efficient for detecting the network based attacks. It is also observed that the multivariate statistical techniques are more effective than the univariate technique, particularly the Bayesian techniques has promising potential in the future IDS research Department of Computer Science and Engineering, Indian Institute of Technology, Kharagpur 45

5. CHAPTER 5 5.1. CONCLUSION Network Intrusion Detection System has a major role to play in safeguarding the network resources against various kinds of attacks. With the advent of new vulnerabilities and sophistications in the nature of attacks, new techniques for intrusion detection have evolved. The main objectives of the research being increasing the detection accuracy while keeping the false positive rate low. As stated earlier, the signature based techniques are good but has the obvious short comings like failure to detect novel attacks, increasing signature database etc. So the viable alternative would be to analyse the behaviour of the network as a whole and trying to build the model based on the observations. So Anomaly based detection has been a wide area of interest for researchers since it provides the base line for developing promising techniques. The Anomaly based detection complements the Signature based technique and helps in identifying the novel attacks which lead to the anomalies in the network traffic. The major concerns in this method are identifying the appropriate network features to characterize the network and build a behavioural model and also the rate of false positives may increase sharply if the IDS is not trained sufficiently in the target network. In the present framework of project, discussed the design and development of Anomaly based intrusion Detection system which is built on top of a existing open source signature based network IDS, called SNORT so to have both the analysis techniques in a single package. The Anomaly based component of IDS is trained in the Computer and Informatics Centre of Indian Institute of Technology (IIT), Kharagpur where the IIT network traffic is sniffed using a port mirrored switch at the gateway. The IDS is trained for more than a month in the IIT network at computer and Informatics centre, to learn the normal traffic pattern. Also it is exposed to the intrusive traffic Department of Computer Science and Engineering, Indian Institute of Technology, Kharagpur 46

for more than 3 weeks, in a simulated environment by replaying the MIT DARPA Intrusion Detection System training datasets (1999). The thesis presented three techniques for detecting anomaly based intrusions at the network level. Statistical based anomaly detection techniques use statistical properties and statistical tests to determine whether "observed behaviour" deviate significantly from the "expected behaviour". The first technique is based on univariate statistic model with mean and variance. The second method uses the multivariate Hotelling s method while the last technique uses the Bayesian classification technique for discriminating attacks from that of normal activities. All the three techniques are evaluated with the DARPA IDS evaluation Data sets (1999) and the results are compared. Bayesian approach proved to be a better solution than the Hotelling s Multivariate technique and the method of Statistical Moments. Presently, the work caters only to identify and classify the events into normal and attack classes. It can be extended to detect and classify the attacks into multiple attack classes. Dynamic updation of the Anomaly Model using Bayesian Network can also be considered for future enhancement. Different Analysis techniques like HMM and Fuzzy Logic can also be tried as alternative techniques for anomaly detection. Department of Computer Science and Engineering, Indian Institute of Technology, Kharagpur 47

BIBLIOGRAPHY [1]. R.Coolen, Intrusion Detection: Generics and State of the Art, RTO Technical Report 49, http://www.tno.nl/instit/fel/div2/resources/rto-tr-049-ids.pdf [2]. J. P. Anderson, Computer Security Threat Monitoring and Surveillance, Technical Report April 1980, http://csrc.nist.gov/publications/history/ande80.pdf [3]. Martin Roesch : Snort Documents, http://www.snort.org/docs/ [4]. Net Optics, Inc. White Paper: Deploying Network Taps with Intrusion Detection Systems, http://www.netoptics.com/products/downloads.asp?pageid=150&section=res [5]. Jack Koziol, Intrusion Detection with Snort, Pearson publications, 2003 [6]. Basic Analysis and Security Engine project, http://base.secureideas.net/ [7]. White papers on Basic Analysis and Security Engine (BASE), http://whitepapers.techrepublic.com.com/abstract.aspx?docid=266711 [8]. Q. Zhao, J. Sun, S. Zhang, A hybrid and hierarchical NIDS paradigm utilizing naïve Bayes classifier, Canadian conference on Electrical and Computer Engineering, 2004, http://ieeexplore.ieee.org/iel5/9317/29618/01344977.pdf?tp=&isnumber=&arnumber=1344977 [9]. Javitz HS, Valdes A. The NIDES statistical component description of justification Technical Report A010, SRI International, Menlo Park, CA, March 1994. http://www.cs.ucdavis.edu/~wu/ecs236/papers/hw2_nides-sta-description.pdf [10]. Javitz HS, Valdes A. The SRI statistical anomaly detector, Proceedings of the 1991 IEEE Symposium on Research in Security and Privacy, May 1991 http://ieeexplore.ieee.org/iel2/349/3628/00130799.pdf?tp=&isnumber=&arnumber=130799 [11]. V. Paxson, Bro: A System for Detecting Network Intruders in Real-Time, Computer Networks, 1999, http://bro-ids.org/publications.html [12]. D. Barbar a and S. Jajodia and N. Wu and B. Speegle, The ADAM project, http://www.isse.gmu.edu/dbarbara/adam.html [13]. Nong Ye and Qiang Chen, An anomaly detection technique based on a chi-square statistic for detecting intrusions into information systems, Quality and Reliability Engineering International, 17:105--112, 2001, http://citeseer.ist.psu.edu/ye01anomaly.html [14]. Ye, N., Li, X., Chen, Q., Emran, S. M., and Xu, M. Probabilistic Techniques for Intrusion Detection Based on Computer Audit Data, IEEE Transactions on Systems, Man and Cybernetics, vol.31(4), pp.266--274, July 2001., http://ieeexplore.ieee.org/iel5/3468/20237/00935043.pdf?tp=&isnumber=&arnumber=935043 [15]. A. Qayyum, M. H. Islam, and M. Jamil, Taxonomy of Statistical Based Anomaly Detection Techniques for Intrusion Detection, IEEE International Conference on Emerging Department of Computer Science and Engineering, Indian Institute of Technology, Kharagpur 48

Technologies, September 17-18,2005 http://ieeexplore.ieee.org/iel5/10430/33125/01558893.pdf?tp=&isnumber=&arnumber=1558893 [16]. M. Mahoney and P. Chan, PHAD: Packet header anomaly detection for identifying hostile network traffic, Technical report, Florida Tech., technical report CS-2001-4, April 2001, http://citeseer.ist.psu.edu/mahoney01phad.html [17]. Mahoney M. and P. Chan, Learning models of network traffic for detecting novel attacks", Technical report, Florida Tech 2002, http://cs.fit.edu/~mmahoney/paper5.pdf [18]. D. Barbara, N. Wu and S. Jajodia, Detecting Novel Network Intrusions using Bayes Estimators, Proceedings of the 1st SIAM International Conference on Data Mining, 2001, http://www.cs.ubc.ca/local/reading/proceedings/siam_datamining2001/pdf/sdm0129.pdf [19]. Jack Koziol, Intrusion Detection with Snort, Pearson publications, 2003 [20]. R. Dan Reid & Nada R. Sanders, Operations Management, 3 rd edition., Wiley,2007 [21]. P. Cisar, S. M Cisar, Quality Control in Function of Statistical Anomaly Detection in Intrusion Detection Systems, SISY 2006-4th Serbian-Hungarian Joint Symposium on Intelligent Systems, www.bmf.hu/conferences/sisy2006/19_cisar.pdf [22]. DARPA Intrusion Detection Evaluation, Data Sets and Documentation, 1999 http://www.ll.mit.edu/mission/communications/ist/corpora/ideval/docs/detections_1999.html [23]. Giorgio Giacinto, Fabio Roli, Luca Didaci, Fusion of multiple classifiers for intrusion detection in computer networks. Pattern Recognition Letters 24(12): 1795-1803 (2003) http://www.diee.unica.it/informatica/en/publications/papers-prag/ids-journal-01.pdf [24]. R. Puttini, Z. Marrakchi, and L. Me. Bayesian Classification Model for Real Time Intrusion Detection, in 22th International Workshop on Bayesian Inference and Maximum Entropy Methods in Science and Engineering, 2002. http://www.rennes.supelec.fr/ren/rd/ssir/publis/maxent02_puttini_marrakchi_me.pdf Department of Computer Science and Engineering, Indian Institute of Technology, Kharagpur 49

6. APPENDIX A 6.1. CHARTS OF DIFFERENT NETWORK PARAMETERS OBTAINED WHILE EXPERIMENTATION 3500 3000 2500 Packet count 2000 1500 1000 500 0 21-01-08 00:10 21-01-08 00:50 21-01-08 01:30 21-01-08 02:10 21-01-08 02:50 21-01-08 03:30 21-01-08 04:10 21-01-08 04:50 21-01-08 05:30 21-01-08 06:10 21-01-08 06:50 21-01-08 07:30 21-01-08 08:10 21-01-08 08:50 21-01-08 09:30 21-01-08 10:10 21-01-08 10:50 21-01-08 11:30 21-01-08 12:10 21-01-08 12:50 21-01-08 13:30 21-01-08 14:10 21-01-08 14:50 21-01-08 15:30 21-01-08 16:10 21-01-08 16:50 21-01-08 17:30 21-01-08 18:10 21-01-08 18:50 21-01-08 19:30 21-01-08 20:10 21-01-08 20:50 21-01-08 21:30 21-01-08 22:10 21-01-08 22:50 21-01-08 23:30 Time Figure 23: Traffic pattern in the course of a day (Monday) 3500 3000 2500 TCP Packets 2000 1500 1000 500 0 21-01-08 00:00 21-01-08 00:40 21-01-08 01:20 21-01-08 02:00 21-01-08 02:40 21-01-08 03:20 21-01-08 04:00 21-01-08 04:40 21-01-08 05:20 21-01-08 06:00 21-01-08 06:40 21-01-08 07:20 21-01-08 08:00 21-01-08 08:40 21-01-08 09:20 21-01-08 10:00 21-01-08 10:40 21-01-08 11:20 21-01-08 12:00 21-01-08 12:40 21-01-08 13:20 21-01-08 14:00 21-01-08 14:40 21-01-08 15:20 21-01-08 16:00 21-01-08 16:40 21-01-08 17:20 21-01-08 18:00 21-01-08 18:40 21-01-08 19:20 21-01-08 20:00 21-01-08 20:40 21-01-08 21:20 21-01-08 22:00 21-01-08 22:40 21-01-08 23:20 Time Figure 24: TCP packet count in the course of a day (Monday) Department of Computer Science and Engineering, Indian Institute of Technology, Kharagpur 50

3500 3000 Total TCP packets TCP Packet count 2500 2000 1500 1000 500 0 21-01-08 15:10 21-01-08 00:00 21-01-08 01:10 21-01-08 02:20 21-01-08 03:30 21-01-08 04:40 21-01-08 05:50 21-01-08 07:00 21-01-08 08:10 21-01-08 09:20 21-01-08 10:30 21-01-08 11:40 21-01-08 12:50 21-01-08 14:00 Time 21-01-08 16:20 21-01-08 17:30 21-01-08 18:40 21-01-08 19:50 21-01-08 21:00 21-01-08 22:10 21-01-08 23:20 Figure 25: TCP statistics in the course of a day ( Monday ) TCP packets sent TCP packets receive d TCP Packets in LAN 180 160 140 UDP Packets 120 100 80 60 40 20 0 21-01-08 00:00 21-01-08 01:00 21-01-08 02:00 21-01-08 03:00 Figure 26: 21-01-08 04:00 21-01-08 05:00 21-01-08 06:00 21-01-08 07:00 21-01-08 08:00 21-01-08 09:00 21-01-08 10:00 21-01-08 11:00 21-01-08 12:00 21-01-08 13:00 21-01-08 14:00 Time 21-01-08 15:00 21-01-08 16:00 21-01-08 17:00 21-01-08 18:00 21-01-08 19:00 21-01-08 20:00 21-01-08 21:00 21-01-08 22:00 21-01-08 23:00 UDP packet count in the course of a day (Monday) Department of Computer Science and Engineering, Indian Institute of Technology, Kharagpur 51

UDP Packet count 180 160 140 120 100 80 60 40 20 0 21-01-08 00:00 21-01-08 01:20 21-01-08 02:40 21-01-08 04:00 21-01-08 05:20 21-01-08 06:40 21-01-08 08:00 21-01-08 09:20 21-01-08 10:40 21-01-08 12:00 21-01-08 13:20 21-01-08 14:40 Time 21-01-08 16:00 21-01-08 17:20 21-01-08 18:40 21-01-08 20:00 21-01-08 21:20 21-01-08 22:40 Figure 27: UDP statistics in the course of a day ( Monday ) Total UDP Packets UDP Packets sent UDP Packets received UDP Packets in LAN 30 25 ICMP Packets 20 15 10 5 0 21-01-08 14:20 21-01-08 00:20 21-01-08 01:20 21-01-08 02:20 21-01-08 03:20 21-01-08 04:20 21-01-08 05:20 21-01-08 06:20 21-01-08 07:20 21-01-08 08:20 21-01-08 09:20 21-01-08 10:20 21-01-08 11:20 21-01-08 12:20 21-01-08 13:20 Time 21-01-08 15:20 21-01-08 16:20 21-01-08 17:20 21-01-08 18:20 21-01-08 19:20 21-01-08 20:20 21-01-08 21:20 21-01-08 22:20 21-01-08 23:20 Figure 28: ICMP packet count in the course of a day ( Monday ) Department of Computer Science and Engineering, Indian Institute of Technology, Kharagpur 52

ICMP Packets 30 25 20 15 10 5 0 21-01-08 00:00 21-01-08 01:10 21-01-08 02:20 21-01-08 03:30 21-01-08 04:40 21-01-08 05:50 21-01-08 07:00 21-01-08 08:10 21-01-08 09:20 21-01-08 10:30 21-01-08 11:40 21-01-08 12:50 21-01-08 14:00 Time 21-01-08 15:10 21-01-08 16:20 21-01-08 17:30 21-01-08 18:40 21-01-08 19:50 21-01-08 21:00 21-01-08 22:10 21-01-08 23:20 Total ICMP Packets ICMP Packets sent ICMP Packets received Figure 29: ICMP packet count in the course of a day ( Monday ) ICMP Packets in LAN 350 300 250 Connections count 200 150 100 50 0 21-01-08 00:00 21-01-08 00:50 21-01-08 01:40 Figure 30: 21-01-08 02:30 21-01-08 03:20 21-01-08 04:10 21-01-08 05:00 21-01-08 05:50 21-01-08 06:40 21-01-08 07:30 21-01-08 08:20 21-01-08 09:10 21-01-08 10:00 21-01-08 10:50 21-01-08 11:40 21-01-08 14:10 21-01-08 12:30 21-01-08 13:20 Time 21-01-08 15:00 21-01-08 15:50 21-01-08 16:40 21-01-08 17:30 21-01-08 18:20 21-01-08 19:10 21-01-08 20:00 21-01-08 20:50 21-01-08 21:40 21-01-08 22:30 21-01-08 23:20 Number of connections in the course of a day (Monday) Department of Computer Science and Engineering, Indian Institute of Technology, Kharagpur 53

900 800 700 Connection count 600 500 400 300 200 100 0 21-01-08 00:00 21-01-08 06:10 21-01-08 12:20 21-01-08 18:30 22-01-08 00:40 22-01-08 06:50 22-01-08 13:00 22-01-08 19:10 23-01-08 01:20 23-01-08 07:30 23-01-08 13:40 23-01-08 19:50 24-01-08 02:00 24-01-08 08:10 24-01-08 14:20 24-01-08 20:30 25-01-08 02:40 25-01-08 08:50 25-01-08 15:00 25-01-08 20:50 26-01-08 03:00 26-01-08 09:10 26-01-08 15:20 26-01-08 21:30 27-01-08 03:40 27-01-08 09:50 27-01-08 16:00 27-01-08 22:10 Time Figure 31: Connection statistics in the course of a day (Monday) 4500 4000 3500 Packet count 3000 2500 2000 1500 1000 500 0 2-02-08 00:00 2-02-08 01:00 2-02-08 02:00 2-02-08 03:00 2-02-08 04:00 2-02-08 05:00 2-02-08 06:00 2-02-08 07:00 2-02-08 08:00 2-02-08 09:00 2-02-08 10:00 2-02-08 11:00 2-02-08 12:00 2-02-08 13:00 2-02-08 14:00 Time 2-02-08 15:00 2-02-08 16:00 2-02-08 17:00 2-02-08 18:00 2-02-08 19:00 2-02-08 20:00 2-02-08 21:00 2-02-08 22:00 2-02-08 23:00 Figure 32: Traffic statistics in the course of a day ( Saturday ) Department of Computer Science and Engineering, Indian Institute of Technology, Kharagpur 54

2000 1800 1600 1400 Packet count 1200 1000 800 600 400 200 0 27-01-08 00:00 27-01-08 01:00 27-01-08 02:00 27-01-08 03:00 27-01-08 04:00 27-01-08 05:00 27-01-08 06:00 27-01-08 07:00 27-01-08 08:00 27-01-08 09:00 27-01-08 10:00 27-01-08 11:00 27-01-08 12:00 27-01-08 13:00 27-01-08 14:00 Time 27-01-08 15:00 27-01-08 16:00 27-01-08 17:00 27-01-08 18:00 27-01-08 19:00 27-01-08 20:00 27-01-08 21:00 27-01-08 22:00 27-01-08 23:00 Figure 33: Traffic statistics in the course of a day ( Sunday ) 7000 6000 5000 4000 3000 2000 1000 0 21-01-08 00:00 21-01-08 06:00 21-01-08 12:00 21-01-08 18:00 22-01-08 00:00 22-01-08 06:00 22-01-08 12:00 22-01-08 18:00 23-01-08 00:00 23-01-08 06:00 23-01-08 12:00 23-01-08 18:00 24-01-08 00:00 24-01-08 06:00 24-01-08 12:00 24-01-08 18:00 25-01-08 00:00 25-01-08 06:00 25-01-08 12:00 25-01-08 17:40 25-01-08 23:40 26-01-08 05:40 26-01-08 11:40 Packet count 26-01-08 17:40 26-01-08 23:40 27-01-08 05:40 27-01-08 11:40 27-01-08 17:40 27-01-08 23:40 Time Figure 34: Traffic statistics in the course of a week Department of Computer Science and Engineering, Indian Institute of Technology, Kharagpur 55

8000 7000 6000 Packet count 5000 4000 3000 2000 1000 0 19-01-08 10:00 20-01-08 16:20 21-01-08 22:40 23-01-08 05:00 24-01-08 11:20 25-01-08 17:20 26-01-08 23:40 28-01-08 06:00 29-01-08 12:20 30-01-08 18:50 1-02-08 02:00 2-02-08 08:20 3-02-08 14:40 4-02-08 21:00 6-02-08 03:20 7-02-08 09:40 8-02-08 16:00 13-02-08 22:40 15-02-08 05:00 16-02-08 11:30 19-02-08 17:50 26-02-08 00:20 27-02-08 06:40 28-02-08 13:00 2-03-08 19:20 Time Figure 35: Average Traffic statistics in the course of a month 120000 100000 Packet count 80000 60000 40000 20000 0 17-03-08 00:31 17-03-08 01:41 Figure 36: 17-03-08 02:51 17-03-08 04:01 17-03-08 14:32 17-03-08 05:11 17-03-08 06:22 17-03-08 07:32 17-03-08 08:42 17-03-08 09:52 17-03-08 11:02 17-03-08 12:12 17-03-08 13:22 Time 17-03-08 15:42 17-03-08 16:52 17-03-08 18:02 17-03-08 19:12 17-03-08 20:22 17-03-08 21:32 17-03-08 22:42 17-03-08 23:52 Intrusive Traffic statistics in the course of a day (Monday) Department of Computer Science and Engineering, Indian Institute of Technology, Kharagpur 56

140000 120000 100000 Packet count 80000 60000 40000 20000 0 17-03-08 00:01 17-03-08 07:02 17-03-08 14:02 17-03-08 21:02 18-03-08 04:03 18-03-08 11:04 18-03-08 18:05 19-03-08 01:05 19-03-08 08:05 19-03-08 15:05 19-03-08 22:06 20-03-08 05:06 20-03-08 12:07 20-03-08 19:07 21-03-08 02:07 21-03-08 09:08 21-03-08 16:08 21-03-08 23:08 22-03-08 06:08 22-03-08 13:09 22-03-08 20:10 23-03-08 03:10 23-03-08 10:10 23-03-08 17:11 Time Figure 37: Intrusive Traffic statistics in the course of a week 2500 2000 Average TCP Packet count Packet count 1500 1000 Average UDP Packet count 500 Average ICMP Packet count 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 Time Figure 38: Average Traffic statistics in the course of a day (Monday ) Department of Computer Science and Engineering, Indian Institute of Technology, Kharagpur 57

6.2. SCREENSHOTS OF BASE CONSOLE Figure 39: BASE Console displaying the Traffic statistics by protocol Figure 40: BASE console displaying the alerts statistics Department of Computer Science and Engineering, Indian Institute of Technology, Kharagpur 58

Figure 41: BASE console displaying unique alerts Department of Computer Science and Engineering, Indian Institute of Technology, Kharagpur 59

6.3. TYPICAL VALUESOF NETWORK PARAMETERS FOR NORMAL TRAFFIC IN THE TARGET NETWORK 23-01-08 00:00,Wed,395,154,237,4,69,28,36,5,11,2,4,4,47,85,95,4,7,0.43,0.73,0.23,0.38,0.08,0.15,0.01,0.01 23-01-08 00:10,Wed,405,158,243,4,71,28,37,5,11,2,5,5,49,87,97,4,7,0.44,0.75,0.24,0.38,0.08,0.15,0.01,0.01 23-01-08 00:20,Wed,313,122,188,3,55,22,29,4,9,2,4,4,38,67,75,3,6,0.34,0.58,0.21,0.34,0.07,0.13,0.01,0.02 23-01-08 00:30,Wed,290,113,174,3,51,20,27,4,8,2,3,3,35,62,70,3,5,0.32,0.54,0.20,0.32,0.07,0.13,0.02,0.02 23-01-08 00:40,Wed,248,97,149,2,43,17,23,3,7,1,3,3,30,53,60,3,5,0.34,0.58,0.23,0.37,0.08,0.14,0.02,0.03 23-01-08 00:50,Wed,189,74,113,2,33,13,17,2,5,1,2,2,23,41,45,2,3,0.26,0.44,0.20,0.33,0.07,0.13,0.04,0.05 23-01-08 01:00,Wed,208,81,125,2,36,15,19,3,6,1,2,2,25,45,50,2,4,0.28,0.48,0.21,0.34,0.07,0.13,0.03,0.04 23-01-08 01:10,Wed,180,70,108,2,32,13,17,2,6,1,3,3,22,39,43,2,3,0.25,0.42,0.20,0.32,0.07,0.12,0.04,0.05 23-01-08 01:20,Wed,122,48,73,1,24,10,13,2,5,1,2,2,15,26,29,1,3,0.17,0.28,0.16,0.26,0.05,0.09,0.05,0.07 23-01-08 01:30,Wed,105,41,63,1,21,8,11,2,4,1,2,2,13,23,25,1,2,0.14,0.24,0.15,0.24,0.05,0.09,0.06,0.09 23-01-08 01:40,Wed,89,35,53,1,18,7,9,1,4,1,1,1,11,19,21,1,2,0.12,0.21,0.14,0.22,0.05,0.08,0.08,0.11 23-01-08 01:50,Wed,91,35,55,1,18,7,10,1,4,1,1,1,11,20,22,1,2,0.12,0.21,0.14,0.23,0.05,0.08,0.08,0.11 23-01-08 02:00,Wed,79,31,47,1,16,6,8,1,3,1,1,1,9,17,19,1,2,0.43,0.73,0.52,0.84,0.17,0.31,0.02,0.03 23-01-08 02:10,Wed,81,32,49,1,16,6,9,1,3,1,1,1,10,17,19,1,2,0.44,0.75,0.53,0.85,0.17,0.31,0.01,0.02 23-01-08 02:20,Wed,60,23,36,1,12,5,6,1,2,1,1,0,7,13,14,1,1,0.33,0.56,0.46,0.73,0.15,0.27,0.02,0.03 23-01-08 02:30,Wed,55,21,33,1,11,4,6,1,2,1,1,0,7,12,13,1,1,0.30,0.51,0.44,0.70,0.14,0.25,0.02,0.03 23-01-08 02:40,Wed,61,24,37,1,12,5,6,1,2,1,1,0,7,13,15,1,1,0.33,0.57,0.46,0.74,0.15,0.27,0.02,0.03 23-01-08 02:50,Wed,56,22,34,1,11,4,6,1,2,1,1,0,7,12,13,1,1,0.31,0.52,0.44,0.71,0.14,0.26,0.02,0.03 23-01-08 03:00,Wed,42,16,25,0,8,3,4,1,2,1,1,0,5,9,10,1,1,0.23,0.39,0.38,0.61,0.13,0.22,0.03,0.05 23-01-08 03:10,Wed,44,17,26,0,9,4,5,1,2,1,1,0,5,9,11,1,1,0.24,0.41,0.39,0.63,0.13,0.23,0.03,0.04 23-01-08 03:20,Wed,43,17,26,0,9,3,5,1,2,1,1,0,5,24,10,1,1,0.23,0.40,0.24,0.62,0.13,0.23,0.03,0.04 23-01-08 03:30,Wed,40,16,24,0,8,3,4,1,2,1,1,0,5,22,10,1,1,0.44,0.74,0.47,1.20,0.24,0.43,0.07,0.10 23-01-08 03:40,Wed,28,11,17,0,6,2,3,0,1,1,1,0,3,15,7,0,1,0.31,0.52,0.39,1.00,0.20,0.36,0.12,0.17 23-01-08 03:50,Wed,34,13,20,0,7,3,4,1,2,1,1,0,4,19,8,1,1,0.37,0.63,0.43,1.10,0.23,0.40,0.09,0.13 23-01-08 04:00,Wed,32,13,19,0,6,3,3,0,2,1,1,0,4,18,8,1,1,0.36,0.61,0.43,1.11,0.22,0.40,0.10,0.14 23-01-08 04:10,Wed,25,10,15,0,5,2,3,0,1,0,0,0,3,14,6,0,1,0.28,0.48,0.38,0.98,0.20,0.35,0.15,0.21 23-01-08 04:20,Wed,36,14,21,0,7,3,4,1,2,1,1,0,4,20,8,1,1,0.40,0.69,0.45,1.18,0.24,0.42,0.09,0.12 23-01-08 04:30,Wed,39,16,23,0,8,3,4,1,2,1,1,0,5,21,9,1,1,0.44,0.74,0.47,1.22,0.25,0.44,0.08,0.11 23-01-08 04:40,Wed,37,15,22,0,7,3,4,1,2,1,1,0,4,20,9,1,1,0.41,0.70,0.46,1.19,0.24,0.43,0.08,0.12 23-01-08 04:50,Wed,45,18,27,0,9,4,5,1,2,0,1,1,5,25,11,1,1,0.13,0.21,0.13,0.33,0.07,0.12,0.02,0.02 23-01-08 05:00,Wed,49,20,29,0,10,4,4,1,2,0,1,1,6,27,12,1,1,0.14,0.23,0.13,0.34,0.07,0.13,0.01,0.02 23-01-08 05:10,Wed,42,17,25,0,8,3,4,1,2,0,1,1,5,23,10,1,1,0.12,0.20,0.12,0.32,0.06,0.12,0.02,0.02 23-01-08 17:40,Wed,03,42,60,1,72,29,43,1,10,2,4,4,12,57,24,6,9,0.10,0.17,0.07,0.18,0.02,0.03,0.00,0.00 23-01-08 17:50,Wed,91,37,53,1,64,25,37,1,9,2,4,4,11,50,21,5,7,0.09,0.15,0.06,0.17,0.02,0.03,0.00,0.00 23-01-08 18:00,Wed,95,39,55,1,67,27,39,1,9,2,4,4,11,52,22,5,8,0.14,0.24,0.10,0.26,0.03,0.05,0.00,0.00 23-01-08 18:10,Wed,107,44,62,1,75,30,44,1,10,2,4,4,12,59,25,6,9,0.16,0.27,0.11,0.28,0.03,0.05,0.00,0.00 23-01-08 18:20,Wed,82,34,48,1,57,23,34,1,8,2,3,3,10,45,19,5,7,0.12,0.21,0.09,0.24,0.03,0.04,0.00,0.00 23-01-08 18:30,Wed,198,81,115,2,139,55,83,0,19,4,8,8,23,109,46,11,17,0.30,0.51,0.14,0.38,0.04,0.07,0.00,0.00 23-01-08 18:40,Wed,211,87,122,2,53,21,30,1,7,1,3,3,24,127,49,4,6,0.32,0.54,0.14,0.39,0.07,0.12,0.01,0.01 24-01-08 09:00,Thu,2164,844,1298,22,108,32,68,7,13,5,5,3,325,464,519,6,14,0.26,0.44,0.06,0.10,0.05,0.06,0.00,0.00 24-01-08 09:10,Thu,5349,2086,3209,53,267,80,177,10,32,13,13,6,802,1043,1284,16,35,0.64,1.08,0.10,0.15,0.07,0.10,0.00,0.00 24-01-08 09:20,Thu,2896,1129,1738,29,145,43,90,12,17,7,7,3,434,565,695,9,18,0.35,0.59,0.07,0.11,0.05,0.07,0.00,0.00 24-01-08 09:30,Thu,3398,1325,2039,34,170,51,107,12,20,8,8,4,510,663,816,10,21,0.41,0.69,0.08,0.12,0.06,0.08,0.00,0.00 24-01-08 09:40,Thu,4100,1599,2460,41,103,31,59,13,12,5,5,2,615,800,984,6,12,0.49,0.83,0.09,0.13,0.09,0.13,0.00,0.01 24-01-08 09:50,Thu,2954,1152,1772,30,74,22,39,12,9,4,4,2,354,576,709,4,8,0.35,0.60,0.07,0.11,0.07,0.11,0.01,0.01 Department of Computer Science and Engineering, Indian Institute of Technology, Kharagpur 60

6.4. TYPICAL VALUES OF NETWORK PARAMETERS FOR ANOMALOUS TRAFFIC IN THE TARGET NETWORK 12/3/2008 17:11,Wed, 4028,1175,2451,402,290,131,107,52,30,11,13,6,346,148,192,35,30,0.59,0.89,0.39,0.5,0.17,0.25,0.08,0.01 12/3/2008 17:21,Wed, 1454,270,1040,144,271,122,100,49,16,6,7,3,135,46,44,33,27,0.57,0.85,0.39,0.49,0.17,0.25,0.08,0.01 12/3/2008 17:32,Wed, 3686,1370,1928,388,330,149,122,59,48,18,21,10,623,178,224,94,82,0.88,1.32,0.48,0.61,0.22,0.29,0.09,0.04 12/3/2008 17:42,Wed, 7322,2130,4422,770,466,210,172,84,86,32,37,17,1331,253,349,33,29,0.52,0.78,0.37,0.47,0.17,0.22,0.07,0.09 12/3/2008 17:52,Wed,601,7504,3851,2864,790,498,224,184,90,32,12,14,6,1925,442,630,17,14,0.36,0.54,0.31,0.39,0.14,0.19,0.06,0.15 12/3/2008 18:02,Wed, 9143,1857,6324,961,298,134,110,54,2,1,1,0,844,222,304,22,18,0.41,0.61,0.32,0.41,0.15,0.19,0.06,0.12 12/3/2008 18:12,Wed,601,9454,1212,7249,993,282,127,104,51,34,13,15,7,865,149,198,20,16,0.38,0.57,0.31,0.39,0.14,0.19,0.06,0.13 12/3/2008 18:22,Wed, 8045,614,6587,843,273,123,101,49,12,4,5,2,279,84,101,26,22,0.44,0.66,0.33,0.42,0.15,0.2,0.06,0.1 12/3/2008 18:32,Wed, 13065,1149,10543,1372,456,205,169,82,19,7,8,4,638,154,188,73,64,0.76,1.14,0.44,0.55,0.2,0.27,0.08,0.05 12/3/2008 18:42,Wed,601,13086,1669,10042,1376,487,219,180,88,39,14,17,8,758,206,273,42,36,0.57,0.85,0.38,0.48,0.17,0.23,0.07,0.07 12/3/2008 18:52,Wed, 10534,729,8700,1105,415,187,154,75,20,7,9,4,405,97,119,27,23,0.45,0.67,0.33,0.42,0.15,0.2,0.06,0.02 12/3/2008 19:02,Wed, 24285,2408,19323,2554,556,250,206,100,38,14,16,8,1338,284,367,23,19,0.4,0.6,0.35,0.44,0.14,0.19,0.06,0.03 12/3/2008 19:12,Wed, 17342,14452,1065,1825,340,153,126,61,11,4,5,2,6569,1635,2203,16,13,0.33,0.5,0.32,0.4,0.13,0.17,0.05,0.04 12/3/2008 19:22,Wed, 25755,7473,15572,2710,388,175,144,70,11,4,5,2,4152,850,1139,15,12,0.97,1.45,0.93,1.19,0.39,0.51,0.16,0.07 12/3/2008 19:32,Wed, 27803,2970,21909,2924,319,144,118,57,49,18,21,10,1350,345,453,17,14,1.03,1.55,0.96,1.22,0.4,0.53,0.17,0.26 12/3/2008 19:42,Wed, 28156,8169,17024,2963,476,214,176,86,45,17,19,9,4084,945,1245,107,94,2.68,4.01,1.55,1.97,0.65,0.86,0.27,0.06 12/3/2008 19:52,Wed, 22538,1735,18433,2370,447,201,165,80,8,3,3,2,723,207,265,23,19,1.2,1.79,1.03,1.31,0.43,0.57,0.18,0.1 12/3/2008 20:02,Wed,601,32062,9301,19387,3374,431,194,159,78,2,1,1,0,6643,1055,1418,16,13,0.99,1.48,0.94,1.2,0.39,0.52,0.16,0.14 12/3/2008 20:12,Wed, 38772,6810,27882,4080,353,159,131,64,10,4,4,2,3095,773,1038,10,8,0.79,1.18,0.84,1.06,0.35,0.46,0.14,0.19 12/3/2008 20:22,Wed, 42546,2651,35419,4475,20379,9171,7540,3668,10,4,4,2,1326,305,404,8,6,0.67,1,0.78,0.99,0.32,0.43,0.13,0.25 12/3/2008 20:32,Wed, 29981,4164,22662,3155,397,179,147,71,13,5,6,3,2974,477,635,13,10,0.89,1.34,0.9,1.14,0.37,0.49,0.15,0.21 12/3/2008 23:32,Wed, 28394,23662,1743,2989,364,164,135,66,12,4,5,2,1816,2671,3608,352,313,0.78,1.17,0.64,0.82,0.16,0.18,0.06,0.02 12/3/2008 23:42,Wed, 39968,1461,34304,4203,492,221,182,89,8,3,3,2,150,174,223,198,176,0.58,0.87,0.56,0.71,0.14,0.16,0.05,0.03 12/3/2008 23:52,Wed,601,7845,329,6695,822,155,70,57,28,31,11,13,6,40,45,50,140,124,0.98,1.47,1.02,1.3,0.25,0.29,0.09,0.09 13-03-08 00:02,Thu, 392,327,65,0,23,10,9,4,0,0,0,0,48,43,50,97,85,0.81,1.22,0.93,1.18,0.23,0.27,0.08,0.07 13-03-08 00:12,Thu, 0,0,0,0,67,30,25,12,0,0,0,0,0,8,0,133,117,0.95,1.43,1.01,1.28,0.25,0.29,0.09,0.05 13-03-08 00:22,Thu, 9,6,3,0,91,41,34,16,1,0,0,0,0,10,1,226,201,1.19,1.77,1.12,1.43,0.28,0.31,0.1,0.04 13-03-08 00:32,Thu,601,5,4,2,0,91,41,34,16,0,0,0,0,0,10,1,241,214,1.22,1.83,1.14,1.44,0.28,0.32,0.1,0.04 13-03-08 00:42,Thu, 0,0,0,0,43,19,16,8,0,0,0,0,0,9,0,214,190,1.15,1.72,1.11,1.41,0.27,0.31,0.1,0.01 13-03-08 00:52,Thu,601,0,0,0,0,56,25,21,10,0,0,0,0,0,7,0,142,125,0.94,1.4,0.99,1.26,0.25,0.28,0.09,0.01 13-03-08 01:02,Thu,601,0,0,0,0,101,46,38,18,0,0,0,0,0,6,0,93,81,0.75,1.13,0.89,1.14,0.22,0.25,0.08,0.01 13-03-08 01:12,Thu, 7,1,6,0,86,39,32,15,0,0,0,0,0,7,0,121,107,0.87,1.3,0.96,1.22,0.24,0.27,0.08,0.01 13-03-08 01:22,Thu, 34,1,33,0,30,13,11,5,2,1,1,0,0,10,0,227,202,1.17,1.76,1.11,1.41,0.27,0.31,0.1,0.01 13-03-08 01:32,Thu, 14,10,4,0,70,32,26,13,0,0,0,0,1,9,2,164,145,1,1.49,1.02,1.3,0.25,0.28,0.09,0.01 13-03-08 01:42,Thu,601,11,9,2,0,80,36,30,14,0,0,0,0,1,9,1,175,156,1.03,1.54,1.04,1.32,0.26,0.29,0.09,0.01 13-03-08 01:52,Thu,601,0,0,0,0,46,21,17,8,0,0,0,0,0,7,0,117,103,0.42,0.63,0.4,0.51,0.12,0.13,0.04,0.01 13-03-08 02:02,Thu, 0,0,0,0,69,31,26,12,0,0,0,0,0,9,0,215,191,0.57,0.85,0.47,0.59,0.13,0.15,0.05,0 13-03-08 02:12,Thu, 0,0,0,0,57,26,21,10,0,0,0,0,0,14,0,61,53,0.9,1.35,0.59,0.75,0.29,0.33,0.1,0.02 13-03-08 02:22,Thu, 8,6,2,0,59,27,22,11,0,0,0,0,0,22,1,73,64,0.99,1.48,0.61,0.77,0.31,0.35,0.11,0.02 13-03-08 02:33,Thu,601,0,0,0,0,60,27,22,11,0,0,0,0,0,39,0,7,5,1.76,2.64,0.81,1.03,1.06,1.2,0.37,1.04 13-03-08 02:43,Thu, 0,0,0,0,48,22,18,9,0,0,0,0,0,61,0,14,11,2.75,4.12,1.02,1.29,1.32,1.5,0.47,0.53 13-03-08 02:53,Thu,601,39,24,15,0,39,18,14,7,2,1,1,0,1,32,4,5,3,1.32,1.98,0.68,0.87,0.9,1.02,0.32,1.47 13-03-08 03:03,Thu,602,43,20,23,0,80,36,30,14,0,0,0,0,1,19,3,49,43,0.78,1.17,0.53,0.67,0.27,0.3,0.09,0.03 13-03-08 03:13,Thu,601,9,8,1,0,51,23,19,9,4,1,2,1,1,11,1,17,14,0.89,1.33,0.93,1.19,0.4,0.46,0.14,0.13 13-03-08 03:23,Thu, 0,0,0,0,100,45,37,18,0,0,0,0,0,5,0,21,17,0.5,0.74,0.7,0.89,0.21,0.24,0.08,0.06 13-03-08 03:33,Thu, 109,100,9,0,47,21,18,9,2,1,1,0,12,17,15,19,16,0.48,0.72,0.69,0.88,0.21,0.24,0.07,0.06 Department of Computer Science and Engineering, Indian Institute of Technology, Kharagpur 61

7. APPENDIX B 7.1. GLOSSARY OF TECHNICAL TERMS Alert A message generated by IDS whenever it detects an event of interest. An alert typically contains information about the attack or some unusual activity that was detected Anomaly Any significant deviations from the normal behaviour/pattern Attack An intelligent act that is a deliberate attempt (especially in the sense of a method or technique) to evade security services and violate the security policy of a system In other words, an intrusion attempt Event Activity detected by the IDS which may result in an alert. For example, N failed logins in T seconds might indicate a bruteforce login attack False negative occurs if the IDS does not identify an event that is part of an attack as being malicious False positive occurs if the IDS identifies an event that is not part of an attack as being malicious Intrusion Any set of actions that attempt to compromise the confidentiality, integrity or availability of system or network resources. Any intrusion is a consequence of an attack, but not all attacks lead to an intrusion Intrusion Detection System Network Security Monitors computer systems and/or network and analyzes the data for possible hostile attacks originating from external world and also for system misuse or attacks originating from inside the enterprise Protection of Integrity, Availability and Confidentiality of Network Assets and services from associated threats and vulnerabilities so as to maintain the service availability, avoid Department of Computer Science and Engineering, Indian Institute of Technology, Kharagpur 62

financial losses, damage to image, protect personnel, customer and business secrets etc Normalizing Plug-in Promiscuous Mode Sensor Session Removal of unwanted strings from the data to reconstruct the application layer payload E.g. Telnet sessions contains telnet negotiation strings like IAC (Interpret as Command), NOP (No Operation) etc, which can disrupt the signature matching at Detection Engine. These strings need to be normalized before passing them on to Detection Module A plug-in is a piece of code (written to comply with a particular API) which extends the capability of a existing program or tool like snort. Plug-in provide the ability to make snort do new and interesting things without directly modifying the internal architecture. SNORT has three kinds of plug-ins. They are pre-processor plug-in, detection plug-ins, and output plug-ins. Each of these acts at a different point in the detection scheme. The preprocessor plug-ins work on packets before they are passed to the detection engine. The detection plug-ins are employed as part of the rules used to match packets. The output plug-ins work with either the alert messages or the packets to be logged Network Interface card when set in promiscuous mode, not only accepts the packets intended to it but also receives and processes all other packets which are moving around in the network Sensor is a part of the network Intrusion Detection that collects data about activities from data sources, detects events, and forwards them to the analyzer A session is a series of interactions between two communication end points that occur during the span of a single connection. Typically, one end point requests a connection with another specified end point and if that end point replies agreeing to the Department of Computer Science and Engineering, Indian Institute of Technology, Kharagpur 63

connection, the end points take turns exchanging commands and data (talking to each other). The session begins when the connection is established at both ends and terminates when the connection is ended Signature / Pattern based intrusion detection SPAN (Switched Port Analyzer) Spoofing True Negative True Positive Vulnerability Security Policy The intrusion detection system contains a database of known vulnerabilities in the form a sequence of strings. It monitors traffic and seeks a pattern or a signature match SPAN copies incoming and outgoing packets from multiple sources, VLANs or ports, to a single destination port A technique used to gain unauthorized access to computers, whereby the intruder sends messages to a computer with an IP address indicating that the message is coming from a trusted host. To engage in IP spoofing, an attacker must first use a variety of techniques to find an IP address of a trusted host and then modify the packet header so that the packets appear to be coming from the trusted host They occur when no alerts are triggered for events which are not part of an attack(s) They occur when alerts are triggered for events which are part of an attack(s) A flaw or weakness in a system s design, implementation, or operation and management that could be exploited to violate the system s security posture A set of rules and practices that specify or regulate how a system or organization provides security services to protect sensitive and critical system resources Department of Computer Science and Engineering, Indian Institute of Technology, Kharagpur 64

8. APPENDIX C 8.1. ATTACK DESCRIPTION Apache2 ARPpoison DoS attack Fragment overlap attack This attack exploits the inability of some versions of the Apache web server to handle very long HTTP requests. A typical attack contains multiple requests each with thousands of lines and looking something like this: GET / HTTP/1.1 User-Agent: sioux User-Agent: sioux An attacker who has compromised a host on the local network disrupts traffic by listening for ARP-who-has packets and sending forged replies. ARP (address resolution protocol) is used to resolve IP addresses to Ethernet addresses. Thus, the attacker disrupts traffic by misdirecting traffic at the data link layer A denial-of-service attack or distributed denial-of-service attack (DDoS attack) is an attempt to make a computer resource unavailable to its intended users. Although the means to, motives for, and targets of a DoS attack may vary, it generally consists of the concerted, malevolent efforts of a person or persons to prevent an Internet site or service from functioning efficiently or at all, temporarily or indefinitely by choking the network bandwidth, and/or consuming computing resources like memory and CPU. A TCP/IP Fragmentation Attack is possible because IP allows packets to be broken down into fragments for more efficient transport across various media. The TCP packets (and its header) are carried in the IP packet. In this attack the second fragment contains incorrect offset. When packet is reconstructed, the port number will be overwritten Department of Computer Science and Engineering, Indian Institute of Technology, Kharagpur 65

IPsweep Land Mailbomb Neptune Phf attack PoD Smurf TCPreset An IPsweep attack is a surveillance sweep to determine which hosts are listening on a network. This information is useful to an attacker in staging attacks and searching for vulnerable machines This is a Denial of service attack where a remote host is sent a UDP packet with the same source and destination This attack floods a user with thousands of junk emails. This type of attack can be detected by the fact that the SMTP mail command is lowercase. It is normally uppercase but not required to be Floods the target machine with SYN requests on one or more ports, thus causing Denial of service The Phf attack abuses a badly written CGI script to execute commands with the privilege level of the http server. Any CGI program which relies on the CGI function escape_shell_cmd() to prevent exploitation of shell-based library calls may be vulnerable to attack. In particular, this vulnerability is manifested by the "phf" program that is distributed with the example code for the Apache web server This attack, also known as ping of death, crashes some older operating system by sending an oversize fragmented IP packet that reassembles to more than 65,535 bytes, the maximum allowed by the IP protocol. It is called ping of death because some older versions of Windows 95 could be used to launch the attack using ping -l 65510 This is a distributed network flooding attack initiated by sending ICMP ECHO REQUEST packets to a broadcast address with the spoofed source address of the target. The target is then flooded with ECHO REPLY packets from every host on the broadcast address This attack listens for TCP SYN packets on a compromised host on the local network and immediately sends a spoofed RST Department of Computer Science and Engineering, Indian Institute of Technology, Kharagpur 66

(connection refused) packet, disrupting traffic Teardrop UDPstorm This attack reboots the Linux host by sending a fragmented IP packet that cannot be reassembled because of a gap between the fragments An attacker floods the local network by setting up a loop between an echo server and a Client machine or another echo server by sending a UDP packet to one server with the spoofed source address of the other Department of Computer Science and Engineering, Indian Institute of Technology, Kharagpur 67

9. APPENDIX D 9.1. THE TCP/IP PROTOCOL STACK Source : http://www.tcpipguide.com/free/t_datalinklayertechnologiesandprotocols.htm Department of Computer Science and Engineering, Indian Institute of Technology, Kharagpur 68

9.2. IP HEADER Source: http://www.visi.com/~mjb/drawings/ Department of Computer Science and Engineering, Indian Institute of Technology, Kharagpur 69

9.3. TCP HEADER Source: http://www.visi.com/~mjb/drawings/ Department of Computer Science and Engineering, Indian Institute of Technology, Kharagpur 70

9.4. UDP HEADER 9.5. ICMP HEADER Source: http://www.visi.com/~mjb/drawings/ Department of Computer Science and Engineering, Indian Institute of Technology, Kharagpur 71

9.6. TCP CONNECTION ESTABLISHMENT Source: http://www.tcpipguide.com/free/t_datalinklayertechnologiesandprotocols.htm Department of Computer Science and Engineering, Indian Institute of Technology, Kharagpur 72

9.7. TCP CONNECTION TERMINATION Source : http://www.tcpipguide.com/free/t_datalinklayertechnologiesandprotocols.htm Department of Computer Science and Engineering, Indian Institute of Technology, Kharagpur 73