Defending Against Distributed Denial of Service Attacks

Transcription

1 Defending Against Distributed Denial of Service Attacks By Tao Peng A thesis submitted to the University of Melbourne in total fullfillment for the degree of Doctor of Philosophy Department of Electrical and Electronic Engineering April 2004

2 ii c Tao Peng, Produced in L A TEX 2ε.

3 Abstract The Denial of Service attack, especially the Distributed Denial of Service (DDoS) attack, has become one of the major threats to the Internet. Generally, attackers launch DDoS attacks by directing a massive number of attack sources to send useless traffic to the victim. The victim s services are disrupted when its host or network resources are occupied by the attack traffic. The threat of DDoS attacks has become even more severe as attackers can compromise a huge number of computers by spreading a computer worm using vulnerabilities in popular operating systems. This thesis investigates DoS attacks (including DDoS attacks), and is divided into three parts. In the first part, we categorize existing defense mechanisms, and analyze their strengths and weaknesses. In particular, we design a countermeasure for each defense mechanism from the attacker s point of view. In the second part of our investigation, we develop and evaluate three defense models for DoS attacks: the Victim Model, the Victim-Router Model, and the Router- Router Model. Each of these models provides defense in a different part of the network, and has different resource requirements. The Victim Model provides defense at the target or victim of an attack. We develop a novel technique for identifying attack traffic based on the connection history at the victim. We then present a history-based IP filtering algorithm to filter attack traffic in an accurate and efficient manner. A key advantage of iii

4 this technique is that it can filter attack traffic while allowing the majority of normal traffic to reach the service under attack. The Victim-Router Model uses cooperation between the victim and its upstream routers to locate attack sources and filter attack traffic close to its source. We propose a novel method for tracing the attack path called adjusted probabilistic packet marking. In contrast to previous packet marking schemes, we can minimize the number of packets needed to trace the attack path. We also present a selective pushback scheme that uses the path information provided by packet marking in order to filter attack traffic close to its source. The Router-Router Model is a distributed defense architecture that can detect attack traffic close to its source. This model is based on a cooperative scheme in which routers can efficiently share evidence of attacks. In order to minimize the communication overhead of this approach, we apply a machine learning technique to decide when to share evidence between routers. A major advantage of this scheme is that it can detect and filter highly distributed attacks before the malicious traffic can congest the network. In each case, we use both analytical results and simulations based on real-life packet traces in order to demonstrate the effectiveness of our models. In the third part of our investigation, we assess the effectiveness of our defense models for different types of DoS attack. We categorize existing DoS attacks, and evaluate the advantages and disadvantages of our defense models in comparison to existing defense techniques. Finally, we demonstrate how our three defense models complement each other, and can be integrated into a robust solution for DoS attacks. iv

5 v Declaration This is to certify that (i) the thesis comprises only my original work towards the PhD, (ii) due acknowledgement has been made in the text to all other material used, (iii) the thesis is less than 100,000 words in length, exclusive of tables, maps, bibliographies and appendices. Tao Peng

6 vi

7 Acknowledgements I would like to thank my supervisors, Dr. Chris Leckie and Professor Rao Kotagiri for their many suggestions, generous help and constant support during my Ph.D research. I also want to thank Chen Zhenzhong for introducing me to the University of Melbourne, and Dr. Steven Low for encouraging me to pursue my study in the University of Melbourne. I feel grateful for the support I have received from the University of Melbourne through both MIRS and MIFRS scholarships. I also appreciate the ARC Special Research Center for Ultra-Broadband Information Networks (CUBIN) for sponsoring my conference trips. I feel extremely grateful for the strong and selfless support from my parents and my dear sister Peng Xiaowen. Without the sacrifices from my parents, I could never have the opportunity to undertake this study. It is my sister who devoted her encouragement, trust, and love to me during the most difficult time of my Ph.D study, which I will never forget. I also want to thank the Waikato Applied Network Dynamics Research Group for making their traces publicly available and Dr. Hai Vu for his comments to this thesis. Finally, I wish to thank Laurence, Malcolm, Alistair, Rami, Marija, Nicolas, Jolyon, Boon, John, Jun, Andrew, Bartek, Brian for their help and making CUBIN such a fun place. vii

8 viii

9 Contents Abstract Acknowledgements List of Figures List of Tables iii vii xv xxi 1 Introduction Background and Problem Statement The Internet Denial of Service (DoS) Attacks Technical Problems Research Objective Basic Concepts Source, Router and Victim Definition of Attacks Flow, Traffic Aggregate and Internet Protocols Firewall and IDS False Positive Rate and Detection Accuracy Defense Mechanism ix

10 x Contents Victim Model (VM) Victim-Router Model (VRM) Router-Router Model (RRM) Overview of the Thesis Contributions of the Thesis List of Publications A Survey of DoS Attacks Introduction Bandwidth Attacks Impacts of the Bandwidth Attack Inherent Vulnerabilities of the Internet Architecture Typical Bandwidth Attacks Existing DoS Attack Defense Proposals Attack Prevention Attack Detection Attack Source Identification Attack Reaction Summary History-based Attack Detection and Reaction Introduction Background Problem Definition Detecting Attacks in the Vicinity of the Attack Source Detecting Attacks Quickly in the Victim s Network Reacting to the Detected DDoS Attacks Motivation for History-based Attack Detection and Reaction

11 Contents xi Stopping DoS Attacks Using Traffic Control Algorithms Intrinsic Attack Feature Attack Detection and Reaction Our Solution: History-based Attack Detection and Reaction Overview of the HADR Scheme New Address Detection Engine Flow Rate Detection Engine Decision Engine Filtering Engine: History-based IP Filtering Placement of the HADR HADR Design The Choice of Detection Feature: New IP Addresses IP Address Database Design Abrupt Change Detection Hash Techniques An Example of IP Address Database Design Normal Traffic Behavior Consistency of the IP Addresses How to Build an Efficient IP Address Database Performance Evaluation DDoS Detection Using the New Address Detection Engine DDoS Detection Using the Flow Rate Detection Engine Performance of the History-based IP Filtering Complexity of Using History-based IP Filtering in Routers Attacks Against the HADR Infiltrating Attacks Countermeasures Against Sophisticated Infiltrating Attacks.. 134

12 xii Contents DDoS Attacks from Infiltrated Sources Discussion Conclusion Adjusted Probabilistic Packet Marking Introduction Motivation Background on Probabilistic Packet Marking (PPM) Passive Victim-Router Model: APPM Number of Hops Traversed by Packet (d 1 ) Number of Hops Traversed Since the Packet was Last Marked (d 2 ) Number of Hops from Current Router to Destination (d 3 ) Summary Evaluation for APPM Methodology Results Discussion APPM Against DDoS Attacks APPM and Marking Field Spoofing Conclusion Selective Pushback Introduction Background Definitions Limitation of Router-based Pushback Selective Pushback

13 Contents xiii Overview of Selective Pushback An Example of Selective Pushback Evaluation for Selective Pushback Simulation Methodology Results Discussion Implementation Overhead for Selective Pushback Other Related Issues Conclusion Distributed Detection by Sharing Combined Beliefs Introduction Problem Definition Distributed Denial of Service Attacks Reflector Attacks Motivation Motivation for Distributed DDoS Attack Detection Motivation for Distributed Reflector Attack Detection Methodology Detecting Abnormal Network Behavior Combining Beliefs for Attack Detection Learning When to Broadcast the Warning Message Evaluation Evaluation for DDoS Attack Detection Evaluation for Reflector Attack Detection Discussion Conclusion

14 xiv Contents 7 Analysis of DoS Defense Schemes Introduction Challenges for DoS Defense Schemes The Tragedy of the Commons The Power of Many Versus the Strength of Few Implementation Cost DoS Attack Category Victim Type The Parameters of Attack Power Average Flow Rate and Number of Flows Attack Traffic Rate Dynamics Impact of Attack Comparison Between Our Defense Models How to Use Our Defense Models DoS Attacks Versus Our Defense Models Integrate the VM with the VRM Other Related Issues Computer Crime Laws IP Version Conclusion Conclusion 235 Appendix A: Abbreviations and Glossary of terms 239 References 243 Index 256

15 List of Figures 1.1 The number of Internet security incidents reported from 1988 to 2003 (The data is collected from CERT [1]) A simple model of the Internet The relation of different types of attacks The number of vulnerabilities reported each year according to CERT [1] TCP 3-Way Handshake The UDP flooding is initiated by a single packet A smurf attack, using an intermediary network to amplify a Ping flood Structure of a typical DDoS attack (based on [2]) Structure of a distributed reflector denial of service (DRDoS) attack (based on [2]) Defending against IP source address spoofing using ingress filtering Router-based packet filtering An example of SAVE message update A model of DoS attack reaction schemes Intermediate network reaction: controller-agent scheme Basic SOS architecture The architecture of History-based Attack Detection and Reaction.. 75 xv

16 xvi List of Figures 3.2 The hash table for the detection engine The placement of the HADR scheme Effect of choice of detection feature on detecting the occurrence of an attack The sampling intervals for History-based Attack Detection and Reaction New Address Detection Engine algorithm Illustration of the CUSUM algorithm CUSUM algorithm For each IP packet, the Bloom filter computes k independent N-bit digests of the 32-bit source IP address, and sets the corresponding bits in the 2 N -bit table Number of IP addresses that appeared in at least d days Distribution of IP addresses that generated at least u packets The trace-driven simulation experiment Auck-IV-in Trace: the ratio of new IP addresses calculated in the time intervals of 10 seconds for each packet trace Auck-IV-out Trace: the ratio of new IP addresses calculated in the time intervals of 10 seconds for each packet trace Bell-I Trace: the ratio of new IP addresses calculated in the time intervals of 10 seconds for each packet trace Auck-IV-in Trace: CUSUM test statistics under normal operation Auck-IV-out Trace: CUSUM test statistics under normal operation Bell-I Trace: CUSUM test statistics under normal operation DARPA Dataset: DDoS Attack Scenario The DDoS attack detection sensitivity in the first-mile router using the Auck-IV-out trace: attacks with 10 new IP addresses

17 List of Figures xvii 3.21 The DDoS attack detection sensitivity in the first-mile router using the Auck-IV-out trace: attacks with 4 new IP addresses The DDoS attack detection sensitivity in the first-mile router using the Auck-IV-out trace: attacks with 2 new IP addresses The DDoS attack detection sensitivity for the last-mile router using the Auck-IV-in trace: attacks with 200 new IP addresses The DDoS attack detection sensitivity for the last-mile router using the Auck-IV-in trace: attacks with 40 new IP addresses The DDoS attack detection sensitivity for the last-mile router using the Auck-IV-in trace: attacks with 18 new IP addresses The flow rate distribution of the Auck-IV-in Trace Detection thresholds for the flow with source IP address DDoS attacks on a two-dimensional attack detection space The filtering accuracy of History-based IP Filtering Accuracy of the combined rule F c = p 1 (d) p 2 (u) on the Auckland Trace Memory requirements for the IP Address Database on the Auckland trace The relation between the IP Address Database and the source IP addresses of attack traffic Probabilistic Packet Marking Definition of different distance measures Adjusted Probabilistic Packet Marking Scheme One Adjusted Probabilistic Packet Marking Scheme Two Adjusted Probabilistic Packet Marking Scheme Three APPM Schemes 1, 2, and 3 compared with uniform marking probability p=0.01, 0.04, and

18 xviii List of Figures 4.7 Effect of Spoofing the Marking Field (Fake sub-path: v f1 to v f3, true path: v 1 to v 3 ) Router map showing attack traffic in bold An example of the Selective Pushback scheme Simulation topology The traffic distribution of router R Detecting a single attack source between 2AM and 3AM Detecting a single attack source between 11AM and 12PM Detecting a DDoS attack with one of the 6 distributed attack sources at router R3.0 between 2AM and 3AM Detecting a DDoS attack with one of 6 distributed attacks sources at router R3.0 between 11AM and 12PM Detecting a DDoS attack with one of the 6 distributed attack sources at router R3.6 between 2AM and 3AM Detecting a DDoS attack with one of 6 distributed attacks sources at router R3.6 between 11AM and 12PM Detecting a DDoS attack with one of the 6 distributed attack sources at router R3.3 between 2AM and 3AM Detecting a DDoS attack with one of 6 distributed attacks sources at router R3.3 between 11AM and 12PM A simple topology to show the advantage of Selective Pushback over Router-based Pushback The challenge of first-mile HADR detection Overview of detecting reflector attacks Combining Beliefs for DDoS Attack Detection The algorithm for learning when to broadcast the warning message.. 205

19 List of Figures xix 6.5 Performance of decision functions of the Router-Router Model for DDoS attack detection Convergence of the broadcast threshold optimization in the Router- Router Model for DDoS attack detection The CUSUM statistics for L in distributed reflector attack detection Performance of decision functions of the Router-Router Model for reflector attack detection Convergence of the broadcast threshold optimization in the Router- Router Model for reflector attack detection The accuracy of distributed detection of reflector attacks Categorization of DoS attacks according to victim type Categorization of DDoS attacks according to the parameters of attack power The architecture for combining the Victim Model and the Victim- Router Model

20 xx List of Figures

21 List of Tables 2.1 Comparison between bandwidth attacks and flash crowds Basic assumptions for different attack detection techniques Percentage of IP addresses in a single day that have previously appeared in the past fortnight Summary of the packet traces used for testing Detection performance of the first-mile router Detection performance of the last-mile router The rule for the decision engine The false positive rate for router R3.0, R3.6, and R3.3 when there are 6 uniformly distributed attack sources The detection performance of our scheme against DDoS attacks with different number of attack sources Comparison between our defense models Summary: DoS attacks versus DoS defense models xxi

22 xxii List of Tables

23 Chapter 1 Introduction The Internet was initially designed for openness and scalability. The infrastructure is certainly working as envisioned by that yardstick. However, the price of this success has been poor security. On the Internet, anyone can send any packet to anyone without being authenticated, while the receiver has to process any packet that arrives to a provided service. The lack of authentication means that attackers can create a fake identity, and send malicious traffic with impunity. All systems connected to the Internet are potential targets for attacks since the openness of the Internet makes them accessible to attack traffic. A Denial of Service (DoS) attack aims to stop the service provided by a target. It can be launched in two forms. The first form is to exploit software vulnerabilities of a target by sending malformed packets and crash the system. The second form is to use massive volumes of useless traffic to occupy all the resources that could service legitimate traffic. While it is possible to protect the first form of attack by patching known vulnerabilities, the second form of attack cannot be so easily prevented. The targets can be attacked simply because they are connected to the public Internet. When the traffic of a DoS attack comes from multiple sources, we call it a Distributed Denial of Service (DDoS) attack. By using multiple attack sources, the power of a DDoS attack is amplified and the problem 1

24 2 Introduction of defense is made more complicated. This thesis presents several techniques for defending against DDoS attacks, and evaluates their effectiveness against a variety of DDoS attacks. 1.1 Background and Problem Statement The Internet The Internet (originally known as ARPANET) was first created in 1969 as a research network sponsored by the Advanced Research Projects Agency (ARPA) of the Department of Defense (DoD) in the United States of America. The original aim was to provide an open network for researchers to share their research resources [3]. Therefore, openness and growth of the network were the design priorities while security issues less of a concern [3]. The occurrence of the Morris Worm [4] in 1988 marked the first major computer security incident on the Internet. However, the world was not so dependent on the Internet as it is now. The Internet was still limited to research and educational communities until the late 1990s. Hence, not much attention was paid to Internet security. In the last decade, the phenomenal growth and success of the Internet is changing its traditional role. The Internet is no longer just a tool for the researchers. It has become the main infrastructure of the global information society. Governments use the Internet to provide information to the citizens and the world at large, and they will increasingly use the Internet to provide government services. Companies share and exchange information with their divisions, suppliers, partners and customers efficiently and seamlessly. Research and educational institutes depend more on the Internet as a platform for collaboration and as a medium for disseminating their research discoveries rapidly. Unfortunately, with the growth of the Internet, the

25 1.1 Background and Problem Statement 3 16 x Estimated number of incidents reported in 2003 is 153,140 Number of incidents reported Number of incidents reported in 1988 was 6 Number of incidents reported in 2002 was 82, Year Figure 1.1: The number of Internet security incidents reported from 1988 to 2003 (The data is collected from CERT [1]). attacks to the Internet have also increased incredibly fast. According to CERT [1], a center of Internet security expertise located in the U.S., the number of reported Internet security incidents has jumped from 6 in 1988 to 82,094 in 2002, and the estimated number of Internet security incidents in 2003 is 153,140. The growth in the number of incidents reported between 1998 to 2003 is shown in Figure 1.1. More importantly, traditional operations in essential services, such as banking, transportation, power, medicine, and defense are being progressively replaced by cheaper, more efficient Internet-based applications. Historically, an attack to a nation s critical services involves actions that need to cross a physical boundary. These actions can be intercepted and prevented by a nation s security services. However, the

26 4 Introduction global connectivity of the Internet renders physical boundaries meaningless. Internetbased attacks can be launched anywhere in the world, and unfortunately no Internetbased services are immune from these attacks. Therefore, the reliability and security of the Internet not only benefits on-line businesses, but is also an issue for national security Denial of Service (DoS) Attacks A DoS attack is a malicious attempt by a single person or a group of people to disrupt an online service. DoS attacks can be launched against both services, e.g., a web server, and networks, e.g., the network connection to a server. The impact of DoS attacks can vary from minor inconvenience to users of a website, to serious financial losses for companies that rely on their on-line availability to do business. On February 9, 2000, Yahoo, ebay, Amazon.com, E*Trade, ZDnet, Buy.com, the FBI, and several other Web sites fell victim to DoS attacks resulting in substantial damage and inconvenience [5]. As emergency and essential services become reliant on the Internet as part of their communication infrastructure, the consequences of DoS attacks could even become life-threatening. Hence, it is crucial to deter, or otherwise minimize, the damage caused by DoS attacks Technical Problems There are four different ways to defend against DoS attacks: (1) attack prevention; (2) attack detection; (3) attack source identification; and (4) attack reaction. Attack prevention aims to fix security holes, such as insecure protocols, weak authentication schemes and vulnerable computer systems, which can be used as stepping stones to launch a DoS attack. This approach aims to improve the global security level and is the best solution to DoS attacks in theory. However, the disadvantage is that it needs

27 1.2 Research Objective 5 global cooperation to ensure its effectiveness, which is extremely difficult in reality. Hence, the challenge is how to develop a scalable mechanism with low implementation cost. Attack detection aims to detect DoS attacks in the process of an attack. Attack detection is an important procedure to direct any further action. The challenge is how to detect every attack quickly without misclassifying any legitimate traffic. Attack source identification aims to locate the attack sources regardless of the spoofed source IP addresses. It is a crucial step to minimize the attack damage and provide deterrence to potential attackers. The challenge for attack source identification is how to locate attack sources quickly and accurately without changing current Internet infrastructure. Attack reaction aims to eliminate or curtail the effects of an attack. It is the final step in defending against DoS attacks, and therefore determines the overall performance of the defense mechanism. The challenge for attack reaction is how to filter the attack traffic without disturbing legitimate traffic. 1.2 Research Objective The objective of this research is to develop practical and scalable mechanisms to detect and react to DoS attacks. These defense mechanisms should detect the DoS attack quickly and accurately, ensure reasonable performance for the networks or systems under attack, and track the attack sources accurately with low computational overhead. Our research also includes a classification of different defense models according to their implementation cost and cooperation levels. We investigate the strengths and weaknesses of each model, and provide extensive analysis of the methods for DoS attack defense.

28 6 Introduction 1.3 Basic Concepts Figure 1.2: A simple model of the Internet. In this section, we introduce some basic concepts on which this thesis is based Source, Router and Victim We define a source as a device that can generate Internet traffic. The source could be a university s mail server, a company s web server or a home PC connected to the Internet. When the source is used to generate attack traffic, it becomes an attack source. In the rest of the thesis, unless otherwise stated, source refers to attack source. We define a third party as a device that is not compromised but is used by an attacker to generate attack traffic without notice. We define a victim as a system that provides an Internet service and whose service is disrupted during an attack. We define a target as a system that is being attacked or will be attacked by an attacker. If the services of a target are damaged during an attack, then the target becomes a victim. The victim could be a government s web server, a regional DNS server or an ISP s router. Depending on actual network conditions, a connected device could be a source or a victim or both. The end host can be defined as a device that connects to the end of the Internet. We use the term edge router to refer to the router that provides access to the Internet

29 1.3 Basic Concepts 7 for the subnetwork that we are defending. For incoming traffic, the edge router can be described as the last-mile router. For outgoing traffic, the edge router can be described as the first-mile router. We define a user s upstream routers as the routers that connect the user to the Internet. Given two routers A and B, if A is B s upstream router, then B is A s downstream router. These definitions are illustrated in Figure Definition of Attacks In general, a denial of service (DoS) attack is any attack which makes an on-line service (e.g., Web Service) unavailable. The attack could involve a single packet (e.g., the land attack [6]) exploiting software bugs in a server, or a traffic stream with a tremendous number of packets that congest the target s server or network. We define a bandwidth attack as any attack that consumes a target s resources through a massive traffic volume. In this thesis, we focus on bandwidth attacks, and henceforth we mean bandwidth attack when we refer to denial of service attacks unless otherwise stated. The distributed denial of service (DDoS) attack is a bandwidth attack whose attack traffic comes from multiple sources. To launch a DDoS attack, an attacker usually compromises many insecure computers connected to the Internet first. Then a DDoS attack is launched from these compromised computers. The reflector attack is an attack where innocent third-parties (reflectors) are used to bounce attack traffic from the attacker to the target. A reflector can be any network device that responds to any incoming packet, for example, a web server. The attacker can make the attack traffic highly distributed by using many reflectors. The reflector attack is a type of DDoS attack. To summarize, the relations between different types of attacks are illustrated in Figure 1.3.

30 8 Introduction Figure 1.3: The relation of different types of attacks Flow, Traffic Aggregate and Internet Protocols We define the IP flow as a sequence of packets with the same source IP address. We define the traffic aggregate as a group of IP flows that share the same feature, for example, the same destination address. We define the Internet protocols as all the protocols that are used in the Internet, such as IP, TCP and UDP Firewall and IDS Firewall is an access control device that admits incoming traffic according to a set of rules. An Intrusion Detection System (IDS) is a traffic monitoring system that analyzes the network traffic and reports any suspicious network behavior False Positive Rate and Detection Accuracy We define a false positive as a normal operation that is misdiagnosed by the detection scheme as an attack. The false positive rate is defined as the number of false positives

31 1.4 Defense Mechanism 9 divided by the total number of detection decisions made. We define a false negative as an attack that has not been detected by the detection scheme. The false negative rate is defined as the number of false negatives divided by the total number of detection decisions made. We define the detection accuracy as the number of attacks detected divided by the total number of attacks. 1.4 Defense Mechanism The defense mechanisms taken by this thesis can be categorized into three basic models: (1) the Victim Model (VM), (2) the Victim-Router Model (VRM), (3) the Router-Router Model (RRM). Each model is classified according to where the defense mechanism is employed, and how the network components, such as the victim and router, cooperate together Victim Model (VM) The VM is a traditional defense model that identifies and filters attack traffic at a single location, namely, the victim. The key issue of the VM is to be able to identify the attack traffic pattern accurately and efficiently. Our proposal consists of a detection agent and a packet filter that are based on the analysis of the connection history to the victim. We define a new IP address as an IP address that does not appear in the target s connection history. The premise of our approach is that attack traffic is most likely to contain new IP addresses. Therefore, we identify the attack traffic by checking whether it has many new IP addresses. The high proportion of new IP addresses indicates an attack. Once a DoS attack is detected, the VM filters the traffic that contains new IP addresses. The purpose of the research for this model is to demonstrate a stand-alone defense mechanism that does not require cooperation with other network systems.

32 10 Introduction Victim-Router Model (VRM) The VRM is a cooperative model that identifies and filters the attack traffic at multiple locations. The defense process is triggered by the signal from a victim and accomplished with the cooperation from participating routers. There are two key points for this model. The first is that the victim identifies the attack source using information inserted by the upstream routers. The second is that the victim directs the routers close to the attack sources to filter attack traffic. To implement this model, routers need to run a lightweight packet marking process to include path information into the packets. In addition, the victim needs to analyze the incoming packets to locate the attack sources. Once the victim identifies the attack sources, control messages will be sent to the routers that are adjacent to the attack sources. The routers will then start to filter attack traffic according to the received control messages. The objective of this research is to investigate the feasibility of a victim-based defense architecture that can be implemented incrementally Router-Router Model (RRM) The RRM is a distributed defense model that detects the attack traffic by sharing information among participating routers. The ultimate goal of DDoS defense is to filter attack traffic close to the attack sources so that both network and server resources will be saved. Therefore, routers close to attack sources should be able to identify attack traffic quickly and accurately. However, if the attack sources for a DDoS attack are highly distributed (for instance, a reflector attack), little attack traffic will be observed by a single router. Due to the scarce attack traffic, it is nearly impossible for an individual router to detect an attack, and an effective solution must be formulated with a distributed approach. In the RRM model, each router reports any suspicious network behavior to other routers. At the same time, each router

33 1.5 Overview of the Thesis 11 combines the reports from other routers with network statistics observed locally to decide whether an attack has happened. The aim of this research is to study how distributed detection can improve the detection accuracy and reduce the detection time. Each of these three models has its own trade-off between defense performance and implementation overhead. The defense performance includes detection time, detection accuracy, and what proportion of resources are protected at the victim. The implementation overhead includes computational overhead, level of manual intervention, and cost of deployment. The VM provides a short-term and basic solution to DDoS attacks. Its defense scheme can be extended by adding the VRM and RRM. The VRM and RRM can be regarded as long-term solutions that need cooperation from multiple network devices, such as upstream routers. In an ideal situation, all three models can be integrated to achieve better performance. 1.5 Overview of the Thesis This thesis is composed of eight chapters. Chapter 1 introduces the problem of the DoS attack problem and defines some basic concepts related to the DoS attack defense. Chapter 2 gives a detailed literature review of DoS attack prevention, detection and reaction. Chapter 3 discusses our Victim Model (VM), where we present our history-based scheme to detect attacks and filter the attack traffic. Chapters 4 and 5 discuss our Victim-Router Model (VRM), where Chapter 4 gives a traceback scheme to locate the attack sources and Chapter 5 presents a selective pushback scheme to filter the attack traffic close to the attack sources. Chapter 6 discusses our Router- Router Model (RRM), where we present a distributed detection scheme, as well as a machine learning approach for sharing information between distributed detection systems. Chapter 7 gives a detailed analysis of all three models, and discusses how

34 12 Introduction to combine these models to provide an integrated solution to DoS attacks. Chapter 8 concludes the thesis by presenting a summary of our proposed defense mechanisms. All the abbreviations and glossary terms are listed in the Appendix. 1.6 Contributions of the Thesis In this thesis, the following contributions have been made. Chapter 2 We categorize the existing solutions to DoS attacks according to their operation. We highlight the limitations of each defense mechanism and formulate a set of attacks against each scheme. Chapter 3 We propose an independent defense model that can detect and filter attack traffic locally. We propose a novel attack detection model based on the connection history at the victim. We use trace-driven simulations to prove the efficiency and accuracy of this defense model. We propose a history-based IP filtering algorithm to filter attack traffic accurately with low computational overhead. The results of this chapter are presented in [7] and [8].

35 1.6 Contributions of the Thesis 13 Chapter 4 We propose an adjusted probabilistic packet marking (APPM) scheme to identify the IP sources reliably and efficiently. We propose three different methods for implementing our APPM scheme. The results of this chapter are presented in [9]. Chapter5 We propose a network defense architecture to detect the attack at the victim and filter attack traffic at the routers close to attack sources. By analyzing the variance of the traffic distribution, our defense model can quickly and accurately detect DoS attacks and locate the attack sources. We conduct extensive simulations to verify the analytical results of our model and gain further insight into its operation. The results of this chapter are presented in [10]. Chapter 6 We propose a distributed detection architecture that can detect DDoS attacks close to the attack sources and apply a machine learning scheme to improve the performance of the distributed approach. We analyze the traffic characteristics of reflector attacks, and propose a distributed approach to detect them. The results of this chapter are presented in [8] and [11].

36 14 Introduction Chapter 7 We compare the strengths of each model and investigate the feasibility of combining these three models together to provide an integrated solution. We analyze the limits of our proposed DDoS defense approaches and their effectivenesses against different types of DoS attacks. 1.7 List of Publications The following list of publications were generated in the course of conducting the research that contributed to this thesis. Published Papers T. Peng, C. Leckie, and K. Ramamohanarao. Adjusted probabilistic packet marking for IP traceback. In Proceedings of the Second IFIP Networking Conference (Networking 2002), pp (Pisa, Italy, 2002). T. Peng, C. Leckie, and K. Ramamohanarao. Defending against distributed denial of service attack using selective pushback. In Proceedings of 9th IEEE International Conference on Telecommunications (ICT 2002), pp (Beijing, China, 2002). T. Peng, C. Leckie, and K. Ramamohanarao. Prevention from distributed denial of service attacks using history-based IP filtering. In Proceeding of 38th IEEE International Conference on Communications (ICC 2003), pp (Anchorage, Alaska, USA, 2003) T. Peng, C. Leckie, and K. Ramamohanarao. Detecting distributed denial of service attacks by sharing distributed beliefs. In Proceedings of 8th Australasian

37 1.7 List of Publications 15 Conference on Information Security and Privacy (ACISP 2003), pp (Wollongong, Australia, 2003). T. Peng, C. Leckie, and K. Ramamohanarao. Detecting reflector attacks by sharing beliefs. In Proceedings of IEEE 2003 Global Communications Conference (Globecom 2003) (San Francisco, California, USA, 2003). Other Papers Under Preparation T. Peng, C. Leckie, and K. Ramamohanarao. Proactively detecting DDoS attack using source IP address monitoring. (to appear in Networking 2004, Athens, Greece.). T. Peng, C. Leckie, and K. Ramamohanarao. Information Sharing for Distributed Intrusion Detection Systems. (submitted to the Journal of Computer Communications) T. Peng, C. Leckie, and K. Ramamohanarao. A survey on DDoS defense mechanisms. (in preparation, to be submitted to ACM Computing Survey) T. Peng, C. Leckie, and K. Ramamohanarao. Victim-Router Mode DDoS Defense Mechanism. (in preparation, to be submitted to IEEE/ACM Transactions on Networking)

38 16 Introduction

39 Chapter 2 A Survey of DoS Attacks This chapter presents a survey of denial of service attacks. In this survey, we analyze the fundamental weaknesses of the Internet in terms of its vulnerabilities for denial of service attacks. We review the proposed methods for defending against denial of service attacks, discuss the strengths and weaknesses of each proposal, and present countermeasures that an attacker may employ to defeat the protection provided by each proposal. 2.1 Introduction Network intrusion has been a growing concern since the invention of the Internet. Problems such as viruses, worms, and hackers are widely reported [12]. Although there is no clear definition of computer and network intrusions, we define two main categories according to the attacker s motivations, namely, unauthorized access and denial of service attacks. Unauthorized access can take a number of forms, which include the user-to-root attack, the remote-to-local attack, and the scan attack [13]. The user-to-root attack occurs when a normal user gains privileged (root) access to a computer by exploiting 17

40 18 A Survey of DoS Attacks a vulnerability of either the operation system or the installed software. A classic example occurred in the early UNIX systems using the finger daemon [14]. The finger daemon neglected to limit of the size of an input string, which has a potential risk of causing buffer overflow. Since the finger daemon has root privileges, by carefully designing a special input string, the attacker is able to exploit the buffer overflow to execute any command as root. The remote-to-local attack occurs when a user of a remote system gains local access to the computer by exploiting a vulnerability of the system. For example, the code red worm gained access to the Microsoft IIS server by sending a malicious HTTP GET request [15]. The scan attack occurs when an attacker makes a reconnaissance of the target network, such as the type of operating system and any open ports on the hosts of the target network. The attacker uses this knowledge to launch an attack using any known vulnerabilities of the operating system and network services that are running at the target hosts. For example, the attacker can send an unusual TCP packet where the SYN flag, which is used to indicate the beginning of a connection, and the FIN flag, which is used to indicate the end of a connection, are both set at the same time. By analyzing the reply of the target host, the attacker can check whether the reply matches the fingerprints of a specific operating system. The second category of security problems is the denial of service (DoS) attack. In this case, the attacker s aim is to make the service provided by the victim unavailable to legitimate users rather than obtain unauthorized access. There are two types of DoS attacks. The first type of DoS attack has the aim of disrupting the services provided by the victim by exploiting a software vulnerability of the system. For example, the ping-of-death attack [6] sends a packet with an illegal payload (i.e., longer than 64K bytes), which causes some operating systems to lock up or reboot due to buffer overflow. The second type of DoS attack is based on the volume of traffic, which is known

41 2.2 Bandwidth Attacks 19 as a bandwidth attack. Bandwidth attacks became a major security concern after massive bandwidth attacks paralyzed many high profile web sites, such as CNN and Yahoo, causing substantial financial loss in February 2000 [5]. After this severe incident, defending against bandwidth attacks has become a very important research issue for both academia and industry. Many schemes have been proposed and many commercial products have been produced to tackle this problem. After three years, one might ask whether the threat of bandwidth attack has been eliminated. Is it safe to run a mission-critical business via the Internet? This chapter presents a survey of the proposed technologies to defend against bandwidth DoS attacks. The rest of the chapter is organized as follows. Section 2.2 gives a definition of the bandwidth attack and the fundamental vulnerabilities of the Internet that facilitate bandwidth attacks. Section 2.3 gives a detailed review of the proposed solutions to DoS attacks. Section 2.4 discusses the remaining threats and open issues in solving denial-of-service problems. 2.2 Bandwidth Attacks The bandwidth attack can be defined as any activity that aims to disable the services provided by the victim by sending an excessive volume of useless traffic. This is in contrast to the flash crowd which occurs when a large number of legitimate users access a server at the same time. The comparison between bandwidth attacks and flash crowds is shown in Table Impacts of the Bandwidth Attack There are two major impacts of bandwidth attacks. This first is the consumption of the host s resources. Generally, the victim could be a web server or proxy connected to the Internet. The victim has limited resources to process the incoming packets.

42 20 A Survey of DoS Attacks Table 2.1: Comparison between bandwidth attacks and flash crowds Bandwidth Attack Flash Crowd Network impact congested congested Server impact overloaded overloaded Traffic illegitimate genuine Response to traffic control unresponsive responsive Traffic type any mostly web Number of flows any large number of flows Predictability unpredictable mostly predictable When the traffic load becomes high, the victim will drop packets to inform senders, which consist of both legitimate users and attack sources, to reduce their sending rates. Legitimate users will slow down their sending rates while the attack sources will maintain or increase their sending rates. Eventually, the victim s resources, such as CPU and memory, will be used up and the victim will be unable to service legitimate traffic. The second impact is consumption of the network bandwidth, which is more threatening than the first. If the malicious flows are able to dominate the communication links that lead to the victim, then the legitimate flows will be blocked. Therefore, not only the intended victim of the attack is disabled, but also any system which relies on the communication links of the attack path. Although a congested router can control the traffic flow by dropping packets, legitimate traffic will also be discarded if there is no clear mechanism to differentiate legitimate traffic from attack traffic Inherent Vulnerabilities of the Internet Architecture Bandwidth attacks are the result of several fundamental weaknesses of the Internet architecture.