DDoS Attacks Evolution, Detection, Prevention, Reaction, and Tolerance

Transcription

1 DDoS Attacks Evolution, Detection, Prevention, Reaction, and Tolerance

2

3 DDoS Attacks Evolution, Detection, Prevention, Reaction, and Tolerance Dhruba Kumar Bhattacharyya Jugal Kumar Kalita Boca Raton London New York CRC Press is an imprint of the Taylor & Francis Group, an informa business

4 CRC Press Taylor & Francis Group 6000 Broken Sound Parkway NW, Suite 300 Boca Raton, FL CRC Press is an imprint of Taylor & Francis Group, an Informa business No claim to original U.S. Government works Printed on acid-free paper Version Date: International Standard Book Number-13: (Hardback) This book contains information obtained from authentic and highly regarded sources. Reasonable efforts have been made to publish reliable data and information, but the author and publisher cannot assume responsibility for the validity of all materials or the consequences of their use. The authors and publishers have attempted to trace the copyright holders of all material reproduced in this publication and apologize to copyright holders if permission to publish in this form has not been obtained. If any copyright material has not been acknowledged please write and let us know so we may rectify in any future reprint. Except as permitted under U.S. Copyright Law, no part of this book may be reprinted, reproduced, transmitted, or utilized in any form by any electronic, mechanical, or other means, now known or hereafter invented, including photocopying, microfilming, and recording, or in any information storage or retrieval system, without written permission from the publishers. For permission to photocopy or use material electronically from this work, please access ( or contact the Copyright Clearance Center, Inc. (CCC), 222 Rosewood Drive, Danvers, MA 01923, CCC is a not-for-profit organization that provides licenses and registration for a variety of users. For organizations that have been granted a photocopy license by the CCC, a separate system of payment has been arranged. Trademark Notice: Product or corporate names may be trademarks or registered trademarks, and are used only for identification and explanation without intent to infringe. Visit the Taylor & Francis Web site at and the CRC Press Web site at

5 Contents List of Figures List of Tables Preface Acknowledgments Authors xiii xvii xix xxi xxiii 1 Introduction Anomalies in Networks Distributed Denial-of-Service (DDoS) Attacks Causes of DDoS Attacks Targets of DDoS Attacks Launching of DDoS Attacks Current Trends in Botnet Technology Machine Learning in DDoS Attack Handling Traffic Attributes and User-Parameter Selection Selection of Metrics or Measures Analysis of Data Mode of Detection Generation of Alarm Information and Reaction DDoS Defense Modules of a DDoS Defense System Types of DDoS Defense Systems Based on Approach DDoS Detection DDoS Prevention DDoS Response v

6 vi CONTENTS DDoS Tolerance Based on Nature of Control Centralized DDoS Defense Hierarchical DDoS Defense Distributed DDoS Defense Based on Defense Infrastructure Host-Based DDoS Defense Network-Based DDoS Defense Based on Defense Location Victim-End DDoS Defense Source-End DDoS Defense Intermediate Network DDoS Defense Based on Technique Used Misuse Detection Anomaly Detection DDoS Tools and Systems DDoS Defense Evaluation Prior Work Contribution of This Book Organization of This Book DDoS, Machine Learning, Measures Issues in Internet Design Complex Edge but Simple Core Link Bandwidth Mismatch between Core and Edge Routing Principles Lack of Centralized Network Management Sharing of Reserved Resources across Data Centers DDoS Attacks and Their Types Agent-Handler and IRC-Based DDoS Attack Generation Types of DDoS Attacks Layer-Specific DDoS Attacks Direct and Reflector-Based DDoS Attacks Direct and Indirect DDoS Attacks High-Rate and Low-Rate DDoS Attacks Attack Types Based on Rate Dynamics DDoS Attack Targets On Infrastructure On Link

7 CONTENTS vii On Router On OS On Defense Mechanism Current Trends in DDoS Attacks Strength of DDoS Attackers Desired Characteristics of DDoS Defense System Recent DDoS Attacks Machine Learning Background Supervised and Unsupervised Machine Learning Measures: Similarity and Dissimilarity Dissimilarity Measures Correlation Measures f-divergence Measures Information Metrics Discussion Some Empirical Studies Using Information Metrics Testbed Used Datasets Used Results of Empirical Study Discussion Using Correlation Measures An Example Using f-divergence Measures Results Discussion Chapter Summary Botnets: Trends and Challenges DDoS Attacks Using Stationary Botnets Botnet Characteristics Botnet Models Agent Handler Model IRC-Based Model Web-Based Model Botnet Formation Life Cycle Stationary Botnet Architecture Botnet Topology Protocols Used Botnet C&C Systems

8 viii CONTENTS Some Stationary Botnets DDoS Attacks Using Mobile Botnets Mobile Botnet Characteristics C&C Mechanisms in Mobile Botnet Some Mobile Botnets Chapter Summary and Recommendations DDoS Detection Modules of a DDoS Defense Solution Monitoring Detection Reaction Types of DDoS Defense Solutions Based on Approach Used Based on Nature of Control Centralized DDoS Defense Hierarchical DDoS Defense Distributed DDoS Defense Based on Defense Infrastructure Host-Based DDoS Defense Network-Based DDoS Defense Based on Defense Location Victim-End DDoS Defense Source-End DDoS Defense Intermediate Network DDoS Defense Based on Techniques Used DDoS Detection Techniques Misuse Detection Signature-Based DDoS Detection Rule-Based Detection State-Transition Techniques Anomaly-Based DDoS Detection Statistical Techniques Machine Learning and Data Mining Techniques Soft Computing Techniques Knowledge-Based Techniques Chapter Summary

9 CONTENTS ix 5 DDoS Prevention DDoS Prevention Techniques IP Traceback Link Testing Packet Marking Packet Logging ICMP Traceback Messages Discussion Filtering Techniques Ingress and Egress Filtering Router-Based Packet Filtering (RPF) Source Address Validity Enforcement (SAVE) Protocol Rate Control Chapter Summary DDoS Reaction and Tolerance Intrusion Response System (IRS) Intrusion Response (IR) and Its Types A Model to Demonstrate Relationships among Responses Development of IRSs: Approaches, Methods, and Techniques Based on the Degree of Automation Based on the Approach Used for Triggering Responses Based on Adaptability Based on Promptness in Response Generation Based on the Level of Cooperation Based on Versatility in Reacting to Unseen Situations Some Example Intrusion Response Systems Cooperative Intrusion Traceback and Response Architecture (CITRA) Distributed Management Architecture for Cooperative Detection and Reaction EMERALD CSM Adaptive, Agent-Based IRS (AAIRS). 175

10 x CONTENTS ALPHATECH SITAR Discussion DDoS Tolerance Approaches and Methods Multi-Level IDS-Based Approaches Middleware Algorithm-Based Approaches Recovery-Based Approaches Discussion Chapter Summary Tools and Systems Introduction Types of Network Security Tools Information Gathering Tools Sniffing Tools Network Mapping/Scanning Tools Attack Launching Tools Trojans Transport and Network Layer Denialof-Service Attacks Application Layer Attack Tools Additional Attack Tools Network Monitoring Tools Visualization and Analysis Tools Observations TUCANNON+: DDoS Attack-Generation and Monitoring Tool TUCannon: Attack-Generation Module Server Sub-module of TUCannon Client Sub-module Scalability of TUCannon Speed of TUCannon Reflector Attack TUCannon Architecture Server Architecture Client Architecture TUMonitor TUMonitor: An Overview TUMonitor Architecture Visualization with TUMonitor

11 CONTENTS xi 7.7 DDoS Defense Systems Systems that Respond to Intrusion Architectures of Some Well-Known Defense Systems Some Commercial and Academic Defense Systems Discussion Chapter Summary Conclusion and Research Challenges Conclusion Source IP Spoofing Degree of Randomization Isolation vs. Combination Realistic TCP SYN Flooding Removal of Unique Characteristics Low-Cost and Limited Bandwidth Attack Research Challenges Developing a Generic DDoS Defense Mechanism Integration of Packet/Flow Monitoring and Detection Developing DDoS-Tolerant Architecture Developing a Cost-Effective Source-End Defense Developing an Efficient Dynamic Firewall Hybridization Issues to Support Real-Time Performance with QoS Heuristics for Accurate Estimation of Defense Parameters Developing a Robust and Cost-Effective Proximity Measure Standard for Unbiased Evaluation of Defense Solutions Large-Scale Testbed for Defense Validation Index 285

12

13 List of Figures 1.1 Number of Internet users up to DDoS attack statistics up to the year 2014 (DDoS attack percentage is shown on the y-axis) DDoS strategy: Recruiting, exploiting, infecting, and attacking Agent handler and IRC architectures Direct DDoS attack Reflector-based DDoS attack Constant-rate, increasing, pulsing, and sub-group attacks Current trends in DDoS attacks DDoS attack frequency Statistics of attack durations TUIDS testbed architecture with the DMZ Spacing results for low-rate and high-rate attacks Differentiating high- and low-rate attack traffic from normal in CAIDA dataset using Generalized Entropy Distinguishing high-rate attack traffic from normal in TUIDS dataset using Generalized Entropy Distinguishing low-rate attack traffic from normal in TU- IDS dataset using Generalized Entropy Distinguishing high-rate attack traffic from normal in CAIDA dataset using Information Distance Distinguishing low-rate attack traffic from normal in CAIDA dataset using Information Distance Distinguishing high-rate attack traffic from normal in TUIDS dataset using Information Distance Distinguishing low-rate attack traffic from normal in TU- IDS dataset using Information Distance xiii

14 xiv LIST OF FIGURES 2.18 Correlation plot of example objects Threshold ranges for various f-divergence measures f-divergence for ICMP flood detection using connection size in Trace f-divergence for ICMP flood detection using connection size in TUIDS dataset f-divergence for TCP SYN flooding attack detection using TCP SYN ACK ratio in Trace II f-divergence for UDP flood detection using destination port change in Trace III Botnet attack Botnet life cycle C&C server approaches Mobile botnet architecture Major factors of botnet design Modules of a DDoS defense system Intrusion detection system: a generic view Structure of centralized DDoS defense Structure of hierarchical DDoS defense Structure of a distributed DDoS defense Source-end DDoS detection: a generic architecture Intermediate DDoS detection: a generic architecture Architecture of DCP system Architecture of the D-WARD system Xie et al. s filtering strategy based on behavior model The NetShield system and its components The conceptual framework of a DDoS container Machine learning in DDoS attack detection The architecture of FireCol The hybrid neuro-fuzzy inference system Framework of the rule-based DDoS detection mechanism Simulated network environment with web clients, attackers, server, and the proposed agent Intrusion prevention system: a generic view An example network for traceback demonstration Tracing attack flows at the router

15 LIST OF FIGURES xv 5.4 An example network to demonstrate ingress and egress filtering A demonstration of router-based packet filtering Intrusion response system: a generic view IRS model A taxonomy of IRSs Intrusion tolerance system: a generic view The architecture of SITAR for intrusion tolerance Three crucial dimensions of MAFTIA MAFTIA: a middleware algorithm-based tolerance architecture ITUA architecture SCIT architecture SCIT/HES architecture for a single server X Taxonomy of attack-related tools Types of port scans TUIDS testbed architecture with DMZ for TUCANNON Direct attack strategy adopted by TUCannon GUI of TUCannon server sub-module Reflector attack TUCannon server architecture Client architecture of TUCannon Traffic capture from NIC using TUMonitor Traffic reading from file in TUMonitor Traffic feature selection in TUMonitor Visualization of multiple graphs in TUMonitor TUMonitor architecture Arithmetic expression in TUMonitor Visualization under normal conditions in TUMonitor Visualization of TCP flooding in TUMonitor Visualization of UDP flooding in TUMonitor Bro IDS cluster architecture [33] VMI-based IDS architecture Network-based intrusion prevention system Architecture of dynamic intelligence cloud firewall An example architecture of IDPS [211] EMERALD architecture

16

17 List of Tables 2.1 Information metrics and variables used Traffic features and details of CAIDA DDoS dataset Example objects Computed distance and correlations on example objects Execution time performance of f-divergence measures Comparison of stationary botnets Comparison of mobile botnets Comparison between mobile botnet and stationary botnet Infrastructure-based defense: a comparison DDoS defense based on locations: a general comparison Misuse detection techniques: a comparison Some sniffing tools and their features Some scanning tools and their features Types of trojans Some attacking tools Additional relevant tools Some visualization tools and their features Category-wise information for some important tools xvii

18

19 Preface Rapid technological advances have made the Internet ubiquitous around the globe. Access speeds and reliability of access are always improving, and as a result, diverse services provided on the Internet are greatly impacting every aspect of our day-to-day lives. Using these services, people routinely depend on the Internet to share confidential and valuable personal and professional information. Because smooth functioning of society depends highly on the Internet, individuals with bad intentions routinely exploit inherent weaknesses of the Internet to paralyze targeted services all over the net. With increasing incidences of network attacks, detecting such unwelcome intrusions has become an important research area. Among all the threats for which network defenders need to watch out, Distributed Denial-of-Service (DDoS) attacks are among the most common and most devastating. In this attack, people with malice use tools that are frequently available on the net to disrupt Websites, databases or enterprise networks by first gathering information on their weaknesses and later exploiting them. DDoS is a coordinated attack, launched using a large number of compromised hosts. A DDoS attack is considered high-rate when it generates a large number of packets or extremely high-volume traffic within a very short time, say a fraction of a minute, to disrupt service. An attack is referred to as a low-rate attack, if it is mounted over minutes or hours. To counter DDoS attacks, several significant defense mechanisms have been developed. This book discusses the evolution of DDoS attacks, how to detect a DDoS attack when one is mounted, how to prevent such attacks from taking place, and how to react when a DDoS attack is in progress with the goal of possibly tolerating the attack and doing the best under the circumstances without failing completely. It introduces types of DDoS attacks, characteristics that they demonstrate, reasons why such attacks can take place, what aspects of the network infrastructure are xix

20 xx PREFACE usual targets, and how these attacks are actually launched. The book elaborates upon the emerging botnet technology, current trends in the evolution and use of this technology, and the role of this technology in facilitating the launching of DDoS attacks, and challenges in countering the role of botnets in the proliferation of DDoS attacks. The book introduces statistical and machine learning methods applied in the detection and prevention of DDoS attacks in order to provide a clear understanding of the state of the art. It presents DDoS reaction and tolerance mechanisms with a view to studying their effectiveness in protecting network resources without compromising the quality of services. Further, the book includes a discussion of a large number of available tools and systems for launching DDoS attacks of various types and for monitoring the behavior of the attack types. The book also provides a discussion on how to develop a custom testbed that can be used to perform experiments such as attack launching, monitoring of network traffic, detection of attacks, as well as for testing strategies for prevention, reaction and mitigation. Finally, the reader will be exposed to additional current issues and challenges that need to be overcome to provide even better defense against DDoS attacks. Dhruba Kumar Bhattacharyya Jugal Kumar Kalita MATLAB R and Simulink are registered trademarks of The MathWorks, Inc. For product information, please contact: The MathWorks, Inc. 3 Apple Hill Drive Natick, MA USA Tel: Fax: info@mathworks.com Web:

21 Acknowledgments This humble work would not have been possible without the constant support, encouragement and constructive criticism of a large number of academicians, scientists and professionals. We are grateful to the panel of reviewers for their constructive suggestions and critical evaluation. Special thanks and sincere appreciation are due to Prof Sukumar Nandi of IITG, Prof R K Agrawal of JNU, Prof S K Gupta of IITD, Dr P N R Rao of DeitY and our dedicated faculty members and research group members: Prof N Sarma, Prof U Sarma, Prof S M Hazarika, Dr Sanjib Deka, Dr M H Bhuyan, Mr Debojit Boro, Mr Nazrul Hoque, Mr R C Baishya, Mr Hasin A Ahmed, Mr Hirak J Kashyap and Mr R K Deka. The constant support and cooperation received from our colleagues and students during the period of writing this book is sincerely acknowledged. Dhruba Kumar Bhattacharyya Jugal Kumar Kalita xxi

22

23 Authors Dhruba Kumar Bhattacharyya received his Ph.D. degree from Tezpur University in 1999 in cryptography and error-control coding. He is a professor in Computer Science and Engineering at Tezpur University. Professor Bhattacharyya s research areas include network security, data mining and bioinformatics. He has published more than 200 research articles in leading international journals and peer-reviewed conference proceedings. Dr. Bhattacharyya has authored three technical reference books and edited eight technical volumes. Under his guidance, thirteen students have received their Ph.D. degrees in the areas of machine learning, bioinformatics and network security. He is Chief Investigator of several major research grants, including the Centre of Excellence, Tezpur University of Ministry of HRD, Government of India under Frontier Areas of Science and Technology and Centre for Advanced Computing, Tezpur University funded by Ministry of IT, Government of India. He is on the editorial board of several international journals and has also been associated with several international conferences. More details about Dr. Bhattacharyya can be found at dkb/index.html. Jugal Kumar Kalita teaches computer science at the University of Colorado, Colorado Springs. He received his M.S. and Ph.D. degrees in computer and information science from the University of Pennsylvania in Philadelphia in 1988 and 1990, respectively. Prior to that he received an M.Sc. from the University of Saskatchewan in Saskatoon, Canada, in 1984 and a B.Tech. from the Indian Institute of Technology, Kharagpur, in His expertise is in the areas of artificial intelligence and machine learning, and the application of techniques in machine learning to network security, natural language processing xxiii

24 xxiv AUTHORS and bioinformatics. He has published 150 papers in journals and refereed conferences. He is the author of a book on Perl titled On Perl: Perl for Students and Professionals. He is also a co-author of a book titled Network Anomaly Detection: A Machine Learning Perspective with Dr. Dhruba K Bhattacharyya. He received the Chancellor s Award at the University of Colorado, Colorado Springs, in 2011, in recognition of lifelong excellence in teaching, research and service. More details about Dr. Kalita can be found at kalita.