SERIES IN ELECTRICAL AND COMPUTER ENGINEERING Intrusion Detection A Machine Learning Approach Zhenwei Yu University of Illinois, Chicago, USA Jeffrey J.P. Tsai Asia University, University of Illinois, Taiwan Chicago, USA Imperial College Press
Contents Preface vii 1. Introduction 1 1.1 Background 1.2 Existing Problems 3 1.2.1 Alarm management 3 1.2.2 Performance maintenance 4 1 2. Attacks and Countermeasures in Computer Security 7 2.1 General Security Objectives 7 2.1.1 Accountability 7 2.1.2 Assurance 8 2.1.3 Authentication 8 2.1.4 Authorization 8 2.1.5 Availability 8 2.1.6 Confidentiality 9 2.1.7 Integrity 9 2.1.8 Non-repudiation 9 2.2 Types of Attacks 10 2.2.1 Attacks against availability.... 10 2.2.2 Attacks against confidentiality... 11 2.2.3 Attacks against integrity 12 2.2.4 Attacks against miscellaneous security objectives 13 2.3 Countermeasures of Attacks 14 2.3.1 Authentication 15 2.3.2 Access control 16 2.3.3 Audit and intrusion detection 20 ix
x Intrusion Detection: A Machine Learning Approach 2.3.4 Extrusion detection 22 2.3.5 Cryptography 23 2.3.6 Firewall 26 2.3.7 Anti-virus software 28 3. Machine Learning Methods 31 3.1 Background 31 3.2 Concept Learning 31 3.3 Decision Tree 32 3.4 Neural Networks 32 3.5 Bayesian Learning 32 3.6 Genetic Algorithms and Genetic Programming 33 3.7 Instance-Based Learning 33 3.8 Inductive Logic Programming 34 3.9 Analytical Learning 34 3.10 Inductive and Analytical Learning 34 3.11 Reinforcement Learning 35 3.12 Ensemble Learning 35 3.13 Multiple Instance Learning 36 3.14 Unsupervised Learning 36 3.15 Semi-Supervised Learning 36 3.16 Support Vector Machines 37 4. Intrusion Detection System 39 4.1 Background 39 4.1.1 Security defense in depth 39 4.1.2 A brief history of intrusion detection 41 4.1.3 Classification of intrusion detection system... 41 4.1.4 Standardization efforts 43 4.1.5 General model of intrusion detection system... 43 4.2 Available Audit Data 44 4.2.1 System features 44 4.2.2 User activities 45 4.2.3 Network activities 46 4.3 Preprocess Methods 47 4.4 Detection Methods 49 4.4.1 Statistical analysis 49 4.4.2 Expert system 51
Contents x\ 4.4.3 Model-based system 51 4.4.4 State transition-based analysis 52 4.4.5 Neural network-based system 53 4.4.6 Data mining-based system 54 4.5 Architecture for Network Intrusion Detection System 56 Part A: Intrusion Detection for Wired Network 5. Techniques for Intrusion Detection 61 5.1 Available Alarm Management Solutions 61 5.1.1 Alarm correlation 61 5.1.2 Alarm filter 62 5.1.3 Event classification process 63 5.2 Available Performance Maintenance Solutions 63 5.2.1 Adaptive learning 63 5.2.2 Incremental mining 64 6. Adaptive Automatically Tuning Intrusion Detection System 65 6.1 Architecture 65 6.2 SOM-Based Labeling Tool 65 6.2.1 Training algorithm 66 6.2.2 Pre-cluster by symbolic features 68 6.2.3 Cluster by SOM 68 6.2.4 Label data in clusters 70 6.3 Hybrid Detection Model 71 6.3.1 Binary SLIPPER rule learning system 71 6.3.2 Binary classifiers 74 6.3.3 Final arbiter 74 6.3.4 Detection model tuning 79 6.3.5 Fuzzy prediction filter 86 6.3.6 Fuzzy tuning controller 96 7. System Prototype and Performance Evaluation 101 7.1 Implementation of Prototype 101 7.1.1 Fuzzy controller 101 7.1.2 Binary prediction and model tuning thread 101... 7.1.3 Final arbiter and prediction filter thread 102
xjj Intrusion Detection: A Machine Learning Approach 7.1.4 User simulator thread 102 7.1.5 Interface for fuzzy knowledge base 103 7.2 Experimental Data set and Related Systems 103 7.2.1 KDDCup'99 intrusion detection data set 103 7.2.2 Performance evaluation method 105 7.2.3 Related IDSs on KDDCup'99 ID data set 108 7.3 Performance Evaluation 112 7.3.1 SOM-based labeling tool performance 112 7.3.2 Build hybrid detection model 114 7.3.3 The MC-SLIPPER system and test performance 116 7.3.4 The ATIDS system and test performance 125 7.3.5 The ADAT IDS system and test performance... 133 Part B: Intrusion Detection for Wireless Sensor Network 8. Attacks against Wireless Sensor Network 141 8.1 Wireless Sensor Network 141 8.2 Challenges on Intrusion Detection in WSNs 142 8.3 Attacks against WSNs 143 9. Intrusion Detection System for Wireless Sensor Network 147 9.1 Architecture of IDS for WSN 147 9.2 Audit Data in WSN 149 9.2.1 Local features for LIDC in WSN 150 9.2.2 Packet features for PIDC in WSN 152 9.3 Detection Model and Optimization 153 9.4 Model Tuning 155 10. Conclusion and Future Research 157 Cited Literature 159 Index 169