CASCADAS Imperial College. Work part of WP4 related to Autonomic Infrastructure Protection

Transcription

1 CASCADAS Imperial College Work part of WP4 related to Autonomic Infrastructure Protection Erol Gelenbe, Georgios Loukas Gulay Oke Intelligent Systems and Networks Group Imperial College London Earlier Work Funded by EPSRC, BT and UK MoD

2 1996. Panix Analyzer attacks the Pentagon Mafiaboy attacks Amazon, Yahoo etc Port of Houston Root Servers American hackers (?) attack Al Jazeera Industrial attacks on P2P Network Sites

3 What is a DoS Attack An attack with the purpose of preventing legitimate users from using a specific network resource

4 Is it a new threat? 1985, R.T. Morris writes: The weakness in the Internet Protocol is that the source host itself fills in the IP source host id, and there is no provision in TCP/IP to discover the true origin of a packet.. IP Spoofing SYN Flood Attack

5 Distributed DoS

6 Issues that have been Examined On-Line Detection Pattern detection Anomaly detection Hybrid detection Third-party detection Autonomic Response Agent identification Rate-limiting Filtering Reconfiguration

7 What is a DoS Attack? A Denial of Service (DoS) attack can be characterized as an attack with the purpose of preventing legitimate users from using a specific network resource. What is a DDoS Attack? A Distributed Denial of Service (DDoS) attacks is one in which a multitude of compromised systems is the source of the attack, thereby causing denial of service for its legitimate users of the targeted system(s).

8 Figure 1. The agent-handler DDoS model

9 Why is detection necessary? A combination of detection and response mechanisms are used to defend against such attacks.. Detection would not be necessary in the ideal case of a response architecture with proactive qualities that would render impossible any DoS attack. However: - No response system is perfect to date. - Denial of Service attacks against one s network do not happen very often and at least resource-wise a proactive protection system is usually too expensive to operate in the absence of an attack. Therefore, a detection mechanism can trigger the response procedure to overcome the weaknesses stated above.

10 Detection of DoS Attacks 1. Methods Based on Identification of the Source Address 2. Methods Based on Analysis of Traffic A robust DoS detection scheme must satisfy the following: High detection rates Minimal false alarm rates Real-time detection with low memory and CPU-time requirements Invariance in evolutionary trends in DoS attacks

11 Methods Based on Identification of the Source Address Ingress Filtering Route-Based Filtering IP Traceback (Probabilistic Packet Marking) Hop-Count Filtering

12 Methods Based on Analysis of Traffic a. Methods Based on Learning Techniques (NNs, RBFs, etc) Jalili, Imani-Mehr,Amini, Shahriari (2005) They proposed the use of an unsupervised neural network for the detection of DoS attacks. A statistical pre-processor is used to extract some features from packets using statistical techniques. The extracted feature vector is converted to numerical form and then it is fed to an unsupervised neural network, namely Adaptive Resonance Theory Net (ART). The ART is first trained with normal or intrusive type of input vectors. In testing phase, it is expected to classify the packets using the adjusted cluster weights.

13 (Jalili, Imani-Mehr,Amini, Shahriari (2005), cont d) The features used in detection: NICMP: the percent of ICMP packets NUDP: the percent of UDP packets NTCP: the percent of TCP packets NTCPSYN: the percent of SYN packets in TCP packets NTCPSYNACK: the percent of SYN+ACK packets in TCP packets NTCPACK: the percent of ACK packets in TCP packets APacket Header Sizes: the packet header sizes average APacket Data Sizes: the packet data sizes average They reported a detection rate of 94.5 percent (0.7 second in best case).

14 Gavrilis, Dermatas (2004) The total scheme consists of a data collector, a feature estimator and a RBF-NN detector. The data collector captures the appropriate data fields for each packet, The feature estimator estimates the frequency of occurrences for the encoded data. The feature vector is passed onto a RBF-NN detector for classification as either normal traffic or DoS attack. For TCP, the source port, SEQ number of the client, window size, and the SYN, ACK, FIN, PSH, URG, RST flags. For UDP,only the source port and TTL have been used. In experiments, it was seen that the set of 9 statistical features surpassed 98% of correct classification. It was observed that with a set of 3 inputs (Source Port, SEQ number, SYN flag), the correct classification rate in most cases was close to the 9 features rate.

15 Gavrilis, Tsoulos, Dermatas (2004) They proposed an optimum feature selection problem for robust detection of DoS attacks using a genetic algorithm. They determined which input features to be considered in detection are more important relative to others and which features have no relevance. Out of a complete set of 44 statistical features, they found out that SYN and URG do play a major role, while TTL and window size provide no information. The total scheme consists of a data collector, a features estimator and a two-layer feed-forward neural network detector.

16 Noh, Lee, Choi and Jung (2003) They utilize three machine learning algorithms, namely C4.5 (represents output as a decision tree), CN2 (ordered set of if-then rules) and a Bayesian classifier for detecting DoS and gave experimental results in a simulated TCP-based network setting. The features used in the detection are the TCP flag rate and the protocol rate. A packet collecting agent captures IP packets and classifies them into TCP, UDP or ICMP packets. If it is a TCP packet, it is further separated into TCP header and payload, the total number of set flags SYN, FIN, RST, ACK, PSH and URG are summed up. TCP flag rate is the ratio of each of these flags to the total number of TCP packets. Protocol rate is the ratio of the number of TCP, UDP or ICMP packets to the total number of IP packets. Best performance was obtained by the rules compiled using Bayesian classifier. No missed alarms were observed, all measured errors were caused by false alarms.

17 b. Methods Based on Wavelet Transform Analysis It is experimentally verified that normal traffic exhibits a remarkably stationary energy distribution, while energy distribution variance changes markedly as traffic behavior changes due to a DoS attack. Wavelet analysis is used to extract information about the energy content of the packets.

18 Li and Lee (2005) x() t Suppose x () t and x( t +τ ) d x are the wavelet coefficients Eg ΔEg t j j = 1 n j k = log Eg d t j byte counts in a fixed time interval t x ( j, k) 2 log Eg t+ τ j two time series and Eg t + τ j = Eg = log Eg t j t+ τ j 1 t + τ d x j k Energy distribution variation in the two time series is considered to be the traffic signature. The normal traffic is defined as: { τ < δ, τ T} () t x( ) var( ΔEg ) x j > n ( j, k) 2

19 Yang, Liu, Zeng and Shi (2004) They propose the BDA-CWT (Network traffic burst detecting algorithm based on the continuous wavelet transform) They divide the bursts in the network traffic into three categories (long-bursts, short-bursts and one-point bursts) and then propose an algorithm based on the continuous wavelet transform for the identification of flat bursts in the traffic in real time. The feature used in the algorithm is the number of packets per second.

20 Advantages vs Shortcomings of Wavelet Methods Energy distribution analysis with wavelet methods Are able to catch attacks early, before congestion builds up Computations will be performed in sliding sampling windows and performance changes with varying window sizes and time step increments. A smaller window size may not provide enough samples to build up traffic self-similarity while too large a window may cause unnecessary computation during the analysis. Other deviations from normal traffic can also be captured in the energy distribution variation.

21 c. Methods Based on Statistical Signal Analysis For a random series x, if its autocorrelation function r xx is summable then x is called statistically short-range dependent (SRD) series. Otherwise it is termed as LRD. Network traffic is LRD. Internet traffic also has the property of self-similarity. The Hurst parameter H, represents the degree of self-similarity. A value of H close to 1 means a larger degree of self-similarity (LRD). In case 0<H<=0.5, there is lack of self-similarity.

22 Li (2004) x() t n() t y() t () t = x() t n() t y + the number of packets arriving at a site at t as normal traffic the component of attack traffic the abnormal traffic The following scheme is proposed: rxx ryy = ξ > V rxx rxl = ζ > V ξ < V Identification False alarm Miss (failing to recognize DoS)

23 Xiang, Lin, Lei and Huang (2004) Xiang, Lin, Lei and Huang extracted the information about the packet number or packet size (in bytes) arriving at a node from the time series x Then they used statistical methods are to calculate H for consecutive time intervals. If there is a doubling in H in consecutive time intervals, a DoS attack is signalled: DDOS attack = 1 0 Var Var ( H1, H 2,... H n )/ Var( H1, H 2,... H n k ) ( H, H,... H )/ Var( H, H,... H ) 1 2 n 1 2 n k 2 < 2

24 d. Methods Based on Multi-Agents Peng, Leckie and Ramamohanarao (2003) The feature used in the detection is the number of new source IP addresses during a specific time interval. A non parametric change detection scheme, CUSUM (cumulative sum) is used to extract information about the abrupt changes in the number of new IP addresses, denoted by variable y n The decision function is: d N ( y ) n = 0 1 if if y y n n N > N

25 (Peng, Leckie and Ramamohanarao (2003), cont d) Each agent applies the described scheme and then they cooperate with each other by sharing their beliefs about potentially suspicious traffic. The l th agent will broadcast if Nl yl > T Using learning techniques, an optimum value of T that minimizes both the communication overhead and the confirmation delay can be obtained. Detection accuracy was as high as 99% in the first-mile router. This approach will be more successful for highly distributed DoS attacks. However, the same attack takes longer to detect in a multi-agent system compared to a centralized system since each agent sees only a subset of the attack traffic.

26 Our Proposed Approach to Detection Using the Neyman-Pearson Decision Rule a. Selecting the Input Features [Incoming Packet Rate, Rise in Rate, Congestion] b. Training Phase c. Dropping a Fraction of the Packets as a Precaution during a Suspected Attack d. Detection using te Neyman-Pearson Decision Rule

27 Figure 2. Detection of DoS using Neyman-Pearson detection network

28 a. Selecting the Input Features Initially, we will choose: R x R SYN R & x R & SYN rate of incoming packets rate of incoming SYN packets change in the rate of incoming packets change in the rate of incoming SYN packets

29 b. Training Phase The probability density functions of all the input features will be determined for both DoS traffic and normal traffic p ( x H ), p ( x H ), p ( x H ), p ( x H ),..., p ( x H ), p ( x H ) n 0 n 1 H 0 H 1 hypothesis that there is no attack hypothesis that there is an attack Nonparametric probability density estimation methods will be used: Histogram Method The real line is partitioned into a number of equal-sized cells of width Δx, the estimate of the density at a point x is taken to be: pˆ ( x) = N n jδx j n j

30 Kernel Method (Parzen Method) { } Given a set of observations x 1, x2,..., x n an estimate of the density function in one dimension: pˆ 1 nh n ( x) = i= 1 K x x h i K(z) is the kernel function n is the window size h is the spread or smoothing parameter (or bandwidth) Gaussian Functions are generally used

31 c. Dropping a Fraction of the Packets as a Precaution during a suspected an Attack In an interval of time E i[ R & x ] 2 σ i [ R & x ] E i[ R & SYN ] 2 σ [ ] If i R & SYN E Δti mean of the increase in the rates of incoming packets variance of the increase in the rates of incoming packets mean of the increase in the rates of incoming SYN packets variance of the increase in the rates of incoming SYN packets [ R& ] E [ R& 2 ] > m [ R& ] is true for Δ t... Δt i+ 1 x i x σ i x 0 N we conclude that there is a likelihood of a DDoS attack, and some fraction δ of the incoming packets is dropped If E [ R& ] E [ R& 2 ] > n [ R& ] i+ 1 SYN i SYN σ i SYN is true for Δ t0... ΔtN again a fraction of the packets can be dropped as a precaution.

32 d. Detection with Neyman-Pearson Decision Rule The Neyman-Pearson decision rule minimizes the false alarm rate for a given level of probability of correct detection, or maximizes the probability of correct detection subject to a given level of the false alarm probability p p ( x H1) ( x H ) 0 > μ

33 In our DoS detection scheme, we make use of all the information we have by incorporating it in the probability density functions. The goodput and traffic rates are evaluated at each sampling interval. The information from the incoming packets is used to update the estimates of the probability density functions. Thresholds of the Neyman-Pearson detectors can be made to be variable to maximize the goodput and to minimize false alarm rate. Adaptation and updating of parameters is important since our detection mechanism should give an optimum result, even if there is a change (drift) in the structure of the traffic flowing into the network Future work will implement this scheme using our existing CPN test-bed. Implementation of the defense reaction is complete and tested.

34 The CPN based DDoS Defence Scheme The CPN architecture traces flows using smart and ACK packets A DoS produces QoS degradation The user(s) and victim(s) detect the attack and inform nodes upstream from the victim(s) using ACK packets These nodes drop possible DoS packets The detection scheme is never perfect (false alarms & detection failures)

35 Mathematical model (1) Analyses the impact of DDoS protection on overall network performance Measures traffic rates in relation to service rates and detection probabilities

36 Mathematical Model: Queueing Network with Blocking = = = = 1 1,,, 1 1,,, )) )(1 ((1 )) )(1 ((1 j l d d d d d d j l n n n n n n l l j j l l j j d L I f L I d d d n n n λ λ )) (1 ) (1 (,,,, + = d d d n n n i d i i n i i i d I f I s ρ i B i B i i i i L ρ ρ ρ =

37 Illustration on an Experimental CPN Test-Bed

38

39

40 Predictions of Mathematical Analysis

41 Impact on the nodes (without Defence)

42 Impact of the Defence on the Nodes

43 Experiment 2.4GHz P4 PCs, Linux kernel , CPN Different QoS protocols for normal and attack traffic Delay-based FIFO queuing 60 sec

44 DoS on a streaming video

45 Math. Analysis, Simulation and Experiment Comparison

46 Conclusions In DoS it is very easy to attack very difficult to defend DoS is the top network security threat DoS harms QoS Our defence scheme improves QoS under DoS

47 Future work Defence specialisation Packet drops near the source Detection near the target Sophisticated detection (probabilistic packet dropping) based on: QoS criteria Priorities Source range Congestion detected Overhead Wireless networks More fragile to network attacks Power consumption

48 Questions