Calculation Algorithm for Network Flow Parameters Entropy in Anomaly Detection Theory, practice, applications Oleg Gudkov, BMSTU IT Security for the Next Generation International Round, Delft University of Technology 11-13 May, 2012 The Netherlands
Intrusion Detection Brief review of the intrusion detection methods
Signature based method This method can detect only known attacks Do well PAGE 3
Selected network traffic parameters deviation discovering relative to defined normal values
Network traffic representation Network traffic representation PAGE 5
Scanning process Problem PAGE 6
Scanning process Problem There is network packets sequence PAGE 7
Scanning process Which packets can be regarded as scanning? PAGE 8
Scanning process And in this case? PAGE 9
Scanning process And in this? PAGE 10
Network traffic parameters Network traffic parameters which used to detect anomalies: Intencity One should define threshold of network packets per every single period of time Probability Hypothesys of network traffic distribution law are examinated Entropy Information entropy of different network packets parameters is been analyzed. PAGE 11
Entropy alteration while scanning Representative example of destination IP address entropy alteration while scanning with nmap. nmap -v scanme.org -T3 Parameter «-T3» is used to reduce intencity of scanning. So there is both scanning and "normal" packets in the marked area. Entropy alteration example while scanning with the nmap packet number PAGE 12
Entropy calculation in practice Theory and practice
Theory Entropy calculation in practice Entropy according to Shennon is: Where p i - probability of member a i of the set A={a 1, a 2,, a m }. In this case during anomaly detection process: There exist m different values of the network packet selected parameter. Each network packet in the sequence has probability of occurance. Entropy calculates according to formula above. PAGE 14
Network packets parameters selection Shape Parameter value Source IP:192.168.0.1 Source IP: 38.117.98.208 Source IP: 8.8.8.8 PAGE 15
Network packets parameters selection Shape Parameter value Destination port: 53 Destination port: 21 Destination port: 137 PAGE 16
Network packets parameter selection Shape Parameter value 192.168.0.1:52 38.117.98.208:21 8.8.8.8:137 PAGE 17
Network packets parameters selection Entropy calculation practice Network packets parameters used for analyze entropy Network addresses (source and destination) Ports Another network packets fields and combinations. PAGE 18
Main tasks Entropy calculation practice For each network packet 1. Calculate entropy PAGE 19
Main tasks Entropy calculation practice For each network packet 1. Calculate entropy 2. It is necessary to calculate it FAST!!! PAGE 20
Entropy calculation PAGE 21
Entropy calculation p i =? PAGE 22
Entropy calculation We should use probability estimation PAGE 23
Entropy calculation Select calculation window: W={ω 1, ω 2,,ω N } N elements PAGE 24
Entropy calculation N elements PAGE 25
Entropy calculation N elements PAGE 26
Sequential entropy calculation H 2 H 1 There should every time calculate an entropy value for each window PAGE 27
Entropies difference calculation Proposal: we should calculate entropy difference not entropy for each window H N = H 2 - H 1 H 2 H 1 PAGE 28
Entropies difference calculation Proposal: we should calculate entropies difference W 2 W 1 Hatched and non-hatched letters - probabilities of selected parameters value for windows W 1 and W 2 respectively PAGE 29
Reduce amount of calculations Notice: only elements in and out affect the difference in amount of elements of every type in every following windows. out W 2 W 1 in PAGE 30
Preliminary data for algorithm Entropy calculation practice Array of elements - values of selected network packet parameter Hash-array, maps number of elements for each parameter value Value Number of elements 3 3 6 PAGE 31
Algorithm first step Entropy calculation practice 1. Fill elements with the initial values 2. Fill hash-array with the number of elements from the initial values Value Number of elements 3 3 6 3. Calculate entropy H 0 for the initial values PAGE 32
Main algorithm step Entropy calculation practice 1. For every new parameter value calculate entropies difference 2. Calculate entropy value H i+1 N=H i N+ H N 3. Update hash-array Value Number of elements 3 4 5 PAGE 33
Algorithm rate comparison Entropy calculation practice Comparison for the same data of 22501 packets. Time in second obtained with time utility. Window size (number of packets) Entropies difference algorithm Sequential entropy calculation algorithm 10 0.046 s. 0.122 s. 50 0.050 s. 0.480 s. 100 0.046 s. 0.915 s. 500 0.051 s. 3.644 s. 1000 0.046 s. 6.738 s. PAGE 34
Thank You Oleg Gudkov, BMSTU IT Security for the Next Generation International Round, Delft University of Technology 11-13 May, 2012 The Netherlands