Calculation Algorithm for Network Flow Parameters Entropy in Anomaly Detection

Transcription

1 Calculation Algorithm for Network Flow Parameters Entropy in Anomaly Detection Theory, practice, applications Oleg Gudkov, BMSTU IT Security for the Next Generation International Round, Delft University of Technology May, 2012 The Netherlands

2 Intrusion Detection Brief review of the intrusion detection methods

3 Signature based method This method can detect only known attacks Do well PAGE 3

4 Selected network traffic parameters deviation discovering relative to defined normal values

5 Network traffic representation Network traffic representation PAGE 5

6 Scanning process Problem PAGE 6

7 Scanning process Problem There is network packets sequence PAGE 7

8 Scanning process Which packets can be regarded as scanning? PAGE 8

9 Scanning process And in this case? PAGE 9

10 Scanning process And in this? PAGE 10

11 Network traffic parameters Network traffic parameters which used to detect anomalies: Intencity One should define threshold of network packets per every single period of time Probability Hypothesys of network traffic distribution law are examinated Entropy Information entropy of different network packets parameters is been analyzed. PAGE 11

12 Entropy alteration while scanning Representative example of destination IP address entropy alteration while scanning with nmap. nmap -v scanme.org -T3 Parameter «-T3» is used to reduce intencity of scanning. So there is both scanning and "normal" packets in the marked area. Entropy alteration example while scanning with the nmap packet number PAGE 12

13 Entropy calculation in practice Theory and practice

14 Theory Entropy calculation in practice Entropy according to Shennon is: Where p i - probability of member a i of the set A={a 1, a 2,, a m }. In this case during anomaly detection process: There exist m different values of the network packet selected parameter. Each network packet in the sequence has probability of occurance. Entropy calculates according to formula above. PAGE 14

15 Network packets parameters selection Shape Parameter value Source IP: Source IP: Source IP: PAGE 15

16 Network packets parameters selection Shape Parameter value Destination port: 53 Destination port: 21 Destination port: 137 PAGE 16

17 Network packets parameter selection Shape Parameter value : : :137 PAGE 17

18 Network packets parameters selection Entropy calculation practice Network packets parameters used for analyze entropy Network addresses (source and destination) Ports Another network packets fields and combinations. PAGE 18

19 Main tasks Entropy calculation practice For each network packet 1. Calculate entropy PAGE 19

20 Main tasks Entropy calculation practice For each network packet 1. Calculate entropy 2. It is necessary to calculate it FAST!!! PAGE 20

21 Entropy calculation PAGE 21

22 Entropy calculation p i =? PAGE 22

23 Entropy calculation We should use probability estimation PAGE 23

24 Entropy calculation Select calculation window: W={ω 1, ω 2,,ω N } N elements PAGE 24

25 Entropy calculation N elements PAGE 25

26 Entropy calculation N elements PAGE 26

27 Sequential entropy calculation H 2 H 1 There should every time calculate an entropy value for each window PAGE 27

28 Entropies difference calculation Proposal: we should calculate entropy difference not entropy for each window H N = H 2 - H 1 H 2 H 1 PAGE 28

29 Entropies difference calculation Proposal: we should calculate entropies difference W 2 W 1 Hatched and non-hatched letters - probabilities of selected parameters value for windows W 1 and W 2 respectively PAGE 29

30 Reduce amount of calculations Notice: only elements in and out affect the difference in amount of elements of every type in every following windows. out W 2 W 1 in PAGE 30

31 Preliminary data for algorithm Entropy calculation practice Array of elements - values of selected network packet parameter Hash-array, maps number of elements for each parameter value Value Number of elements PAGE 31

32 Algorithm first step Entropy calculation practice 1. Fill elements with the initial values 2. Fill hash-array with the number of elements from the initial values Value Number of elements Calculate entropy H 0 for the initial values PAGE 32

33 Main algorithm step Entropy calculation practice 1. For every new parameter value calculate entropies difference 2. Calculate entropy value H i+1 N=H i N+ H N 3. Update hash-array Value Number of elements PAGE 33

34 Algorithm rate comparison Entropy calculation practice Comparison for the same data of packets. Time in second obtained with time utility. Window size (number of packets) Entropies difference algorithm Sequential entropy calculation algorithm s s s s s s s s s s. PAGE 34

35 Thank You Oleg Gudkov, BMSTU IT Security for the Next Generation International Round, Delft University of Technology May, 2012 The Netherlands