Statistical Aspects of Intrusion Detection II

Transcription

1 Statistical Aspects of Intrusion Detection II Mgr. Rudolf B. Blažek, Ph.D. Department of Computer Systems Faculty of Information Technologies Czech Technical University in Prague Rudolf Blažek Network Security MI-SIB, ZS 2011/12, Lecture 9 The European Social Fund Prague & EU: We Invest in Your Future

2 Statistické aspekty detekce síťových útoků II Mgr. Rudolf B. Blažek, Ph.D. Katedra počítačových systémů Fakulta informačních technologií České vysoké učení technické v Praze Rudolf Blažek Síťová bezpečnost MI-SIB, ZS 2011/12, Přednáška 9 Evropský sociální fond Praha & EU: Investujeme do vaší budoucnosf

3 Statistical Aspects of Intrusion Detection 3

4 Introduction Statistical Aspects of Intrusion Detection Non-statistical features of network intrusions: Network protocols are deterministic and well understood Protocol anomalies can be detected by stateful analysis Many ad-hoc methods work well to detect various attacks Example of a good deterministic ad-hoc detection rule Suspect a host is using P2P transfers if: it uses network flows via port 6881 uses ports above 50000, with many port changes connects to many IPs, most of them inaccessible at the end many connection finish at the same time 4

5 Introduction Statistical Aspects of Intrusion Detection Statistical features of network intrusions: Network intrusions occur randomly Intrusions occur at unknown points in time Intrusions lead to changes of statistical properties of some observable characteristics Attack detection viewed as a change-point detection (CPD): Detect changes in the distributions (models, parameters) With fixed delays (batch-sequential approach) Or with minimal average delays (sequential approach) While maintaining the false alarm rate at a given level 5

6 Review of Statistical Hypothesis Testing 6

7 Review of Statistical Hypothesis Testing Test if a Coin is Fair: P(H)=1/2 Toss the coin repeatedly n times Count the number of Heads Estimate P(H) as proportion of Heads pn = #Heads / n If pn is close to 1/2, we will believe the coin is fair 7

8 What does it mean? If pn is close to 1/2, we believe the coin is fair? How close? How sure are we? 8

9 Experiments - Fair Coin

10 Experiments - Fair Coin

11 Strong Law of Large Numbers (SLLN) Assume random variables X1, X2, X3,... are independent and identically distributed (i.i.d.) have finite mean µ = E Xi Then the sample means converge to µ: X n = 1 n (X 1 + X X n )! µ as n!1!!!!!! almost surely (for almost all outcomes) 11

12 Cauchy Distribution

13 Proportion of Heads is the Sample Mean Define an indicator for Heads Xi = 0 if the i-th toss is T Xi = 1 if the i-th toss is H Expected value is finite µ = E(Xi) = 1 P(H) + 0 P(T) = P(H) SLLN: Proportion of Heads in n tosses will converge X n = 1 n (X 1 + X X n )! µ = P (H) as n!1 13

14 SLLN for a Fair Coin

15 Testing if a Coin is Fair pn = proportion of heads in n tosses If pn is close to 1/2, we believe the coin is fair How close? SLLN: Eventually pn will approach P(H) So, being close to 1/2 indicates the coin is fair How sure are we that the coin is fair? Far from 1/2 = unusually far from 1/2 We need to learn about the distribution of pn 15

16 Central Limit Theorem (CLT) Assume random variables X1, X2, X3,... are independent and identically distributed (i.i.d.) have finite mean µ = E Xi and variance σ 2 = Var Xi Then the sample means have approximately Gaussian distribution: X n = 1 n (X 1 + X X n ) N(µ, approximately for large n 2 n ) 16

17 Fair Coin n= %

18 Fair Coin n= %

19 Fair Coin n= %

20 Fair Coin n= %

21 Fair Coin n= %

22 Fair Coin n= %

23 Hypothesis Testing for Fair Coin Hypothesis: ) P(H) = 1/2 Alternative: ) P(H) 1/2 Test: Calculate sample mean pn Reject hypothesis P(H)=1/2 if pn is outside of the 99% region of the Normal curve Probability of error 1% Otherwise accept P(H) = 1/2 Probability of error unknown!! (But smallest possible) 23

24 24

25 Standardization of Gaussian Variables Z ~ N(0,1) 68.27%

26 Standardization of Gaussian Variables Y ~ N(μ,σ 2 ) 68.27% μ σ -1 0μ μ+σ 1 26

27 Standardization of Gaussian Variables X n N(µ, 2 /n) 68.27% μ σ -1 0μ μ+σ 1 / p n / p n 27

28 Standardization of Gaussian Variables Z ~ N(0,1) 95.44%

29 Standardization of Gaussian Variables Y ~ N(μ,σ 2 ) 95.44% μ 2σ -2 0μ μ+2σ 2 29

30 Standardization of Gaussian Variables X n N(µ, 2 /n) 95.44% μ 2σ -2 0μ μ+2σ 2 / p n / p n 30

31 Standardization of Gaussian Variables Z ~ N(0,1) 99.73%

32 Standardization of Gaussian Variables Y ~ N(μ,σ 2 ) 99.73% μ 3σ -3 0μ μ+3σ 3 32

33 Standardization of Gaussian Variables X n N(µ, 2 /n) 99.73% μ 3σ -3 0μ μ+3σ 3 / p n / p n 33

34 The Formulas for The Test Hypothesis: ) µ = µ0 (1/2 for Fair coin) CLT: p n = X n = 1 n (X 1 + X X n ) N(µ 0, 2 n ) approximately for large n Alternative: ) µ µ0 (1-α)100% Region for pn: 99.73%:)zα/2 = 3 99%: ) zα/2 = %: ) zα/2 = 1.96 µ 0 ± z /2 p n 34