Nino Pellegrino October the 20th, 2015

Transcription

1 Learning Behavioral Fingerprints from NetFlows... using Timed Automata Nino Pellegrino October the 20th, 2015 Nino Pellegrino Learning Behavioral Fingerprints October the 20th, / 32

2 Use case Nino Pellegrino Learning Behavioral Fingerprints October the 20th, / 32

3 Use case What does behavioral ngerprint exactly mean? How is it possible to detect a malicious host using behavioral ngerprints? Nino Pellegrino Learning Behavioral Fingerprints October the 20th, / 32

4 NetFlows Characterized by several properties derived from the aggregation of packet-based features. FEATURE TYPE VALUES source-ip string protocol string TCP, UDP direction string ->, <-, <->, <?> start-time timestamp :51: duration oat 0.103, total-packets integer 9, 1 total-bytes integer 1030, 66, 43 Nino Pellegrino Learning Behavioral Fingerprints October the 20th, / 32

5 NetFlows PRO: frequently logged by network operators and much easy to obtain. PRO: (more) privacy preserving. In contrast to network packets, NetFlows do not contain content and format elds. PRO: scale smoothly with big data amounts CON: automata learned from NetFlows dene behavior on a high abstraction level. Nino Pellegrino Learning Behavioral Fingerprints October the 20th, / 32

6 Timed Events Timed Events are couples (timestamp, symbol): NetFlow TIMED EVENT , udp, ->, 15:51:09, , 1, 68 (1,a) , udp, <->, 15:52:01, , 5, 590 (52,b) , tcp, <->, 15:53:46, , 3, 479 (150,c) Timed Events represent a symbolic abstraction of actual NetFlows data Nino Pellegrino Learning Behavioral Fingerprints October the 20th, / 32

7 Obtaining Events The symbolic part of a Timed Event is set in a way such that two NetFlows exhibiting the same features get the symbol. Cathegorical features, as direction or protocol, have been mapped to progressive positive numbers basing on their values. EXAMPLE: if protocol=udp then 0 if protocol=tcp then 1 etc. Nino Pellegrino Learning Behavioral Fingerprints October the 20th, / 32

8 Obtaining Events Numerical features, as duration or total packets, have been mapped according to the 20th, 40th, 60th, and 80th percentiles. EXAMPLE (total-packets feature): Nino Pellegrino Learning Behavioral Fingerprints October the 20th, / 32

9 Obtaining TIMED sequences of events The temporal part of a Timed Event t i is set to the dierence in the start-time from the previously seen Timed Event t i 1 prot dir time duration packet byte event udp -> (0,a) udp <-> (5,b) tcp <-> (12,c) udp -> (5,c) tcp <-> (2,d) Sequences of Timed Events are generated by sliding a temporal window of xed duration, i.e. 20 milliseconds. s 1 = (0,a)(5,b)(12,c) s 2 = (5,b)(12,c)(5,b)(2,d) Nino Pellegrino Learning Behavioral Fingerprints October the 20th, / 32

10 Learning a stateful model Positive data: aa, b, bba; Negative data: a, aaa, aabb Nino Pellegrino Learning Behavioral Fingerprints October the 20th, / 32

11 Learning a stateful model Select two nodes Nino Pellegrino Learning Behavioral Fingerprints October the 20th, / 32

12 Learning a stateful model Move input transitions Nino Pellegrino Learning Behavioral Fingerprints October the 20th, / 32

13 Learning a stateful model Move output transitions Nino Pellegrino Learning Behavioral Fingerprints October the 20th, / 32

14 Learning a stateful model Move output transitions Nino Pellegrino Learning Behavioral Fingerprints October the 20th, / 32

15 Learning a stateful model Delete the obsolete state Nino Pellegrino Learning Behavioral Fingerprints October the 20th, / 32

16 Learning a stateful model Determinization Nino Pellegrino Learning Behavioral Fingerprints October the 20th, / 32

17 Learning a stateful model Determinization Nino Pellegrino Learning Behavioral Fingerprints October the 20th, / 32

18 Learning a stateful model Determinization Nino Pellegrino Learning Behavioral Fingerprints October the 20th, / 32

19 Learning a stateful model Determinization Nino Pellegrino Learning Behavioral Fingerprints October the 20th, / 32

20 Learning a stateful model Determinization Nino Pellegrino Learning Behavioral Fingerprints October the 20th, / 32

21 Learning a stateful model Determinization Nino Pellegrino Learning Behavioral Fingerprints October the 20th, / 32

22 Learning a stateful model Select two nodes, iterate Nino Pellegrino Learning Behavioral Fingerprints October the 20th, / 32

23 Learning a stateful model Select two nodes, iterate Nino Pellegrino Learning Behavioral Fingerprints October the 20th, / 32

24 Recognizing a host as infected Two dierent strategies for nding infection on candidate hosts. Both strategies rely on infection symptoms. Infection symptoms are all couples (state, timed-event) collected using a model on testing data. Nino Pellegrino Learning Behavioral Fingerprints October the 20th, / 32

25 Error Based Strategy Evaluates whether a candidate host C shows the same symptoms occurrences as a known malicious host M. Let Countsi M and Countsi C be counts of symptom i in M and C, respectively, host C is classied as infected if Counts M i Counts C i < τ i i.e. if the absolute error between the expected and observed symptom counts if below a pre-computed threshold. Nino Pellegrino Learning Behavioral Fingerprints October the 20th, / 32

26 Fingerprint Based Strategy Uses a conguration dataset to look for distinguishing symptoms. distinguishing symptoms characterize malicious hosts, but never occur in any host in the conguration datatset. Let Countsi F denote occurrences of a symptom i in such dataset, host C is considered malicious if: Counts F i = 0 Counts M i > 0 Counts C i > 0 Nino Pellegrino Learning Behavioral Fingerprints October the 20th, / 32

27 Per-scenario Performances Scenario Conf Size Train Size Eval Size Infected error based ngerprint based TP TN FP FN TP TN FP FN Nino Pellegrino Learning Behavioral Fingerprints October the 20th, / 32

28 Error Based Strategy True Negative (S1) False Positive (S2) True Positive (S1, S2) True Positive (S1, S2) True Negative (S1, S2) True Negative (S1, S2) Observed Occurence Expected Occurence Nino Pellegrino Learning Behavioral Fingerprints October the 20th, / 32

29 Detection performance on unseen malware Scenario TP FP Accuracy F-measure Accuracy = F-Measure = TP + TN TP + FP + TN + FN 2TP 2TP + FP + FN Nino Pellegrino Learning Behavioral Fingerprints October the 20th, / 32

30 Obtaining Events Using mapping values we assign for each NetFlow in a single positive number using the following encoding algorithm: Input: a NetFlow n = a 0, a 1,, a k with k features Input: an attribute mapping M i, i = 0, 1,, k Output: integer code for n code 0; spacesize k i=0 Dom(M i(a i )) ; for i 0 to k do code code + M i (a i ) spacesize Dom(M i ) ; spacesize spacesize Dom(M i ) ; return code; Nino Pellegrino Learning Behavioral Fingerprints October the 20th, / 32

31 Example If we have two features mapping domain size protocol TCP=0, UDP=1, <unknown>=2 3 total-packets 20th: 7, 40th: 22, 60th: 30, 80th: 32 5 Then Encode( TCP, 12 ) = = 1 Encode( UDP, 45 ) = = 9 Encode( ICMP, 25 ) = = 12 Nino Pellegrino Learning Behavioral Fingerprints October the 20th, / 32

32 Host behavior descriptions: PDRTAs root state 1 [ ] [ ] Q-TCP [171,195] TCP Q-UDP [0,203] state 2 [ ] [ ] Q-TCP TCP [0,1] Q-UDP [204,2759] Q-UDP [2760,max] UDP Q-TCP Q-UDP Q-TCP [161,170] state 5 [ ] [ ] TCP [2,max] Q-TCP [31,153] Q-TCP [196,max] Q-UDP Q-UDP TCP Q-TCP [0,30] Q-TCP [154,160] state 3 [ ] [ ] Q-UDP UDP state 6 [ ] [ ] TCP TCP state 9 [ ] [ ] TCP Q-TCP Q-UDP Q-TCP TCP state 4 [ ] [ ] Q-TCP TCP TCP state 8 [ ] [ ] Q-TCP state 7 [ ] [ ] Nino Pellegrino Learning Behavioral Fingerprints October the 20th, / 32