Bypassing Space Explosion in Regular Expression Matching for Network Intrusion Detection and Prevention Systems

Transcription

1 Bypssing Spce Explosion in Regulr Expression Mtching for Network Intrusion Detection n Prevention Systems Jignesh Ptel, Alex Liu n Eric Torng Dept. of Computer Science n Engineering Michign Stte University

2 Prolem Sttement Core opertion in IDS/IPS is Deep Pcket Inspection Pst DPI string mtching Current DPI regulr expression (RE) mtching Exmple SNORT, Bro Prolem given set of REs, how to quickly scn pcket pylo to etermine which REs re mtche? Jignesh Ptel - Michign Stte University 2

3 Solution using Automt Common solution is to uil n equivlent Finite Stte Automt se on DFA. DFA size grows exponentilly with numer of REs. Severl lternte utomt hve een propose D 2 FA, XFA, δfa etc. Jignesh Ptel - Michign Stte University 3

4 Limittions of Prior Work Prior solution Union then Minimize frmework. First comine DFA for the whole RE set is uilt. Compression technique is pplie to the comine DFA to get the lternte utomt. Merge RE 1 RE 2 NFA 1 NFA 2 NFA Suset construc4on DFA min. DFA Compression Algorithm Alternte Automt RE n NFA n Prolems The minimiztion/compression is pplie on lrge comine utomt, hence requires too much time n memory. The intermeite DFA might e too lrge to fit in memory. Whole utomt nees to e reuilt if new RE is e to set. Jignesh Ptel - Michign Stte University 4

5 Our Approch Our pproch Minimize then Union frmework. Buil iniviul DFAs for ech RE in the RE set. Compress ech DFA to get iniviul lternte utomt. Merge the ll compresse lternte utomt together. RE to Min DFA Compression Algorithm Merge RE 1 DFA 1 Alt. Automt 1 RE 2 DFA 2 Alt. Automt 2 Alternte Automt RE n DFA n Alt. Automt n Avntges The compression lgorithm is pplie to smll DFAs. Lrge intermeite DFA oes not nee to e uilt. Esy to new RE to the set with one merge. In this work we focus on the D 2 FA. Jignesh Ptel - Michign Stte University 5

6 D 2 FA Overview D 2 FA [Kumr et l., 26] uses common trnsitions etween sttes to reuce the numer of trnsitions. s 8 s 7 c 1 s 6 To uil D 2 FA 1. We choose eferre stte for ech stte in the DFA. 2. For ech stte, remove trnsitions tht re common with its eferre stte. s 2 s 5 s 1 c 2 c 1 c 3 c 3 c 2 c 1 s 4 s 3 Jignesh Ptel - Michign Stte University 6

7 D 2 FA Construction Buil Spce Reuction Grph (SRG) Fin mximum spnning tree (MST) in SRG. Use the MST to set eferre sttes. from [,3] -{,} fil 1 3/1 c -{,} 2 c 4 6/2 from [4,7] fil 5 7/ SRG DDFA 2 for RE set {, c.*} Trnsi4ons Jignesh Ptel - Michign Stte University

8 DFA for RE Mtching in DPI from [,3] fil 1 3/1 2 c 4 6/2 from [4,7] c fil 5 7/1 DFA for RE set {, c.*} Jignesh Ptel - Michign Stte University 8

9 Merging DFAs (1) DFA for RE from [,2] fil 1 2/1 from,1 fil from [,3], fil 1, 1 2, fil c 1 2 3/2 from [4,7] fil,1 2 c,2 4,3 6/2 1,1 c 1,2 5 1,3 2,1 3/1 2,2 7/1 2,3 DFA for RE c.* Merge DFA for RE set {, c.*} Jignesh Ptel - Michign Stte University 9

10 Merging DFAs (2) Jignesh Ptel - Michign Stte University 1

11 Merging D 2 FA We exten the UCP construction for merging DFAs to merge D 2 FAs. To generte D 2 FA, we nee to set eferre stte for ech stte. Set the eferre stte s soon s new stte is crete. Since eferre stte is set when stte is crete, we only nee to store the non-eferre trnsitions for the stte. The whole DFA is never uilt since we lwys store the D 2 FA. Jignesh Ptel - Michign Stte University 11

12 Setting Deferre Stte D 1 v v 1 v 2 v i D 3 u D 2 w w 1 w 2 w j Jignesh Ptel - Michign Stte University 12

13 Merging D 2 FA Exmple For most sttes, one of the first pir is the est pir. D 2 FA for RE - 1 2/1 In our experiments, verge numer of comprisons neee < 1.5 -, -{,} 1, 1 1,1 2 2,1 3/1 c - c -{,} D 1 2 2,2 4 1,2 5 2,2 7/1 D 3 D /2 D 2 FA for RE c.*,3 6/2 Merge D 2 FA for RE set {, c.*} Jignesh Ptel - Michign Stte University 13

14 Experimentl Results Min We use rel worl 8 RE sets tht were use in prior work for our experiments. We group the 8 RE sets into three groups ccoring to type of REs in the sets STRING, WILDCARD, SNORT We compre D 2 FA Merge lgorithm with the Originl D 2 FA lgorithm. RE set group # Sttes / ASCII len. Trns increse Def. epth rtio Avg. Mx. Spce rtio Speeup fctor All % STRING.7 44.% WILDCARD 36 3.% SNORT % Jignesh Ptel - Michign Stte University 14

15 Experimentl Results Scle To test sclility we use synthetic RE set with REs of the form /c 1 c 2 c 3 c 4.*c 5 c 6 c 7 c 8 / We one RE t time until memory estimte goes over 1GB. Originl D 2 FA lgorithm # REs e 12 # sttes in finl D 2 FA 397,312 Time to uil D 2 FA 71 hours D 2 FA Merge lgorithm # REs e 19 # sttes in finl D 2 FA 8,216,64 Time to uil D 2 FA 1.2 hours For 12 REs, D 2 FA Merge only nees 1 secons to uil. D 2 FA Merge results in sme D 2 FA size s the originl lgorithm. Jignesh Ptel - Michign Stte University 15

16 Questions? Thnk you for listening! Jignesh Ptel - Michign Stte University 16