Enhanced TCP SYN Attack Detection

Transcription

1 Enhanced TCP SYN Aack Deecion Vrizlynn L. L. Thing Morris Sloman Naranker Dulay Deparmen of Compuing Imperial College London 80 Queen s Gae, London, SW72AZ Absrac - In his paper, we analyze he saeless SYN- SYN&ACK and SYN-FIN/RST deecion mechanisms for TCP SYN aacks. We indicae he inheren vulnerabiliy of he SYN-FIN/RST deecion mechanism caused by he compuaion of he RST packe couns. We indicae why SYN- SYN&ACK is a more efficien and reliable deecion mechanism han SYN-FIN/RST. We come up wih Bo Buddies for TCP SYN aacks and explain how he use of hem can compromise boh mechanisms. We propose an enhanced deecion mechanism incorporaing he Bloom filer o handle hese variaions of TCP SYN aacks. We show ha our enhanced mechanism overcomes he problems of he use of Bo Buddies and analyse is efficiency. Keywords - Disribued Denial of Service Aacks, TCP SYN Flood, DDoS Deecion, Nework Securiy.. INTRODUCTION Since is firs appearance in 999, Disribued Denial of Service (DDoS) aacks [] have coninued o become more prevalen in he Inerne, wih aacks argeing banking and financial companies, online gambling firms, web reailers and governmens. The 2007 Symanec Threa Repor [2] indicaes ha over 5000 DoS aacks were observed worldwide on a daily basis. In 2006, backscaer analysis [3] was conduced where DDoS aack raffic was capured. I shows ha over a period of 3 years from 200 o 2004, 22 colleced disinc races revealed 68,700 aacks on over 34,700 disinc Inerne hoss, wih 95% of he aacks using TCP as heir choice of proocol. A recen survey [4] of 55 ier, ier 2 and hybrid IP nework operaors in Norh America, Europe and Asia repored ha DDoS aacks remain he mos significan ISP securiy hrea wih TCP SYN aacks leading he pack. Figure (a) TCP 3-way handshake (b) TCP SYN Aack In TCP, o esablish a connecion (before being able o carry ou daa ransmission beween he server and he clien), he clien sends a SYN o he server. The server allocaes a buffer for he clien and replies wih a SYN&ACK packe. A his sage, he connecion remains in he half-open sae while waiing for he ACK reply from he clien o complee he connecion seup, afer which he 3-way handshake is achieved. TCP SYN DDoS aacks exploi he TCP 3-way handshake. The aackers spoof he source IP addresses of he massive amoun of SYN packes hey send o he vicim servers. As a resul, he SYN&ACK response packes do no reach he aackers machines and he final ACK packes are no sen o he vicim server o complee he 3-way handshake. Therefore, resources a he vicim server are ied up for hese half-open sae connecions creaed by he aackers prevening services o be graned o oher legiimae requess. Previous work on miigaing TCP SYN aacks include SYN cache [5], SYN cookie [6], SYNDefender [7] and Synkill [8]. These work aim o miigae he flooding effec on he vicim server. Previous work on deecing SYN aacks include Specral analysis based deecion [9], SYN arrival rae based deecion [0], SYN-SYN&ACK based deecion [], and SYN-FIN/RST based deecion [2]. Deails are in he nex secion. In his paper, we analyze he wo exising saeless and efficien deecion mechanisms, he SYN-SYN&ACK and SYN-FIN/RST schemes, in view of curren and new variaions of TCP SYN aacks. We find ha hese schemes are vulnerable o exising and new forms of aacks, and propose an enhanced SYN aack deecion scheme o overcome hese vulnerabiliies. 2. RELATED WORK In SYN cache [5], a hash able keeps rack of he halfopen sae connecions insead of relying on he backlog queue provided for each applicaion. Therefore, a higher number of half-open sae connecions are possible bu, during an aack, his is sill insufficien. In addiion, iems in he hash able would have o be dropped consanly o allow for new requess and may resul in even higher overhead and overwhelming he vicim server during an aack. SYN&ACK is used o represen ha boh he SYN and ACK flags are se.

2 SYN cookie [6] eliminaes he need for he backlog queue o keep rack of each SYN reques. Insead, a SYN cookie, also used as he iniial sequence number in he SYN&ACK sen back o he clien, is compued based on a couner a he server, he Maximum segmen size in he SYN queue enry and he TCP header of he reques packe. The ACK reurned from he clien mus conain a sequence number of he SYN cookie plus one o be valid. However, he drawbacks of his scheme are he overhead of compuing he cookies during an aack. Reransmission required in he siuaion of los ACK packe is also no possible as he sae of he connecion reques is no sored. In addiion, TCP opions which are no encoded in he SYN cookie are los, denying he use of cerain TCP performance enhancemen. In SYNDefender [7], he firewall inerceps he SYN reques from he clien and sends he SYN&ACK packe on he behalf of he server. Afer he firewall receives he ACK packe, he reques is hen le hrough o he server. In his way, he server does no need o hold he half-open saes and so does no deplee is resources. However, he weakness is he addiional workload and processing wihin he firewall which migh no cope during a high rae aack. In Synkill [8], source IP addresses are classified in a daabase as good or bad based on observed nework raffic and adminisraively supplied inpu. Bad source addresses are sen he RST packe o erminae heir requess while good ones are allowed o carry on wih he handshaking. However, he above mehods only deal wih miigaing he flooding effec of he SYN aacks and mos of hem are saeful mechanisms, which could be subjeced o DDoS aacks hemselves. Specral analysis based SYN aack deecion [9] moniors he arrival raes of he raffic flows as a signal. The power specral of he signal for a normal TCP flow is found o exhibi srong periodiciy around is round-rip ime (RTT) in boh direcions, whereas ha of he DDoS aack raffic flows are no regulaed in such a way. However, he scheme deals wih long TCP flows. For TCP flows wih shor duraions, he effec of heir saisical muliplexing may ouweigh heir inrinsic periodiciy and be deeced as aack flows. RTTs of flows also vary from rip o rip which makes i difficul o obain a reliable model o represen he normal raffic for differen raffic condiions. Anoher limiaion of he scheme is ha i canno idenify TCP flows wih very small RTTs. The SYN arrival rae based deecion scheme [0] models he arrival rae of he normal SYN packes as a normal disribuion. A very reliable model of he normal raffic paern has o be mainained. I allows a high rae SYN aack o be deeced quickly and accuraely. However, compared o non-parameric approaches such as he SYN-FIN/RST deecion [2], i was no able o deec low rae aack (e.g. < 4 SYNs/sec). Therefore, a low rae aack would sill be able o bring down he vicim server wihou being deeced. Non-parameric approaches such as he SYN- SYN&ACK [] and SYN-FIN/RST [2] deecion mechanisms allow aack deecion even in he case of any variance of normal raffic making hem insensiive o sie and access paerns. The SYN-SYN&ACK deecion mechanism is based on he inheren TCP SYN SYN&ACK pairs behavior. Wih a SYN reques sen ou by a clien o a server, i mus also receive a SYN&ACK reply. An aack hos spoofing as a clien would only send ou massive amoun of SYN requess and no receive any SYN&ACK replies from he vicim server due o is spoofed address. The SYN-SYN&ACK agen moniors he difference of he number of ougoing SYN and incoming SYN&ACK packes. I hen uses he non-parameric Cumulaive Sum (CUSUM) approach o deec any abrup rise in he difference. The SYN-FIN/RST approach proposed by he same auhors allows deecion boh a he aackers and vicim s ends. For a normal TCP connecion, a saring SYN reques will be ended by a FIN/RST o close he connecion. Therefore, correlaion is performed beween he number of SYN and FIN/RST packes insead. However, as RST can be classified as acive (i.e. iniiaed o abor TCP connecion) and passive (i.e. ransmied in response o packes desined o a closed por), and could no be easily differeniaed a he monioring agens, he scheme couns 75% of all RST packes o be acive and he res o be passive (i.e. background noise). Non-parameric CUSUM is also used o deec he abrup rise in he difference beween he SYN and he FIN/RST packes. We analyze boh schemes in he nex secion a heir curren sae and in view of proposed new variaions of TCP SYN aacks. 3. SYN-FIN/RST AND SYN-SYN&ACK ANALYSIS In SYN-FIN/RST, SIN, FIN and RST packes in boh direcions are moniored and couned. To address he issue of passive RST packes, only 75% are couned as valid RST. These valid RST are added o he FIN packes and he difference beween he number of SYN and FIN/RST packes for each monioring inerval is compued. Any abrup posiive flucuaion in his difference would signal he occurrence of an aack. We find ha classifying 75% of RST packes as valid ones weaken he deecion mechanism. RST packes can be generaed by a hos for he following evens: i. arrival of daa packes for which no connecion has been esablished ii. arrival of TCP segmens wih inappropriae sequence numbers iii. arrival of SYN&ACK packes for which no SYN has been iniiaed

3 iv. arrival of TCP packes for closed pors In he even of a SYN aack, aackers could arge random pors as in [3]. In his case, he vicim server would generae RST packes due o (iv). In addiion, source address spoofing is used by he aackers. Therefore, he SYN&ACK packes generaed by he vicim server would be delivered o hoss locaed a he spoofed addresses. They would generae RST packes o he vicim server due eiher o he evens in (iii) and (iv) as well. Whichever he case, RST packes generaed by he above evens should no be classified as valid packes. In [3], DDoS aack ools in Bos could send a mixure of SYN and ACK packes o he vicim. Insead, a new variaion of aack could be easily creaed by sending ou a mixure of SYN, FIN and RST packes o he vicim server. This would resul in balancing he SYN and FIN/RST packes, and hus weakens or even defeas he SYN- FIN/RST deecion mechanism. As menioned in he previous secion, he SYN- SYN&ACK approach is applicable a he source of he aack insead of he vicim s end due o he deecion which is based on he behaviour of SYN and SYN&ACK pairs. Having he deecion mechanism closer o he aack sources allows a speedier deecion before a damaging impac could be a he he vicim server. As he aackers send ou SYN packes and do no receive any SYN&ACK from he vicim due o source IP address spoofing, his approach is very efficien in aack deecion. In comparison o he SYN-FIN/RST deecion scheme, he SYN-SYN&ACK scheme also allows a higher degree of correlaion and accuracy deecion due o he shorer round rip ime beween he SYN-SYN&ACK pairs insead of he ime difference beween he SYN-FIN/RST pairs, which las for he whole duraion of he TCP session. Therefore, a shorer monioring inerval and deecion ime could be achieved in he case of he SYN-SYN&ACK deecion approach. As menioned above, i would be possible o creae a new variaion of he aack by sending ou a mixure of SYN and SYN&ACK packes as well. However, as only he ougoing SYN and incoming SYN&ACK packes are couned, even if he aackers were o make such modificaions o he aack code, i would no have any impac on he deecion mechanism. Insead, he SYN&ACK packes would only weaken he aack (by reducing he SYN aack raffic sen o he vicim due o resources used for sending ou SYN&ACK packes). Insead, we come up wih anoher new variaion of coordinaed aack o defea he SYN-SYN&ACK deecion mechanism. We named his aack he Bo Buddy Aack (shown in Figure 2), as i requires he co-operaion of bos wihin he bone carrying ou he SYN aack. For each SYN packe sen ou o he vicim server, a SYN&ACK packe wih he source address spoofed o he vicim server is sen o anoher bo wihin he bone. In his case, each ougoing SYN packe has an incoming SYN&ACK reply. This aack will herefore circumven he SYN- SYN&ACK deecion. Alhough he diagram shows a 2- buddy bone sysem which is he safes case for he aacker, i is also feasible o have bo responsible for sending o muliple bos (i.e. o Many Bo Buddies Aack) as shown in Figure 3. The reason is ha only he ougoing SYN and incoming SYN&ACK couns are moniored. In addiion, as he Bo maser has a lis of all he bos in he bone, his aack could be easily implemened. Figure 2: Bo Buddy Aack SYN Figure 3: o Many Bo Buddies Aack As he SYN-FIN/RST deecion mechanism has an inheren flaw and he SYN-SYN&ACK deecion mechanism proved o be a more efficien and effecive for SYN aack deecion, we propose an enhancemen o he SYN-SYN&ACK deecion mechanism o resolve he above vulnerabiliies. 4. ENHANCED TCP SYN ATTACK DETECTION As in he original SYN-SYN&ACK deecion mechanism, he packe sniffing agens are locaed a he leaf rouers ha connec end hoss o he Inerne. We consider he SYN and SYN&ACK packes sen by he bos. We assume ha he source IP addresses of he SYN packes are randomly spoofed (sysemaic spoofing will be considered in fuure work), wheher all 4 byes or jus he hos suffix of he IP address. Therefore, he following siuaion shown in Figure 4 arises. SYN

4 availabiliy of sorage space on he leaf rouer. The Bloom filer has a zero false negaive (i.e. if an elemen has been sored, i will be found in he filer) bu a non-zero false posiive (i.e. if an elemen has no been sored, i migh sill reurn he saus of found ). Assuming ha he hash funcions spread he elemens evenly across he filer array, le p 0 be he probabiliy ha a bi in he array is no se (i.e. 0) by any of he hash funcions afer soring n elemens. Le p be he probabiliy ha he bi is se (i.e. ). Figure 4: Aack Packe s Header Since he source IP address used in he SYN packe does no mach he desinaion IP address in he SYN&ACK packe received a Bo_, we could perform SYN-SYN&ACK pair maching o eliminae he effec caused by he Bo Buddy Aack on he SYN-SYN&ACK deecion mechanism. Insead of soring he flow addresses o perform maching, we propose using Bloom filers [4] o achieve space and ime efficiency. We describe he algorihm of our mechanism as follows. In our deecion mechanism, we define our Bloom filer, F[0 m-], as an m-bi array which is iniialized o 0. We define each elemen o be sored in he filer as e ou, which corresponds o he SYN packe being sen ou. e ou = src des k hash funcions, h () h k (), are used o compue k key values for e ou and e ou is sored ino he filer by, F[ h ( e ) mod m] =, for i 0o k i ou = () (2) For he incoming SYN&ACK packe, he same equaion () o compue e ou is used o compue he corresponding elemen, e in. In a normal TCP 3-way handshake, here will be an ougoing SYN wih elemen value of e ou which is equal o an incoming SYN&ACK wih elemen value of e in. To be couned as a valid SYN&ACK packe for inclusion ino he CUSUM deecion algorihm, all he bis a posiions, { hi ( ein ) mod m} in he filer array mus be se o. In he nex secion, we perform evaluaions and analysis of our deecion mechanism. 5. EVALUATION AND ANALYSIS Using he Bloom filer o validae he SYN&ACK replies allows space and ime efficiency. The ime required o sore and search for an elemen in he filer is a fixed consan, O(k), which is independen of he size of he filer and he number of sored elemens. The space allocaed o he Bloom filer is m bis. This allocaion depends on he p 0 p = m = p0 kn (3) (4) For a false posiive error o occur, during a search of an elemen, each of he k array posiions compued by he hash funcions for he elemen mus be se o. Therefore, he probabiliy of a false posiive error, p e, is given by: k p = ( p ) (5) e We see ha, as m, he array size, increases, p e will decrease. However, as n, he number of elemens o be sored, increases, p e will increase oo. Oher han he false posiive error from he Bloom filer, we may consider he possibiliy of error coming from he address spoofing algorihm. Tha is, if he spoofed source IP address of he ougoing SYN packe happens o be he same as he desinaion of he incoming SYN&ACK reply generaed by he Bo Buddy. This address would also be he real address of he Bo sending ou he SYN aack packe. Alhough he SYN packe would no consiue an aack packe in his case as a SYN&ACK reply would be received from he vicim hus compleing he 3-way handshake and esablishing he connecion, we have o ake ino consideraion he addiional SYN&ACK reply ha would come from he bo buddy. Assuming ha all 32 bis of address are spoofed, he probabiliy of collision (wih he real source address) is given as: p c = 32 2 (6) However, if parial address spoofing whereby nework prefix of he real address is preserved when performing spoofed address generaion, and q is he number of bis of he preserved nework prefix, pc = 2 32 q (7) We now analyze our enhanced deecion mechanism aking ino consideraion he false posiive error of he Bloom filer and he collision of he spoofed address. As in he original SYN-SYN&ACK deecion mechanism, le {, =0,, } be he number of ougoing SYN packes

5 minus ha of he incoming SYN&ACK packes colleced from each sampling period. = SYN SYN & ACK (8) To alleviae is dependence on he ime, access paern and size of he nework, { } is normalized by he average number K of incoming SYN&ACK packes during he sampling period. The recursive esimaion of K is given by: K = α K + ( α) SYN & ACK (9) where is he discree ime index and α is a consan beween 0 and o represen he memory in he esimaion. We define X = / K whereby he mean of X, denoed by c, is much less han. In general, E( X ) = c <. A parameer a > c is chosen ~ and X = X a is defined so ha a negaive mean is achievable during normal operaion. When an aack occurs, X ~ quickly become a large posiive number. The abrup rise deecion is based on he observaion of h >> c, whereby he increase in he mean of X ~ can be lower bounded by h. y y 0 = 0 ~ X = ( + y ) + (0) () y is defined as he maximum coninuous incremen unil ime n. A large y is a srong indicaion of an aack. ~ Equaion () indicaes ha y is se o ( y + X ) if his value is 0 else i is se o 0. N is defined as he aack hreshold, i.e. y N indicaes an aack is deeced. Taking ino consideraion he false posiive errors in he Bloom filer search and he collision error of he spoofed address, we ge = SYN, norm SYN & ACK, norm + SYN, a Err, a (2) during a Bo Buddy SYN aack. SYN,norm are he number of SYN packes and SYN&ACK,norm are he number of SYN&ACK packes from he legiimae raffic respecively, in inerval. SYN,a are he number of SYN packes and Err,a are he number of SYN&ACK packes from he Bo Buddy aack respecively, a inerval. Noe ha in he original SYN-SYN&ACK deecion mechanism, Err,a will be large (i.e. SYN,a ) and he aack will no be deecable. In our deecion approach, Err,a is given by: Err SYN, a, a = i= (2 p c + ( p c )( p e )) (3) As in he case of collision, 2p c represens one SYN&ACK reurned from he vicim server and he oher from he Bo Buddy. Only when here is no collision is here a possibiliy of false posiive error in he Bloom filer (i.e. here is no maching SYN elemen sored bu searching reurns rue). This is represened by (-p c )(p e ) in he equaion. n in equaion (3), which is used o derive p e, refers o he number of sored elemens already presen in he bloom filer. A he beginning of he firs sampling period, n sars wih 0. Subsequenly, n is incremened as SYN packes, boh legiimae and aack, arrive. We will consider elemen removal in he fuure work (plan in Secion 6). Using he experimen parameers in he paper describing he original SYN-SYN&ACK deecion mechanism, he sampling period is se o 20 seconds. We choose he aack rae o be 60 SYN packes/second. In [], he normal raffic races daed Sepember 2000, of he ougoing SYN and incoming SYN&ACK were obained from he Universiy of Norh Carolina (moniored on he high-speed OC-2, 622Mbps link connecing is Chapel Hill campus nework o he res of he world). The races show ha he normal ougoing SYN packes flucuaed from around 200 o 900, while he incoming SYN&ACK packes flucuaed from around 050 o 700, in 0-second sampling inervals. The races show consisen synchronizaion beween he SYN and SYN&ACK packes. Therefore, is consisenly around 200 during normal operaions in 0 seconds, which is 400 in 20 seconds, he deecion mechanism s sampling period. As for our mechanism, we se k= (i.e. one hash funcion), m=33,554,432 (i.e. Bloom filer size of 4MB) and assume ha he bo spoofs 32 bis of he source address. In [5], he RTTs for FTP downloads a 6 differen Inerne sies have been found o average o 65.27ms. Therefore, we assume ha each SYN&ACK packe would ake 65.27ms o reach he leaf rouer of he aack bo. The ougoing SYN packes would no incur much delay as he leaf rouer is locaed very close o he aack source. The aack arrival raes are se o 60 packes/second (i.e. around 3 packes every 50ms assuming 5.27ms incurred for he generaion of he aack packes and minor rip ime o he leaf rouer), and he normal SYN raffic arrival rae is averaged a 55 packes/second (i.e. ( )/(2*0)) or around 0 packes every 65.27ms; no delay is assumed here as he daa is obained from races and is he acual arrival rae a he leaf rouer). Therefore, we assume ha n a each arrival of a Bo Buddy SYN&ACK o be 20x550=3000 (i.e. legiimae SYN packes in sampling period) plus he ime slos of 65.27ms ha have passed

6 muliplied by 3, as we assume saring he aack one sampling period laer han he legiimae raffic. The firs compuaion of Err,a will begin only afer he arrival of he firs SYN&ACK packe (i.e. comes afer SYN packes and n will be > 0). Figure 5 shows he Err,a for he samples. We observed ha afer 0 minues of aack, he number of incorrecly validaed SYN&ACK packes is jus abou. Therefore, we can be assured ha our enhanced deecion mechanism will deec SYN aacks effecively even in he face of he Bo Buddy aack. Error Raes (packes) Error Raes for he Sampling Periods Sampling Periods (secs) Figure 5: Error Raes Compuaion 6. FUTURE WORK Boh developmen of he original SYN/SYN&ACK deecion mechanism and our enhanced deecion mechanism is near compleion. We plan o run experimens o compare he performance of boh approaches in he siuaions of he original TCP SYN aack and new variaions of he aack. As hash funcions are used o compue he key values for sorage of he elemens in he Bloom filer, we would like o find ou he overhead incurred in an experimenal scenario. The selecion of k (he number of hash funcions) and m (he size of he Bloom filer), and heir effec on he deecion speed and false posiive rae will also be sudied. We have no considered he removal of elemens from he Bloom filer, which is an essenial feaure of he scheme, in his paper. We will include he sudy in our fuure work as well, considering Couning Bloom Filer. efficien and reliable compared o he SYN-FIN/RST, i fails o deec our Bo Buddy aack. We proposed an enhanced deecion mechanism incorporaing he Bloom filer o handle he aack. We analyzed and evaluaed our enhanced mechanism and found i o work as effecively as he original SYN&ACK, as if he Bo Buddy aack is no presen. 8. ACKNOWLEDGEMENTS This research is coninuing hrough paricipaion in he Inernaional Technology Alliance sponsored by he U.S. Army Research Laboraory and he U.K. Minisry of Defence. 9. REFERENCES. Diane E. Levine and Gary C. Kessler, "Chaper - Denial of Service Aacks, Compuer Securiy Handbook, 4 h Ediion", Ediors - Seymour Bosworh, Michel E. Kabay, Symanec Inerne Securiy Threa Repor, Volume XI, Mar David Moore, e al., "Inferring Inerne Denial-of-Service Aciviy ", ACM Transacions on Compuer Sysem (TOCS), May 2006, 24(2), pp Arbor Neworks, "Worldwide ISP Securiy Repor", Sep Jonahan Lemon, "Resising SYN flood DoS aacks wih a SYN cache", USENIX BSDCon, Daniel J. Bernsein and Eric Shenk, hp://cr.yp.o/syncookies.hml, Check Poin Sofware Technologies Ld., SynDefender, hp:// 8. Chrisoph L. Schuba, e al., "Analysis of a Denial of Service Aack on TCP", IEEE SYmposium on Securiy and Privacy, May Chen-Mou Cheng, e al., "Use of Specral Analysis in Defense Agains DoS Aacks", IEEE GLOBECOM, Yuichi Ohsia, e al., "Deecing Disribued Denial-of-Service Aacks by Analyzing TCP SYN Packes Saisically", IEICE Trans. Comm, Oc. 2006, E89-B(0), pp Haining Wang, e al., "SYN-dog: Sniffing SYN Flooding Sources", Inernaional Conference on Disribued Compuing Sysems, Jul Haining Wang, e al., "Deecing SYN flooding aacks", IEEE INFOCOM, Jun Vrizlynn L. L. Thing, e al., "A Survey of Bos Used for Disribued Denial of Service Aacks", IFIP Inernaional Informaion Securiy Conference (SEC), May Buron H. Bloom, "Space/ime rade-offs in hash coding wih allowable errors", Communicaions of he ACM, Jul. 970, 3(7), pp Bryan Veal, e al., "New Mehods for Passive Esimaion of TCP Round-Trip Times", Passive and Acive Measuremens (PAM), CONCLUSION We have analyzed he saeless SYN-SYN&ACK and SYN-FIN/RST deecion mechanisms. We discover he inheren vulnerabiliy of he SYN-FIN/RST deecion mechanism caused by he RST packe couns. Boh mechanisms also suffered in erms of reliabiliy in view of our new variaions of TCP aacks. Alhough, he SYN- SYN&ACK deecion mechanism is found o be more