Polymorphic Shellcodes vs. Application IDSs

Transcription

1 Plymrphic Shellcdes vs. Applicatin IDSs 1. Intrductin. 2. Shellcde types and recgnitin techniques. 3. Intrusin Detectin Systems. 4. NGSecureWeb. 5. References. 6. Credits. Page 1 f 6

2 1. Intrductin. This dcument fcuses n hw IDS, under certain circumstances, can detect Plymrphic shellcdes. We will g thrugh the three main parts f a plymrphic shellcde and analyze IDS cmmn prblems t detect them. 2. Shellcde types and recgnitin techniques. We will discuss shellcdes referring t IA32 platfrms. Please nte that all these techniques can be implemented n ther platfrms such as SPARC, HPPA, MIPS, etc. Befre plymrphic appeared, regular shellcdes had tw well-defined sectins: NOP sectin: This is the sectin where the prgram will jump when a cmmn buffer verflw is successfully explited. It is just a huge amunt f NOP instructins. Shellcde paylad: This is the sectin where gets executed /bin/sh, binds a shell t a TCP prt, etc. These shellcdes are very easy t detect. On the past years it was very cmmn t use 0x90 instructins (np) n IA32 platfrms in the NOP sectin. S IDS just searched fr an X amunt f 0x90s and triggered a shellcde alarm. Als, sme IDS have signatures (such as /bin/sh) t detect the shellcde paylad. When plymrphic shellcdes appeared these techniques became bslete. Plymrphic shellcdes have three well-defined sectins: NOP sectin: This sectin nw is a randm mix f n-effect instructins such as inc %eax, inc %ebx, pp %eax, np, dec %eax, Decrypter engine: This sectin cntains an engine t decrypt the shellcde paylad. This engine is nt the same frm ne shellcde t anther, it varies randmly using sme virii plymrphic techniques. The cipher used nrmally is a xr mechanism r a duble xr mechanism, it culd be implemented a better ciphering mechanism such as RJINDAEL, but shellcde wuld grw in sme thusands f bytes and wuld nt be useful anymre in many buffer verflw explitatins. Encrypted shellcde paylad: This sectin has the riginal shellcde encrypted. Page 2 f 6

3 This kind f shellcde is harder, thugh nt impssible t detect. There has been sme discussin n hw t detect such shellcdes: Shellcde paylad decrypt and detectin with ld shellcde paylad signatures: This technique, used by antivirus t detect viral cde, can be an apprached t detect these shellcdes. But it has sme pen issues: Hw d yu detect it is an encrypted shellcde paylad? Which cipher mechanism uses? Which key(s) are used in the cipher mechanism? Can it be brute-frced in lw time? Signatures t detect the decrypter engine: This technique culd be a better apprach, but has sme prblems t: Since decrypter engine mutates and t many instructins are invlved, IDS wuld have t check t many signatures (mask). Lts f false psitives. T many CPU cycles and time needed. Decrypter engine emulatin: IDS is emulating the cde s if it finds cde that seems t decrypt smething in memry it raises a shellcde alarm. This technique raises lw number f false psitives but has a strng weakness: T many CPU cycles and time needed. NOPS sectin detectin: INMHO this is the best technique, it just tries t detect a NOP_NUMBER number f n-effect instructins. It has sme issues such as many false psitive, but if yu set NOP_NUMBER t a range between 50-60, recgnizes almst every shellcde. Weaknesses: Lts f CPU cycles. False psitive when NOP_NUMBER is lw. Since sme nn-effect instructins have ascii representatin, sme character strings such as AAAAA 60times A, wuld be recgnized as shellcdes. Page 3 f 6

4 It is very imprtant t set NOP_NUMBER t a reasnable value in rder t avid t many false psitives. Thrughut NGSEC s benchmark test we fund that a number ranging was a gd value t detect shellcdes withut t many false psitives. Yu can grab a free simple Netwrk IDS that implements this technique at: 3. Intrusin Detectin Systems: There are mainly three types f Intrusin Detectin Systems: Netwrk IDS: This type f IDS grabs datagrams frm a netwrk interface and lks fr attack patterns in them (such as prt scanning, cgi explitatin, etc). Hst IDS: This type f IDS lks fr patterns in lcal user actins; e.g. if a user is trying t view such file as /rt/.rhsts, this actin culd clearly be identified as an attack pattern. Applicatin IDS: This type f IDS has recently appeared. It just lks fr attack patterns t all the input data that cmes t the applicatin. Implementing the NOPS sectin detectin n these types f IDS has, as usual, pen issues: Netwrk IDS: Since it has t grab as many datagrams as it can, it can t waste t much time lking at every packet, because it culd drp t many datagrams while lking fr NOPS. This wuld cause a big decrease n IDS perfrmance. Hst IDS: This technique can be implemented with these type f IDS s, but there are better ways f buffer verflw explitatin detectin, since yu can watch the flw f the prgram and see when return pinter is changed. Applicatin IDS: This is the best kind f IDS t implement this technique. Since all input data is checked, n drp f data is pssible, and nrmally nt many data is checked (lw CPU and time t check) yu can recgnize almst all plymrphic shellcdes at input data. Yu have t set the NOP_NUMBER t a value that fits better with yur prtcl, and nrmal data input. Page 4 f 6

5 4. NGSecureWeb : The NOP detectin technique was successfully implemented in NGSecureWeb : An Applicatin IDS and firewall fr Web Servers. NGSecureWeb is the result f the fusin f tw security technlgies: Firewalls and IDSs. NGSW filters all traffic frm client t the web-server, lking fr well-knwn attacks t web-servers and t its third party applicatins. When NGSW detects a pssible attack (acts as an Applicatin IDS), it will refuse t frward it t the web server (acts as an Applicatin Firewall). NGSW will prtect yur system frm bth knwn and unknwn vulnerabilities, since its IDS engine lks fr patterns f well-knwn attacks. Future security flaws in this web-server and its applicatins will nt be explitable. Currently NGSW IDS engine checks fr the fllwing patterns: Directry traversal attacks: During the past few years many CGI applicatins have suffered this kind f vulnerabilities, especially the nes invlved in file managing. NGSW Firewall engine will prtect bth fr knwn and unknwn vulnerabilities. Frbidden Wrds: NGSW will search the entire request fr frbidden wrds (defined by the Administratr). Fr example, it isn t cmmn t find the wrd "/bin/sh" in an HTTP request. NGSW IDS engine detects these wrds and NGSW Firewall engine blcks the request t the web-server. Shellcde: NGSW IDS engine uses the newest techniques in shellcde recgnitin (even plymrphic nes). Lng Headers (buffer verflws): NGSW IDS engine will check the length f all HTTP headers lking fr unusual values. Lng GET (buffer verflws): NGSW IDS engine will check the length f the GET arguments lking fr unusual high values. These high values culd be due f buffer verflw explitatin. Lng POST (buffer verflws): NGSW IDS engine will check the length f the URL arguments lking fr unusual high values. These high values culd be due f buffer verflw explitatin. Lng URL (buffer verflws): NGSW IDS engine will check the length f the URL lking fr unusual high values. These high values culd be due f buffer verflw explitatin. Current versin is Please direct any enquiries t ngsw@ngsec.cm Page 5 f 6

6 5. References: [1] NGSecureWeb at [2] ADM mutate at [3] Fcus-IDS at 6. Credits: This dcument was brught t yu by: Fermín J. Serna < fjserna@ngsec.cm > Chief Technlgy Officer Next Generatin Security Technlgies NGSEC Research Team < labs@ngsec.cm > Page 6 f 6