DSIE 10 5 th Doctoral lsymposium on Informatics Engineering i January 28 29, 2010 Porto, Portugal Self Protection Techniques in Malware Tiago Santos
Overview Introduction Malware Types Why Self Protection? Stealth, Evasion, Obfuscation Code Obsfuscation Entry Point Obfuscation Compression / Packers Encryption Anti Debugger Anti Emulation Anti Disassemble Retrovirus Conclusions
Introduction
Malware Software that can disrupt the normal functioning of a system. Installed without the knowledge or approval of the user Why? Experimental or Research Anti social bh behaviour Prank Ideological Profit/Extortion Warfare
Malware Software that can disrupt the normal functioning of a system. Installed without the knowledge or approval of the user Why? Experimental or Research Anti social bh behaviour Prank Ideological Profit/Extortion Warfare
Malware Software that can disrupt the normal functioning of a system. Installed without the knowledge or approval of the user Why? Experimental or Research Anti social bh behaviour Prank Ideological Profit/Extortion Warfare
Types Adware Ransomware Browser Hijack Rootkit Backdoor Scumware Botnet Spam Bugs Cryptovirus Dialer Exploit e Bomb Keylogger Hoaxes Logic Bombs Rabbit Spyware Trojans Virus Worms
Why Self Protection? Increase thespace betweenthetemporal the temporal points of the malware s life: infection > detection > analysis > defense
Why Self Protection? Increase thespace betweenthetemporal the temporal points of the malware s life: infection > detection > analysis > defense
Stealth, Evasion, Obfuscation
Self Protection Methods PASSIVE Code Obfuscation Entry Point Obfuscation Encryption Compression Packing Oligomorphism Polymorphism Metamorphism ACTIVE Anti emulation Anti debugging Anti disassembling Retro virus
Self Protection Methods PASSIVE Code Obfuscation Entry Point Obfuscation Encryption Compression Packing Oligomorphism Polymorphism Metamorphism ACTIVE Anti emulation Anti debugging Anti disassembling Retro virus
Self Protection Methods PASSIVE Code Obfuscation Entry Point Obfuscation Encryption Compression Packing Oligomorphism Polymorphism Metamorphism ACTIVE Anti emulation Anti debugging Anti disassembling Retro virus
Source: Alisa Shevchenko
Code Obsfuscation Inserting junk code Code transposition Exchanging registers Instruction substitution Commom problem: the code can increase significantly! Solution: opcode confusion undocumented instructions LEA EAX, DWORD PTR [040200H] JMP EAX MOV AX, 0200H MOVBX BX, 0004H AND EAX, 0XFFFF SHL EBX, 16 OR EAX, EBX JMP EAX
Code Obsfuscation Inserting junk code Code transposition Exchanging registers Instruction substitution Commom problem: the code can increase significantly! Solution: opcode confusion undocumented instructions LEA EAX, DWORD PTR [040200H] JMP EAX MOV AX, 0200H MOVBX BX, 0004H AND EAX, 0XFFFF SHL EBX, 16 OR EAX, EBX JMP EAX
Code Obsfuscation Inserting junk code Code transposition Exchanging registers Instruction substitution Commom problem: the code can increase significantly! Solution: opcode confusion undocumented instructions LEA EAX, DWORD PTR [040200H] JMP EAX
Code Obsfuscation Inserting junk code Code transposition Exchanging registers Instruction substitution Commom problem: the code can increase significantly! Solution: opcode confusion undocumented instructions LEA EAX, DWORD PTR [040200H] JMP EAX MOV AX, 0200H MOVBX BX, 0004H AND EAX, 0XFFFF SHL EBX, 16 OR EAX, EBX JMP EAX
Entry Point Obfuscation Anti heuristic method Hard to detect, desinfect and remove Can get control in any point (e.g. in the end => replace API ExistProcess() by a jump to the viral section )
Entry Point Obfuscation Anti heuristic method Hard to detect, desinfect and remove Can get control in any point (e.g. in the end => replace API ExistProcess() by a jump to the viral section )
Entry Point Obfuscation Anti heuristic method Hard to detect, desinfect and remove Can get control in any point (e.g. in the end => replace API ExistProcess() by a jump to the viral section )
Compression / Packers Reduces significantly the malware size Makes the code analysis much more difficult and tedious Efficient commercial andfree software (Problem: compression/decompression algorithm are known by the AV software Can use available software in the infected system It is possible to change the entire packed code by just changing gone single byte Some packers incorporates in the packed file antidebugging capabilities.
Compression / Packers Reduces significantly the malware size Makes the code analysis much more difficult and tedious Efficient commercial andfree software (Problem: compression/decompression algorithm are known by the AV software Can use available software in the infected system It is possible to change the entire packed code by just changing gone single byte Some packers incorporates in the packed file antidebugging capabilities.
Compression / Packers Reduces significantly the malware size Makes the code analysis much more difficult and tedious Efficient commercial andfree software (Problem: compression/decompression algorithm are known by the AV software Can use available software in the infected system It is possible to change the entire packed code by just changing gone single byte Some packers incorporates in the packed file antidebugging capabilities.
Compression / Packers Reduces significantly the malware size Makes the code analysis much more difficult and tedious Efficient commercial andfree software (Problem: compression/decompression algorithm are known by the AV software Can use available software in the infected system It is possible to change the entire packed code by just changing gone single byte Some packers incorporates in the packed file antidebugging capabilities.
Compression / Packers Reduces significantly the malware size Makes the code analysis much more difficult and tedious Efficient commercial andfree software (Problem: compression/decompression algorithm are known by the AV software Can use available software in the infected system It is possible to change the entire packed code by just changing gone single byte Some packers incorporates in the packed file antidebugging capabilities.
Compression / Packers Reduces significantly the malware size Makes the code analysis much more difficult and tedious Efficient commercial andfree software (Problem: compression/decompression algorithm are known by the AV software Can use available software in the infected system It is possible to change the entire packed code by just changing gone single byte Some packers incorporates in the packed file antidebugging capabilities.
Encryption why? To hide the malware s fixed signature, making it unrecognizable to a scanner Camouflage constants (e.g.,.exe exe, windows/system32, email addresses, etc) To intricatei the code analysis
Encryption why? To hide the malware s fixed signature, making it unrecognizable to a scanner Camouflage constants (e.g.,.exe exe, windows/system32, email addresses, etc) To intricatei the code analysis
Encryption why? To hide the malware s fixed signature, making it unrecognizable to a scanner Camouflage constants (e.g.,.exe exe, windows/system32, email addresses, etc) To intricatei the code analysis
Encryption Key Points Strenght Key origin and distribution Publicvs private key Speed and reliability (e.g. RC4 faster than RSA) Implementation
Encryption Key Points Strenght Key origin and distribution Publicvs private key Speed and reliability (e.g. RC4 faster than RSA) Implementation
Encryption Key Points Strenght Key origin and distribution Publicvs private key Speed and reliability (e.g. RC4 faster than RSA) Implementation
Encryption Key Points Strenght Key origin and distribution Publicvs private key Speed and reliability (e.g. RC4 faster than RSA) Implementation
Encryption Key Points Strenght Key origin and distribution Publicvs private key Speed and reliability (e.g. RC4 faster than RSA) Implementation
Anti Debugger Different techniques to different debuggers Modified code based detection breakpoint detection : Checksum comparation Checking instructions Software interrupts : INT 3 (for standard Debug Breakpoint 0xCC) Timing based detection check for slowdown in runtime API (e.g. isdebuggerpresent()) Exception based detection many times difficult to bypass (INT 2Dh, INT 0x03, 0xF1, 0Fh 0Bh) Decryption based Instructions prefetching
Anti Debugger Different techniques to different debuggers Modified code based detection breakpoint detection : Checksum comparation Checking instructions Software interrupts : INT 3 (for standard Debug Breakpoint 0xCC) Timing based detection check for slowdown in runtime API (e.g. isdebuggerpresent()) Exception based detection many times difficult to bypass (INT 2Dh, INT 0x03, 0xF1, 0Fh 0Bh) Decryption based Instructions prefetching
Anti Debugger Different techniques to different debuggers Modified code based detection breakpoint detection : Checksum comparation Checking instructions Software interrupts : INT 3 (for standard Debug Breakpoint 0xCC) Timing based detection check for slowdown in runtime API (e.g. isdebuggerpresent()) Exception based detection many times difficult to bypass (INT 2Dh, INT 0x03, 0xF1, 0Fh 0Bh) Decryption based Instructions prefetching
Anti Debugger Different techniques to different debuggers Modified code based detection breakpoint detection : Checksum comparation Checking instructions Software interrupts : INT 3 (for standard Debug Breakpoint 0xCC) Timing based detection check for slowdown in runtime API (e.g. isdebuggerpresent()) Exception based detection many times difficult to bypass (INT 2Dh, INT 0x03, 0xF1, 0Fh 0Bh) Decryption based Instructions prefetching
Anti Emulation Use of triggers Sparse infection Sparse payload delivery Decryption: Nested decryption loops Brute force decryption algorithms Non linear decryption algorithms Overextending: Undocumented instructions Instructions not implemented dby the emulator Fool the emulator Network connection
Anti Emulation Use of triggers Sparse infection Sparse payload delivery Decryption: Nested decryption loops Brute force decryption algorithms Non linear decryption algorithms Overextending: Undocumented instructions Instructions not implemented dby the emulator Fool the emulator Network connection
Anti Emulation Use of triggers Sparse infection Sparse payload delivery Decryption: Nested decryption loops Brute force decryption algorithms Non linear decryption algorithms Overextending: Undocumented instructions Instructions not implemented dby the emulator Fool the emulator Network connection
Anti Emulation Use of triggers Sparse infection Sparse payload delivery Decryption: Nested decryption loops Brute force decryption algorithms Non linear decryption algorithms Overextending: Undocumented instructions Instructions not implemented dby the emulator Fool the emulator Network connection
Anti Disassemble Trick the disassembler by generating an incorrect disassembled code
Anti Disassemble Trick the disassembler by generating an incorrect disassembled code
Anti Disassemble Trick the disassembler by generating an incorrect disassembled code MOV BX, 0xF0F0 CMP BX, 0 JE @OPAQUE JNE @REALCODE @OPAQUE: DB 50H @REALCODE: ADD CX,BX XOR EAX, EAX...
Anti Disassemble Trick the disassembler by generating an incorrect disassembled code MOV BX, 0xF0F0 CMP BX, 0 JE @OPAQUE JNE @REALCODE @OPAQUE: DB 50H @REALCODE: ADD CX,BX XOR EAX, EAX... 401000 MOV BX,0F0F0 401004 CMP BX,0 401008 JE SHORT 0040100C 40100A JNZ SHORT 40100D 40100C PUSH EAX 40100D XOR CX,BX 401010 ADD EAX,EAX...
Fighting AV Software Retrovirus Actively attacks the AV software by trying to bypass or to block the AV s operations, personal firewall or other security programs Easy to implement Attacks: Modify the AV code Delete theav Change the infected system in a way that affects the AV Remove CPU time to the AV Kill pre determinated processes
Fighting AV Software Retrovirus Actively attacks the AV software by trying to bypass or to block the AV s operations, personal firewall or other security programs Easy to implement Attacks: Modify the AV code Delete theav Change the infected system in a way that affects the AV Remove CPU time to the AV Kill pre determinated processes
Fighting AV Software Retrovirus Actively attacks the AV software by trying to bypass or to block the AV s operations, personal firewall or other security programs Easy to implement Attacks: Modify the AV code Delete theav Change the infected system in a way that affects the AV Remove CPU time to the AV Kill pre determinated processes
Conclusions Can a system be safe with an up to date AV software? Due the continue evolution techniques and the development of new ones, it is impossible to have a complete protection even with an up to dt date state tt of the art AV software, making prudent to always assume that a system is unprotected and already infected. It s a never ending conflict between malware writers and AV developers.
Conclusions Can a system be safe with an up to date AV software? Due the continue evolution techniques and the development of new ones, it is impossible to have a complete protection even with an up to dt date state tt of the art AV software, making prudent to always assume that a system is unprotected and already infected. It s a never ending conflict between malware writers and AV developers.
Conclusions Can a system be safe with an up to date AV software? Due the continue evolution techniques and the development of new ones, it is impossible to have a complete protection even with an up to dt date state tt of the art AV software, making prudent to always assume that a system is unprotected and already infected. It s a never ending conflict between malware writers and AV developers.
Conclusions... virus writers don t have to achieve perfect infection. It isn t crucial that infection attempts sometimes fail, or if a virus can t reliably tell whether a file is already infected. Viruses are messy and an imperfect virus can spread quite well in the real world. Conversely, an anti virus product must be extremelyreliable reliable. Unreliable detection, either by identifying a benign file as infected or vice versa, is a fatal flaw flaw. Richard Ford