Network Payload Analysis for Advanced Persistent Threats Charles Smutz, Lockheed Martin CIRT 0000 7/6/2010 1
About Speaker Name Background Current Job Employer Education Charles Smutz Sysadmin, Networking, C&A Lead Software Developer Lockheed Martin CIRT Pursuing PhD at GMU 2
Background Understanding of APT Persistent, Organized, Targeted CNE Typical APT Attack Sequence Importance of Threat Focused CND/Security Intelligence You ll have this by end of Summit 3
Topics Motivation Why do network payload analysis Suggestions for Capabilities What data to collect Importance of Normalized Payload Analysis Importance of Information Retrieval How to implement Capabilities COTS/FOSS Build Your Own 4
Why Network Analysis Important Data Source 4n6 and Detection Intertwined 4n6 identifies and vets indicators Detections feed 4n6 Facilitate Pre-Compromise Detection Strong Compliment to Host Analysis Complete Attack Sequence Analysis 5
Network Analysis Pros/Cons Benefits Passive nature limits impact to network Omniscience at network tap points Control over data retention Drawbacks Network forensics requires explicit data retention Encryption 6
Net vs. Host Compromise IR Predominately Host Predominately Network Detection Malware C2 Beacon Collection Artifacts Damage Assessment Host Logs, Memory Image, Disk Images Malware, (Deleted) Tools and Staged Data, Anything in Memory/Swap/Hyberfil Commands Passwords Lateral Movement Dropped Tools Exfilled Data Days/Weeks Network Logs, Packet Captures Full Command and Control Decodes Commands Passwords Lateral Movement Dropped Tools Exfilled Data Hours/Days 7
Beyond FPC FPC is expensive, unwieldy Strategies for Targeted Data Collection Network Transaction Logs Payload Collection Payload Metadata Information Retrieval For Accessibility 8
Network Transaction Logs Situational Awareness--Inbound of HTTP Requests Direct Attacks (SQL injection etc) Attacker Reconnaissance Options: Sift through FPC Collect, normalize, centralize all webserver logs Snarf and reconstruct web activity Lots of tools to do this Bro, Suricata, HTTPry, etc What about other protocols? 9
Attacks Moving Up Stack Document and Multimedia Viewers, Browsers http://www.sans.org/top-cyber-security-risks/ 10
Attacks Moving Up Stack Users Highly Targeted Social Engineering Exploits 11
Attacks Moving Up the Stack From: spoofed@partner.com Received: from open.relay.com ([10.10.10.10]) by mx.company.com Received: from now.bad.com ([172.16.1.1]) by mx.relay.com Date: Thu, 17 Jun 2010 12:03:41-0700 (PDT) Message-Id: <1.1.2.3.5.8@mailer> X-Mailer: SillyMailer v3.14 Subject: All your Base are belong to us Please review attached. Edward Spoofed Spoofed Inc. 301-867 5309 InfoKey: Creator InfoValue: Acrobat PDF Printer InfoKey: Author InfoValue: TK421 InfoKey: Producer InfoKey: ModDate InfoValue: D:20100616+08'00' PdfID1: 8d23f593e67be992ff3470d PdfID0: 798f9d8e3966ac586a61dc0 for(fqchp=0;fqchp<inxnh;fqchp++) {dnysj[fqchp]=dtkrx + hjnoa;} if(ingmh){hsbsd();hsbsd();try {this.media.newplayer(null);} catch(e) {}hsbsd();} <Obfuscated Embedded Malware> 12
Attacks Moving up Stack Email from legitimate email relay with Trojan Document Attachment Layer Protocol Badness Embedded Object Application Transport Internet Link PDF SMTP/MIME Spoofed Sender, Social Engineering TCP - IP - Ethernet - Exploit/Social Engineering, Malware 13
Indicators Moving Up Stack Users Useful Indicators 14
Indicators Moving Up the Stack 12:03:31.165239 tcp 10.10.10.10.59170 -> 192.168.0.10.25 276 29770 FIN From: spoofed@partner.com Received: from open.relay.com ([10.10.10.10]) by mx.company.com Received: from now.bad.com ([172.16.1.1]) by mx.relay.com Date: Thu, 17 Jun 2010 12:03:41-0700 (PDT) Message-Id: <1.1.2.3.5.8@mailer> X-Mailer: SillyMailer v3.14 Subject: All your Base are belong to us Please review attached. Edward Spoofed Spoofed Inc. 301-867 5309 InfoKey: Creator InfoValue: Acrobat PDF Printer InfoKey: Author InfoValue: TK421 InfoKey: Producer InfoKey: ModDate InfoValue: D:20100616+08'00' PdfID1: 8d23f593e67be992ff3470d PdfID0: 798f9d8e3966ac586a61dc0 for(fqchp=0;fqchp<inxnh;fqchp++) {dnysj[fqchp]=dtkrx + hjnoa;} if(ingmh){hsbsd();hsbsd();try {this.media.newplayer(null);} catch(e) {}hsbsd();} <Obfuscated Embedded Malware> 15
Targeted Collection and Analysis Email Web USB Targeted attacks warrant targeted data collection 16
Email Data Collection Options Basic Email Transaction Data Network Flow Data Full Packet Capture Normalized Emails Reassembled, Decoded, Indexed Extended Email Metadata Headers: Subject, X-Mailer, Received MIME Metadata: Names, Size, md5 Links Attachments (specific type?) Attachment Metadata: Author, Creator, Dates 17
Usability Is Nice 18
Tiered Collection Data Retention Length Size / Day Total Size FPC (entire network) 1 week 1 TB 7 TB Network Flow (entire network) 1 year 4 GB 1.5 TB Standard Mail Logs 2 year 50 MB 36 GB Normalized, Indexed Emails 6 weeks 20 GB 800 GB Extended Email Metadata 6 months 500 MB 100 GB Attachment Metadata 6 months 100 MB 20 GB 19
Accessibility Is Critical Rapid accessibility is critical: Historical Detections Identifying and vetting indicators Time to research an indicator matters 1s, 1 minute, 1 hour, 1 day? The faster you can research activity over large spans of time, the faster you ll build threat intelligence 20
From: spoofed@partner1.com Received: from open.relay1.com ([10.10.10.10]) by mx.company.com Received: from now.bad.com ([172.16.1.1]) by mx.relay.com Date: Mon, 28 Dec 2009 5:48:02 +0800 Message-Id: <1.1.2.3.5.8@mailer> X-Mailer: SillyMailer v3.14 <Malware 1.3> From: spoofed@partner2.com Received: from mx.openrelay2.com ([10.20.30.40]) by mx.company.com Received: from now.bad.com ([192.168.2.2]) by mail.openrelay2.com Date: Mon, 5 Mar 2010 13:35:28-0700 (PDT) Message-Id: <1.1.2.3.6.9@mailer> X-Mailer: SillyMailer v3.14 <Malware 2.0> From: spoofed@partner3.com Received: from relay.all.com ([10.70.50.60]) by mx.company.com Received: from now.bad.com ([172.16.1.1]) by mx.relay.com Date: Thu, 17 Jun 2010 12:03:41-0700 (PDT) Message-Id: <1.1.2.3.7.2@mailer> X-Mailer: SillyMailer v3.14 <Malware 2.01> 21
Ultra Light Weight Indexing Rapidly Search Key Indicator Types IP addresses, Domains, etc Low Resolution Log Type: proxy, email, etc Time: ~Day Per Device: proxy1, proxy2, proxy3 Huge Scope Time: indefinite retention Data Sources: All Performance Fast, << 1s response times 22
Ultra Light Weight Indexing Example search for 172.16.1.1: Data Type Source Date Indicator email-metadata mx1 2009-12-28 172.16.1.1 inbound-http sensor1 2010-03-04 172.16.1.1 email-metadata mx2 2010-06-17 172.16.1.1 23
Implementing Payload Analysis Tools Passive Collection: Adapt an FPC Tail collection, filter normalize, extract Adapt an IDS Filter, normalize, extract, archive Inline Collection Milter, ICAP, etc Differences probably nuances, End goal is the same 24
Payload Analysis Issues Issues to be addressed: Latency Computational Expense Implementing Payload Specific Capabilities 25
Payload Analysis: Latency IDS/IPS bound by real time FPC provides on-demand data/processing (arbitrarily long) High Latency Analysis to be preformed (lookups) Payload analysis for 4n6 usually should be somewhere in between Usually no benefit to be quicker than minute For some applications slower than hour can slow down response Often daily processing makes sense 26
Payload Analysis: Complexity Expensive Tasks Decoding, decompression, etc Parsing, tokenizing, metadata extraction Normalized archival (buffer copies) Payload Identification Any inherently computationally expensive things Statistical analysis Compression Etc 27
Latency and Complexity Heavy Buffering 1 Gpbs * 60s = 7.5 GB RAM (dirt cheap) True Parallelism Load balancing needs to move up stack also Example later 28
Implementing Payload Specific Capabilities Use existing network capabilities Protocol Parsers HTTP::Parser, Mime::Parser, etc Use payload capabilities Payload Analyzers pdftk, pdf-parser, Officecat, etc Use your in-house tools on extracted payloads Build network tools that work on objects (Abstraction) 29
Near Real Time IDS Platforms vortex (Lockheed Martin) http://sourceforge.net/projects/vortex-ids/ Abstracts capture and TCP stream reassembly, simple method for multithreading snort-nrt (Sourcefire VRT) http://labs.snort.org/nrt/ Commitment to payload analysis Ruminate (George Mason University) http://mason.gmu.edu/~csmutz/ruminate/ Focus on efficiency, scalability, completeness of parsing 30
Vortex Overview Captured Network Traffic Vortex Stream Management, Flow Control Libnids TCP Stream Reassembly Libpcap Stream Metadata (STDOUT) Analyzer Program Reads Metadata, Loads Stream Data, Analyzes, optionally Purges Stream Data Packet Capture/Filtering Stream Data File System 31
Vortex Multithreaded Captured Network Traffic Vortex Stream Metadata (STDOUT) Xpipes Load Balancing Analyzer Program Analyzer Program Analyzer Program Analyzer Program Stream Data File System 32
Conclusions Network Data is important source for 4n6 Strategies for Network Data Collection Conventional (netflow, logs, FPC) Targeted (playloads, payload metadata) Importance of data accessibility Normalization Search and Retrieval Ideas on Implementation 33
Questions? charles.smutz@lmco.com Personal Blog: http://smusec.blogspot.com 34
35
APT Attack Sequence Pre-Compromise Reconnaissance Weaponization Delivery Exploit Installation Reconnaissance Initial Intrusion Post-Compromise Command & Control Actions on Intent Establish Backdoor Obtain User Credentials Install Various Utilities Priv. Escalation, Lateral Move., Data Exfil. Maintain Persistance 36