Berkley Packet Filters and Open Source Tools a tranched approach to packet capture analysis at today s network speeds 1
Agenda Packet Capture overview Bro description Security Onion description The problem ingesting at line rate Berkeley Packet Filter (BPF) overview The solution using BPF to neck down traffic 2
Packet Capture Leverages an API for capturing network traffic Unix-like systems implement in libpcap; Windows uses WinPcap, port of libpcap PCAP (Packet Capture) data are produced MIME type for the file format created and read by libpcap and WinPcap is application/vnd.tcpdump.pcap. Complete record of network activity Layers 2 7 Capture adaptor is in promiscuous mode 3
Who Uses PCAP s Researchers: access to raw data. Administrators: debug network problems. Analysts: define malicious activity. Incident Responders: forensic investigation. Remediation Teams: identify affected assets and remediate. 4
Bro: Not A Traditional NIDS Efficient: Bro targets high-performance networks and is used operationally at a variety of large sites. Flexible: Bro is not restricted to any particular detection approach and does not rely on traditional signatures. Forensics: Bro comprehensively logs what it sees and provides a highlevel archive of a network's activity. In-depth Analysis: Bro comes with analyzers for many protocols, enabling high-level semantic analysis at the application layer. Highly Stateful: Bro keeps extensive application-layer state about the network it monitors. Open Interfaces: Bro interfaces with other tools for real-time exchange of information. 5
BRO Log Output #path /extraction/logdata/int0/6_747/1447703473733426325_2.pcap.unusual #open 2015-11-16-21-18-51 ts uid id.orig_h id.orig_p id.resp_h id.resp_p name 1447703474 ClUTWe2CPmzJYl8jQ 172.16.9.171 3387 213.254.244.30 80 inflate_failed 1447703474 Cvwwtm1jEXT08mMQX4 172.16.9.171 2561 128.241.21.163 80 unescaped_special_uri_char 1447703474 Cvwwtm1jEXT08mMQX4 172.16.9.171 2561 128.241.21.163 80 window_recision above_hole_data_without_any 1447703474 CXGnsl1wpuZgH8vMZ4 172.16.9.171 2780 17.254.0.91 80 _acks 1447703474 CeOrPg17TZP6gLgd9k 172.16.9.171 2837 66.249.91.83 80 active_connection_reuse 1447703474 CSUvRB2SYZBnEhTXJl 172.16.9.171 3116 84.53.136.167 80 premature_connection_reuse 1447703474 CA6xhkcd9XtaLPWr8 172.16.9.171 3368 84.53.136.167 80 SYN_inside_connection #close 2015-11-16-21-19-21 6
The Security Onion Security Onion is a Linux distro for intrusion detection, network security monitoring, and log management. It's based on Ubuntu and includes the following Security Tools: Snorby, OSSEC, Sguil, Squert, ELSA, PADS, and NetworkMiner. It is Snort centric. One key component is using BRO to pre-process, parse, and flag anomalous traffic that is potentially malicious prior to any other analysis or activities. 7
Sguil 8
ELSA 9
Squert 10
Snorby 11
Network Miner 12
The Problem Analysis tools choke at line rates High performance platforms exist, but are too costly for most enterprises 13
Berkley Packet Filters a.k.a. BPF. Conceptually similar to Wireshark filters. Filter on layer 2+. Richest in layers 2 4. Berkley Packet Filters handle packets from different types of network interfaces and apply a standard 5 Tuple format for indexing: Source IP, Source Port, Destination IP, Destination Port, and Protocol Using 5 Tuples is a critical component for capturing bidirectional connections, allows for real time indexing, and provides a fast search of the data-store with defined parameters when forensic review is needed. 14
Why BPF? Extremely Fast. Advanced Filters for TCP, UDP, ICMP, etc. Provides access to raw packet bytes. Combine BPF Primitives with Logical Operators Types are host, net, port and portrange; Dir or Direction are src, dst, src or dst and src and dst; and, Proto, which restricts the match to a particular protocol NOT, AND, OR. 15
Use BPF Primitives to Neck Down Results (src net 192.168.5.0/24 and port 443) Include only traffic originating from this network And only for this port (host 10.0.0.1 && host 10.0.0.2) Traffic between these hosts 16
BRO Weird: Anomalous Behavioral redef Weird::actions: table[string] of Action = { ["data_before_established"] = ACTION_LOG_ONCE, Before the connection was fully established, a TCP endpoint sent some data. ["possible_split_routing"] = ACTION_LOG_ONCE, Bro appears to be seeing only one direction of some bi-directional connections. This can also occur due to certain forms of stealth-scanning. }; 17
How Does A Capture Platform Look Forward? Using BPF-translated BRO attributes, the filter can identify a potentially malicious flow. BRO detects intrusions by parsing network traffic to extract applicationlevel semantics and executes an event-oriented analysis that compares behavioral patterns that are deemed abnormal from the expected behavior. Traffic that exhibits anomalous behavior is defined as a possible attack and gets flagged as an event or unusual activity. Alternatively, if traffic doesn't conform to the expected RFC Policy (as one example), it also is defined as potentially malicious 18
What Happens Next When BRO flags traffic as exhibiting abnormal behavior, a log entry is generated that is tagged with a unique identifier that can be indexed and correlated with the associated packets prior to archiving the PCAP and compressing it in the data-store. This is a critically important step for two reasons: 1) It gives a practitioner more than just the ability to search using BPF primitives. A practitioner can pass the BRO logs to their preferred security tool (like Splunk or LogoRhythm) or they can pivot ELSA, which is included in the Security Onion. 2) As this happens in real time, a practitioner has the opportunity to interdict and stop a malicious attack before its actions are completed. 19
To Recap Berkley Packet Filters breakdown traffic into a standardized format during ingestion, in real time. 330 total filters are used to parse that filtered traffic and create an alert when anomalous behavior or non-conformant to RFC Policy is observed. All traffic has been filtered, indexed, pre-processed in real time at line rate by BRO, and BRO has parsed all the traffic to generate correlating logs with unique identifiers for: Unusual Log, Event Log, HTTP Log, SMTP Log, File Log, and Connections Log 20
Expediting Investigation and Response Time Time is a valuable resource for a security practitioner. Using BPF primitives and BRO to filter and index network ip traffic frees up Security Onion resources to focus only the traffic that is flagged as malicious. As Snort inspects and parses the pre-processed traffic, it s not spinning cycles on inspecting packets that don t contain flags to trigger on--because this traffic is pre-qualified as abnormal Snort is able to process more effectively. Snort generates an insightful NIDS log that is accessible through ELSA 21
Unfiltered and filtered 22
References http://biot.com/capstats/bpf.html http://www.tcpdump.org/ https://www.bro.org/ http://blog.securityonion.net/ http://tcpreplay.synfin.net/wiki/tcpreplay http://www.sentrywire.com Or email josh.williams@alliance-it.com 23