DPI and Metadata for Cybersecurity Applications

Transcription

1 White Paper DPI and Metadata for Cybersecurity Applications How vendors can improve solutions for new market demands by filling the gap between COTS cybersecurity and raw data analysis

2 Executive Summary According to the Verizon 2013 Data Breach Investigations Report, 78% of breaches take weeks or months to discover. The authors stress the importance of a strong strategy for detection and response: Prevention is crucial, and we can t lose sight of that goal. But we must accept the fact that no barrier is impenetrable, and detection/response represents an extremely critical line of defense. Let s stop treating it like a backup plan if things go wrong, and start making it a core part of THE plan. 1 This highlights the need for situational awareness, now a necessary pillar of effective cyberdefense. In today s world, organizations must assume that their networks will be compromised. In order to accelerate breach detection and mitigation, they need to improve their understanding and monitoring of normal network behavior. Vendors of cybersecurity solutions are the most likely sources for help. This paper offers a viable strategy for vendors to increase the effectiveness of products such as nextgeneration firewalls and solutions for NBAD, SIEM and DDoS attacks by using traffic metadata. It explains how metadata can be leveraged to see good from bad network behavior faster than current COTS products and raw data analysis using data logs, full packet capture and traditional Deep Packet Inspection (DPI). Metadata strengthens cybersecurity solutions by providing behavioral context to traffic monitoring. Vendors can use metadata to: Establish good and bad behavior for services, and how they communicate Rapidly define/map behavioral patterns for a host or user Build custom protocol validators to look for system exploits Allow analysts to query and investigate data in new and more effective ways Reduce data storage requirements by 10x 150x compared to full packet capture (enabling storage of a year s worth of metadata, which is unfeasible with full packet capture) Deliver products with higher quality security and fewer false positives By enabling cybersecurity tools to search through data faster, and with fewer demands on IT organizations to improve their situational awareness, vendors stand to increase their market relevance and value to customers as partners in cyberdefense Data Breach Investigations Report, Verizon,

3 Contents Executive Summary... 2 Cyberdefense Trends: The Need for Situational Awareness... 4 How Conventional Security Analysis Limits Detection and Mitigation... 4 Shifting Mindsets to Situational Awareness... 5 The Value of Traffic Metadata... 6 Use Case: Firewalls with Metadata... 8 Use Case: NBAD with Metadata... 9 Use Case: SIEM with Metadata... 9 Use Case: DDoS Mitigation with Metadata Qosmos DPI Metadata for Cybersecurity Vendors Improving Your Value Proposition to Customers

4 Cyberdefense Trends: The Need for Situational Awareness Networks today must support increases in data size and availability. Organizations have more network data to analyze and less certainty about what data they need to analyze. Data sets are more complex, making it harder to extract meaningful information. Cybersecurity teams can no longer afford to police all their data manually and must outsource all or part of their security analysis to third parties, where analysis can easily lose context for a specific business environment or security objective. The volume of data and proliferation of threats change how organizations should now approach cyberdefense. Recommendations from cybersecurity experts prioritize situational awareness and breach detection over impossible 100% prevention. Strategy should emphasize network intelligence gathering and analysis, and smart network monitoring as the best defense against threats. Vendors of cybersecurity solutions can drive emerging defense trends by engineering intelligence for situational awareness into their offerings. How Conventional Security Analysis Limits Detection and Mitigation Figure 1 shows a representative example of conventional security analysis. Billions of raw data elements collected over a period of time are screened in stages down to a few thousand investigated events. Figure 1. Conventional Security Analysis Within the Security Operations Center (SOC), traffic records are analyzed using conventional tools of choice. The time and resources required to validate data, examine events, identify breaches and mitigate them, as revealed in the Verizon report, can take weeks to months at substantial costs for the tools and talent. This is largely due to a gap between conventional COTS tools and the manual analysis performed by security teams (Figure 2). The inability to search for data patterns in the context of user behavior and application usage makes useful pattern detection (and quick breach mitigation) difficult. 4

5 Figure 2. Gap between COTS Tools and Raw Data Analysis Shifting Mindsets to Situational Awareness The use of metadata to improve the situational awareness of cybersecurity requires a shift in mindsets among vendors and their customers. Instead of preventing attacks, the premise should be that breaches will occur. Objectives should shift to detecting breaches faster by understanding the behavior and use of applications in traffic flows to recognize anomalies. Instead of thinking of cybersecurity as a discrete solution, the approach should be the integration of security with applications and web logs. Instead of relying only on protocol signatures to monitor traffic, vendors must enable products with real-time visibility into traffic patterns based on user behavior and application usage made possible with metadata. 5

6 The Value of Traffic Metadata What security teams need, and what vendors should seek to provide, are capabilities to examine traffic data with the quality of full packet inspection coupled with indexed searching of protocol attributes to find meaningful user and application behavior patterns. This would improve the situational awareness of customers cyberdefense, and is within reach for vendors through the use of traffic metadata. Metadata bridge the gap between conventional COTS and raw data analysis, enabling vendors to improve their customers situational awareness for cyberdefense. Metadata bridge the gap between conventional tools and raw analysis by enabling detection and differentiation of good and bad behavior patterns in network traffic flows. Traffic metadata provide the following advantages for vendors of cybersecurity solutions and their customers: Full classification and decoding of network protocols Layers 4-7, describing as many protocol and application attributes as needed Extraction from traffic in real-time without the need for data aggregation, formatting and database searches; they are therefore more precise, faster and easier to use than data logs Analysis of traffic without the need to store full, raw data packets, reducing storage requirements by a ratio of 1000:1, compared to processing packet captures and/or Syslog Application and session aware, and capable of tracking multiple flows with a single protocol (e.g. an FTP connection and data channels) Figure 3 shows examples of traffic metadata. Some protocols and applications can have more than 50 metadata attributes totaling thousands of attributes collectively which can be selected, correlated and analyzed to provide a complete understanding of the quality and purpose of network events. Figure 3. Examples of Application and Protocol Metadata Application Examples and Webmail Social Networking (Facebook, Twitter, Baidu, etc.) IM (MSN, Yahoo, Skype, QQ, etc.) Web Apps (YouTube, ebay, etc.) Business Apps (CRM, ERP, Citrix, MS Exchange, etc.) Protocol Examples HTTP (Hypertext Transfer Protocol) GTP (GPRS Tunneling Protocol) UDP (User Datagram Protocol) IP (Internet Protocol) RTSP (Real Time Streaming Protocol) SIP (Session Initiation Protocol) Typical Metadata Sender, receiver, login, subject, message, attachments, date and time User login, application URLs/activity, posting activity, tweets, search engine queries, and resulting URL clicks, chats, video streams, file attachments. User login, login / logoff date and time, data transfer sessions (type, content, time), volume (per user, IP, subnet, application) Typical Metadata URL, browser, cookies, URI, referrer, Device, user location, quality of service (QoS) metrics, time, duration Source port, destination port, client port, server port Source address, destination address, source port, destination port, data Play/pause, streaming file, URL, duration Caller, entity called, codec 6

7 Figure 4 compares records from Netflow, an industry standard for IP traffic monitoring, with and without metadata from Qosmos. Netflow alone is fast and repeatable but, because it is neither application nor protocol aware, the standard Netflow record doesn t disclose potential threats. Security specialists must still screen and analyze full data packets and logs manually to find behavioral context using increasingly outdated tools. Event correlations and differentiation of abnormal from normal behavior are difficult. Figure 4. Extended Visibility into Layers 4 through 7 with Metadata Qosmos metadata parse traffic in real time for user behavior and application usage, providing insight into what actually occurred between source and destination. In Figure 4, the Qosmos metadata additions to the Netflow record in this example reveal: 1. A referring party (chicaroo.cc) Why would chicaroo.cc be referring users to our site? 2. A suspicious URL ( login.php) and no cookies Why would anyone go directly to a failed login page without a session cookie? 3. A suspicious browser (curl2.x) not Internet Explorer, Firefox or Chrome, etc., but a command line version of a browser typically used in malicious scripts. 4. The server code is giving a positive result (200) despite the record s irregularities Is someone exploiting a vulnerability? Without the Qosmos metadata, the Netflow record shows how much data was transferred, between what ports and when, but security specialists must still make assumptions when screening traffic data for suspicious activity. The same record enhanced with the Qosmos metadata tells security specialists what actually transpired in the communication. It enables accurate real-time traffic monitoring of both normal and abnormal behavior i.e. situational awareness. Security specialists can work with useful pattern detection and know specifically which records to investigate. In this way, the use of metadata can reduce breach detection from weeks and months to hours and minutes. 7

8 Use Case: Firewalls with Metadata Firewalls historically employ five-tuple filtering of ports and Internet protocols. Basic port inspection, stateful inspection and protocol detection are typically based on predefined ports. Traffic protocols and applications cannot be identified unless ports and IP address are known. By enhancing their solutions with metadata, firewall vendors can enable their customers to filter traffic based on true protocol recognition from Layers 1 to 7. Firewalls today need full traffic visibility independent of ports to block security breaches initiated, for example, through social networking applications, instant messaging and . By enhancing solutions with metadata (Figure 5), firewall vendors can enable their customers to filter traffic in real time based on true protocol detection from Layers 1 to 7, so firewalls act with application awareness. For example, they may wish to allow access to Facebook, but not allow access to Facebook game applications like Farmville. Without metadata, vendors risk considerable time and money developing technology such as Deep Packet Inspection (DPI) and keeping up with changing protocols and applications to defend against new threats. And firewall customers must wrestle with the maintenance overhead of trying to block applications with IP addresses and ports. Figure 5. Firewall Improvements with Metadata 8

9 Use Case: NBAD with Metadata Network Behavior Anomaly Detection (NBAD) requires visibility into normal network behavior as a baseline in order to flag abnormal behavior. To be fully effective, this should include the real-time monitoring of protocols, payloads, virus detection, bandwidth and connection rates. Metadata improve the ability of NBAD solutions to query traffic for suspicious activity and investigative analysis by providing information on: Services or encrypted traffic on non-standard ports Hosts connecting to port 443 and not using the SSL protocol Metadata improve the ability of NBAD vendors to design solutions that query traffic for suspicious activity and investigative analysis. Hosts that started a flow with one protocol, and then changed mid stream (started with SSL then changed to something else) File attachments and user names in any supported protocol (Webmail, HTTP, FTP, file sharing, IM, etc.) A referrer or URL related to a phishing campaign The longest session with high error responses from the server Sessions with traffic that falls out of protocol spec Metadata are essential to effective cybersecurity because they provide the capability to see traffic patterns and to detect cookies that can reveal abnormal network behavior to know where a visitor is on a website at any given time. Metadata are a perfect complement to logs since they can be mixed with log information for a single, more robust view of traffic that can be indexed with data collection and indexing tools. And metadata improve investigative capabilities by requiring less storage than full packet capture, leading to faster data searches and the archiving of historical data for much longer periods of time. Use Case: SIEM with Metadata Security Information and Event Management (SIEM) is constantly required to process and analyze increasing quantities of data, and has a difficult time keeping up with the rising volume of network traffic. At the same time, vendors of Intrusion Protection Systems (IPS) and Intrusion Detection Systems (IDS) are trying to reduce the number of false alerts false negatives from IPS and false positives from 9

10 IDS which diminish their effectiveness. Today, SIEM lacks scalability. And since most solutions use Netflow as an index to correlate events, SIEM also lacks the context of user behavior and application usage to establish situational awareness for allowing or blocking traffic accurately. Metadata (Figures 6 and 7) enable events to be validated before going to full packet capture and provide an accurate and scalable method to build behavioral rules for more reliable alerts. For example, to better qualify IDS alerts, they can be weighted with metadata to screen for traffic parameters such as browser type, URL length, referrer, cookies, connection time, protocol, protocol change and more. Metadata can be used to: Show all inbound IDS/WAF (Web Application Firewall) alerts for a cross site scripting (XSS) vulnerability hidden in good traffic (correlated server response of 200) Reveal the least common URL in traffic and any related IDS/IPS alerts The use of metadata enables IPS/IDS vendors to deliver solutions with better performance, stronger capabilities and a competitive advantage against solutions that rely only on protocol signature sets. Classify protocols on any port and process only the rules that apply for each protocol (10 to 200 rules per traffic flow instead of 2,000 rules per flow) Make more informed decisions on which traffic to allow and block with in-line IPS solutions Figure 6. IDS Improvement with Metadata 10

11 Figure 7. IPS Improvement with Metadata The use of metadata enables IPS/IDS vendors to deliver solutions with better performance, stronger capabilities and a competitive advantage against solutions that rely only on protocol signature sets. For example, metadata: Reduce the noise that IDS produces, and the tuning and filtering time of false positives by 50% Reduce the false negative rate of rules in IPS by 50%, which enables customers to make more efficient use of their IPS without dramatic changes in the way they manage it Detect protocol changes in traffic flow beyond the first 5-10 MB, which eliminates a huge limitation of purely signature-based IDS/IPS solutions Use Case: DDoS Mitigation with Metadata The use of metadata enables vendors and Managed Security Service Providers (MSSPs) to dramatically improve protection against all types of Distributed Denial of Service (DDoS) attacks especially emerging attacks like application-level DDoS at significantly reduced cost per customer. SYN floods are easy to detect and block for most vendors today, but application-level DDoS detection and mitigation (Figure 8) remain difficult since they require better visibility and understanding of protocol and application behavior. Metadata improve visibility and understanding of network traffic and applications, giving DDoS detection the ability to clearly differentiate good from bad traffic, especially for application-level attacks. 11

12 Figure 8. Application-Level DDoS Detection and Mitigation with Metadata Protocol decoding and metadata extraction up to Layer 7 provide complete visibility of all network traffic and applications, giving DDoS detection solutions the ability to clearly differentiate good from bad traffic, especially for application-level attacks. For example, detection can be based on sessions connecting without a session key established over a given time, a URL, referrer, browser, etc. Even SSL renegotiation attacks become easy to identify on a per flow basis when based on metadata attributes. For vendors, the competitive advantage comes from giving customers better protection against attacks without having to increase the expertise or headcount of security specialists. The time to tune solutions is less per site and defense against DDoS is stronger with fewer chances of false alerts. Qosmos DPI Metadata for Cybersecurity Vendors Qosmos specializes in software libraries and tools for cybersecurity vendors to enhance their solutions with DPI and Metadata Data Extraction. Qosmos technology enables vendors to make their solutions applications aware and thereby increase their value to customers by responding to today s need for greater situational awareness in cyberdefense. Vendors who use Qosmos benefit from market-leading traffic parsing technology that can accelerate the delivery of application-aware solutions. As shown in Figure 9, the Qosmos ixengine Software Development Kit (SDK) and Qosmos Labs support services easily integrate into new or existing solutions. Vendors benefit from market-leading traffic parsing technology to accelerate the delivery of applicationaware solutions. Using Qosmos reduces a vendor s time to market with next-generation solutions; 12

13 development time, cost and resource requirements; and the time to update constantly changing protocols. Figure 9. Qosmos Embedded DPI and Metadata Technology Vendors Requirements Support for Network Processors Performance & Scalability Figure 10. Qosmos Technology Alignment with Cybersecurity Vendors Qosmos Technology Qosmos ixengine DPI and Metadata Extraction SDK optimized on Cavium, Intel, NetLogic, and Tilera Handles up to 10 Gbps of traffic on a single chassis Easy Integration C Library APIs (Can also share state table from another application ) Robust Protocol Extraction Wide Range of Protocols & Metadata Integration Support Standardize the Output for Metadata Rapidly extracts metadata from traffic flows, and can change protocol configuration on the fly. (Supports many protocols active at once.) 2,500 application protocols classified and 4,300 metadata extracted Quickly integrates into most systems (90 day acceleration program offered). Provides differentiating technology in security areas where security metadata are just being adopted. Provide help in standardizing output for syslog/netflow/ipfix, (provide a standards framework) 13

14 Figure 10 summarizes how Qosmos technology directly aligns with vendors needs. Qosmos offers market-leading protocol and application expertise, featuring: A library of plugins for classifying more than 2,500 protocols and applications An industry-best 4,300 metadata attributes Advanced traffic parsing capabilities for tunneling, unidirectional flows, fragmented and partial traffic, and packet by packet inspection Robust architecture to handle abnormal and forged traffic Support for core network speeds A custom protocol plugin SDK that allows vendors to develop and update their own protocols On-demand protocol and application development by Qosmos Improving Your Value Proposition to Customers Cybersecurity experts and industry analysts recognize that not all data breaches can be prevented. Adversaries motivated by espionage, fraud, terrorism, socio-political agendas and simply mischief are too many in number and too creative with easy access to increasingly sophisticated tools. The new strategy for All next-generation cybersecurity solutions will leverage traffic metadata. effective cyberdefense is to assume some breaches will occur, but to promptly detect and mitigate them with greater situational awareness of network activity. Traffic metadata provide vendors of cybersecurity solutions with the power to improve their value proposition to customers with solutions that increase their customers situational awareness. This encompasses much better capabilities to 1) differentiate good traffic from bad, and 2) detect and mitigate data breaches faster. All next generation cybersecurity solutions will leverage traffic metadata. The technology, in the form of software development kits and intelligent IP probes backed by constantly updated libraries of protocol signatures and metadata attributes, already exists through specialist companies like Qosmos. Without having to invest substantial time and resources, vendors can integrate pre-developed technology into solutions, complete with robust technical support. Many already have or are in the process of doing so. ### Copyright 2014 Qosmos S.A. All rights reserved. Qosmos, the Qosmos logo, Qosmos Service Aware Module, Qosmos SAM and Qosmos ixengine are trademarks of Qosmos. Other names and brands may be claimed as the property of others. 14