Bridging the gap between COTS tool alerting and raw data analysis

Transcription

1 Article Bridging the gap between COTS tool alerting and raw data analysis An article on how the use of metadata in cybersecurity solutions raises the situational awareness of network activity, leading to faster detection and mitigation of data breaches Kurt Neumann - Director, Cybersecurity Applications, Qosmos April 2012 Executive Summary Effective cyberdefense requires security teams to identify and validate traffic events quickly. Until now, they had to choose between searching through system logs, NetFlow, or full packet captures. New network probes leveraging the advantages of traffic metadata now enable NBAD, SIEM, DDoS and Network Analytics solutions to search though data faster with fewer demands on users.

2 According to the Verizon 2011 Data Breach Investigations Report (a study conducted by the Verizon RISK Team in cooperation with the U.S. Secret Service and the Dutch National High Tech Crime Unit), nearly 75% of breaches take weeks to months to discover and 70% take days to weeks to contain. Alarmingly, 86% of data beaches are discovered by third parties (customers and partners not the guarding Security Operations Center). This highlights the need for situational awareness of network activity and the ability to distinguish bad from good traffic in minimal time, which is now a necessary pillar of effective cyberdefense. In today s world, organizations must assume that their networks will be compromised. In order to accelerate breach detection and mitigation, they need to improve their understanding and monitoring of normal network behavior. Vendors of cybersecurity solutions are the most likely sources for help. The effectiveness of products for next-generation firewalls, NBAD, SIEM, DDoS and Network Analytics can be dramatically improved by incorporating the use of traffic metadata into solutions. Metadata strengthens solutions by providing behavioral context to traffic monitoring. As Qosmos demonstrates with its new line of DeepFlow cybersecurity probes, vendors can use metadata to bridge the gap between COTS tool alerting and the raw data analysis that security teams still must perform. The need for situational awareness Organizations today have more network data to analyze and less certainty about what data they need to analyze. Data sets are more complex, making it harder to extract meaningful information. Security teams can no longer afford to police all their data manually and must outsource all or part of their security analysis to third parties, where analysis can easily lose context for a specific business environment or security objective. The volume of data and proliferation of threats change how organizations should now approach cyberdefense. Recommendations from cybersecurity experts prioritize situational awareness and breach detection over impossible 100% prevention. Strategy should emphasize network intelligence gathering and analysis, and smart network monitoring as the best defense against threats. The gap between COTS tools and raw data analysis Figure 1 shows a representative example of conventional security analysis. Billions of raw data elements collected over a period of time are screened in stages down to a few thousand investigated events. Within the Security Operations Center, traffic records are analyzed using conventional tools of choice. The time and resources required to validate data, examine events, identify breaches and mitigate them can take weeks to months at substantial costs for the tools and talent. This is largely due to a gap between conventional COTS tools and the manual analysis performed by security teams (Figure 2). The inability to search for data patterns in the context of user behavior and application usage makes useful pattern detection (and quick breach mitigation) difficult. 2

3 Figure 1. Conventional Security Analysis Figure 2. Gap between COTS Tools and Raw Data Analysis 3

4 Shifting mindsets to situational awareness Improving the situational awareness of cybersecurity requires a shift in mindsets among vendors and their customers. Instead of preventing attacks, the premise should be that breaches will occur. Objectives should shift to detecting breaches faster by understanding the behavior and use of applications in traffic flows to recognize anomalies. Instead of thinking of cybersecurity as a discrete solution, the approach should be the integration of security with applications and web logs. Instead of relying only on protocol signatures to monitor traffic, vendors must enable products with real-time visibility into traffic patterns based on user behavior and application usage. The value of traffic metadata What security teams need, and what vendors should seek to provide, are capabilities to examine traffic data with the quality of full packet inspection coupled with indexed searching of protocol attributes to find meaningful user and application behavior patterns. This would improve the situational awareness of customers cyberdefense, and is within reach for vendors through the use of traffic metadata. Metadata can bridge the gap between conventional tools and raw analysis by enabling detection and differentiation of good and bad behavior patterns in network traffic flows. The advantages of metadata for vendors of cybersecurity solutions and their customers include: Full classification and decoding of network protocols Layers 4-7, describing as many protocol and application attributes as needed Extraction from traffic in real time without the need for data aggregation, formatting and database searches (making metadata more precise, faster and easier to use than data logs) Analysis of traffic without the need to store full, raw data packets, reducing storage requirements by a ratio of 1000:1, compared to processing packet captures and/or Syslog Application and session awareness, and capable of tracking multiple flows with a single protocol (e.g. an FTP connection and data channels) Figure 3 compares records from Netflow, an industry standard for IP traffic monitoring, before and after enhancement with metadata as used in Qosmos DeepFlow probe appliances. Netflow alone is fast and repeatable but, because it is neither application- nor protocol-aware, the standard Netflow record doesn t disclose potential threats. Security specialists must still screen and analyze full data packets and logs manually to find behavioral context using increasingly outdated tools. Event correlations and differentiation of abnormal from normal behavior are difficult. The Qosmos metadata parse traffic in real time for user behavior and application usage, providing insight into what actually occurred between source and destination. In Figure 3, the metadata additions to the Netflow record in this example reveal: 1. A referring party (chicaroo.cc) Why would chicaroo.cc be referring users to this website? 2. A suspicious URL ( login.php) and no cookies Why would anyone go directly to a failed login page without a session cookie? 3. A suspicious browser (curl2.x) not Internet Explorer, Firefox or Chrome, etc., but a command line version of a browser typically used in malicious scripts. 4. The server code is giving a positive result (200) despite the record s irregularities Is someone exploiting a vulnerability? 4

5 Without the metadata, the Netflow record shows how much data was transferred, between what ports and when, but security specialists must still make assumptions when screening traffic data for suspicious activity. The same record enhanced by the Qosmos metadata tells security specialists what actually transpired in the communication. It enables accurate real-time traffic monitoring of both normal and abnormal behavior i.e. situational awareness. Security specialists can work with useful pattern detection and know specifically which records to investigate. In this way, the use of metadata can reduce breach detection from weeks and months to hours and minutes. Figure 3. Extended Traffic Visibility into Layers 4-7 with Metadata The impact on network security Raising situational awareness for effective network protection requires security teams to identify and validate events quickly. Until now, they had the choice between searching through system logs, NetFlow, or full packet captures. Traffic metadata, as designed by Qosmos into its DeepFlow cybersecurity probes, combine the essence of all three into a single message flow, normalized to be easily used by SIEM, NBAD, DDoS, and Network Analytics tools. As shown in Figure 4, vendors can enable their customers to search through data faster, and with fewer IT demands, increasing their value to customers as partners in cyberdefense. 5

6 Figure 4. Deep, fast traffic visibility with fewer IT demands For more information: 6