5 Things You Need to Know About Deep Packet Inspection (DPI)

Transcription

1 White Paper: 5 Things You Need to Know About Deep Packet Inspection (DPI) By Safa Alkateb Updated April 2011

2 White Paper: 5 Things You Need to Know about Deep Packet Inspection (DPI) 2 5 Things You Need to Know About Deep Packet Inspection (DPI) By Safa Alkateb Network and telecommunications engineers face stark challenges in the coming years. Analysts predict sharp increases in demand for network bandwidth and speed, as well as the proliferation of sophisticated security risks. YouTube video already accounts for about one fifth of all Internet data, and Cisco forecasts that by 2014 online video use will increase seven fold. Peer to peer networking, VoIP, video chat and conferencing, online gaming, cloud computing and other data-intensive activities are also expected to grow dramatically, straining physical and wireless infrastructure across the globe. On top of these bandwidth concerns are the ever-changing security threats that jeopardize government and corporate networks, individual computers and mobile devices. According to WhiteHat Security, the number of security threats doubled in the past year and a half, and the pace of cyber crime is quickening. To combat these pressures and meet future demand for data services, governments, enterprises and carriers are not only upgrading their network infrastructure for greater speed and quality of service, but they are looking for ways to manage their data flows more intelligently. And the key to maintaining the integrity and efficiency of a multi-gbps network is a technology called deep packet inspection (DPI). In this white paper, we explore five critical issues related to DPI, helping companies that are interested in adding DPI to their products or networks better understand DPI, what it can achieve, what best practices look like and what implications DPI has on privacy and net neutrality.

3 White Paper: 5 Things You Need to Know about Deep Packet Inspection (DPI) 3 1. What is DPI? DPI is a hardware and software solution that monitors a network's data stream and identifies protocols and applications, inappropriate URLs, intrusion attempts and malware by looking deep into data packets. DPI provides important security and translation functions by inspecting incoming packets, reassembling and decompressing them, analyzing the code and passing data to appropriate applications and services. If malicious URLs or code are detected, the system can block them entirely. DPI can also be used by service providers to offer subscribers different levels of access (such as type of usage, data limits or bandwidth level), comply with regulations, prioritize traffic, adjust loads and gather statistical information. As more and more software moves off the desktop and onto the enterprise network or into the cloud, network performance becomes critical to productivity. DPI can recognize applications as data passes through the system, allocating each the resources they need.

4 White Paper: 5 Things You Need to Know about Deep Packet Inspection (DPI) 4 To offer such a wide array of services, DPI examines not only a packet s originating port and IP address (sometimes called shallow inspection ) which provide limited and sometimes misleading information but looks deep into the Application layer of the OSI model (the seven layered model that describes the structure of packet data), where it can use a variety of techniques, including signature- and heuristicsbased detection, to identify the nature of the packet s payload. Today, the DPI industry is growing rapidly, with product revenue expected to reach $1.5 billion by DPI is an important part of a larger network security appliance and software market that is expected to reach $7 billion by What are the critical applications of a DPI system? In most situations, a DPI system needs to be able to provide four major services: Protocol Analysis & Application Recognition Anti-malware and Anti-virus Protection IDS and/or IPS URL Filtering Protocol Analysis & Application Recognition To make sense of the data that flows through a network, a DPI system must be able to distinguish between many different protocols. Today s sophisticated DPI systems can identify hundreds of protocols covering almost every type of application and service. For instance, strong DPI systems should be able to distinguish between services, including IMAP, POP3 and SMTP. They should identify web protocols, such as HTTP, FTP and TCP, as well as multimedia types, such as Flash, QuickTime, Real, YouTube and Windows Media. In fact, DPI systems need to be able to identify a wide variety of web 2.0, tunneling, session, peer-to-peer, messaging and voice over IP protocols in order to route the data to appropriate detection and processing engines. DPI can also extract a payload s meta data, including attachment formats, file names, phone numbers and more.

5 White Paper: 5 Things You Need to Know about Deep Packet Inspection (DPI) 5 The ability to quickly and easily update detection profiles without disrupting the system is important in a DPI solution, particularly for data centers and carriers. For this reason, detection engines should be user configurable without requiring a system reboot. A credible DPI system should be able to detect protocols and applications using all of the following methods: Port Detection Signature Detection Heuristics Detection Other characteristics of a high-performance DPI system include flow-based detection (for TCP, UDP and WAP), support for IPv4 and IPv6, TCP/IP normalization and reassembly and rules-based metadata extraction. Anti-malware and Anti-virus DPI is an ideal environment for detecting and filtering a wide range of malware and viruses, such as worms, Trojan horses, spyware, adware and other malicious applications. Most DPI systems can be configured to detect and eliminate the vast majority of these threats or the systems can be extended with third-party solutions. Almost all threats can be intercepted if the system employs a three-pronged security approach: Normalized URL Detection Comparing incoming and embedded URLs against a database of known malicious sites Object Detection Searching the data flow for potentially harmful executables or objects (such as JPEG images), then analyzing them Signature Detection Using a signature database to detect certain kinds of malware, especially viruses that mutate upon replication Each of these detection approaches can and should be updated with third-party signature subscriptions (such as those from security service provider Kaspersky). IDS / IPS Intrusion detection systems (IDS) and intrusion prevention systems (IPS) both detect intrusion attempts and share many characteristics. They are used to detect hackers and unauthorized people trying to access a network or computer, usually by exploiting a vulnerability in an application. But the two systems differ in one important aspect: IDS is primarily an out-of-band logging tool used for forensic analysis. IPS, on the other hand, runs inline and automatically takes action when malicious activity is detected.

6 White Paper: 5 Things You Need to Know about Deep Packet Inspection (DPI) 6 DPI systems can provide one or both of these services. To provide optimal performance, IDS and IPS should support PCRE syntax, SNORT rules, normalized URL detection and TCP normalization. Third party signature databases are available to detect thousands of threats. URL Filtering URL filtering is a basic security feature, blocking unauthorized or inappropriate URLs. But to work in a carrier-grade DPI environment it must be able to perform at a high level. Specifically, the filtering function must be able to handle millions of URLs at real-time speeds. To achieve these speeds, the system must be able to support both literal strings and wildcards. To reduce the complexity of the rules that govern it, the filtering system should provide URL normalization. 3. Why speed and efficiency matter. Until recently, most DPI systems weren t able to keep up with modern, multi-gigabit network speeds. Latency and quality of service were serious problems. But the introduction of multi-core processors and hardware acceleration of important functions have made DPI practical and affordable enough for wide deployment. In fact, many of today s carrier-grade DPI systems can be housed in a single enclosure and run at wirespeed, processing tens of billions of bits of information in real time. Without the hardware advantages of modern systems, DPI would become a bottleneck in high-traffic circumstances. Raw throughput speed is only part of the picture. Advanced DPI systems are also highly efficient, so they consume fewer resources and can run on less expensive equipment. Until recently, DPI had to run on power-hungry, dedicated systems. Today, it can be integrated efficiently into a larger system. What do these advances mean to you? To provide DPI, you no longer need deep expertise in the technology. Standardization has made DPI relatively easy to add to many OEM and enterprise systems.

7 White Paper: 5 Things You Need to Know about Deep Packet Inspection (DPI) 7 4. How do you achieve high-performance DPI? Recent technical advances have made line-speed DPI a practical and affordable option for many enterprise and carrier-grade networks. Today s state-of-the-art multi-gigabit DPI systems include many hardware and software innovations. If you are considering implementing a DPI system, you will want to look out for these high-performance features: Hardware Features: Multi-core processor technology On-chip or on-board hardware acceleration for common functions Code compaction to reduce l-cache misses Normal path prediction to reduce execution cycles Data structure consolidation to improve flow setup performance Pre-fetching to sustain performance through data flow spikes Software Features: TCP-IP reassembly for accurate payload scanning An abstracted centralized flow manager to allow for additional DPI engines In-line decompress/gzip support to decompress HTTPS payloads HTML and MIME parsing to allow URL and object extraction Minimal packet rescanning for 3x to 4x performance improvement Ability to dynamically update rules Optimized signatures 5. The implications of DPI on privacy DPI is a powerful technology. And with great power comes the potential for abuse. Because DPI can search through the contents of Internet traffic including , http requests and chat some privacy advocates are worried that individuals civil liberties are at risk. For instance, DPI can scan all of a network s unencrypted traffic, searching for and logging specific keywords, identification characteristics and Internet use. (In fact, this exactly the sort of snooping that is allowed under the Communications Assistance to Law Enforcement Act (CALEA), the federal law that allows law enforcement under a warrant to tap into networks.) Fortunately, few cases of this type of abuse have been discovered in the private sector, to date. In fact, there is little reason to look into the data portion of a packet s payload, as signatures, meta data and rules can usually identify an application without that information. Companies that deploy DPI can combat privacy concerns with clearly written, enforceable policies that lay out what information can be collected and what cannot. They should also remind themselves on a regular basis that intrusive behavior, if discovered, can have serious repercussions on their reputation and revenues.

8 White Paper: 5 Things You Need to Know about Deep Packet Inspection (DPI) 8 There is nothing inherent in DPI that compromises people s privacy, of course. In fact, DPI provides features and benefits to network communications that are available nowhere else. For instance, DPI s ability to feed data to applications at different bit rates allows service provides to make optimal use of limited bandwidth and dramatically improve the end user s experience. Using DPI, a provider can discriminate in favor of applications that require smooth data delivery. In this way, a streaming video can be allocated more bandwidth than a video download. This technology allows companies like Netflix and YouTube to deliver high performance even during peak hours. And now lower speed options are becoming available at commodity prices, putting DPI within reach of consumer-facing products. A Comprehensive Approach to DPI What does a comprehensive approach to DPI look like? A number of companies build carrier-grade DPI devices, but in an attempt to describe a fully-featured product, we will look at the solution with which we are most familiar. Cavium Solutions and Services TurboDPI TurboDPI, a network-based multi-function software platform, is designed to take advantage of Cavium Networks multi-core OCTEON II processors and their built-in packet inspection engines. The product is designed for OEM and ODM customers who either 1) don t have their own DPI product and want to add carrier-grade performance to a new or existing product; or 2) want to enhance the performance and functionality of their existing DPI product. TurboDPI can be adapted to any of several standard form factors, including AMC modules and ATCA blades. Architecture The TurboDPI system is designed to simultaneously support multiple functions, such as protocol detection, URL filtering and IDS/IPS, and anti-malware. Packets passing through the system first undergo on-the-fly IP and TCP reassembly and decompression before being passed to the flow manager. HTTP, MIME and URL normalization are applied and the data flow is checked against a variety of signatures and rules. Packets flagged as positives are then routed to appropriate applications (such as anti-malware) for further processing. TurboDPI s patented Uni-Scan technology offers an additional three-fold performance boost by performing multiple detection scans in a single pass. The system is able to achieve this efficiency by taking advantage of OCTEON s hardware acceleration features, such as HFA.

9 White Paper: 5 Things You Need to Know about Deep Packet Inspection (DPI) 9 Key Functions The TurboDPI system comes with built-in support for all four critical detection functions: Protocol detection and application recognition, anti-malware and anti-virus, IDS/IPS and URL filtering. The protocol detection engine is supported by signature-, port- and heuristics-based detection systems, all of which can be updated dynamically. Similarly, the anti-malware and anti-virus system can be easily updated, either manually or using an automated third-party profiling service (such as Kaspersky).

10 White Paper: 5 Things You Need to Know about Deep Packet Inspection (DPI) 10 Performance TurboDPI was designed for performance. It s state-of-the-art OCTEON II processor with on-board HFA can process packets at a data rate of up to 40 Gbps. In addition, the solution s hardware-based decompression and checksum engines, together with its Uni- Scan technology, provide industry-leading performance in a compact form factor. About Cavium Solutions and Services Cavium Solutions and Services (CSS) is the leading authority on software application development for the Cavium platform. With insider access to Cavium s chip designers and engineers, CSS is able to achieve the greatest possible performance from Cavium parallel processors. CSS has been developing multi-core software for over nine years, and it has helped many brand-name manufacturers bring top-performing products to market.