EBOOK. The Network Comes of Age: Access and Monitoring at the Application Level

Transcription

1 EBOOK The Network Comes of Age: Access and Monitoring at the Application Level Rev. A, January 2014

2 2

3 Table of Contents How Flow Analysis Grows Into Total Application Intelligence... 4 The Secret Life of Packets:... 4 Why application specific metrics are critical to performance monitoring... 4 The Foundation of Network Monitoring: Flows, Applications, Sessions and DPI... 5 Achieving Superior Monitoring and Analysis... 7 Application Specific Metrics: Examples of Web Based Applications

4 How Flow Analysis Grows Into Total Application Intelligence The Secret Life of Packets: Companies need to control or reduce complexity without sacrificing the level of detail that keeps the network running smoothly Why application specific metrics are critical to performance monitoring. Today the explosion of huge numbers of applications, many of which converge on the same port numbers, calls for revolutionary network monitoring architecture. This need goes well beyond traditional performance monitoring to provide a richer set of data than ever before. The growing numbers of applications bring with them a concurrent need for a costeffective application-awareness architecture that delivers full network visibility and availability. It would reveal the deep, application-specific metrics necessary to detect, isolate, diagnose and resolve problems. At the same time, companies need to control or reduce complexity without sacrificing the level of detail that keeps the network running smoothly. Small and medium-size businesses in particular need application metrics so that they can maintain quality of service without major investment in IT personnel. The needs of SMBs vary from those of large corporations: SMB resources are more limited and manpower both management time and financial means are constraints. SMBs need to focus resources on their core business and daily core business operations. They are looking for user friendliness so that they don t have to employ costly on-site engineers. SMBs are also concerned with achieving lowest total cost of ownership (TCO), which includes maintenance and support. 4

5 Thus, SMBs seek to create a total network monitoring and access architecture that extends visibility and control to the critical application layer and enables them to plug visibility holes in their infrastructure. Comprehensive visibility, application intelligence and real-time monitoring can help these companies address critical business needs. Acquiring the ability to look inside a payload and see key performance metrics is critical to achieving true performance management functionality. This feature is unfortunately lacking in many Network Performance Monitoring (NPM) and Application Performance Monitoring (APM) offerings. This ebook, through a thorough explanation of traditional and emerging monitoring techniques explores the sophisticated architecture behind Net Optics network monitoring architecture, a new route to the very capabilities needed to support a wide range of applications. This route leads to true business intelligence, reliability, and optimization of resource use in your network. The Foundation of Network Monitoring: Flows, Applications, Sessions and DPI Today, there is a lot of confusion around the idea of flow monitoring, application monitoring, session analysis and the place of Deep Packet Inspection (DPI). Vendors use these terms prolifically to describe their own product sets but unfortunately they mean different things to different people. So for the purposes of this discussion, let s define some of the terms. Fundamentals Packet Structure For obvious reasons, many flow monitoring vendors today are claiming to be application performance vendors and to some extent it could be argued that they are, as there is no generally accepted definition of APM. To truly clarify the value and power of monitoring applications let s look at flow monitoring as a starting point, in terms of its technology the packets and IP protocol itself. These fundamental concepts are actually quite simple. Today, there is a lot of confusion around the idea of flow monitoring, application monitoring, session analysis and the place of Deep Packet Inspection (DPI). As most are aware, information is transmitted across modern networks as discrete units of information called packets by means of a common protocol called Transmission Control Protocol / Internet Protocol (TCP/IP). Networking used to be a wilderness of literally dozens of protocols, but TCP/IP has emerged as the predominant approach. The term itself is less important than the fact that TCP/IP is actually two different protocols cobbled together. Each packet on the network has an IP portion and TCP portion (there are others, but for the sake of this discussion, we will consider only TCP/IP). Each packet looks like this: The packet is broken into basic parts: an IP header, a TCP header and a payload. The IP header contains the addressing information where to send the packet. This IP header is analogous to the recipient address and return address on a letter, letting the routers and switches know where to direct the packet. The TCP header describes what type of packet it is, using Port Numbers to define different types of packets; for example, Port 80 usually defines a packet to or from a web server. 5

6 The payload contains the data itself; in the web context, it might hold the URL string or the actual data from the web page itself. Let s consider one more item before moving on to flow monitoring; A Brief TCP/IP Primer Vendors often refer to Layer 3 or Layer 4 or even to the ubiquitous Layer 7. These layers originate from a theoretical model of networking called the OSI model. This model defined seven layers for networking: Layer 1 is the physical medium, such as the actual copper or fiber; Layer 7 is the application itself. Although these layers have fallen into common parlance, many people do not actually understand the origins of the OSI model, which is illustrated and explained by the graphic referenced below. Deriving statistical information from packet headers is easy, but packets do not exist in isolation. TCP/IP does not strictly conform to the OSI model, but rather merges some layers together. Again for simplicity, with respect to the layers, the TCP/IP packet looks as follows: In modern networks, there are essentially three layers above the Data Link Layer, i.e. Layer 2. These are Layers 3, 4 and 7. Layer 3 corresponds to the IP header, Layer 4 to the TCP header and Layer 7 to the payload itself. The Importance of Basic Packet Structure to the Flow So now that we have described TCP/IP, let s consider flow monitoring itself. The good news is that both TCP and IP are protocols; that is, they are well defined and documented in Internet documents called Requests for Comments (RFCs), which are openly available for anyone to implement. Protocols themselves are very strict; for example, we know that at a particular byte offset we will see the source or sender s address; at another we will see the destination or recipient s address; and at yet another, the TCP port number. In fact, the protocols define many fields, each at a specific and well-known location. Monitoring the Flow Deriving statistical information from packet headers is easy, but packets do not exist in isolation; most applications rely on a sequence of packets to transfer data. The mechanism that groups together multiple packets associated with an application is commonly referred to as a flow. Each unique flow may be identified by fields contained in the IP and TCP header. Five fields are commonly used to identify a flow (referred to as 5-tuple identification). Grouping multiple packets into flows further reduces the processing and storage overhead on monitoring systems, as we need to collect statistical information only for each flow rather than for each packet. Think of a flow as a telephone conversation: Each telephone call typically has two parties, each of whom can communicate at the same time. In the same way that a phone call is 6

7 bidirectional, an IP flow is typically made up of two unidirectional flows. Monitoring tools perform a process called flow stitching to put the two unidirectional flows back together and form a single bi-directional flow (similar to a phone call). Achieving Superior Monitoring and Analysis Flow monitoring occurs at layer 4; it provides reasonable information about network traffic and can be carried at high speed. Why then do we need to go any further? Historically it has been sufficient to examine network traffic at layer 4, as most applications used a distinct TCP port number (contained in the TCP header). This meant that we could usually identify applications based on information contained in the TCP header alone. However, there are two main problems with this approach: First, many applications use the same, or common, TCP port number for example, many web applications use TCP Port 80, making it impossible to differentiate applications based on port number alone. Secondly and perhaps more importantly the statistical information needed to characterize performance for one application may be different from that used for another, causing confusion and inaccuracy. The only way to really examine application-specific performance is to look beyond the IP and TCP header into the payload; that is, to examine the packet at Layer 7 using deep packet inspection (DPI). After inspecting the header, one would think it is easy to assess the payload as well, but this perceived simplicity is deceptive. Examining the payload is actually very difficult. For one, no standards determine how applications store data in the payload, so every application takes a different approach, often proprietary. This makes the task of identifying individual applications within the payload very difficult indeed. It becomes even more challenging to employ DPI and application identification at wire speed. The only way to really examine application-specific performance is to look beyond the IP and TCP header into the payload. The upside is that if we can inspect and interpret within the payload, this information is a far richer source of performance data. The challenge is large but the rewards are great! Application Specific Metrics: Examples of Web Based Applications To diverge for a moment, let s consider a common and relatively well-understood application of web browsing. Example: Web Browsing The diagram below shows a normal web browsing session as a TCP/IP flow using TCP port 80. However the payload is where it becomes interesting: The payload contains the URL of the website along with a wealth of other statistical data such as web server type, client browser type, server response code, page download time and much more. 7

8 The ability to use DPI to look inside a payload and identify applicationspecific metrics is a cornerstone of Net Optics APM solutions. Example: Yahoo Mail Now let s take another completely different application this time a web-based program such as Yahoo Mail, looking at the diagram again we see that Yahoo Mail shares commonality with web browsing up to and including Layer 4 and even uses the same transport protocol, HTTP. Traditional flow-based tools cannot differentiate between web browsing and Yahoo mail, as they look identical up to layer 4. However, these are clearly different applications; one is browsing, and the other is . Whilst we might be interested in URL, page download and browser type for web browsing, these metrics are meaningless for Yahoo Mail. metrics are more likely to be to and from address, attachment name, subject, and so on. More similarities and differences appear on the diagram below. Note that there are many flow-type attributes in common between Web browsing and Yahoo Mail that are of interest, including source and destination address, bytes transferred, latency and so on. There are also many more disparate application-specific attributes or metrics. The ability to use DPI to look inside a payload and identify application-specific metrics is a cornerstone of Net Optics APM (Application Performance Monitoring) solutions. The concept extends well beyond web browsing and Yahoo Mail, of course. There are literally thousands of applications that need consideration, making the tree diagram very complex indeed. The current Net Optics architecture can support identify well over 1000 applications within the payload, represented as a tree with 1000 branches in the diagram above (from the IP/TCP trunk, as opposed to application-specific metrics, which would be represented as leaves numbering in the tens of thousands). Net Optics has the architecture in place to potentially cover any application solution 8

9 Summary The Net Optics approach to network monitoring is engineered to reach beyond conventional performance monitoring or simple flow monitoring to provide a far deeper and richer set of data. The advantage of such pinpoint accuracy and granularity cannot be overstated. Net Optics monitoring architecture is designed to support and analyze a wide range of applications a full tree this differentiates it profoundly from other offerings. Extracting application-specific metrics tailored to the needs of each application is critical yet many offerings are severely deficient by comparison in this essential capability. Customers must make certain that they are investing in a truly capable solution in order to derive full benefit from the leading edge of this exciting and productive technology. About the Author David Britt, Director of APM Technology, Net Optics Inc. David Britt, Director of APM Technology for Net Optics, is the lead developer behind Net Optics application performance monitoring solutions. Dave helped companies extend network visibility to remote sites and branch offices with the creation of the apptap. This network and application performance monitoring tool provides packet capture, flow analysis and Deep Packet Inspection (DPI) in traditional network blind spots. apptap is ideally suited to working within an entire network topology such as a remote branch or satellite office. Dave brings to Net Optics over 20 years experience in networking building solutions for network performance monitoring, security and lawful interception. 9

10 EBOOK Ixia Worldwide Headquarters Agoura Rd. Calabasas, CA (Toll Free North America) (Outside North America) (Fax) Ixia European Headquarters Ixia Technologies Europe Ltd Clarion House, Norreys Drive Maidenhead SL6 4FL United Kingdom Sales (Fax) Ixia Asia Pacific Headquarters 21 Serangoon North Avenue 5 #04-01 Singapore Sales Fax Rev. A, January 2014