October 2010 Consolidating Multiple s Space and power are major concerns for enterprises and carriers. There is therefore focus on consolidating the number of physical servers in data centers. Application server consolidation is already taking place via virtualization, but what about the various network appliances used to monitor, analyze, measure, secure and optimize IP networks? These are typically stand-alone devices installed at critical points in the network. How can we consolidate these appliances into fewer servers? This paper will look at alternative consolidation approaches based on data sharing mechanisms and virtualization, offering advice on when to use each approach.
Consolidating Multiple s WHITE PAPER THE NEED FOR CONSOLIDATION OF NETWORK APPLIANCES More and more network appliances are being used to monitor, manage, test and secure IP networks. Network appliances are dedicated, built for purpose systems that provide value by off-loading data- and compute-intensive operations from IP routers. However, this comes at the cost of several extra systems to support. Depending on the application, the network appliance is installed at critical points in the network where data needs to be analyzed. For example, network firewalls and Intrusion Prevention Systems (IPS) are installed where the enterprise Local Area Network (LAN) connects to the carrier s Wide Area Network (WAN). The firewall and the IPS analyze data traffic from the WAN to ensure that there are no security threats entering the LAN. One of the major costs associated with installing multiple network appliances is the space and power cost. Many network appliances are based on standard servers, so installing a new network appliance has the same space and power cost as installing a new server. This is typically 500 W per server. In data centers with several hundred thousand servers, power cost is an issue, but so is space and power budget. Many data centers already struggle with power supply issues there is simply no more power to be had! It is for this reason that consolidation of application servers using virtualization is one of the main initiatives for data centers. But what about consolidation of network appliances? How can this be achieved and how can we reduce the number of systems to be supported while still ensuring that we have access to the valuable information that these systems provide? This paper will investigate alternatives for consolidating multiple network monitoring applications. WHAT IS A NETWORK APPLIANCE? A network appliance is a dedicated, built-for-purpose system designed to analyze data in real time. Network appliances differ in their application, which can be: Network and application performance monitoring Network test and measurement Network security Network optimization A common requirement for network appliances is a need to analyze all the data on a connection. For example, for a network measurement application it can be crucial that no IP packets or Ethernet frames are lost. The architecture of a network appliance is straightforward in most cases: A (GUI) for presentation, reporting and configuration of the network appliance implementing the data analysis algorithms A hardware platform for processing data Network interfaces for receiving and (possibly) transmitting data Figure 1: Anatomy of a network appliance 2 DN-0431 Rev. 3
The hardware platform can either be custom-built or based on a standard PC server. The data input/output can thus be custombuilt or based on network adapters that conform to standards used in PC servers (e.g. PCI-Express). Network appliances can either be installed off-line in packet capture mode or in-line as part of the connection: Off-Line Packet Capture MULTIPLE NETWORK APPLIANCES Since each network appliance provides specific functionality, there is often a need for several different types of network appliance at the same location. Often these network appliances need to operate on the same data at the same time. For in-line applications, the network appliances can be serially connected with each appliance working on the data in turn. For example, Firewalls and IPS are usually installed with the Firewall closest to the edge router followed by the IPS: In-Line Packet Analysis Network Firewall Network IPS Figure 2: Off-Line Packet Capture and In-Line Packet Analysis Figure 3: Examples of in-line network appliances Examples of off-line packet capture network appliances are Network Analysis & Troubleshooting Systems, Network Performance Monitors, Intrusion Detection Systems (IDS), Lawful Intercept Systems, Latency Measurement Systems etc. Examples of in-line packet analysis network appliances are firewalls, Intrusion Prevention Systems (IPS), Policy Enforcement Systems etc. However, for off-line packet capture network appliances, each network appliance needs to analyze the same data at the same time. Therefore, the data to be analyzed must be distributed to the different network appliances using a separate system, typically a Load Balancer: 3
Data Distribution to Multiple Appliances Load Balancer Off-Line Packet Capture Network Appliance Graphical Network User Interface Appliance Graphical Network User Interface Appliance Graphical User Interface Data Hardware Input/Output Platform Data Hardware Input/Output Platform Data Hardware Input/Output Platform This configuration works as long as the amount of data that each network appliance has to analyze does not exceed 2 Gbps. However, if the traffic pattern changes and 80% of the traffic needs to be analyzed by one of the network appliances, then this network appliance will be overloaded (i.e. anything above 2 Gbps will be dropped). Another major disadvantage of the load balancing approach is that it introduces an extra box into the solution. This is adding to the space and power concerns already mentioned. Let s look at these in more detail. SPACE AND POWER CONCERNS As mentioned earlier, there are more and more network appliances being introduced into networks to allow efficient network monitoring, management, measurement and security. However, each of these network appliances take up space and consume power and this is a major concern for many organizations. Figure 4: Load Balancing network appliances According to Eaton 1, a typical 2U standard PC server in 2007 consumed 370 Watts of power. By June 2009, this had risen to 530 Watts or an increase of 43% in 2 years! Incidentally, for blade servers, the consumption can be up to 5,000 Watts! This is an effective means of distributing data amongst multiple network appliances, but is it efficient? ADVANTAGES AND DISADVANTAGES OF THE LOAD-BALANCING APPROACH Load balancing provides multiple applications with access to the same data at the same time. It can also filter data so that network appliances need only receive the data they require rather than all data. This can be an advantage in reducing the data load for each network appliance. It also allows each network appliance to be independent. Load balancing can also be used to support redundant network appliances. However, load balancers can only balance the load across physical servers and ports. If a server is supporting multiple applications, a load balancer cannot balance the load intelligently across these applications. This type of application load balancing requires intelligence within the physical server itself. There is also the potential issue of overloading network appliances. Consider an example with 5 network appliances each with 2 x 1 Gbps ports. The line being monitored is a 10 Gbps line. According to Gartner 2, servers only account for 15% of the direct energy in data centers, but have a knock-on effect on cooling requirements leading to a far larger in-direct energy requirement. Application servers are typically under-utilized (i.e. typically less than 15%), but the servers themselves still consume 60% to 70% of the total power consumption even at these low rates 3. Gartner has estimated that virtualization, and thereby consolidation of application servers, can reduce server energy consumption by up to 82% and floor space consumption by up to 85% 3. This is why virtualization has proven to be a popular solution in these environments, as it not only reduces the number of servers required, but optimizes use of those servers remaining allowing the most efficient use of space and power. 1. The Vector Approach to Data Center Power Planning, Eaton, June 2009 2. Data Center Power, Cooling and Space: A Worrisome Outlook for the Next Two Years, Gartner, May 2010 3. Energy Savings via Virtualization: Green IT on a Budget, Gartner, November 2008 4
The challenge is therefore to find a solution that will allow multiple network appliances to be consolidated into a single solution thereby optimizing space and power requirements for these devices. CONSOLIDATING NETWORK APPLIANCES There are two basic approaches to consolidating network appliances: 1. Functionality consolidation: Develop super network appliances that can address multiple applications at the same time 2. Appliance consolidation: Consolidate multiple independent network appliances to a single hardware platform APPLIANCE CONSOLIDATION Supporting multiple applications on the same server is possible today thanks to the multiple Central Processing Unit (CPU) cores in modern servers. Applications can be run at the same time or assigned an affinity to one or more dedicated CPU cores. Intelligent network adapters can provide Intelligent Flow Identification and Distribution features that can provide multiple applications running on different CPU cores with exactly the data they require. For example, a VoIP monitoring application can receive only VoIP frames, an IPTV monitoring application can receive only IPTV frames etc. Functionality consolidation involves the development of super network appliances that include a super-set of features that can address many applications. Examples of such solutions include Universal Threat Management (UTM) systems, which combine firewall, IPS and various other network security functions in a single solution. However, the span of applications for network appliances is so wide, that creating a single network appliance that can be used for all conceivable applications is difficult to achieve in the short term. As the UTM example shows, this approach is best used when addressing related applications. Intelligent Flow Distribution to multiple applications PC Server Hardware Same App 1 App 2 App 3 App 4 Intelligent Network Adapter Applications running on 1 or more CPU cores Applications based on the same operating system Intelligent network adapter distributes required data to each application Appliance consolidation, on the other hand, focuses on the appliance application software level. The focus here is to consolidate multiple applications within a single hardware platform. Each application can remain independent, which allows porting of existing applications from stand-alone network appliances to a consolidated platform. We will focus on appliance consolidation and look at alternatives for implementing such a solution. It is assumed that the consolidated hardware platform is based on a standard server, which is the case for many network appliances. It is also assumed that an intelligent network adapter is used that is capable of capturing packet data at full line rate without packet loss and with a low CPU load. Figure 5: Using Intelligent Flow Distribution to multiple applications The example shown in Figure 5 above is for a packet capture scenario where each application is working on its own set of data. It is equally applicable to in-line scenarios. This solution works well for applications running on the same Operating System () (e.g. Linux, FreeBSD or Windows). However, what about applications running on different or legacy operating systems? In the first instance, we will look at off-line packet capture scenarios, where multiple applications need to access the same data at the same time. 5
This is where virtualization is ideal, as Virtual Machines (VM) can be created to emulate the and environment that each application expects. In Figure 6, a solution is described based on the generic principles of virtualization (i.e. the ability to accommodate applications based on different operating systems each working on their own set of data). Indeed, data separation is a key principle of virtualization solutions. Virtualization solutions, such as VMware, KVM and others, are ideal for situations where the network appliance application software cannot easily be ported to a single server solution as described above. Intelligent Flow Distribution to multiple applications appliances are dedicated to a specific network monitoring, analysis, measurement, security or optimization task, it is common that multiple applications need to access the same data, at the same time, at the same location. In the MULTIPLE NETWORK APPLIANCES section earlier, we saw how load balancers are used to distribute traffic to multiple appliances. Load balancers can also be used to replicate data to these network appliances, which is often a primary use case for load balancers. Nevertheless, in order to consolidate these multiple network appliances onto the same hardware platform, we need to find a mechanism within the physical server of sharing data between multiple applications. PC Server Hardware Virtualization SW (e.g. VMware) 1 App 1 2 App 2 3 App 3 4 App 4 Intelligent Network Adapter Applications running on 1 or more Virtual Machines Virtual Machines emulate different 's including legacy 's Intelligent network adapter distributes required data to each application Figure 6: Using virtualization to distribute to multiple applications There are two ways in which this can be achieved: 1. Data sharing allowing multiple applications on the same to access the same data at the same time 2. Virtualization allowing multiple applications on different 's to access the same data at the same time DATA SHARING TO MULTIPLE APPLICATIONS BASED ON THE SAME In the data sharing solution, data traffic is captured and stored in a single memory buffer. Instead of copying this buffer for each application that needs to analyze the data, a data sharing mechanism is implemented whereby each application can access the single memory buffer at the same time. Each application runs on one or more CPU cores and has access to the captured packet data in a common memory cache. Virtualization can therefore provide a means of supporting multiple 's and even legacy 's making consolidation of existing network appliances easier. Several vendors are currently working on implementing such a solution. SHARING DATA BETWEEN MULTIPLE APPLICATIONS In the examples so far, the assumption has been that each application will analyze its own set of data. In other words, there is no need to share data between applications. However, this is not always the case for network appliances. Since network The data input/output hardware (intelligent network adapter) ensures that the packet data is provided to the memory cache in real time with zero packet loss. The data sharing is managed either by the intelligent network adapter software or separate application software. 6
Data Sharing to multiple applications Data Flow Distribution to multiple applications PC Server Hardware PC Server Hardware Same Applications running on 1 or more CPU cores Virtualization SW (e.g. VMware) Applications running on 1 or more Virtual Machines App 1 App 2 Applications based on the 1 App 1 2 App 2 Applications based on same operating system different operating systems App 3 App 4 3 App 3 4 App 4 Data Sharing SW Data Distribution SW allows all applications provides required data Data Sharing Software to see the same data at the same time Data Distribution Software to each application Intelligent Network Adapter Intelligent Network Adapter Figure 7: Using Data Sharing to distribute to multiple applications Figure 8: Using Virtualization and Data Distribution to multiple applications Using this approach, application software that previously was installed on multiple servers can be installed on a single server. The scale of the solution is limited by the processing power required by each application and the available CPU cores in the server. In this scenario, the data sharing software is replaced by data distribution software used to provide dedicated data to each virtual machine and supported application. This software requires a dedicated CPU core, but all other CPU cores are available to support virtual machines. VIRTUALIZATION FOR SUPPORT OF MULTIPLE APPLICATIONS BASED ON DIFFERENT 'S The solution in this case is to use Virtual Machines (VM) running on one of more CPU cores to emulate the operating system required for the network appliance application software in question. This allows the hardware platform to run the latest operating system independent of the various applications to be supported. This solution is ideal for organizations that are dependent on network appliances that are difficult to upgrade and require their own environment and operating system. IN-LINE CONSOLIDATION In-line consolidation can use the same approaches as above. However, in-line network appliances introduce some additional concerns that should be taken into consideration: Typically, there is a desire to reduce latency as much as possible in in-line devices. Therefore, serial transfer of data between applications can lengthen latency. Equally, shared data access can lead to longer latency, as the total latency will be dictated by the slowest application (i.e. the frame will not be re-transmitted until the last application has finished analyzing the frame). Most in-line applications require a large amount of computing power in order to process and analyze frames as quickly as possible. Therefore, throughput can be affected if these applications are lacking processing power when they need it. 7
CONSOLIDATION PSIBLE, BUT BEWARE! We have now shown that it is possible to consolidate multiple network appliances into a single server using intelligent flow distribution, data sharing and virtualization. Therefore, consolidation is a good solution for lower-bandwidth network appliance applications that are not compute-intensive, but for higher speed rates and more demanding network appliances, a dedicated system approach is still recommended. Nevertheless, one should carefully consider which network appliances should be consolidated. Consolidation is ideal when the connections being monitored/tested/secured are of low bandwidth (i.e. up to 1 Gbps) or have low line utilization. As mentioned earlier, the reason why virtualization provides tangible benefits for application servers is due to the fact that there are typically very low levels of utilization on these servers. However, if network appliances are analyzing high-bandwidth connections with high utilization, literally millions of frames need to be analyzed per second. This typically consumes the available bandwidth and processing power of high-end network appliances. These network appliances can be consolidated into much larger servers, but the benefit of this needs to be analyzed in relation to power and space savings. ABOUT NAPATECH Napatech is the leading OEM supplier of 1 GbE to 40 GbE intelligent adapters for real-time network analysis. Napatech network adapters enable OEM vendors to build affordable, yet high-performance network analysis systems using standard servers. The network adapters provide real-time packet capture and transmission with full line rate throughput and zero packet loss no matter the packet size. Intelligent features enable off-load of data traffic processing and packet analysis normally performed in the CPU. This results in more processing power for the network monitoring, analysis, management, test, measurement, security or optimization application being supported. Napatech has sales, marketing and R&D offices in Mountain View (CA), Andover (MA), Washington D.C., Tokyo ( Japan), and Copenhagen (Denmark). Europe, Middle East and Africa Napatech A/S Tobaksvejen 23 A, 1 DK-2860 Soeborg Denmark Americas Napatech, Inc. One Tech Drive Suite 110 Andover, MA 01810 US APAC Napatech 1/F Place Canada 7-3-37 Akasaka Minato-ku 107-0052 Tokyo Japan Tel. +45 4596 1500 Fax +45 6980 2970 www.napatech.com nteusales@napatech.com Tel. +1 888 318 8288 Fax +1 978 824 9414 www.napatech.com ntamericassales@napatech.com Tel. +81 3 6894 7678 Fax +81 3 6894 7701 www.napatech.com ntapacsales@napatech.com Disclaimer: This document is intended for informational purposes only. Any information herein is believed to be reliable. However, Napatech assumes no responsibility for the accuracy of the information. Napatech reserves the right to change the document and the products described without notice. Napatech and the authors disclaim any and all liabilities. Napatech is a trademark used under license by Napatech A/S. All other logos, trademarks and service marks are the property of the respective third parties. Copyright Napatech A/S 2011. All rights reserved. 8