White Paper Meeting the Five Key Needs of Next-Generation Cloud Computing Networks Cloud computing promises to bring scalable processing capacity to a wide range of applications in a cost-effective manner. Some applications like large social networking sites (Web 2.0) are natural fits for cloud computing s scale and efficiency. Other emerging cloud computing applications include software as a service (SaaS), platform as a service (PaaS), and even infrastructure as a service (IaaS). Regardless of the deployment model, cloud computing is likely to play a role in many information technology decisions in the future. July 2009
This white paper will show the significant role that the Broadcom NetXtreme II 10 Gigabit Ethernet (10 GbE) controllers and Arista Networks 10 GbE switches play in enabling efficient cloud computing and how this technology addresses the five key attributes required to build out a viable infrastructure: High scalability is used to accommodate the enormous intracloud throughput required. The technology promises guaranteed performance for diverse applications such as simultaneous voice, video, and Web traffic. Low latency is provided to maintain performance. Simple management is inherent in spite of its vast complexity. A resilient architecture is built in to handle live updates and server failure. Understanding Cloud Computing Maintaining hardware and software in a business environment can be daunting. As Web traffic grows, enterprise networks face a challenge: how can the network use the resources at its disposal in the most cost-effective manner without compromising future growth? Cloud computing addresses this challenge by shifting the workload from the client s front end (local computers) to a third-party provider s back end (services). It replaces the burden of finite local system management with a highly scalable hosting remote environment solution (see Figure 1). To the point, cloud computing provides a cost-effective, virtual hardware and software solution: a client contracts with third-party providers. The providers supply the most current hardware and software (such as network servers, switches, routers, back-up systems, and various applications) and the client scales the services according to fit the enterprise s requirement. Figure 1: Cloud Computing Networks Consolidate Scalable, Abstracted Processing Capacity. Through Virtualization, These Resources are Accessible as Services to Users. Page 2
There are several applications driving the cloud computing market. While social networking applications with their tremendous bandwidth consumption are clear candidates for cloud computing, few companies have historically had a business need to drive so much data. However, with increasing network complexity and the need for more flexibility in the data center, a cloud computing infrastructure may provide an efficient solution for large enterprises to accommodate ever-changing computing needs. Cloud computing will also serve companies with modest or erratic capacity needs that do not want to constantly change their data center hardware to meet evolving computing needs. For example, virtual servers can be created from a homogenous pool of high-performance servers equipped with a 10 GbE network to scale processing and network capacity on demand instead of deploying servers that would be used only occasionally or at low utilization levels. In this way, capacity can be reallocated between different departments throughout an enterprise based on changing or seasonal needs. As the cost of virtual servers drops, the number of applications that can make use of them increases. Temporary access to servers makes substantial processing capacity an entire cloud s worth available to users for a fraction of the cost of operating an independent data center. For example, a video editing company could reallocate additional virtual servers for frame rendering, making delivery time more a factor of how many servers are running concurrently than how many servers were purchased for a specific task. In the engineering sector, developers can commandeer a block of servers for a 24-hour period to quickly and thoroughly test a new software release. Researchers could use university or lab servers offpeak processing times to crunch large data sets. Scalability While aggregating servers alleviates processing capacity limitations, the fabric connecting them must also scale if the fabric is not to become a bottleneck and throttle throughput of data. Given the potential number of servers that can be brought together, a cloud computing fabric must conceivably be able to scale enough to efficiently handle billions of packets per second. The importance of scalability can be illustrated using a generic social networking application (see Figure 2 on page 4). When data arrives into the cloud, it first passes through a Web serving tool, then through a database, followed by a distributed memory caching application. The result of all these transitions is that each incoming byte of data can propagate to 20 bytes to 30 bytes of intracloud traffic. Implementing 10 GbE in the cloud fabric is key to ensuring that intracloud traffic does not create a bottleneck through the fabric. Since 10 GbE supports 10 times the network traffic as 1 GbE to 10 GbE, this enables users to create higher capacity layer 2 subnets, leading to reduced system cost and lower latency. Consider that the basic structure of a cloud computing network consists of pools of servers. If these servers are connected across a 1 GbE fabric using 48-port switches, then each pool can support enough servers to consume 96 Gbps bidirectionally. To build out a cloud computing infrastructure with greater capacity requires the connection of each pool, typically via a router. Page 3
Figure 2: Hundreds of Data Center Servers may be Involved per Transaction/Request. A Small Amount of External Network Traffic Creates a Large Amount of Intracloud Network Traffic. Moving from 1 GbE to 10 GbE enables users to create much more fabric capacity per deployed switch. For example, the Arista Networks 7148SX 48-port 10 GbE switch can support a fabric of servers consuming 960 Gbps bidirectionally. A larger fabric capacity efficiently supports a greater number of virtual servers. Depending upon the architecture employed, utilizing a 10 GbE-based layer 2 subnet also reduces the number of routers required to create larger clouds, lowering initial capital expenses as well (see Figure 1 on page 2). Developers must also consider server resources when evaluating scalability. Depending upon the implementation, TCP and iscsi processing overhead at 10 GbE line rates can consume a significant amount of resources and overwhelm servers. While these two tasks can be implemented in software, doing so consumes those CPU resources that could instead be used to run applications within the cloud computing network. Given the cost differential between hardware-based offloading and available CPU resources, it is prudent to utilize hardware-based offloading where possible. Page 4
Figure 3: (Top) Layer 2 Subnet Created by a 1 GbE Fabric using a 48-port Switch Supports a Server Pool Capable of Consuming 96 Gbps Bidirectionally. (Bottom) Layer 2 Subnet Created by Moving from 1 GbE to 10 GbE Enables the Creation of a Much Larger Server Pool Capable of Consuming 960 Gbps Bidirectionally. Utilizing a 10 GbE-based Layer 2 Subnet (using, for example, an Arista Networks 7148SX 10 GbE Switch) Significantly Increases the Network Capacity in Large Cloud Networks. Page 5
Broadcom NetXtreme II Ethernet controllers, for example, maximize server capacity by offloading TCP/IP protocol processing from server CPUs. In addition, by implementing a full iscsi host bus adapter (HBA), the NetXtreme II controllers are able to directly take SCSI control and data blocks and convert them to iscsi Ethernet packets, thus offloading iscsi processing as well. By more efficiently managing 10 GbE links in this way, servers are freed to run more applications. Likewise, the Arista Networks Extensible Operating System (EOS TM ) enables customers to customize their IT environment to converge storage and data networks into a 10GbE environment. Guaranteed Performance For cloud computing to serve across as diverse a range of applications as possible, it must be able to guarantee performance based on the specific needs of each application. However, to support voice, video, Web, and other types of traffic predictably and reliably, the fabric must have a means for differentiating between them. When networks are first deployed, administrators often rely upon the overprovisioning of links to guarantee the performance of all traffic types. However, as these networks mature, actual utilization begins to more closely match available bandwidth. As this occurs, the differing priority of the various traffic types begins to result in contention and loss of performance for higher priority flows. Rather than making further capital investments by deploying new equipment to overprovision links that are nearing capacity, administrators can employ Quality of Service (QoS) mechanisms such as Broadcom s Service Aware Flow Control (SAFC). SAFC tags the priority of traffic and makes more efficient use of available bandwidth by ensuring that high-priority traffic is given access to the network ahead of lowerpriority traffic. Going forward, SAFC is being adapted to conform to the emerging Data Center Bridging standard defined by the Institute of Electrical and Electronic Engineers (IEEE). Users will experience an immediate increase in performance when moving to 10-gigabit speeds, even if only replacing existing 1 GbE links, due to the 10X availability of bandwidth. As a result, there may be a tendency to underestimate the need for QoS. However, all networks reach congestion levels sooner than anticipated and, if QoS has not been implemented in the core fabric infrastructure, users will be forced to rely upon overprovisioning, with its corresponding capital investment, to maintain performance. By recognizing that overprovisioning and QoS are both powerful tools to guarantee performance at different stages of a network s life cycle, developers can be sure to create cloud computing networks that will scale reliably as utilization and congestion increase. Latency In addition to increased scalability and guaranteed performance, 10 GbE also delivers much lower latency than 1 GbE when used as an interconnect between servers. As with performance, latency also degrades as the network scales and, to maximize server utilization, the cloud network must deliver microsecond latency across the entire fabric. By its basic nature, 10 GbE introduces one tenth the wire line latency of 1 GbE. Because of the increased processing requirements for operating at line rate, 10 GbE adapters and switches are also faster than their 1 GbE counterparts. Page 6
Full Remote Management Maintaining a network requires sufficient visibility into the fabric as well as control over how the network configures, operates, and repairs itself. Traditionally, networks have been managed using a dedicated network reserved solely for management functions. Network elements would provide heartbeats that, if missed, would alert a systems operator to dispatch a technician to investigate the problem. While this approach protects management commands from being delayed by data traffic, it requires substantial capital investment. By supporting the Network Controller Sideband Interface (NC-SI) defined by the Desktop Management Task Force (DMTF), it becomes possible to have system management traffic coexist with standard traffic in a way that maximizes the utility of each 10 GbE connection. NC-SI allows a system management controller to reside on each server motherboard that operates on its own power and functions independently of the OS running on the server. As a consequence, NC-SI extends full management capabilities to remote administrators as if they were in the room with the server, including provisioning, monitoring, maintenance, upgrading, and troubleshooting all regardless of the state of the server. In fact, remote administrators are able to update the OS or even force a reset when the server has gone down. Such a lights out approach simplifies network management as well as brings significant cost savings. A cloud of 10,000 servers, for example, no longer needs 10,000 cables and corresponding switches to drive a separate management network. NC-SI for 10 GbE is an open standard, making it compatible and interoperable with existing network management environments. Resilience Cloud networks operate 24/7, so downtime for maintenance is not an option. With sideband management, real-time software updates, images, and patches can be made transparently, and the fabric can heal itself when servers go down. The availability of full remote management capabilities results in a simpler, more cost-effective approach to network management, enabling administrators to reliably maximize the capacity of computing clouds of even the largest size. Cloud computing promises to change the way businesses allocate and scale processing capacity by eliminating some CAPEX and reducing OPEX costs. 10 GbE network controllers from Broadcom and switches from Arista Networks are key enabling technologies for cloud computing that address the five major factors required for designing cloud computing networks. With the ability to scale throughput, maintain performance, control latency, extend full remote management capabilities, and ensure the highest network resiliency, 10 GbE and the NetXtreme II are well-suited to meet the needs of today s cloud computing networks. Page 7
Broadcom, the pulse logo, Connecting everything, the Connecting everything logo, and NetXtreme II are among the trademarks of Broadcom Corporation and/or its affiliates in the United States, certain other countries and/or the EU. Arista Networks and EOS are trademarks of Arista Networks Inc in the U.S. and other countries. Any other trademarks or trade names mentioned are the property of their respective owners. BROADCOM CORPORATION 5300 California Avenue Irvine, CA 92617 2009 by BROADCOM CORPORATION. All rights reserved. Phone: 949-926-5000 Fax: 949-926-5203 E-mail: info@broadcom.com Cloud_Computing-WP101-D1 July 2009