Lustre Networking BY PETER J. BRAAM

Save this PDF as:
 WORD  PNG  TXT  JPG

Size: px
Start display at page:

Download "Lustre Networking BY PETER J. BRAAM"

Transcription

1 Lustre Networking BY PETER J. BRAAM A WHITE PAPER FROM CLUSTER FILE SYSTEMS, INC. APRIL 2007

2 Audience Architects of HPC clusters Abstract This paper provides architects of HPC clusters with information about Lustre networking that they can use to make decisions about performance and scalability relevant to their deployments. We will review Lustre message passing, Lustre Network Drivers, and routing in Lustre networks and describe how these can be used to improve cluster storage management. The final section of this paper describes some new Lustre networking features that are currently under consideration or planned for release. Contents Challenges in Cluster Networking Lustre Networking - Architecture and Current Features Lustre Networking Architecture Network Types Supported in Lustre Networks Routers and Multiple Interfaces in Lustre Networks Lustre Networking Applications Lustre Support for RMDA Using Lustre Networking to Implement a Site-Wide File System Using Lustre Routers for Load Balancing Anticipated Features in Future Releases New Features For Multiple Interfaces Server-Driven QoS A Router Control Plane Asynchronous I/O Conclusion Lustre Networking

3 Challenges in Cluster Networking Today's data centers provide many challenges on the networking front, which few systems address. File systems require native storage networking over different types of networks and must be able to exploit features like remote direct memory access (RDMA). In large installations, multiple networks must be able to simultaneously access all storage from all locations through routers and multiple network interfaces. Storage management nightmares, such as handling multiple copies of data as they are staged on file systems local to a cluster, can be avoided only if such features are available. We will first describe how Lustre networking addresses almost all of these challenges today. We will also describe how Lustre networking is expected to evolve to provide further levels of load balancing, control, quality of service (QoS) and high availability in networks on a local and global scale. Lustre Networking - Architecture and Current Features Lustre Networking Architecture Based on extensive research, Lustre networking has evolved into a set of protocols and APIs to support high performance, high availability file systems. Key features of Lustre networking are: RDMA, when supported by underlying networks Support for a number of commonly-used network types such as InfiniBand and IP High availability and recovery built into the Lustre networking stack Simultaneous availability of multiple network types with routing between them Figure 1 shows how these network features are implemented in a cluster deployed with Lustre. Figure 1. A Lustre cluster Cluster File Systems, Inc

4 Lustre networking is implemented with layered APIs and software modules. The file system uses a remote procedure API with facilities for recovery and bulk transport. This API in turn uses the LNET TM Message Passing API, which has been derived from the Sandia Portals message passing API, a well known API in the HPC community. The LNET API has a pluggable driver architecture, similar in concept to the Portals network abstraction layer (NAL), to support multiple network types individually or simultaneously. The drivers, called Lustre Network Drivers (LND), are loaded into the driver stack, one for each network that is in use. A feature that allows routing between the different network types has been implemented as a result of a suggestion early in the Lustre product cycle by a key customer, Lawrence Livermore National Laboratories (LLNL). Figure 2 shows how the software modules and APIs are layered. Figure 2. Modular Lustre networking implemented with layered APIs In a Lustre network, configured interfaces are named using network identifiers (NIDs). The NID is a string that has the form id>. A Lustre network is a set of configured interfaces on nodes that can send traffic directly from one interface on the network to another. Examples of NIDs are designating an address on the 0th Lustre TCP network, and designating address 4 on the 8th Lustre Elan network. Network Types Supported in Lustre Networks Lustre provides LNDs to support many networks including: InfiniBand: OpenFabrics, Mellanox Gold, Cisco, Voltaire and Infinicon TCP: Any network carrying TCP traffic, including GigE, 10GigE, and IPoIB Quadrics: Elan3, Elan4 Myricom: GM, MX Cray: Seastar, RapidArray The networks are supported by LNDs, which are pluggable modules for the LNET interfaces. 4 Lustre Networking

5 Routers and Multiple Interfaces in Lustre Networks Lustre networks consist of interfaces on nodes configured with a NID and communicating without the use of intermediate router nodes with their own NIDs. A Lustre network is not required to be physically separated from another, although that is possible. LNET can conveniently define a Lustre network by naming the IP addresses of the interfaces forming the Lustre network. When more than one Lustre network is present, LNET can route traffic between networks using routing nodes in the network. An example is shown in Figure 3. If multiple routers are present between a pair of networks, they offer both load balancing and high availability through redundancy. Figure 3. Lustre networks connected through routers If multiple interfaces of one type are available, they should be placed on different Lustre networks, unless the underlying network software for the network type supports interface bonding resulting in one address. Such interface bonding is available for IP networks and Elan4. Later in this paper, we will describe features that may be developed in future releases to allow LNET to manage multiple network interfaces. Figure 4 shows how multiple Lustre networks can make effective use of multiple server interfaces in the presence of multiple clients. Figure 4. A Lustre server with multiple network interfaces offering load balancing to the cluster Cluster File Systems, Inc

6 Lustre Networking Applications Lustre Support for RMDA With the exception of TCP, LNET provides support for RDMA on all network types. The LND driver automatically uses this feature for large message sizes. When RDMA is used, nodes can achieve almost full bandwidth with extremely low CPU utilization. This is advantageous, particularly for nodes that are busy running other software, such as Lustre server software. However, provisioning with sufficient CPU power and high performance motherboards may justify TCP networking as a trade-off to using RDMA. On 64-bit processors, LNET can saturate several gige interfaces with relatively low CPU utilization, and with the recently released dual-core Intel Xeon processor 5100 series ("Woodcrest"), the bandwidth on a 10 GigE network can approach a gigabyte per second. Lustre networking provides extraordinary bandwidth utilization of TCP networks. For example, end-to-end I/O over a single GigE link routinely exceeds 110 MB/sec. Using Lustre Networking to Implement a Site-Wide File System Site-wide file systems are typically used in HPC centers where many clusters exist on different high speed networks. Such networks are usually not easy to extend or connect to other networks. An increasingly popular approach is to build a storage island at the center of such an installation. The storage island contains storage arrays and servers and is connected on a network such as an InfiniBand or TCP network. Multiple clusters can connect to this island through routing nodes. The routing nodes are simple Lustre systems with at least two network interfaces, one to the internal cluster network and one to the network used in the storage island. Figure 5 shows an example of a global file system. Figure 5. A global file system implemented using Lustre networks A global file system, also referred to as a site-wide file system, provides transparent access from all clusters to file systems located in the storage island. The benefits are not to be underestimated. Traditional data management for multiple clusters involves staging data from one cluster on the file system to another. By using Lustre as a site-wide file system, multiple copies of the data are no longer needed and substantial savings can be achieved, from a manageability and from a capacity perspective. 6 Lustre Networking

7 Using Lustre Routers for Load Balancing Lustre routers are commodity server systems and can be used in a load-balanced, redundant router configuration. For example, consider an installation with servers on a network with 10 GigE interfaces and many clients attached to a GigE network. It is possible, but typically costly, to purchase IP switching equipment that can connect to both the servers and the clients. With a Lustre network, the purchase of such costly switches can be avoided. For a more cost-effective solution, two separate networks can be created. A smaller server network contains the servers on the fast network and a set of router nodes with sufficient aggregate throughput. A second client network with slower interfaces contains all the client nodes and is also attached to the router nodes. If this second network already exists and has sufficient free ports to add the Lustre router nodes, no changes to this client network are required. Figure 6 shows an installation with this configuration. Figure 6. An installation combining slow and fast networks using Lustre routers The routers provide a redundant, load-balanced path between the clients and the servers. This network configuration allows many clients to utilize together the full bandwidth of a server, even if individual clients have insufficient network bandwidth. The routers collect the throughput of the slow network on the client side and forward the data stream to the servers. Because multiple routers stream data to the server network simultaneously, the network on the server side can see data streams that are in excess of those seen on the client networks. Cluster File Systems, Inc

8 Anticipated Features in Future Releases Although Lustre networking offers many features today, more are coming in future releases. Some possible directions for the development of new features include support of multiple network interfaces, implementation of server-driven QoS guarantees, asynchronous I/O, and a control interface for routers. New Features For Multiple Interfaces LNET can currently exploit multiple interfaces by placing them on different Lustre networks. For example, consider a server with two network interfaces with a subset of the clients connected to one interface and the remaining clients connected to the other interface. LNET can define two networks, each network consisting of the subset of clients connected to one of the server network interfaces. This configuration provides reasonable load balancing for a server with many clients. However, it is a static configuration that does not handle link-level failover or dynamic load balancing. We plan to address these shortcomings with the following design. First, LNET can virtualize multiple interfaces and offer the aggregate as one NID to the users of the LNET API. In concept, this is quite similar to the aggregation (also referred to as bonding or trunking) of Ethernet interfaces using protocols like 802.3ad Dynamic Link Aggregation. The key features that a future LNET release may offer are: Load balancing: All links are used based on availability of throughput capacity. Link-level high availability: If one link fails, the other channels transparently continue to be used for communication. These features are shown in Figure 7. Figure 7. Link-level load balancing and failover From a design perspective, these load-balancing and high-availability features are similar to the features offered with LNET routing. A challenge in developing these features is providing a simple way to configure the network. Assigning and publishing NIDs for the bonded interfaces should be simple and flexible, and work even if not all links are available at startup. We expect to use the management server protocol to resolve this issue. 8 Lustre Networking

9 Server-Driven QoS QoS is an issue in various scenarios. A prevalent one is when multiple clusters are competing for bandwidth from the same storage servers. A primary QoS goal is to avoid thrashing server systems in which conflicting demands from multiple clusters or systems result in performance degradation for all clusters. Setting and enforcing policies is one way to avoid this. For example, a policy can be established that guarantees that a certain minimal bandwidth is allocated to resources that must respond in real-time, such as a display session of visual streaming data. Or a policy can be defined that gives systems or clusters doing mission critical work priority for bandwidth over less important clusters or systems. Lustre's role is not to determine an appropriate set of policies, however, but to provide site management capabilities to implement policies to be set and enforced. Figure 8. Using server-driven QoS to schedule video rendering and visualization The Lustre QoS scheduler will have two components: a Local Request Scheduler (LRS) and a global Epoch Handler (EH). The LRS is responsible for receiving and queueing requests according to a local policy as shown in Figure 8. The EH supports the concept of a virtual shared time slice among all servers. This time slice can be relatively large to avoid overhead due to excessive server-to-server networking and latency. For example, a slice might be one second. The LRS and EH together allow a cluster of servers to all execute the same policy during the same time slice. Note that the policy may subdivide the EH time slice and use the subdivision advantageously. The LRS also provides summary data to the EH so that global knowledge and adaptation can be established. Cluster File Systems, Inc

10 A Router Control Plane Lustre is expected to be used in vast worldwide file systems that traverse networks with up to hundreds of routers. To achieve wide area QoS guarantees that cannot be achieved with static configurations, these networks must provide for dynamically implemented configuration changes. To handle these situations, a rich control interface is required between the routers and outside administrative systems. Such a rich control interface is the Lustre Router Control Plane. An example where the Lustre Router Control Plane can be useful is when data packets are being routed by routers from A to B and also from C to D and for operational reasons a preference needs to be given to routing the packets from C to D. The control plane would apply a policy to the routers so that packets would be sent from C and D before packets are sent from A to B. The technology to be used for the control interface remains under discussion but could be similar to what is being discussed elsewhere for global network management. Asynchronous I/O In large compute clusters, significant I/O optimization is still a possibility. When a client writes large amounts of data, a truly asynchronous I/O mechanism would allow the client to register the memory pages that need to be written for RDMA and allow the server to transfer the data to storage without causing interrupts on the client. This makes the client CPU fully available to the application again, which is a significant benefit for some situations. Figure 9. Network-level DMA with handshake interrupts and without handshake interrupts LNET supports RDMA, but currently a handshake at the operating system level is required to initiate the RDMA. The handshake exchanges the network-level DMA addresses to be used. The proposed change would eliminate the handshake and include the network-level DMA addresses in the initial requests to transfer data as shown in Figure Lustre Networking

11 Conclusion Lustre networking provides an exceptionally flexible and innovative infrastructure. Among the many features and benefits that have been discussed, the most significant are: Native support for all commonly used HPC networks Extremely fast data rates through RDMA and unparalleled TCP throughput Support for site-wide file systems through routing, eliminating staging, and copying of data between clusters Load-balancing router support to eliminate low-speed network bottlenecks Lustre networking will continue to evolve with features to handle link aggregation, server-driven QoS, a rich control interface to large routed networks and asynchronous I/O without interrupts. Legal Disclaimer Lustre is a registered trademark of Cluster File Systems, Inc. and LNET is a trademark of Cluster File Systems, Inc. Other product names are the trademarks of their owners. Although CFS strives for accuracy, we reserve the right to change, postpone, or eliminate features at our sole discretion. Cluster File Systems, Inc

InfiniBand Software and Protocols Enable Seamless Off-the-shelf Applications Deployment

InfiniBand Software and Protocols Enable Seamless Off-the-shelf Applications Deployment December 2007 InfiniBand Software and Protocols Enable Seamless Off-the-shelf Deployment 1.0 Introduction InfiniBand architecture defines a high-bandwidth, low-latency clustering interconnect that is used

More information

Solving I/O Bottlenecks to Enable Superior Cloud Efficiency

Solving I/O Bottlenecks to Enable Superior Cloud Efficiency WHITE PAPER Solving I/O Bottlenecks to Enable Superior Cloud Efficiency Overview...1 Mellanox I/O Virtualization Features and Benefits...2 Summary...6 Overview We already have 8 or even 16 cores on one

More information

Quantum StorNext. Product Brief: Distributed LAN Client

Quantum StorNext. Product Brief: Distributed LAN Client Quantum StorNext Product Brief: Distributed LAN Client NOTICE This product brief may contain proprietary information protected by copyright. Information in this product brief is subject to change without

More information

Sockets vs. RDMA Interface over 10-Gigabit Networks: An In-depth Analysis of the Memory Traffic Bottleneck

Sockets vs. RDMA Interface over 10-Gigabit Networks: An In-depth Analysis of the Memory Traffic Bottleneck Sockets vs. RDMA Interface over 1-Gigabit Networks: An In-depth Analysis of the Memory Traffic Bottleneck Pavan Balaji Hemal V. Shah D. K. Panda Network Based Computing Lab Computer Science and Engineering

More information

Exploiting Remote Memory Operations to Design Efficient Reconfiguration for Shared Data-Centers over InfiniBand

Exploiting Remote Memory Operations to Design Efficient Reconfiguration for Shared Data-Centers over InfiniBand Exploiting Remote Memory Operations to Design Efficient Reconfiguration for Shared Data-Centers over InfiniBand P. Balaji, K. Vaidyanathan, S. Narravula, K. Savitha, H. W. Jin D. K. Panda Network Based

More information

Accelerating High-Speed Networking with Intel I/O Acceleration Technology

Accelerating High-Speed Networking with Intel I/O Acceleration Technology White Paper Intel I/O Acceleration Technology Accelerating High-Speed Networking with Intel I/O Acceleration Technology The emergence of multi-gigabit Ethernet allows data centers to adapt to the increasing

More information

Next Generation HPC Storage Initiative. Torben Kling Petersen, PhD Lead Field Architect - HPC

Next Generation HPC Storage Initiative. Torben Kling Petersen, PhD Lead Field Architect - HPC Next Generation HPC Storage Initiative Torben Kling Petersen, PhD Lead Field Architect - HPC 2 Forward Looking Statement The following information contains, or may be deemed to contain, "forward-looking

More information

High Speed I/O Server Computing with InfiniBand

High Speed I/O Server Computing with InfiniBand High Speed I/O Server Computing with InfiniBand José Luís Gonçalves Dep. Informática, Universidade do Minho 4710-057 Braga, Portugal zeluis@ipb.pt Abstract: High-speed server computing heavily relies on

More information

Private cloud computing advances

Private cloud computing advances Building robust private cloud services infrastructures By Brian Gautreau and Gong Wang Private clouds optimize utilization and management of IT resources to heighten availability. Microsoft Private Cloud

More information

SMB Advanced Networking for Fault Tolerance and Performance. Jose Barreto Principal Program Managers Microsoft Corporation

SMB Advanced Networking for Fault Tolerance and Performance. Jose Barreto Principal Program Managers Microsoft Corporation SMB Advanced Networking for Fault Tolerance and Performance Jose Barreto Principal Program Managers Microsoft Corporation Agenda SMB Remote File Storage for Server Apps SMB Direct (SMB over RDMA) SMB Multichannel

More information

Auspex Support for Cisco Fast EtherChannel TM

Auspex Support for Cisco Fast EtherChannel TM Auspex Support for Cisco Fast EtherChannel TM Technical Report 21 Version 1.0 March 1998 Document 300-TC049, V1.0, 980310 Auspex Systems, Inc. 2300 Central Expressway Santa Clara, California 95050-2516

More information

3G Converged-NICs A Platform for Server I/O to Converged Networks

3G Converged-NICs A Platform for Server I/O to Converged Networks White Paper 3G Converged-NICs A Platform for Server I/O to Converged Networks This document helps those responsible for connecting servers to networks achieve network convergence by providing an overview

More information

Intel Ethernet Switch Load Balancing System Design Using Advanced Features in Intel Ethernet Switch Family

Intel Ethernet Switch Load Balancing System Design Using Advanced Features in Intel Ethernet Switch Family Intel Ethernet Switch Load Balancing System Design Using Advanced Features in Intel Ethernet Switch Family White Paper June, 2008 Legal INFORMATION IN THIS DOCUMENT IS PROVIDED IN CONNECTION WITH INTEL

More information

Brocade Solution for EMC VSPEX Server Virtualization

Brocade Solution for EMC VSPEX Server Virtualization Reference Architecture Brocade Solution Blueprint Brocade Solution for EMC VSPEX Server Virtualization Microsoft Hyper-V for 50 & 100 Virtual Machines Enabled by Microsoft Hyper-V, Brocade ICX series switch,

More information

From Ethernet Ubiquity to Ethernet Convergence: The Emergence of the Converged Network Interface Controller

From Ethernet Ubiquity to Ethernet Convergence: The Emergence of the Converged Network Interface Controller White Paper From Ethernet Ubiquity to Ethernet Convergence: The Emergence of the Converged Network Interface Controller The focus of this paper is on the emergence of the converged network interface controller

More information

I/O Virtualization Using Mellanox InfiniBand And Channel I/O Virtualization (CIOV) Technology

I/O Virtualization Using Mellanox InfiniBand And Channel I/O Virtualization (CIOV) Technology I/O Virtualization Using Mellanox InfiniBand And Channel I/O Virtualization (CIOV) Technology Reduce I/O cost and power by 40 50% Reduce I/O real estate needs in blade servers through consolidation Maintain

More information

Running Native Lustre* Client inside Intel Xeon Phi coprocessor

Running Native Lustre* Client inside Intel Xeon Phi coprocessor Running Native Lustre* Client inside Intel Xeon Phi coprocessor Dmitry Eremin, Zhiqi Tao and Gabriele Paciucci 08 April 2014 * Some names and brands may be claimed as the property of others. What is the

More information

Using Multipathing Technology to Achieve a High Availability Solution

Using Multipathing Technology to Achieve a High Availability Solution Using Multipathing Technology to Achieve a High Availability Solution Table of Contents Introduction...3 Multipathing Technology...3 Multipathing I/O Implementations...5 Storage Redundancy...5 Infortrend

More information

Microsoft SQL Server 2012 on Cisco UCS with iscsi-based Storage Access in VMware ESX Virtualization Environment: Performance Study

Microsoft SQL Server 2012 on Cisco UCS with iscsi-based Storage Access in VMware ESX Virtualization Environment: Performance Study White Paper Microsoft SQL Server 2012 on Cisco UCS with iscsi-based Storage Access in VMware ESX Virtualization Environment: Performance Study 2012 Cisco and/or its affiliates. All rights reserved. This

More information

Can High-Performance Interconnects Benefit Memcached and Hadoop?

Can High-Performance Interconnects Benefit Memcached and Hadoop? Can High-Performance Interconnects Benefit Memcached and Hadoop? D. K. Panda and Sayantan Sur Network-Based Computing Laboratory Department of Computer Science and Engineering The Ohio State University,

More information

Broadcom Ethernet Network Controller Enhanced Virtualization Functionality

Broadcom Ethernet Network Controller Enhanced Virtualization Functionality White Paper Broadcom Ethernet Network Controller Enhanced Virtualization Functionality Advancements in VMware virtualization technology coupled with the increasing processing capability of hardware platforms

More information

Why Compromise? A discussion on RDMA versus Send/Receive and the difference between interconnect and application semantics

Why Compromise? A discussion on RDMA versus Send/Receive and the difference between interconnect and application semantics Why Compromise? A discussion on RDMA versus Send/Receive and the difference between interconnect and application semantics Mellanox Technologies Inc. 2900 Stender Way, Santa Clara, CA 95054 Tel: 408-970-3400

More information

Introduction to Cloud Design Four Design Principals For IaaS

Introduction to Cloud Design Four Design Principals For IaaS WHITE PAPER Introduction to Cloud Design Four Design Principals For IaaS What is a Cloud...1 Why Mellanox for the Cloud...2 Design Considerations in Building an IaaS Cloud...2 Summary...4 What is a Cloud

More information

LS-DYNA Best-Practices: Networking, MPI and Parallel File System Effect on LS-DYNA Performance

LS-DYNA Best-Practices: Networking, MPI and Parallel File System Effect on LS-DYNA Performance 11 th International LS-DYNA Users Conference Session # LS-DYNA Best-Practices: Networking, MPI and Parallel File System Effect on LS-DYNA Performance Gilad Shainer 1, Tong Liu 2, Jeff Layton 3, Onur Celebioglu

More information

Question: 3 When using Application Intelligence, Server Time may be defined as.

Question: 3 When using Application Intelligence, Server Time may be defined as. 1 Network General - 1T6-521 Application Performance Analysis and Troubleshooting Question: 1 One component in an application turn is. A. Server response time B. Network process time C. Application response

More information

NETWORK ISSUES: COSTS & OPTIONS

NETWORK ISSUES: COSTS & OPTIONS VIDEO CONFERENCING NETWORK ISSUES: COSTS & OPTIONS Prepared By: S. Ann Earon, Ph.D., President Telemanagement Resources International Inc. Sponsored by Vidyo By:S.AnnEaron,Ph.D. Introduction Successful

More information

Site2Site VPN Optimization Solutions

Site2Site VPN Optimization Solutions XROADS NETWORKS WHITE PAPER Site2Site VPN Optimization Solutions XROADS NETWORKS - WHITE PAPER Site2Site VPN Optimization Solutions The purpose of this paper is to provide an understanding of how XRoads

More information

InfiniBand in the Enterprise Data Center

InfiniBand in the Enterprise Data Center InfiniBand in the Enterprise Data Center InfiniBand offers a compelling value proposition to IT managers who value data center agility and lowest total cost of ownership Mellanox Technologies Inc. 2900

More information

RDMA over Ethernet - A Preliminary Study

RDMA over Ethernet - A Preliminary Study RDMA over Ethernet - A Preliminary Study Hari Subramoni, Miao Luo, Ping Lai and Dhabaleswar. K. Panda Computer Science & Engineering Department The Ohio State University Outline Introduction Problem Statement

More information

VIDEO SURVEILLANCE WITH SURVEILLUS VMS AND EMC ISILON STORAGE ARRAYS

VIDEO SURVEILLANCE WITH SURVEILLUS VMS AND EMC ISILON STORAGE ARRAYS VIDEO SURVEILLANCE WITH SURVEILLUS VMS AND EMC ISILON STORAGE ARRAYS Successfully configure all solution components Use VMS at the required bandwidth for NAS storage Meet the bandwidth demands of a 2,200

More information

Integration Guide. EMC Data Domain and Silver Peak VXOA 4.4.10 Integration Guide

Integration Guide. EMC Data Domain and Silver Peak VXOA 4.4.10 Integration Guide Integration Guide EMC Data Domain and Silver Peak VXOA 4.4.10 Integration Guide August 2013 Copyright 2013 EMC Corporation. All Rights Reserved. EMC believes the information in this publication is accurate

More information

Isilon IQ Network Configuration Guide

Isilon IQ Network Configuration Guide Isilon IQ Network Configuration Guide An Isilon Systems Best Practice Paper August 2008 ISILON SYSTEMS Table of Contents Cluster Networking Introduction...3 Assumptions...3 Cluster Networking Features...3

More information

Boosting Data Transfer with TCP Offload Engine Technology

Boosting Data Transfer with TCP Offload Engine Technology Boosting Data Transfer with TCP Offload Engine Technology on Ninth-Generation Dell PowerEdge Servers TCP/IP Offload Engine () technology makes its debut in the ninth generation of Dell PowerEdge servers,

More information

Introduction to Infiniband. Hussein N. Harake, Performance U! Winter School

Introduction to Infiniband. Hussein N. Harake, Performance U! Winter School Introduction to Infiniband Hussein N. Harake, Performance U! Winter School Agenda Definition of Infiniband Features Hardware Facts Layers OFED Stack OpenSM Tools and Utilities Topologies Infiniband Roadmap

More information

Top Ten Reasons for Deploying Oracle Virtual Networking in Your Data Center

Top Ten Reasons for Deploying Oracle Virtual Networking in Your Data Center Top Ten Reasons for Deploying Oracle Virtual Networking in Your Data Center Expect enhancements in performance, simplicity, and agility when deploying Oracle Virtual Networking in the data center. ORACLE

More information

IBM Global Technology Services September 2007. NAS systems scale out to meet growing storage demand.

IBM Global Technology Services September 2007. NAS systems scale out to meet growing storage demand. IBM Global Technology Services September 2007 NAS systems scale out to meet Page 2 Contents 2 Introduction 2 Understanding the traditional NAS role 3 Gaining NAS benefits 4 NAS shortcomings in enterprise

More information

Post-production Video Editing Solution Guide with Microsoft SMB 3 File Serving AssuredSAN 4000

Post-production Video Editing Solution Guide with Microsoft SMB 3 File Serving AssuredSAN 4000 Post-production Video Editing Solution Guide with Microsoft SMB 3 File Serving AssuredSAN 4000 Dot Hill Systems introduction 1 INTRODUCTION Dot Hill Systems offers high performance network storage products

More information

Driving IBM BigInsights Performance Over GPFS Using InfiniBand+RDMA

Driving IBM BigInsights Performance Over GPFS Using InfiniBand+RDMA WHITE PAPER April 2014 Driving IBM BigInsights Performance Over GPFS Using InfiniBand+RDMA Executive Summary...1 Background...2 File Systems Architecture...2 Network Architecture...3 IBM BigInsights...5

More information

Meeting the Five Key Needs of Next-Generation Cloud Computing Networks with 10 GbE

Meeting the Five Key Needs of Next-Generation Cloud Computing Networks with 10 GbE White Paper Meeting the Five Key Needs of Next-Generation Cloud Computing Networks Cloud computing promises to bring scalable processing capacity to a wide range of applications in a cost-effective manner.

More information

SiteCelerate white paper

SiteCelerate white paper SiteCelerate white paper Arahe Solutions SITECELERATE OVERVIEW As enterprises increases their investment in Web applications, Portal and websites and as usage of these applications increase, performance

More information

Configuring IPS High Bandwidth Using EtherChannel Load Balancing

Configuring IPS High Bandwidth Using EtherChannel Load Balancing Configuring IPS High Bandwidth Using EtherChannel Load Balancing This guide helps you to understand and deploy the high bandwidth features available with IPS v5.1 when used in conjunction with the EtherChannel

More information

Building a Scalable Storage with InfiniBand

Building a Scalable Storage with InfiniBand WHITE PAPER Building a Scalable Storage with InfiniBand The Problem...1 Traditional Solutions and their Inherent Problems...2 InfiniBand as a Key Advantage...3 VSA Enables Solutions from a Core Technology...5

More information

Mellanox Academy Online Training (E-learning)

Mellanox Academy Online Training (E-learning) Mellanox Academy Online Training (E-learning) 2013-2014 30 P age Mellanox offers a variety of training methods and learning solutions for instructor-led training classes and remote online learning (e-learning),

More information

Achieving Real-Time Business Solutions Using Graph Database Technology and High Performance Networks

Achieving Real-Time Business Solutions Using Graph Database Technology and High Performance Networks WHITE PAPER July 2014 Achieving Real-Time Business Solutions Using Graph Database Technology and High Performance Networks Contents Executive Summary...2 Background...3 InfiniteGraph...3 High Performance

More information

Cluster Grid Interconects. Tony Kay Chief Architect Enterprise Grid and Networking

Cluster Grid Interconects. Tony Kay Chief Architect Enterprise Grid and Networking Cluster Grid Interconects Tony Kay Chief Architect Enterprise Grid and Networking Agenda Cluster Grid Interconnects The Upstart - Infiniband The Empire Strikes Back - Myricom Return of the King 10G Gigabit

More information

Mesh VPN Link Sharing (MVLS) Solutions

Mesh VPN Link Sharing (MVLS) Solutions XROADS NETWORKS WHITE PAPER Mesh VPN Link Sharing (MVLS) Solutions XROADS NETWORKS - WHITE PAPER Mesh VPN Link Sharing (MVLS) Solutions The purpose of this paper is to provide an understanding of how XRoads

More information

Data Storage across the MAN by deploying a SAN over Dark Fiber

Data Storage across the MAN by deploying a SAN over Dark Fiber White Paper Data Storage across the MAN by deploying a SAN over Dark Fiber by Pieter Panis, EuroFiber 1/7 1 Introduction The purpose of this document is to illustrate how a storage area network (SAN) can

More information

Unified Fabric: Cisco's Innovation for Data Center Networks

Unified Fabric: Cisco's Innovation for Data Center Networks . White Paper Unified Fabric: Cisco's Innovation for Data Center Networks What You Will Learn Unified Fabric supports new concepts such as IEEE Data Center Bridging enhancements that improve the robustness

More information

ADVANCED NETWORK CONFIGURATION GUIDE

ADVANCED NETWORK CONFIGURATION GUIDE White Paper ADVANCED NETWORK CONFIGURATION GUIDE CONTENTS Introduction 1 Terminology 1 VLAN configuration 2 NIC Bonding configuration 3 Jumbo frame configuration 4 Other I/O high availability options 4

More information

Intel DPDK Boosts Server Appliance Performance White Paper

Intel DPDK Boosts Server Appliance Performance White Paper Intel DPDK Boosts Server Appliance Performance Intel DPDK Boosts Server Appliance Performance Introduction As network speeds increase to 40G and above, both in the enterprise and data center, the bottlenecks

More information

APPLICATION NOTE. Benefits of MPLS in the Enterprise Network

APPLICATION NOTE. Benefits of MPLS in the Enterprise Network APPLICATION NOTE Benefits of MPLS in the Enterprise Network Abstract As enterprises evolve to keep pace with the ever-changing business climate, enterprises networking needs are becoming more dynamic.

More information

POWER ALL GLOBAL FILE SYSTEM (PGFS)

POWER ALL GLOBAL FILE SYSTEM (PGFS) POWER ALL GLOBAL FILE SYSTEM (PGFS) Defining next generation of global storage grid Power All Networks Ltd. Technical Whitepaper April 2008, version 1.01 Table of Content 1. Introduction.. 3 2. Paradigm

More information

Load Balancing for Microsoft Office Communication Server 2007 Release 2

Load Balancing for Microsoft Office Communication Server 2007 Release 2 Load Balancing for Microsoft Office Communication Server 2007 Release 2 A Dell and F5 Networks Technical White Paper End-to-End Solutions Team Dell Product Group Enterprise Dell/F5 Partner Team F5 Networks

More information

Windows 8 SMB 2.2 File Sharing Performance

Windows 8 SMB 2.2 File Sharing Performance Windows 8 SMB 2.2 File Sharing Performance Abstract This paper provides a preliminary analysis of the performance capabilities of the Server Message Block (SMB) 2.2 file sharing protocol with 10 gigabit

More information

Brocade Solution for EMC VSPEX Server Virtualization

Brocade Solution for EMC VSPEX Server Virtualization Reference Architecture Brocade Solution Blueprint Brocade Solution for EMC VSPEX Server Virtualization VMware vsphere 5 for 50 & 100 Virtual Machines Enabled by VMware vsphere 5, Brocade ICX series switch,

More information

Scalable Internet Services and Load Balancing

Scalable Internet Services and Load Balancing Scalable Services and Load Balancing Kai Shen Services brings ubiquitous connection based applications/services accessible to online users through Applications can be designed and launched quickly and

More information

Fault Tolerant Servers: The Choice for Continuous Availability on Microsoft Windows Server Platform

Fault Tolerant Servers: The Choice for Continuous Availability on Microsoft Windows Server Platform Fault Tolerant Servers: The Choice for Continuous Availability on Microsoft Windows Server Platform Why clustering and redundancy might not be enough This paper discusses today s options for achieving

More information

Oracle SDN Performance Acceleration with Software-Defined Networking

Oracle SDN Performance Acceleration with Software-Defined Networking Oracle SDN Performance Acceleration with Software-Defined Networking Oracle SDN, which delivers software-defined networking, boosts application performance and management flexibility by dynamically connecting

More information

Cisco Application Networking for BEA WebLogic

Cisco Application Networking for BEA WebLogic Cisco Application Networking for BEA WebLogic Faster Downloads and Site Navigation, Less Bandwidth and Server Processing, and Greater Availability for Global Deployments What You Will Learn To address

More information

An Oracle Technical White Paper November 2011. Oracle Solaris 11 Network Virtualization and Network Resource Management

An Oracle Technical White Paper November 2011. Oracle Solaris 11 Network Virtualization and Network Resource Management An Oracle Technical White Paper November 2011 Oracle Solaris 11 Network Virtualization and Network Resource Management Executive Overview... 2 Introduction... 2 Network Virtualization... 2 Network Resource

More information

Cisco Application Networking for IBM WebSphere

Cisco Application Networking for IBM WebSphere Cisco Application Networking for IBM WebSphere Faster Downloads and Site Navigation, Less Bandwidth and Server Processing, and Greater Availability for Global Deployments What You Will Learn To address

More information

Cisco Application Networking for Citrix Presentation Server

Cisco Application Networking for Citrix Presentation Server Cisco Application Networking for Citrix Presentation Server Faster Site Navigation, Less Bandwidth and Server Processing, and Greater Availability for Global Deployments What You Will Learn To address

More information

Enhancing Cisco Networks with Gigamon // White Paper

Enhancing Cisco Networks with Gigamon // White Paper Across the globe, many companies choose a Cisco switching architecture to service their physical and virtual networks for enterprise and data center operations. When implementing a large-scale Cisco network,

More information

Networking and High Availability

Networking and High Availability TECHNICAL BRIEF Networking and High Availability Deployment Note Imperva appliances support a broad array of deployment options, enabling seamless integration into any data center environment. can be configured

More information

Cray DVS: Data Virtualization Service

Cray DVS: Data Virtualization Service Cray : Data Virtualization Service Stephen Sugiyama and David Wallace, Cray Inc. ABSTRACT: Cray, the Cray Data Virtualization Service, is a new capability being added to the XT software environment with

More information

Virtual Fibre Channel for Hyper-V

Virtual Fibre Channel for Hyper-V Virtual Fibre Channel for Hyper-V Virtual Fibre Channel for Hyper-V, a new technology available in Microsoft Windows Server 2012, allows direct access to Fibre Channel (FC) shared storage by multiple guest

More information

Achieving Nanosecond Latency Between Applications with IPC Shared Memory Messaging

Achieving Nanosecond Latency Between Applications with IPC Shared Memory Messaging Achieving Nanosecond Latency Between Applications with IPC Shared Memory Messaging In some markets and scenarios where competitive advantage is all about speed, speed is measured in micro- and even nano-seconds.

More information

Cisco Wireless Security Gateway R2

Cisco Wireless Security Gateway R2 Cisco Wireless Security Gateway R2 Product Overview The Cisco Wireless Security Gateway (WSG) is a highly scalable solution for tunneling femtocell, Unlicensed Mobile Access (UMA)/Generic Access Network

More information

Purpose-Built Load Balancing The Advantages of Coyote Point Equalizer over Software-based Solutions

Purpose-Built Load Balancing The Advantages of Coyote Point Equalizer over Software-based Solutions Purpose-Built Load Balancing The Advantages of Coyote Point Equalizer over Software-based Solutions Abstract Coyote Point Equalizer appliances deliver traffic management solutions that provide high availability,

More information

Feature Comparison. Windows Server 2008 R2 Hyper-V and Windows Server 2012 Hyper-V

Feature Comparison. Windows Server 2008 R2 Hyper-V and Windows Server 2012 Hyper-V Comparison and Contents Introduction... 4 More Secure Multitenancy... 5 Flexible Infrastructure... 9 Scale, Performance, and Density... 13 High Availability... 18 Processor and Memory Support... 24 Network...

More information

Cisco Wide Area Application Services (WAAS) Network Module

Cisco Wide Area Application Services (WAAS) Network Module Cisco Wide Area Application Services (WAAS) Network Module The Cisco Wide Area Application Services (WAAS) Network Module for the Cisco Integrated Services Routers (ISR) is a powerful WAN optimization

More information

The functionality and advantages of a high-availability file server system

The functionality and advantages of a high-availability file server system The functionality and advantages of a high-availability file server system This paper discusses the benefits of deploying a JMR SHARE High-Availability File Server System. Hardware and performance considerations

More information

BRIDGING EMC ISILON NAS ON IP TO INFINIBAND NETWORKS WITH MELLANOX SWITCHX

BRIDGING EMC ISILON NAS ON IP TO INFINIBAND NETWORKS WITH MELLANOX SWITCHX White Paper BRIDGING EMC ISILON NAS ON IP TO INFINIBAND NETWORKS WITH Abstract This white paper explains how to configure a Mellanox SwitchX Series switch to bridge the external network of an EMC Isilon

More information

New Features in SANsymphony -V10 Storage Virtualization Software

New Features in SANsymphony -V10 Storage Virtualization Software New Features in SANsymphony -V10 Storage Virtualization Software Updated: May 28, 2014 Contents Introduction... 1 Virtual SAN Configurations (Pooling Direct-attached Storage on hosts)... 1 Scalability

More information

Achieving Mainframe-Class Performance on Intel Servers Using InfiniBand Building Blocks. An Oracle White Paper April 2003

Achieving Mainframe-Class Performance on Intel Servers Using InfiniBand Building Blocks. An Oracle White Paper April 2003 Achieving Mainframe-Class Performance on Intel Servers Using InfiniBand Building Blocks An Oracle White Paper April 2003 Achieving Mainframe-Class Performance on Intel Servers Using InfiniBand Building

More information

Windows TCP Chimney: Network Protocol Offload for Optimal Application Scalability and Manageability

Windows TCP Chimney: Network Protocol Offload for Optimal Application Scalability and Manageability White Paper Windows TCP Chimney: Network Protocol Offload for Optimal Application Scalability and Manageability The new TCP Chimney Offload Architecture from Microsoft enables offload of the TCP protocol

More information

Truffle Broadband Bonding Network Appliance

Truffle Broadband Bonding Network Appliance Truffle Broadband Bonding Network Appliance Reliable high throughput data connections with low-cost & diverse transport technologies PART I Truffle in standalone installation for a single office. Executive

More information

A Dell Technical White Paper Dell Storage Engineering

A Dell Technical White Paper Dell Storage Engineering Networking Best Practices for Dell DX Object Storage A Dell Technical White Paper Dell Storage Engineering THIS WHITE PAPER IS FOR INFORMATIONAL PURPOSES ONLY, AND MAY CONTAIN TYPOGRAPHICAL ERRORS AND

More information

Lustre on Hyperion. Marc Stearman. April, 2009

Lustre on Hyperion. Marc Stearman. April, 2009 Lustre on Hyperion Marc tearman marc@llnl.gov April, 2009 This work performed under the auspices of the U.. Department of Energy by Lawrence Livermore National Laboratory under Contract DE-AC52-07NA27344.

More information

Voice Over IP. MultiFlow 5048. IP Phone # 3071 Subnet # 10.100.24.0 Subnet Mask 255.255.255.0 IP address 10.100.24.171. Telephone.

Voice Over IP. MultiFlow 5048. IP Phone # 3071 Subnet # 10.100.24.0 Subnet Mask 255.255.255.0 IP address 10.100.24.171. Telephone. Anritsu Network Solutions Voice Over IP Application Note MultiFlow 5048 CALL Manager Serv # 10.100.27 255.255.2 IP address 10.100.27.4 OC-48 Link 255 255 25 IP add Introduction Voice communications over

More information

VMWARE WHITE PAPER 1

VMWARE WHITE PAPER 1 1 VMWARE WHITE PAPER Introduction This paper outlines the considerations that affect network throughput. The paper examines the applications deployed on top of a virtual infrastructure and discusses the

More information

Networking and High Availability

Networking and High Availability yeah SecureSphere Deployment Note Networking and High Availability Imperva SecureSphere appliances support a broad array of deployment options, enabling seamless integration into any data center environment.

More information

IP SAN Best Practices

IP SAN Best Practices IP SAN Best Practices A Dell Technical White Paper PowerVault MD3200i Storage Arrays THIS WHITE PAPER IS FOR INFORMATIONAL PURPOSES ONLY, AND MAY CONTAIN TYPOGRAPHICAL ERRORS AND TECHNICAL INACCURACIES.

More information

HBA Virtualization Technologies for Windows OS Environments

HBA Virtualization Technologies for Windows OS Environments HBA Virtualization Technologies for Windows OS Environments FC HBA Virtualization Keeping Pace with Virtualized Data Centers Executive Summary Today, Microsoft offers Virtual Server 2005 R2, a software

More information

Network Performance Management Solutions Architecture

Network Performance Management Solutions Architecture Network Performance Management Solutions Architecture agility made possible Network Performance Management solutions from CA Technologies compliment your services to deliver easily implemented and maintained

More information

CQG/LAN Technical Specifications. January 3, 2011 Version 2011-01

CQG/LAN Technical Specifications. January 3, 2011 Version 2011-01 CQG/LAN Technical Specifications January 3, 2011 Version 2011-01 Copyright 2011 CQG Inc. All rights reserved. Information in this document is subject to change without notice. Windows XP, Windows Vista,

More information

Network Virtualization and Data Center Networks 263-3825-00 Data Center Virtualization - Basics. Qin Yin Fall Semester 2013

Network Virtualization and Data Center Networks 263-3825-00 Data Center Virtualization - Basics. Qin Yin Fall Semester 2013 Network Virtualization and Data Center Networks 263-3825-00 Data Center Virtualization - Basics Qin Yin Fall Semester 2013 1 Walmart s Data Center 2 Amadeus Data Center 3 Google s Data Center 4 Data Center

More information

Testing Challenges for Modern Networks Built Using SDN and OpenFlow

Testing Challenges for Modern Networks Built Using SDN and OpenFlow Using SDN and OpenFlow July 2013 Rev. A 07/13 SPIRENT 1325 Borregas Avenue Sunnyvale, CA 94089 USA Email: Web: sales@spirent.com www.spirent.com AMERICAS 1-800-SPIRENT +1-818-676-2683 sales@spirent.com

More information

Resource Utilization of Middleware Components in Embedded Systems

Resource Utilization of Middleware Components in Embedded Systems Resource Utilization of Middleware Components in Embedded Systems 3 Introduction System memory, CPU, and network resources are critical to the operation and performance of any software system. These system

More information

Cisco Performance Visibility Manager 1.0.1

Cisco Performance Visibility Manager 1.0.1 Cisco Performance Visibility Manager 1.0.1 Cisco Performance Visibility Manager (PVM) is a proactive network- and applicationperformance monitoring, reporting, and troubleshooting system for maximizing

More information

Implementing Network Attached Storage. Ken Fallon Bill Bullers Impactdata

Implementing Network Attached Storage. Ken Fallon Bill Bullers Impactdata Implementing Network Attached Storage Ken Fallon Bill Bullers Impactdata Abstract The Network Peripheral Adapter (NPA) is an intelligent controller and optimized file server that enables network-attached

More information

基 於 SDN 與 可 程 式 化 硬 體 架 構 之 雲 端 網 路 系 統 交 換 器

基 於 SDN 與 可 程 式 化 硬 體 架 構 之 雲 端 網 路 系 統 交 換 器 基 於 SDN 與 可 程 式 化 硬 體 架 構 之 雲 端 網 路 系 統 交 換 器 楊 竹 星 教 授 國 立 成 功 大 學 電 機 工 程 學 系 Outline Introduction OpenFlow NetFPGA OpenFlow Switch on NetFPGA Development Cases Conclusion 2 Introduction With the proposal

More information

Highly-Available Distributed Storage. UF HPC Center Research Computing University of Florida

Highly-Available Distributed Storage. UF HPC Center Research Computing University of Florida Highly-Available Distributed Storage UF HPC Center Research Computing University of Florida Storage is Boring Slow, troublesome, albatross around the neck of high-performance computing UF Research Computing

More information

Preparing Your IP network for High Definition Video Conferencing

Preparing Your IP network for High Definition Video Conferencing White Paper Global Services April 2007 Table of Contents 1.0 OVERVIEW...3 2.0 VIDEO CONFERENCING BANDWIDTH DEMAND...3 3.0 AVAILABLE BANDWIDTH...5 3.1 Converged Network Links... 6 3.2 Dedicated Network

More information

Ethernet: THE Converged Network Ethernet Alliance Demonstration as SC 09

Ethernet: THE Converged Network Ethernet Alliance Demonstration as SC 09 Ethernet: THE Converged Network Ethernet Alliance Demonstration as SC 09 Authors: Amphenol, Cisco, Dell, Fulcrum Microsystems, Intel, Ixia, JDSU, Mellanox, NetApp, Panduit, QLogic, Spirent, Tyco Electronics,

More information

Exhibit n.2: The layers of a hierarchical network

Exhibit n.2: The layers of a hierarchical network 3. Advanced Secure Network Design 3.1 Introduction You already know that routers are probably the most critical equipment piece in today s networking. Without routers, internetwork communication would

More information

Cloud Storage. Parallels. Performance Benchmark Results. White Paper. www.parallels.com

Cloud Storage. Parallels. Performance Benchmark Results. White Paper. www.parallels.com Parallels Cloud Storage White Paper Performance Benchmark Results www.parallels.com Table of Contents Executive Summary... 3 Architecture Overview... 3 Key Features... 4 No Special Hardware Requirements...

More information

IP Telephony: Reliability You Can Count On

IP Telephony: Reliability You Can Count On WHITE PAPER IP Telephony: Reliability You Can Count On Architecture Matters Table of Contents 1. IP Delivers a Foundation for Reliability.................................. 3 2. Compare Three Different

More information

Microsoft Private Cloud Fast Track

Microsoft Private Cloud Fast Track Microsoft Private Cloud Fast Track Microsoft Private Cloud Fast Track is a reference architecture designed to help build private clouds by combining Microsoft software with Nutanix technology to decrease

More information

Extending SANs Over TCP/IP by Richard Froom & Erum Frahim

Extending SANs Over TCP/IP by Richard Froom & Erum Frahim Extending SANs Over TCP/IP by Richard Froom & Erum Frahim Extending storage area networks (SANs) over a distance has become a necessity for enterprise networks. A single SAN island with one storage system

More information