Automated Migration of Port Profile for Multi-level Switches



Similar documents
A Management Method of IP Multicast in Overlay Networks using OpenFlow

Ethernet Virtual Bridging Automation Use Cases

Standardizing Data Center Server- Network Edge Virtualization

Control Tower for Virtualized Data Center Network

Virtual networking technologies at the server-network edge

Network Virtualization for Large-Scale Data Centers

Automating Virtual Machine Network Profiles

Extreme Networks: Building Cloud-Scale Networks Using Open Fabric Architectures A SOLUTION WHITE PAPER

Simplifying Data Center Network Architecture: Collapsing the Tiers

Huawei Enterprise A Better Way VM Aware Solution for Data Center Networks

SDN CENTRALIZED NETWORK COMMAND AND CONTROL

How To Manage A Virtualization Server

Brocade Solution for EMC VSPEX Server Virtualization

Cloud Networks Uni Stuttgart

The Value of Open vswitch, Fabric Connect and Fabric Attach in Enterprise Data Centers

Virtual Networking Management White Paper. Version Status: DMTF Informational Publication Date: DSP2025

Virtualized Access Layer. Petr Grygárek

The Future of Cloud Networking. Idris T. Vasi

Advanced Network Services Teaming

DCB for Network Virtualization Overlays. Rakesh Sharma, IBM Austin IEEE 802 Plenary, Nov 2013, Dallas, TX

Juniper Networks EX Series/ Cisco Catalyst Interoperability Test Results. May 1, 2009

SummitStack in the Data Center

Ethernet Fabrics: An Architecture for Cloud Networking

A Platform Built for Server Virtualization: Cisco Unified Computing System

Abstract. Avaya Solution & Interoperability Test Lab

OVERLAYING VIRTUALIZED LAYER 2 NETWORKS OVER LAYER 3 NETWORKS

Analysis of Network Segmentation Techniques in Cloud Data Centers

Intel Ethernet Switch Converged Enhanced Ethernet (CEE) and Datacenter Bridging (DCB) Using Intel Ethernet Switch Family Switches

Lecture 02b Cloud Computing II

SummitStack in the Data Center

Virtual PortChannels: Building Networks without Spanning Tree Protocol

Application Note Gigabit Ethernet Port Modes

Abstract. Avaya Solution & Interoperability Test Lab

Multi-Chassis Trunking for Resilient and High-Performance Network Architectures

16-PORT POWER OVER ETHERNET WEB SMART SWITCH

Data Center Networking Designing Today s Data Center

Management Software. Web Browser User s Guide AT-S106. For the AT-GS950/48 Gigabit Ethernet Smart Switch. Version Rev.

The Future of Computing Cisco Unified Computing System. Markus Kunstmann Channels Systems Engineer

How To Make A Vpc More Secure With A Cloud Network Overlay (Network) On A Vlan) On An Openstack Vlan On A Server On A Network On A 2D (Vlan) (Vpn) On Your Vlan

Carrier Ethernet: New Game Plan for Media Converters

How To Make A Virtual Machine Aware Of A Network On A Physical Server

Network Discovery Protocol LLDP and LLDP- MED

iscsi Top Ten Top Ten reasons to use Emulex OneConnect iscsi adapters

Network Discovery Protocol LLDP and LLDP- MED

Migrate from Cisco Catalyst 6500 Series Switches to Cisco Nexus 9000 Series Switches

The Lagopus SDN Software Switch. 3.1 SDN and OpenFlow. 3. Cloud Computing Technology

Network Virtualization and Data Center Networks Data Center Virtualization - Basics. Qin Yin Fall Semester 2013

Data Center Convergence. Ahmad Zamer, Brocade

CCNA R&S: Introduction to Networks. Chapter 5: Ethernet

How To Configure Voice Vlan On An Ip Phone

Objectives. The Role of Redundancy in a Switched Network. Layer 2 Loops. Broadcast Storms. More problems with Layer 2 loops

Next Generation Data Center Networking.

Virtual Networking Features of the VMware vnetwork Distributed Switch and Cisco Nexus 1000V Series Switches

VXLAN: Scaling Data Center Capacity. White Paper

Configuring Network Address Translation (NAT)

EVOLVING ENTERPRISE NETWORKS WITH SPB-M APPLICATION NOTE

CSIS CSIS 3230 Spring Networking, its all about the apps! Apps on the Edge. Application Architectures. Pure P2P Architecture

The Impact of Virtualization on Cloud Networking Arista Networks Whitepaper

Nutanix Tech Note. VMware vsphere Networking on Nutanix

Zarząd (7 osób) F inanse (13 osób) M arketing (7 osób) S przedaż (16 osób) K adry (15 osób)

Configuring LLDP, LLDP-MED, and Location Service

Building Tomorrow s Data Center Network Today

SOLUTIONS FOR DEPLOYING SERVER VIRTUALIZATION IN DATA CENTER NETWORKS

Chapter 1 Reading Organizer

Achieve Automated, End-to-End Firmware Management with Cisco UCS Manager

Cloud Networking: A Novel Network Approach for Cloud Computing Models CQ1 2009

Advanced VSAT Solutions Bridge Point-to-Multipoint (BPM) Overview

Implementing Cisco Data Center Unified Computing (DCUCI)

Chapter 7 Configuring Trunk Groups and Dynamic Link Aggregation

White Paper. Best Practices for 40 Gigabit Implementation in the Enterprise

Cloud Infrastructure Planning. Chapter Six

Configure IOS Catalyst Switches to Connect Cisco IP Phones Configuration Example

B&B ELECTRONICS WHITE PAPER. Managed Ethernet Switches - Key Features for a Powerful Industrial Network

STATE OF THE ART OF DATA CENTRE NETWORK TECHNOLOGIES CASE: COMPARISON BETWEEN ETHERNET FABRIC SOLUTIONS

Cisco EtherSwitch Network Modules

BLADE PVST+ Spanning Tree and Interoperability with Cisco

Impact of Virtualization on Cloud Networking Arista Networks Whitepaper

Juniper / Cisco Interoperability Tests. August 2014

Cisco Nexus 1000V Switch for Microsoft Hyper-V

HP Virtual Connect Ethernet Cookbook: Single and Multi Enclosure Domain (Stacked) Scenarios

Lab VI Capturing and monitoring the network traffic

Management of VMware ESXi. on HP ProLiant Servers

基 於 SDN 與 可 程 式 化 硬 體 架 構 之 雲 端 網 路 系 統 交 換 器

Note: This case study utilizes Packet Tracer. Please see the Chapter 5 Packet Tracer file located in Supplemental Materials.

Technical Bulletin. Enabling Arista Advanced Monitoring. Overview

Cloud Computing and the Internet. Conferenza GARR 2010

Enterasys Data Center Fabric

TRILL for Data Center Networks

What s New in VMware vsphere 5.0 Networking TECHNICAL MARKETING DOCUMENTATION

Windows TCP Chimney: Network Protocol Offload for Optimal Application Scalability and Manageability

802.1X Authentication, Link Layer Discovery Protocol (LLDP), and Avaya IP Telephones

Optimize Server Virtualization with QLogic s 10GbE Secure SR-IOV

Aruba Mobility Access Switch and Arista 7050S INTEROPERABILITY TEST RESULTS:

How To Set Up A Virtual Network On Vsphere (Vsphere) On A 2Nd Generation Vmkernel (Vklan) On An Ipv5 Vklan (Vmklan)

Solaris For The Modern Data Center. Taking Advantage of Solaris 11 Features

Transcription:

Automated Migration of Profile for Multi-level Switches Yukihiro Nakagawa, Kazuki Hyoudou, Shinji Kobayashi, Osamu Shiraki, and Takeshi Shimizu Technologies Laboratory, IT Systems Laboratories Fujitsu Laboratories Ltd. Kawasaki, Kanagawa 211-8588, Japan E-mail: yukihiron@jp.fujitsu.com, Tel. +81-44-754-2177 Abstract The use of virtualization technology has been increasing in the data center to consolidate physical servers into virtual machines and allocate computing resources dynamically by moving virtual machines. IEEE 802.1 Data Center Bridging Task Group is developing a standard to reduce complexity of the network management and automate a task to move network state in an adjacent bridge along with the migration of a virtual machine, which is called Automated Migration of Profile (AMPP). However AMPP works between a server and an adjacent bridge only. We propose an extension of AMPP for multi-level switches to further reduce complexity and automate the task. We developed a prototype of the extension and confirmed a network state is moved in a multi-level switch configuration using the standard protocol as it is on the wire. This paper describes IEEE standard based AMPP, our proposed extension of AMPP for multi-level switches, and a prototype of the extension. Keywords- virtual bridging; Automated migration of port profile; Virtual machine migration I. INTRODUCTION A Cloud Data Center requires dynamic infrastructure and flexible allocation of computing resources to satisfy changing requirements of customers. The virtualization technology and virtual machine mobility are important to realize dynamic infrastructure [1], [2]. When a virtual machine is migrated, it is desired that the network state in the physical switches to be moved together for the network resource optimization and for better security. To reduce complexity of the network management and move network state automatically, some venders are implementing their own protocols [3], [4] or utilizing hypervisor-dependent APIs [5]. These technologies are proprietary and cannot be used in a heterogeneous environment with physical switches from multiple venders. Also some technologies are lack of synchronization mechanism with hypervisor and therefore it is not guaranteed that physical switch configuration completes before a virtual machine migration ends. To overcome these problems, IEEE 802.1 Data Center Bridging Task Group is developing a standard for Virtual Bridging (EVB). As a part of EVB, Automated Migration of Profile (AMPP) is being defined and it is expected to reduce complexity of the network management and automate a task to move network state in an adjacent bridge along with the migration of a virtual machine [6], [7]. DMTF is also working on Virtual Bridging and defining CIM data model for management, port profile database XML schema, and OVF extension [9], [10]. A dynamic infrastructure should allow flexible resource allocation across large server pools. However the standardbased AMPP works between a server and an adjacent bridge only. And external bridges other than the adjacent bridges need to be configured by a network manager using a proprietary management interface per switch vender. In addition, a management interface is not fast enough for mobility. For example, a switch takes one second to process a CLI command comparing a switch firmware takes milliseconds to process a message while a migration of with 512MB memory takes only two seconds in 10GbE network in Windows 2008 Hyper-V. We propose an extension of AMPP for multi-level switches to further reduce complexity and automate the task. An adjacent bridge with the extension conditionally forwards control messages to an external bridge based on the internal status. If an adjacent bridge simply forward control messages to an external bridge, a port profile could be removed while a is running and AMPP fails. We developed a prototype of the extension and confirmed a network state being moved in a multi-level switch configuration using the standard protocol as it is on the wire. In other words, a bridge with AMPP extension capability can work with an external bridge with standard AMPP capability. This paper describes IEEE standard-based AMPP, our proposed extension of AMPP for multi-level switches, and a prototype of the extension. II. IEEE STANDARD-BASED AMPP IEEE 802.1Qbg standard newly defines protocols for EVB. These are Discovery and Configuration Protocol (VDP), Control Protocol (), and S-Channel Discovery and Configuration Protocol (CDCP). VDP is an -based protocol to associate port profile to Virtual Station Interface () of. is a transport protocol to carry with re-transmission capability. CDCP is an LLDP-based protocol to configure S-Channel. VDP is the main protocol to realize the standard-based AMPP operation. Therefore we explain the protocol briefly. 978-0-9836283-2-3 c 2011 ITC 22 This paper was peer reviewed by subject matter experts for publication in the Proceedings of DC CaVES 2011

A. Discovery and Configuration Protocol (VDP) VDP simplifies and automates virtual station configuration by enabling the movement of a instance and its related port profile from one EVB bridge to another. VDP protocol defines 4 messages: Pre-Associate, Pre- Associate with resource reservation, Associate, and Deassociate messages. Pre-Associate message is used to pre-associate a Instance Identifier with a bridge port. Pre-Associate enables faster response to an associate, by allowing the bridge to obtain the port profile prior to an association. Pre-Associate with resource reservation message is similar with Pre- Associate but it also reserves resources in the bridge to prepare for a subsequent association. Associate message is used to activate an association between a Instance and a bridge port and configure it with the port profile. Deassociate message is used to remove an association between a Instance and a bridge port. Remove port profile Bridge Source port Migration not running DE-ASSOC DE-ASSOC CONF Source Destination Power-on destination PRE-ASSOC PRE-ASSOC CONF ASSOC ASSOC CONF ARP Bridge Destination port Retrieve port profile from DB Apply port profile VDP message B. Examples of VDP messages in Migration Figure 1 shows an example of VDP messages communicated between a server and an adjacent bridge when a virtual machine is migrated. When a migration is initiated, Pre-Associate message is sent from the destination server to the adjacent bridge at a destination port and the bridge is preparing a port profile for the by fetching it from a database if necessary. Then a successful response or confirmation is returned to the destination server. During the stop and copy phase, Associate message is sent from the destination server to the adjacent bridge at the destination port and the bridge associates the port profile with the. The bridge returns a response to the destination server. On the other hand, De-associate message is sent from the source server to the adjacent bridge at the source port and the adjacent bridge de-associates the port profile with the. Then the is re-started in the destination server. In this way, the port profile is moved together with the and the physical switch is automatically configured. C. AMPP in Multi-level Switch Configuration As described above, the standard-based AMPP using VDP works between a server and an adjacent bridge. Therefore when a is migrated in a multi-level switch configuration, a network manager needs to configure nonadjacent external bridges as described in Figure 2 (a). In a blade server configuration, there is a switch blade in the chassis and a migration to another chassis always requires an intervention of a network manager. Similarly when we use rack servers, a migration to another rack that is connected to another ToR switch requires an intervention of a network manager. Figure 1. Migration An example of VDP messages in migration Profile Management (a) Without AMPP Extension IEEE Standard Protocol (b) With AMPP Extension Network Manager External Bridge External Bridge () Figure 2. AMPP Extension for Multi-level Switches Proceedings of the 2011 3rd Workshop on Data Center Converged and Virtual Ethernet Switching 23

III. AMPP EXTENSION FOR MULTI-LEVEL SWITCHES We propose an extension of AMPP for a multi-level switches so that we can automatically configure physical switches including non-adjacent bridges in addition to adjacent bridges. The extension consists of forwarding of VDP messages and conversion of edge relay mode. Figure 2 (b) shows our idea in which we move the server edge to the outside of the adjacent bridge so that an external bridge works as an adjacent bridge. To realize this, an adjacent bridge forwards VDP messages to an external bridge based on the internal VDP status. Basically we do not forward VDP messages if a is migrated to a server that is connected with the same adjacent bridge. And we forward VDP messages if a is migrated to a server that is not connected with the adjacent bridge. We note that if we forward VDP messages unconditionally, AMPP fails and causes an unexpected result. For example, when we move a to a server within the adjacent bridge, a port profile at the external bridge is removed when De-associate message comes after Associate message because De-associate message and Associate message are sent on the same uplink on the adjacent bridge which is connected to the external bridge. Also EVB bridge capability needs to be advertized on LLDP so that VDP protocol works between an adjacent bridge and an external bridge. The conversion of mode on EVB TLV in LLDP can be done at the adjacent bridge to increase performance. TABLE I shows the combination of modes in a server and an adjacent bridge. The conversion from Virtual Aggregator () to Virtual Bridge (VEB) enables reflective relay at the adjacent bridge and increases the performance. On the other hand, the conversion from VEB to causes packet duplication hence it is illegal. For example a transmit a multicast packet and the packet is forwarded to another via VEB in the server. The multicast packet is also sent to an adjacent bridge and it is reflective relayed to the server, resulting in a duplicated packet. Figure 3 shows handlings of Mode in legal combinations. In our extension, we use standard protocol as it is on the wire. Therefore we can use a standard-compliant AMPP capable bridge as the most outer physical switch, for example, the external bridge in the configuration of Figure 2(b). TABLE I. Combination of Mode Figure 3. External Bridge VEB VEB RR RR (a) VEB -> VEB (b) -> VEB (c) -> RR: Reflective Handling of Mode A. Forwarding of VDP Messages The forwarding decision of is made for each type (Pre-Associate, Pre-Associate with resource reservation, Associate, and De-associate) based on the state (vsistate = PREASSOC, PREASSOCR, ASSOC, DEASSOC) of the VDP state machine. When the adjacent bridge received a from a server, it retrieves Virtual Station Instance ID (ID) in the TLV and search VDP entries with the VSSID. Up to two entries with same VSSID could exist because of transient status in the migration. Table II shows the forwarding conditions of VDP messages. Pre-Associate with resource reservation is omitted in the table for simplicity of explanation. For example, Pre-Associate message is received when vsistate of the reception port is DEASSOC and vsistate of any other port is DEASSOC, the bridge forwards Pre-Associate TLV because the vsistate of the corresponding port of the upper switch is DEASSOC as described in the case P7. When the forwarding decision of is yes, VDP state machine (bridge side) of the ID goes to the next state and is forwarded to the uplink port related to the downlink port. It waits for the response from the upper bridge. Upon the reception of the response, it checks an error in TLV Type and Reason. If an error occurs, it cancels the action on the VDP state machine and goes to the previous state. Then it sends the response to the server. When the forwarding decision of is no, VDP state machine (bridge side) of the ID goes to the next state and the response is returned to the server for. RR RR Note VEB VEB VEB Illegal because of Packet Duplication VEB Performance Optimization 24 Proceedings of the 2011 3rd Workshop on Data Center Converged and Virtual Ethernet Switching

Ca se TABLE II. Forwarding Conditions of VDP Messages vsistate of Adjacent Switch VDP Reception Other than Reception (a) Pre-Associate Forward ing Note Cf. vsistate of Upper Switch Corresponding P1 PREASSOC DEASSOC Yes Keep Alive PREASSOC P2 PREASSOC Yes Keep Alive PREASSOC P3 ASSOC ASSOC P4 ASSOC DEASSOC Yes ASSOC P5 PREASSOC Yes ASSOC P6 ASSOC ASSOC P7 DEASSOC DEASSOC Yes DEASSOC P8 PREASSOC (Yes) PREASSOC P9 ASSOC ASSOC (b) Associate Ca se vsistate of Adjacent Switch VDP Reception Other than Reception Forwardi ng Note Cf. vsistate of Upper Switch Corresponding A1 PREASSOC DEASSOC Yes PREASSOC A2 PREASSOC Yes PREASSOC A3 ASSOC (Yes) ASSOC A4 ASSOC DEASSOC Yes Keep Alive ASSOC A5 PREASSOC Yes Keep Alive ASSOC A6 ASSOC Yes Keep Alive ASSOC A7 DEASSOC DEASSOC Yes DEASSOC A8 PREASSOC Yes PREASSOC A9 ASSOC (Yes) ASSOC (c) De-associate Ca se vsistate of Adjacent Switch VDP Reception Other than Reception Forwardi ng Note Cf. vsistate of Upper Switch Corresponding D1 PREASSOC DEASSOC Yes PREASSOC D2 PREASSOC PREASSOC D3 ASSOC ASSOC D4 ASSOC DEASSOC Yes ASSOC D5 PREASSOC ASSOC D6 ASSOC ASSOC D7 DEASSOC DEASSOC (Yes) DEASSOC D8 PREASSOC PREASSOC D9 ASSOC ASSOC B. Examples of Migration to another Rack Figure 4 shows an example of VDP messages communicated between a server and an adjacent bridge when a virtual machine is migrated to another server in another rack via multiple ToR switches in the path. When a migration is initiated, Pre-Associate message is sent from the destination server to the adjacent bridge at a destination port. This is a case of P7 described above and Pre-Associate TLV is forwarded to the upper ToR switch. During the stop and copy phase, Associate message is sent from the destination server to the adjacent bridge at the destination port. This is the case of A1 and the adjacent bridge forwards Associate TLV to the upper ToR switch. When a response is returned from the upper switch, the adjacent bridge returns a response to the destination server. On the other hand, De-associate message is sent from the source server to the adjacent bridge at the source port. This is the case of D4 and the bridge forwards De-associate message to the upper ToR switch. In this way, the port profile is moved together with the and the physical switch is automatically configured. Live Migration To another rack Rack Forwarding VDP message External Bridge Profile move External Bridge TLV Type= Pre-Associate, Pre-Associate with resource reservation Associate Figure 4. 1. VDP Req 9. VDP Resp 1. VDP Req 9. VDP Resp VDP downlink 2. Entry Search ID ID vsistate, vsistate, ID vsistate, 8. Resp transmit 2. Entry Search 3. State Machine 4. Forwarding 7. Resp check 8. Resp transmit 3. State Machine VDP VDP SM 7. Resp check ID ID vsistate, ID vsistate, vsistate, VDP SM VDP SM uplink 4. Forwarding 5. VDP Req 6. VDP Resp 5. VDP Req 6. VDP Resp Inter-rack Migration and VDP Messages C. Examples of Migration within Rack Figure 5 shows an example of VDP messages communicated between a server and an adjacent bridge when a virtual machine is migrated to a server in the rack via a ToR switch. When a migration is initiated, Pre- Associate message is sent from the destination server to the adjacent bridge at the destination port. This is the case of P9 where vsistate of the reception port (destination port) is DEASSOC and vsistate of another port (source port) is ASSOC, and the bridge does not forward Pre-Associate TLV because the vsistate of the corresponding port of the upper ToR switch is ASSOC. During the stop and copy phase, Associate message is sent from the destination server to the adjacent bridge at the destination port. This is the case of A3 and the adjacent Proceedings of the 2011 3rd Workshop on Data Center Converged and Virtual Ethernet Switching 25

bridge does not forward an unnecessary Associate TLV to the upper ToR switch because vsistate of the upper switch is ASSOC although forwarding of Associate TLV is acceptable. On the other hand, De-associate message is sent from the source server to the adjacent bridge at the source port. This is the case of D6 and the bridge does not forward Deassociate message to the upper ToR switch. In this case, forwarding of De-associate TLV is unacceptable because if De-associate message is forwarded, the associate on the destination port is removed while is running. In this way, the port profile is moved together with the and the physical switch is automatically configured. Live Migration To same rack Rack Req No Forwarding Profile move External Bridge VDP 2. Entry Search 3. State Machine VDP TLV Type= Pre-Associate, Pre-Associate with resource reservation, Associate ID ID vsistate, vsistate, ID vsistate, 4. Resp transmit VDP SM External Bridge IV. PROTOTYPE We developed a prototype of the AMPP extension. The prototype consists of Profile Engine for AMPP with an extension for multi-level switches, EVB Packet Analyzer with Protocol Checker and Visualization Tool of EVB messages. A. Profile Engine Figure 6 shows the module structure of Profile Engine. Profile Engine consists of EVB, VDP and daemons. EVB daemon is a master daemon and initiates VDP and daemons. EVB daemon communicates with an adjacent bridge using EVB TLV in LLDP and configures parameters. VDP daemon communicates with daemon using TCP socket and daemon communicates with Layer 2 protocol daemon using Layer 2 socket. daemon guarantees that in packet is received by the adjacent bridge using L-ACK. TABLE III shows the list of processing in VDP daemon. VDP daemon keeps VDP Instance with related information including Instance ID (ID), port profile ID ( Type ID), VDP state machine status (vsistatus) and bridge port number. In this prototype, we implemented Profile Engine as a program running in a remote server connected to the switch. LLDP packet with EVB TLV and packet with are forwarded to the remote server and they are processed by Profile Engine. Profile Engine specifies a designation port for the packet to be transmitted. And the switch sends out the packet on the designated port. profile engine will be implemented in the switch firmware for the management when the IEEE 802.1Qbg standard completes. Figure 5. 1. VDP Req 5. VDP Resp 1. VDP Req 5. VDP Resp downlink uplink ID ID 2. Entry Search vsistate, ID vsistate, vsistate, 3. State Machine VDP SM 4. Resp Transmit Intra-rack Migration and VDP Messages Type DB EVB d VDP daemon F(VTID, Type Version, Instance ID) VDP Instance #, Instance ID, Mnager ID, Type ID #, #, Instance Instance ID, ID, Mnager Mnager ID, ID, Type Type ID ID Type #, Instance ID, Mnager ID, Type ID MSG HDR VLAN add/delete Layer 2 Management Ioctl (VLAN add/delete) D. Topology for AMPP Extension In the AMPP extension, we forward VDP messages to an upper-level switch. In other words, AMPP extension works in a topology like a tree structure where we can decide an uplink to which we forward VDP messages. Our switch blade has a port grouping feature in Intelligent Blade Panel (IBP) firmware mode and the port group has only one active uplink for downlink ports [11]. Figure 6. daemon Layer 2 Protocol Module Configuration of Prfile Engine 26 Proceedings of the 2011 3rd Workshop on Data Center Converged and Virtual Ethernet Switching

TABLE IV. Features of EVB Packet Analyzer Processing VDP Initialization VDP Data Reception VDP Instance Search VDP State Machine VDP Forwarding VDP Data Transmission VDP Disconnect TABLE III. Procesing in VDP Daemon Description Initialize VDP Data Transmit/Receive Interface to/from module Receive VDP data in receeived from module Search VDP Instance using ID in VDP TLV VDP State Machine When VDP forwarding mode enabled, forward to upper bridge if necessary Transmit VDP data to module for transmission Disconnect VDP Data Transmit/Receive Interface to/from module B. EVB Packet Analyzer with Protocol Checker Figure 7 shows configuration of EVB packet analyzer and a visualization tool. TABLE IV shows features of EVB Packet Analyzer. EVB Packet Analyzer is a standalone program and captures LLDP / Packets and analyzes EVB TLV in LLDP packets and in packets. The analysis results are displayed on the window of EVB Packet Analyzer with packet filtering applied if filtering is configured. The analysis results can be output to Visualization Tool described later. It also has protocol checking capability and checks protocol and VDP protocol if it is enabled. Through the prototyping, we noticed that we cannot distinguish VDP request from VDP response with success by just looking at. To distinguish request from response, we looked at a control tag by which Profile Engine designates an output port for an outgoing frame. From EVB Packet Analyzer point of view, it is better to have an indication on the. Network Interface RawSocket Figure 7. 1. Receive Packet 2. Analyze Packet Update Reference Display Update Packet Data (text) Analyzer Window Analysis Results Config (xml file) 3. Send Results TCP Socket EVB Packet Analyzer and Visualization Tool Animation Features Packet Reception Packet Analysis Packet Filter Display of Analysis Results Output of Analysis Results for Visualization Protocol Checker VDP Protocol Checker Description Receive and LLDP packets and save packet data Analyze and EVB TLV and save the analysis results. Filter packet based on the filter setting Display analysis result with filtering applied on the analyzer window Output analysis results to visualization tool for the animation Check transition of state machine based on packet analysis on Packet Check transition of VDP state machine based on packet analysis on C. Visualization Tool of VDP Messages The visualization tool inputs results of analysis from EVB Packet Analyzer and generates animation of VDP messages in migration based on the inputs. The visualization tool of VDP messages is used to explain the protocol behavior intuitively in a demonstration. V. EVALUATION We evaluate our prototype of AMPP extension in a multilevel switch configuration. For the evaluation system, we use a blade server and a ToR switch where a multi-level switch configuration is typical and migration of interchassis always requires configuration changes both in switch blade and ToR switch. A. Evaluation System Figure 8 shows an evaluation system for AMPP extension prototype. The evaluation system mainly consists of three BX920 server blades equipped with 10GbE NICs and two 10GbE switch blades, one 10GbE ToR switch XG2600 and RX300 rack servers [13], [14]. Profile Engine is running on RX300 and processes EVB packets coming into the switch it manages. The switch blade works as an adjacent bridge with AMPP extension enabled. EVB Packet Analyzer with Protocol Checker and Visualization Tool are also running on one of RX300s and EVB Packet Analyzer captures and analyzes mirrored control packets. Visualization Tool generates animation of VDP messages from the analysis results. Linux LLDP agent with IEEE 802.1Qbg patch [15] is running on the BX920 server blade. We used the agent on the server side to make sure interoperability with our prototype on the switch side. The LLDP agent advertises EVB capabilities on EVB TLV in LLDP packet and supports protocol to carry in packet. Proceedings of the 2011 3rd Workshop on Data Center Converged and Virtual Ethernet Switching 27

ToR 10GbE Switch XG2600 Profile Engine RX300 10GbE Switch Blade 10GbE Switch Blade monitor Blade BX920 Blade BX920 GbE SW Blade BX920 (a) System Configuration Figure 9. Visualization of VDP Messages in Migration (b) ToR 10GbE Switch XG2600 (c) BX900 10GbE Switch Blade Figure 8. Evaluation System B. Protocol Operation In the first evaluation, we evaluate EVB capabilities exchanges using EVB TLV in LLDP. The switch blade communicates with a station and works as an adjacent bridge from the station point of view. Also the switch blade communicates with ToR switch and works as a station from ToR switch point of view. The conversion of Mode from to VEB is recognized by ToR switch correctly. In the second evaluations, we evaluate VDP protocol for port profile movement along with migration. Figure 9 shows captured packets by EVB Packet Analyzer and visualizations of VDP messages in inter-chassis migration by Visualization Tool. In inter-chassis migration, s from a station are forwarded to ToR switch and the port profile is moved both in switch blades and ToR switch along with migration. In intra-chassis migration, s from a station is not forwarded to ToR switch and the port profile is moved within the switch blade only. C. Scalability One concern regarding AMPP extension is scalability of Profile Engines in a large-scale multi-level switch configuration. Since we do not have a prototype at scale, we use measurements from our existing system to project the requirements of large systems. Figure 10 shows the amount of VDP messages Profile Engines (PPE) expected to process at a switch stage in multi-level switch configuration with locality as a parameter. PPE-100 means 100% of locality where VDP messages are processed in the first stage only. PPE-75 means 75% of VDP messages are processed in the first stage locally and 25% of messages are forwarded to an upper switch, and so on. For a comparison, it also shows the amount of VDP messages in a flat configuration using Extension in concept (PEC) where all messages are forwarded and processed in the most upper switch and have nothing to do with locality. Profile Engines need to handle less VDP messages in a large-scale multi-level switch configuration. # of VDP message (normalized) 0.9 1 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0 PEC PPE-50 PPE-75 PPE-100 1 2 3 4 Switch Stage Figure 10. Amount of VDP Messages 28 Proceedings of the 2011 3rd Workshop on Data Center Converged and Virtual Ethernet Switching

VI. CONCLUSION This paper describes an extension of AMPP for multilevel switches to reduce complexity and automate the network management in a Cloud Data Center where mobility is important for the dynamic resource allocation. We developed a prototype of the extension and confirmed a network state being moved in a multi-level switch configuration using the standard protocol as it is on the wire. We will continue developing new functions for dynamic infrastructure as part of its efforts to promote research and development of high-performance data center network. [14] 10GbE Ultra-low Latency Switch XG2600, Fujitsu XG2600 Data Sheet, 2010. [15] open-lldp lldpad (LLDP Agent Daemon). Available: http://www.open-lldp.org/git/?p=lldp/open-lldp.git;a=summary ACKNOWLEDGMENT We would like to thank Y. Mochizuki and M. Kawata at Fujitsu Advanced Engineering Limited, and N. Matsuoka and J. Tanaka at Fujitsu Laboratories Ltd. for their help and insightful comments. We also express our gratitude to T. Horie and A. Hattori at Fujitsu Laboratories Ltd. for their support of this project. REFERENCES [1] N. Carr, The Big Switch: Rewiring the World, from Edison to Google. New York, U.S.: W. W. Norton & Company, 2008, pp. 75-79. [2] P. Dawson and T. Bittman, Virtualization Changes Virtually Everything, Gartner Special Report, March 28, 2008. [3] Cisco Systems, Inc., Cisco VN-Link: Virtualization-Aware Networking, White Paper, 2009. Available: http://www.cisco.com/en/us/solutions/collateral/ns340/ns517/ns224/ ns892/ns894/white_paper_c11-525307.pdf [4] BLADE Network technologies Inc., ready. Available: http://www.bladenetwork.net/vmready.html [5] Arista Networks Inc., Tracer. Available: http://www.aristanetworks.com/en/products/eos/vmtracer [6] IEEE P802.1Qbg/D1.5 Draft Standard for Local and Metropolitan Area Networks-Virtual Bridged Local Area Networks -Amendment XX: Virtual Bridging, IEEE Draft Standard 802.1Qbg, March 2011. [7] R. Recio, S. Krishnasamy, and R. Sharma, Ethernet Virtual Bridging Automation Use Cases, DC CAVES workshop ITC 22, September 2010. [8] IEEE P802.1Qbh/D2.0 Draft Standard for Local and Metropolitan Area Networks-Virtual Bridged Local Area Networks -Amendment: Bridge Extension, IEEE Draft Standard 802.1Qbh, April 2011. [9] DMTF, Virtual Networking Management white Paper, DSP2025, February 2011. [10] H. Shah, Management Standards for Virtual Bridging (EVB) and Profiles, Management Developers Conference 2010, November 2010. [11] Y. Nakagawa, T. Shimizu, Y. Koyanagi, O. Shiraki, S. Kobayashi, K. Hyoudou, T. Miyoshi, Y. Ogata, Y. Umezawa, T. Horie and A. Hattori, A Single-Chip, 10-Gigabit Ethernet Switch LSI for Energy- Efficient Blade s, GreenCom 2010, December 2010. [12] Fujitsu Develops World's First Technology Employing 10 Gbps Virtual Switch to Substitute for On- Virtual Switch Functions, Available: http://www.fujitsu.com/global/news/pr/archives/month/2010/2010061 0-02.html, June 10, 2010. [13] PRIMERGY BX900 S1, State of the Art Blade Chassis the Dynamic Cube, Fujitsu PRIMERGY BX900 Data Sheet, 2009. Proceedings of the 2011 3rd Workshop on Data Center Converged and Virtual Ethernet Switching 29