HP 5920 & 5900 Switch Series

Transcription

1 HP 5920 & 5900 Switch Series FCoE Configuration Guide Part number: Software version: Release2207 Document version: 6W

2 Legal and notice information Copyright 2012 Hewlett-Packard Development Company, L.P. No part of this documentation may be reproduced or transmitted in any form or by any means without prior written consent of Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. HEWLETT-PACKARD COMPANY MAKES NO WARRANTY OF ANY KIND WITH REGARD TO THIS MATERIAL, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. Hewlett-Packard shall not be liable for errors contained herein or for incidental or consequential damages in connection with the furnishing, performance, or use of this material. The only warranties for HP products and services are set forth in the express warranty statements accompanying such products and services. Nothing herein should be construed as constituting an additional warranty. HP shall not be liable for technical or editorial errors or omissions contained herein.

3 Contents FCoE overview 1 Storage area network 1 FC SAN 1 FC protocol 2 Basic concepts 2 Communication flow 3 VSAN 4 FC zone 4 FCoE 4 Basic concepts 5 How FCoE works 6 FCoE modes 8 FCF mode 8 NPV mode 9 Protocols and standards 9 FCoE configuration guidelines 11 FCoE features supported by different FCoE modes 11 Configuring an FCoE mode for a switch 11 Configuring VFC interfaces and FIP 12 VFC interfaces and FIP configuration task list 12 Configuring a VFC interface 12 Enabling FCoE for a VLAN and mapping a VSAN to the VLAN 13 Configuration restrictions and guidelines 13 Configuration procedure 13 Configuring the FC-MAP value 13 Configuring the FKA advertisement period value 14 Role of the FKA advertisement period value 14 Configuration restrictions and guidelines 14 Configuring the FCF priority 15 Configuring the system FCF priority 15 Configuring the VFC interface FCF priority 16 Displaying and maintaining VFC interfaces and FIP 16 VFC interfaces and FIP configuration example 16 Network requirements 16 Configuration procedure 17 Setting up a fabric 19 Overview 19 Principal switch selection 19 Domain ID assignment 20 FC address assignment 21 Fabric setup configuration task list 21 Building a fabric statically 21 Building a fabric dynamically 22 Enabling or disabling the fabric configuration function 22 Setting a fabric name 23 Setting the switch priority 23 Configuring the allowed domain ID list 24 i

4 Configuring a domain ID for a switch 24 Configuring the mapping between the N_Port WWN and the FC address 25 Configuring the fabric timers 25 Configuring the fabric timers in system view 25 Configuring the fabric timers in VSAN view 26 Configuring the fabric reconfiguration 26 Configuring the auto fabric reconfiguration function 27 Manually initiating the fabric reconfiguration 27 Configuring a VFC interface to reject incoming RCF requests 27 Displaying and maintaining a fabric 27 Static fabric building configuration example 28 Network requirements 28 Configuration procedure 28 Verifying the configurations 29 Dynamic fabric building configuration example 30 Network requirements 30 Configuration procedure 30 Verifying the configurations 31 Configuring VSAN 33 Overview 33 VSAN fundamentals 33 Creating a VSAN 34 Configuring a trunk VSAN 34 Displaying and maintaining VSAN 35 VSAN configuration example 35 Network requirements 35 Configuration considerations 35 Configuration procedure 36 Verifying the configurations 37 Configuring FC routing and forwarding 38 Overview 38 Routing table and FIB table 38 Direct routes 39 Static routes 39 FSPF routes 40 Configuring static routes for FC 41 Configuration restrictions and guidelines 41 Configuration procedure 41 Configuring FSPF 41 FSPF configuration task list 42 Enabling FSPF 42 Configuring the shortest SPF calculation interval 42 Configuring the minimum LSR receiving interval 43 Configuring the minimum LSR refresh interval 43 Configuring the FSPF cost for an interface 43 Configuring the hello interval for an interface 44 Configuring the dead interval for an interface 44 Configuring the LSR retransmission interval for interfaces 44 Disabling FSPF for an interface 45 Configuring FSPF GR 45 Displaying and maintaining FC routing and forwarding 46 Static FC routing configuration example 46 Network requirements 46 ii

5 Configuration procedure 47 Verifying the configurations 48 FSPF configuration example 49 Network requirements 49 Configuration procedure 50 Verifying the configurations 51 Configuring FC zones 53 Overview 53 Zone database 53 Distributing zones 55 Zone merge 57 Access control 59 FC zone configuration task list 59 Configuring zone aliases 60 Configuring zones 60 Configuring zone sets 61 Configuring the default zone policy 61 Configuring zone distribution and merge types 61 Activating a zone set and distributing it to the entire fabric 62 Triggering a complete distribution 62 Renaming zone aliases, zones, and zone sets 63 Copying zone aliases, zones, and zone sets 63 Deleting the zone database 63 Displaying and maintaining FC zones 64 FC zone configuration example 64 Network requirements 64 Configuration considerations 65 Configuration procedure 65 Verifying the configurations 66 Configuring NPV 68 Overview 68 Downlink interface and downlink 68 Uplink interface and uplink 68 Downlink-to-uplink interface mappings 69 Disruptive load balancing 69 NPV configuration task list 69 Configuring uplink interfaces and downlink interfaces 69 Configuring uplink interfaces 69 Configuring downlink interfaces 70 Configuring downlink-to-uplink interface mappings 70 Initiating a disruptive load-balancing process 71 Displaying and maintaining NPV 71 NPV configuration example 71 Network requirements 71 Configuration procedure 72 Verifying the configurations 73 Configuring FC ping 75 Overview 75 Configuration procedure 75 FC ping configuration example 75 Network requirements 75 Configuration procedure 75 Verifying the configurations 77 iii

6 Configuring FC tracert 78 Overview 78 Configuration procedure 79 FC tracert configuration example 79 Network requirements 79 Configuration procedure 79 Appendixes 82 Appendix A Fabric address assignment 82 Appendix B Well-known fabric addresses 82 Support and other resources 84 Contacting HP 84 Subscription service 84 Related information 84 Documents 84 Websites 84 Conventions 85 Index 87 iv

7 FCoE overview Storage area network FC SAN According to the Storage Networking Industry Association dictionary, "a storage area network (SAN) is any high-performance network whose primary purpose is to enable disk devices to communicate with computer systems and with each other." A SAN enables the universal connectivity of servers and disk devices. Compared to the conventional client/server computer system, a SAN allows the servers to share data and directly access data created by one another without having to copy it, improves storage scalability, and centralizes the management of data backup, access, and security. Most SANs use Fibre Channel (FC) or Ethernet to interconnect devices. An FC SAN uses the FC protocol suite for communication, and an Ethernet SAN uses the TCP/IP protocol suite for communication. This document covers only the FC SAN. As shown in Figure 1, an FC SAN connects the data sending and receiving entities (network servers and disk devices) with fibers or copper wires in the following ways: Directly connects a server and a disk device, as shown in the point-to-point connection. Connects servers and disk devices to an FCF switched fabric, as shown in the switched fabric. In a switched fabric, the servers and disk devices are called "nodes." A fabric uses 24-bit addressing and supports thousands of devices. Figure 1 FC SAN networking NOTE: An FC SAN refers to a network comprising FCF switches and nodes. A fabric refers to a transmission network comprising FCF switches. 1

8 FC protocol The servers, FCF switches, and disk devices in an FC SAN must all support FC. Basic concepts WWN FC address The World Wide Name (WWN) is a 64-bit address that identifies a fabric or an entity (such as an FCF switch, node, or port) in an FC SAN. The upper-layer protocol of FC uses WWNs for communication. Each entity has a factory-assigned globally unique WWN. The FC protocol accesses communication entities in a SAN through FC addresses. An FC address is also known as an "FC_ID." Figure 2 shows the structure of an FC address. The FC address is 24 bits long and is divided into these 8-bit fields: Domain_ID, Area_ID, and Port_ID. A domain represents a switch and all N_Ports connected to the switch. A Domain_ID, which is in the range of 1 to 239, uniquely identifies an FCF switch. One or more N_Ports on the same node can be assigned to an area, which is identified by an Area_ID. The Port_ID field identifies an N_Port. Figure 2 Structure of an FC address Interface modes A Domain_ID can uniquely identify an FCF switch. Different FCF switches in the same fabric have different Domain_IDs. An FC address can uniquely identify an N_Port on a node. Different N_Ports on the same node have different FC addresses. FCF switches use Domain_IDs to route messages between each other. The FC protocol standardizes the FC address usage. For more information, see "Appendixes." In a switched fabric, nodes and FCF switches communicate through interfaces of different modes. 2

9 Figure 3 Port modes 1. The interface on a node is called an "N_Port." 2. An FCF switch provides the following types of ports: F_Port Connects to an N_Port or an NP_Port on another FCF switch. E_Port Connects to an E_Port on another FCF switch. NP_Port Connects to an F_Port on another FCF switch. For more information about NP_Port, see "Configuring NPV." E_Ports connect FCF switches to form a fabric, and F_Ports connect the nodes to FCF switches in the fabric. Communication flow FCF switches provide data transmission services. Through FCF switches, a server sends instructions and data to disk devices and reads data from disk devices. Figure 4 FC SAN communication model The following takes a server accessing a disk device as an example to see how data communication occurs in an FC SAN. 1. The server and the disk device use the fabric login (FLOGI) protocol to register with the FCF switches, which then assign FC addresses to each directly-connected node. 2. The registered server and disk device send name service registration requests to their respective access FCF switches to register name service information, including their WWNs and FC addresses. Finally, each FCF switch in the fabric stores the name service information for all nodes. 3. To access a disk device, the server needs to send a name service query request to its directly-connected FCF switch to obtain the list of disk devices in the fabric and their WWNs and FC addresses. 3

10 4. After the server obtains the FC address of the disk device, the server can send FC frames (with the FC address of the disk device as the destination FC address) to the FCF switch nearby. 5. When the FCF switch receives the FC frame from the server, it queries its FIB table for a data forwarding path according to the destination FC address, and forwards the FC frame to the next-hop FCF switch. The next-hop FCF switch forwards the FC frame in the same way, until the FCF switch at the last hop forwards the FC frame to the destination disk device. NOTE: A FIB table is generated by the FCF switch through calculation based on the FC routing protocol or configured static routes. VSAN FC zone FCoE In actual applications, the data is insecure because the data of all users is transmitted in the same FC SAN. You can divide one physical FC SAN into multiple Virtual Storage Area Networks (VSANs). In this manner, VSANs are separated from one another and provide independent services, enhancing adaptability and security of the network and offering more effective services for users. For more information about VSAN, see "Configuring VSAN." With VSAN, one physical SAN is divided into multiple logical SANs. A VSAN, however, cannot perform access control over the servers and disk devices (or the N_Ports) connected to a fabric. N_Ports in the same VSAN can access one another only if these N_Ports register name services. This creates data security risks. Zoning can solve the preceding problem by dividing a VSAN into zones and adding N_Ports to different zones for different purposes. In this manner, N_Ports in different zones are separated to implement access control. For more information about FC zones, see "Configuring FC zones." A data center using the FC SAN technology usually comprises separate local area networks (LANs) and SANs. LANs carry traditional Ethernet/IP services, and SANs carry network storage services. To provide services for LANs and use SANs for storage simultaneously, the servers must use independent Ethernet adapters and FC adapters. In addition, the IP switches and the FCF switches are also independent and have independent network connections. Such a network needs many switches, network adapters, and cables, and it brings high investments and maintenance costs and low scalability. FCoE was introduced to solve this problem. FCoE is a protocol that carries FC over Ethernet. In an FCoE solution, the server uses an FCoE-capable Ethernet adapter, and the FCoE switch (FCoE forwarder) integrates the functions of both the traditional IP switch and FCF switch. FCoE reduces the number of network adapters, switches, and cables, and the network operation and maintenance workload. In all, FCoE reduces the total cost. 4

11 Figure 5 FCoE for I/O consolidation As shown in Figure 5, in the traditional network, the server is connected to the LAN through an Ethernet interface and to the SAN through an FC interface. In the FCoE network, the server is connected to the FCoE-capable FCF switch, and then the FCF switch is connected to the LAN through an Ethernet interface and to the SAN through an FC interface. The links between the server and the FCF switch and between FCF switches can transmit both Ethernet frames and FC frames. For more information about FCoE, see "Configuring FCoE." Basic concepts As shown in Figure 6, the links between the FCF switch and the ENode (nodes that can transport FC over Ethernet, such as servers and disk devices) and between FCF switches can be used for receiving and sending both Ethernet frames and FC frames. Figure 6 FCoE network diagram VFC interface and VN interface A virtual fiber channel (VFC) interface is a logical interface manually created on the FCF switch to simulate the function of a physical FC interface. To use a VFC interface, bind it to a physical Ethernet interface. 5

12 FIP protocol FCoE frames You can connect either an ENode or an FCF switch to a VFC interface. VFC interfaces support E mode, F mode (default), and NP mode. The virtual node (VN) interface is a logical interface on an ENode to simulate the function of a physical FC interface. FCoE initialization protocol (FIP) is an FCoE control protocol that establishes and maintains virtual links. FIP establishes a virtual link between the VFC interface of an FCF switch and the VN interface of an ENode or between VFC interfaces of two FCF switches to provide a physical infrastructure for transmitting FC frames over Ethernet. To transmit an FC frame over an Ethernet link, you must encapsulate the FC frame in an FCoE frame by adding an Ethernet frame header to the FC frame. An FCoE frame uses Ethernet II encapsulation, which has the following fields in the Ethernet header: EtherType 0x8906. Destination MAC address/source MAC address For a switch, it is the FCoE MAC address of the switch (which can be displayed by using the display fcoe command). For a node, it is the fabric provided MAC address (FPMA) of the node. As shown in Figure 7, an FPMA is composed of the FC-MAP as the 24 most significant bits and the FC ID of the VN interface as the 24 least significant bits. The FC-MAP takes the value of the switch FC-MAP, 0x0EFC00 by default and confiugrable by using the fcoe fcmap command. Figure 7 FPMA composition How FCoE works Figure 8 Block diagrams of the ENode and the FCF switch ENode FCF VN interface Virtual link VFC interface FC layer Ethernet layer FC layer Ethernet layer Ethernet interface Ethernet interface 6

13 NOTE: This section describes how FCoE works only on the FCF switch, rather than on the ENode. Procedure for receiving and sending FC frames over Ethernet How FIP works An FC frame is transmitted over Ethernet using the following workflow: FIP establishes a virtual link between the VFC interface of the FCF switch and the VN interface of the ENode or between VFC interfaces of two FCF switches. After the virtual link is established, the FCF switch encapsulates the FC frame in an FCoE frame and sends it out. After receiving the FCoE frame, the FCF switch removes its Ethernet header to send the original FC frame to the upper layer for processing. FIP sets up and maintains virtual links between a VFC interface and a VN interface or between VFC interfaces. Two categories of packets are used in FIP: Discovery Solicitation and Discovery Advertisement. There are two types of Discovery Advertisement: Solicited Discovery Advertisement A reply for a Discovery Solicitation. Unsolicited Discovery Advertisement Periodically sent. The following example shows how a virtual link is set up between an FCF switch and an ENode. Figure 9 FIP operation FCF ENode (1) Send Discovery Solicitation Learn FCoE MAC address (2) Send solicited Discovery Advertisement (3) Send solicited Discovery Advertisements periodically (4) Send FLOGI request Check FCoE MAC address (5) Send FLOGI LS_ACC (6) Send solicited Discovery Advertisements periodically As shown in Figure 9, the following workflow is used to set up a virtual link: 1. The ENode sends a Discovery Solicitation containing its FCoE MAC address. 2. After receiving the Discovery Solicitation, the FCF switch acts differently depending on whether the receiving VFC interface is bound to the FCoE MAC address: If it is not bound, the switch learns the FCoE MAC address and replies with a solicited Discovery Advertisement, whose fcf priority field carries the FCF priority of the VFC interface. 7

14 If it is bound, the switch checks whether the FCoE MAC address matches the bound FCoE MAC address. If they match, it replies with a solicited Discovery Advertisement, whose fcf priority field carries the FCF priority of the VFC interface. If they do not match, it discards the Discovery Solicitation. 3. The FCF switch periodically sends unsolicited Discovery Advertisements, whose fcf priority field carries the FCF priority of the system. The sending interval is specified by using the fcoe fka-adv-period command and defaults to 8 seconds. 4. After receiving the Discovery Advertisements, the ENode determines the FCF switch with the highest priority according to the fcf priority field and sends a FLOGI request frame to that switch for login. 5. After receiving the FLOGI request frame, the FCF checks whether the source MAC address matches its learned or bound FCoE MAC address. If they match, it sends a FLOGI LS_ACC, which indicates the setup of the virtual link. Otherwise, it discards the FLOGI frame. 6. The FCF switch also periodically sends unsolicited Discovery Advertisements to maintain established virtual links. If the ENode fails to receive an unsolicited Discovery Advertisement within a period 2.5 times the interval specified by the fcoe fka-adv-period command, it deletes the virtual link. FCoE modes FCF mode The switch supports the following FCoE modes: FCF mode A switch operating in this mode is called an FCF switch. Its VFC interfaces support E mode (E_Port) and F mode (F_Port). NPV mode A switch operating in this mode is called an N_Port Virtualizer (NPV) switch. Its VFC interfaces support F mode (F_Port) and NP mode (NP_Port). An FCoE-capable switch can operate in the following modes: FCF mode When the switch operates in this mode, it can connect to the E_Port on another FCF switch through its E_Port, or connect to the N_Port on a node or the NP_Port on an NPV switch through its F_Port. NPV mode When the switch operates in this mode, it can connect to the N_Port on a node through its F_Port or to the F_Port on an FCF switch through its NP_Port. Non-FCoE mode When the switch operates in this mode, it is a standard switch and does not provide any FCoE capabilities. An FCF switch encapsulates FC frames in Ethernet frames and uses FCoE virtual links to simulate physical FC links. Therefore, it provides standard FC switching capabilities and features on a lossless Ethernet network. 8

15 Figure 10 FCF network diagram In an FCoE environment as shown in Figure 10, different from a pure FC network, the ENode and FCF switch communicate over Ethernet interfaces on a lossless Ethernet network. The FCoE virtual link between the ENode and FCF switch connects a VN interface to a VFC interface, and the FCoE virtual link between FCF switches connects two VFC interfaces. Each FCF switch is assigned a domain ID. Each FC SAN supports a maximum number of 239 domain IDs, so an FC SAN cannot have more than 239 FCF switches. NPV mode An FC SAN needs a large number of edge switches that connect directly to nodes. N_Port Virtualization (NPV) switches are developed to expand the number of switches in an FC SAN. Figure 11 NPV network diagram As shown in Figure 11, the NPV switch resides between nodes and the core switch on the edge of the fabric. The core switch is a switch operating in FCF mode. The NPV switch is connected to the nodes through its F_Ports and to the core switch through its NP_Port. In this manner, the NPV switch forwards traffic from its connected nodes to the core switch. The NPV switch appears as an FCF switch to nodes and as a node to the core switch. For more information about NPV, see "Configuring NPV." Protocols and standards FC-FS-3, Fibre Channel - Framing and Signaling - 3 9

16 FC-SW-5, Fibre Channel - Switch Fabric - 5 FC-LS-2, Fibre Channel - Link Services - 2 FC-GS-6, Fibre Channel - Generic Services - 6 FC-BB-5, Fibre Channel - Back Bone 5 10

17 FCoE configuration guidelines The switch supports FCoE only when operating in advanced mode. For more information about system operating modes, see Fundamentals Configuration Guide. FCoE features supported by different FCoE modes The switch supports two FCoE modes: FCF mode and NPV mode. Each mode has different features as shown in Table 1. You can choose to configure different features based on the FCoE mode of a switch. Table 1 FCoE functions supported by different FCoE modes FCoE feature FCoE mode FCF mode NPV mode Configuring VFC interfaces and FIP Setting up a fabric Supported Supported Supported Only the following function is supported: "Configuring the fabric timers." Configuring VSAN Supported Supported Configuring FC routing and forwarding Supported Only the following functions are supported: Displaying FC routing table information Displaying FC FIB table information Display FC Exchange table information Configuring FC zones Supported Not supported Configuring NPV Not supported Supported Configuring FC ping Supported Not supported Configuring FC tracert Supported Not supported Configuring an FCoE mode for a switch An FCoE-capable switch can operate in FCF mode, NPV mode, or non-fcoe mode. The switch can only convert from non-fcoe mode to one FCoE mode, or vice versa, and it cannot convert directly among the two FCoE modes. To convert among the two FCoE modes, first convert the switch to non-fcoe mode. After converting the switch to non-fcoe mode, FCoE-related configurations in the original FCoE mode are cleared. To configure an FCoE mode for a switch: Step Command Remarks 1. Enter system view. system-view N/A 2. Configure an FCoE mode for the switch. fcoe-mode { fcf npv } By default, a switch operates in non-fcoe mode. 11

18 Configuring VFC interfaces and FIP VFC interfaces and FIP configuration task list Tasks at a glance (Required.) Configuring a VFC interface (Required.) Enabling FCoE for a VLAN and mapping a VSAN to the VLAN (Optional.) Configuring the FC-MAP value (Optional.) Configuring the FKA advertisement period value (Optional.) Configuring the FCF priority Configuring a VFC interface Step Command Remarks 1. Enter system view. system-view N/A 2. Create a VFC interface and enter its view. 3. Configure the VFC interface mode. 4. Bind the VFC interface to the specified Ethernet interface. 5. Assign the VFC interface to the specified VSAN as a trunk interface. 6. (Optional.) Configure a description for the VFC interface. interface vfc interface-number fc mode { e f np } bind interface interface-type interface-number [ mac mac-address ] port trunk vsan vsan-id description text N/A By default, a VFC interface operates in F mode. When an FCF switch operates in FCF mode, VFC interfaces support E and F modes. When an FCF switch operates in NPV mode, FC interfaces support F and NP modes. By default, no Ethernet interface is bound to a VFC interface. The VFC interface sends and receives packets through the Ethernet interface bound to it. By default, a VFC interface is not assigned to any VSAN as a trunk interface. You can assign a VFC interface to a nonexistent VSAN as a trunk interface and then create the VSAN. By default, the description of an interface is Interface name Interface, for example, Vfc1 Interface. 12

19 Step Command Remarks 7. (Optional.) Restore the default settings for the VFC interface. default 8. Bring up the VFC interface. undo shutdown By default, a VFC interface is up. N/A Enabling FCoE for a VLAN and mapping a VSAN to the VLAN When you use a VFC interface to transmit packets, the Ethernet interface bound to the VFC interface may allow multiple VLANs. You should enable FCoE for a VLAN and map a VSAN to the VLAN, so that the packets from the VSAN are tagged with the VLAN tag and transmitted within the VLAN. Configuration restrictions and guidelines Follow these restrictions and guidelines when you configure this feature: FCoE cannot be enabled for VLAN 1. VSANs are mapped to VLANs on a one-to-one basis. You must enable FCoE for the same VLAN and map this VLAN to the same VSAN on the two ends. Make sure the Ethernet interface bound to the VFC interface allows the FCoE-capable VLAN. After you enable FCoE for a VLAN, the following changes apply to the VLAN: An FCoE-capable VLAN allows only FCoE traffic. All member ports in an FCoE-capable VLAN are isolated. For this reason, a Layer 2 protocol enabled in the FCoE-capable VLAN runs based on the port isolation topology. Configuration procedure To enable FCoE for the specified VLAN and map this VLAN to the specified VSAN: Step Command Remarks 1. Enter system view. system-view N/A 2. Enter VLAN view. vlan vlan-id N/A 3. Enable FCoE for the specified VLAN and map this VLAN to the specified VSAN. fcoe enable [ vsan vsan-id ] By default, FCoE for a VLAN is disabled. Make sure that the VSAN to be mapped has been created. Configuring the FC-MAP value The FC-MAP value identifies an FCoE network. Switches in the same FCoE network must have the same FC-MAP value. 13

20 IMPORTANT: After FC-MAP values are configured, VFC interfaces perform a renegotiation. The same FC-MAP value is required for two VFC interfaces to negotiate successfully. To configure an FC-MAP value: Step Command Remarks 1. Enter system view. system-view N/A 2. Configure an FC-MAP value. fcoe fcmap fc-map The default setting is 0x0EFC00. Configuring the FKA advertisement period value Role of the FKA advertisement period value The FKA advertisement period value functions as follows: After setting up a virtual link with a peer switch, a switch sends unsolicited Discovery Advertisements every FKA advertisement period on its VFC interfaces in E mode to maintain the established virtual link. The FKA advertisement period value is carried in unsolicited Discovery Advertisements. After receiving an unsolicited Discovery Advertisement, the peer switch maintains the status of the virtual link and records the FKA advertisement period value. If the peer switch fails to receive an unsolicited Discovery Advertisement within 2.5 FKA advertisement periods, it deletes the virtual link. After setting up a virtual link with a peer ENode, a switch sends unsolicited Discovery Advertisements every FKA advertisement period on its VFC interfaces in F mode to maintain the established virtual link. The FKA advertisement period value is carried in unsolicited Discovery Advertisements. After receiving an unsolicited Discovery Advertisement, the peer ENode maintains the status of the virtual link and records the FKA advertisement period value. If the peer ENode fails to receive an unsolicited Discovery Advertisement within 2.5 FKA advertisement periods, it deletes the virtual link. In addition, the ENode sends keepalive frames to the switch every FKA advertisement period value (this value is obtained from unsolicited Discovery Advertisements received from the switch). After receiving a keepalive frame, the switch maintains the status of the virtual link. If the switch fails to receive a keepalive frame within 2.5 FKA advertisement periods, it deletes the virtual link. The same as ENodes, VFC interfaces in NP mode use the FKA advertisement period value learned from the peer switch instead of that configured on the local switch. Configuration restrictions and guidelines When you configure the FKA advertisement period value, follow these restrictions and guidelines: As specified in FC-BB-5, the upper limit of the FKA advertisement period value is 90 seconds. The switch allows a maximum FKA advertisement period value of 600 seconds. When the switch interoperates with servers, storage devices, or other vendors' switches, you cannot configure the FKA advertisement period to be greater than 90 seconds. In normal cases, use the default FKA advertisement period value (8 seconds). In the case of an active/standby switchover or ISSU reboot on an IRF member switch with subordinate switches, HP 14

21 recommends that you increase the FKA advertisement period to a value between 60 and 90 seconds to avoid service interruption if there are many configurations on the switch. For more information about ISSU, see Fundamentals Configuration Guide. Use values greater than 90 in the case of an ISSU reboot on a switch operating in standalone mode or in IRF mode but without subordinate switches. In this case, the switch cannot send unsolicited Discovery Advertisements or keepalive frames for a period of time. To prevent the peer from deleting the virtual link for failing to receive an unsolicited Discovery Advertisement and avoid service interruption, HP recommends that you set the FKA advertisement period to a value between 300 and 600 seconds. To ensure service continuity during an active/standby switchover or ISSU reboot on an NPV switch, you must also adjust the FKA advertisement period value on the upstream FCF switch. This is because the FKA advertisement period value configured on the NPV switch affects only its VFC interfaces in F mode and connected ENodes, and its VFC interfaces in NP mode use the FKA advertisement period value learned from the upstream FCF switch. FCoE traffic will be interrupted during an ISSU reboot on an access FCF switch or NPV switch operating in standalone mode or in IRF mode but without subordinate switches. This is because an access FCF switch or NPV switch will connect to servers, storage devices, or other vendor's switches and its FKA advertisement period cannot be longer than 90 seconds. An ISSU reboot on a switch operating in standalone mode or in IRF mode but without subordinate switches takes a longer time than 225 (2.5*90) seconds. Therefore, FCoE traffic will be interrupted during the ISSU reboot because the virtual link is deleted after the peer fails to receive an unsolicited Discovery Advertisement within 225 seconds. To configure an FKA advertisement period value: Step Command Remarks 1. Enter system view. system-view N/A 2. Configure an FKA advertisement period value. fcoe fka-adv-period fka-adv-period The default setting is 8 seconds. Configuring the FCF priority The FCF priority includes the VFC interface FCF priority and the system FCF priority, which are used in the following scenarios: The VFC interface FCF priority is used in the fcf priority field in an unsolicited Discovery Advertisement. The system FCF priority is used in the fcf priority field in a solicited Discovery Advertisement. An ENode selects the FCF switch with the highest priority from the FCF switches sending Discovery Advertisements and sends a FLOGI request to it for login. The FCF priority is effective only on a VFC interface connected to an ENode (VFC interface in F mode). Configuring the system FCF priority Step Command Remarks 1. Enter system view. system-view N/A 15

22 Step Command Remarks 2. Configure the system FCF priority. fcoe global fcf-priority priority The default setting is 128. The configuration takes effect on all VFC interfaces in F mode. Configuring the VFC interface FCF priority Step Command Remarks 1. Enter system view. system-view N/A 2. Enter VFC interface view. 3. Configure the FCF priority for the VFC interface. interface vfc interface-number fcoe fcf-priority priority N/A The default setting is 128. The configuration takes effect on a VFC interface only when it operates in F mode. Displaying and maintaining VFC interfaces and FIP Execute display commands in any view. Task Display VFC interface information. Display FCoE global configuration. Command display interface [ vfc [ interface-number ] ] [ brief [ description ] ] display fcoe Clear the statistics for VFC interfaces. reset counters interface [ vfc [ number ] ] VFC interfaces and FIP configuration example Network requirements As shown in Figure 12, use the FCoE solution in a data center combining a LAN and a SAN to reduce the number of devices, network adapters, and cables. 16

23 Figure 12 Network diagram If the FCF switch is connected to the converged network adapter (CNA) of the server or storage device, PFC and DCBX should be configured additionally on the physical interfaces. For information about configuring PFC and DCBX, see Layer 2 LAN Switching Configuration Guide. Configuration procedure This section describes the configurations for VFC interfaces and FIP on the FCF switch. 1. Configure Switch A: # Create VSAN 10 and configure Switch A to operate in FCF mode. <SwitchA> system-view [SwitchA] fcoe-mode fcf [SwitchA] vsan 10 [SwitchA-vsan10] quit # Create interface VFC 1, bind it to interface Ten-GigabitEthernet 1/0/1, and assign it to VSAN 10 as a trunk port. [SwitchA] interface vfc 1 [SwitchA-Vfc1] bind interface ten-gigabitethernet1/0/1 [SwitchA-Vfc1] port trunk vsan 10 [SwitchA-Vfc1] quit # Create interface VFC 2, configure it to operate in E mode, bind it to interface Ten-GigabitEthernet 1/0/2, and assign it to VSAN 10 as a trunk port. [SwitchA] interface vfc 2 [SwitchA-Vfc2] fc mode e [SwitchA-Vfc2] bind interface ten-gigabitethernet1/0/2 [SwitchA-Vfc2] port trunk vsan 10 [SwitchA-Vfc2] quit # Configure interface Ten-GigabitEthernet 1/0/1 to allow VLAN 20. [SwitchA] interface ten-gigabitethernet1/0/1 [SwitchA-Ten-GigabitEthernet1/0/1] port link-type trunk [SwitchA-Ten-GigabitEthernet1/0/1] port trunk permit vlan 20 [SwitchA-Ten-GigabitEthernet1/0/1] quit # Configure interface Ten-GigabitEthernet 1/0/2 to allow VLAN

24 [SwitchA] interface ten-gigabitethernet1/0/2 [SwitchA-Ten-GigabitEthernet1/0/2] port link-type trunk [SwitchA-Ten-GigabitEthernet1/0/2] port trunk permit vlan 20 [SwitchA-Ten-GigabitEthernet1/0/2] quit # Enable FCoE for VLAN 20 and map it to VSAN 10. [SwitchA] vlan 20 [SwitchA-vlan20] fcoe enable vsan Configure Switch B: # Create VSAN 10 and configure Switch B to operate in FCF mode. <SwitchB> system-view [SwitchB] fcoe-mode fcf [SwitchB] vsan 10 [SwitchB-vsan10] quit # Create interface VFC 1, configure it to operate in E mode, bind it to interface Ten-GigabitEthernet 1/0/2, and assign it to VSAN 10 as a trunk port. [SwitchB] interface vfc 1 [SwitchB-Vfc1] fc mode e [SwitchB-Vfc1] bind interface ten-gigabitethernet1/0/2 [SwitchB-Vfc1] port trunk vsan 10 [SwitchB-Vfc1] quit # Configure interface Ten-GigabitEthernet 1/0/2 to allow VLAN 20. [SwitchB] interface ten-gigabitethernet1/0/2 [SwitchB-Ten-GigabitEthernet1/0/2] port link-type trunk [SwitchB-Ten-GigabitEthernet1/0/2] port trunk permit vlan 20 [SwitchB-Ten-GigabitEthernet1/0/2] quit # Enable FCoE for VLAN 20 and map it to VSAN 10. [SwitchB] vlan 20 [SwitchB-vlan20] fcoe enable vsan 10 18

25 Setting up a fabric Overview A fabric transmits data for servers and disk devices. When setting up a fabric, you must assign a domain ID to each FCF switch in the fabric and assign an FC address to each node connected to the fabric. You can build a fabric through one of the following modes: Static mode You must manually assign domain IDs to all switches in the network, and then each switch assigns FC addresses to the N_Ports connected to it. The static mode avoids network flappings, but it is applicable only to simple, small-sized networks. Dynamic mode A principal switch is automatically elected to assign domain IDs to all switches in the network, and then each switch assigns FC addresses to the N_Ports connected to it. The dynamic mode enables centralized network management and is applicable to large-sized networks. Figure 13 Fabric setup workflows The following section details each process in the fabric setup workflows. Principal switch selection During the dynamic fabric building process, it is the principal switch that assigns domain IDs to all switches in the network. The switch with the highest priority is selected as the principal switch. When multiple switches have the same priority, the switch with the smallest WWN wins. The principal switch selection process is as follows: 1. When the principal switch selection starts, each switch considers itself as the principal switch, records itself in the principal switch information, and starts the Principal Switch Selection Timer (PSST), which is 10 seconds. 2. Before the PSST times out, the switches exchange packets carrying the principal switch information to select a principal switch. On receiving such a packet, a switch compares the priority and WWN of the principal switch carried in the packet against those locally recorded. If the priority carried in the packet is higher, or the priority in the packet is the same and the WWN is smaller, the switch 19

26 replaces the locally-record principal switch information with the principal switch information recorded in the packet, and notifies the other switches. Finally, all switches in the network make an agreement on which switch is the principal switch. 3. When the PSST times out, if the locally-recorded principal switch information is the local switch, the switch becomes the principal switch. After the principal switch is selected, the WWN of the principal switch becomes the fabric name. NOTE: During the principal switch selection process, if a switch receives a packet that updates the principal switch information, the switch must record the E_Port receiving the packet. The link relevant to this E_Port is called the "upstream principal link." Domain ID assignment A domain represents a switch and all N_Ports connected to the switch. Each domain must have a domain ID. An FCF switch can automatically assign domain IDs. Alternatively, you can manually configure static domain IDs. If you manually configure static domain IDs, you must assign a unique domain ID to each switch in the fabric. If domain IDs are dynamically assigned, the fabric configuration process is performed to select a principal switch and assign domain IDs. After the principal switch is selected, the principal switch assigns domain IDs to all switches in the fabric. After the fabric configuration process, each switch has a unique domain ID. The dynamic domain ID assignment process is as follows: 1. The principal switch assigns a domain ID to itself. If the principal switch has been configured with a desired domain ID, the principal switch assigns the domain ID to itself. Otherwise, the principal switch assigns the smallest available domain ID to itself. After the principal switch assigns a domain ID to itself, it notifies its downstream switches to request domain IDs from it. 2. After a downstream switch receives the notification, it starts to request a domain ID from the principal switch. If the downstream switch is configured with a desired domain ID, it requests the desired domain ID from the principal switch. 3. After the principal switch receives the domain ID request from the downstream switch, it assigns a domain ID to it and notifies the downstream switch. The principal switch assigns domain IDs following these rules: If the downstream switch requests a desired domain ID that is available, the principal switch assigns the domain ID to the downstream switch. If the downstream switch does not request a desired domain ID or the desired domain ID is not available, the principal switch assigns the smallest available domain ID to the downstream switch. If all available domain IDs have been assigned, the principal switch notifies the downstream switch that no domain ID can be assigned to it. 4. After the downstream switch receives the domain ID assignment notification from the principal switch, it works following these rules: If the downstream switch has been configured with a static domain ID and the static domain ID is different from the one assigned by the principal switch, or if the principal switch notifies the 20

27 downstream switch that no domain ID can be assigned, the downstream switch isolates its upstream principal link and brings down the relevant interface. For more information about domain ID types, see "Configuring a domain ID for a switch." Otherwise, the downstream accepts the domain ID assigned by the principal switch and notifies the nearby downstream switch to request a domain ID from the principal switch. 5. Repeat steps 2 through 4 until all downstream switches have been assigned domain IDs. NOTE: During the dynamic domain ID assignment process, if a switch receives the domain ID request on an E_Port, the switch must record the E_Port. The link relevant to this E_Port is called the "downstream principal link." FC address assignment After a switch obtains a domain ID, it can assign FC addresses to N_Ports directly connected. The Domain_ID field in the FC address is the domain ID of the switch, and it does not need assignment. The switch assigns the Area_IDs and Port_IDs following these guidelines: If you bind the WWN of an N_Port to an FC address, the switch assigns the bound FC address to the N_Port. If the N_Port itself has a desired FC address, the switch assigns the desired FC address, if available. If the N_Port itself does not have a desired FC address or the desired FC address is unavailable, the switch assigns the smallest available Area_ID and Port_ID to the N_Port. Fabric setup configuration task list When you set up a fabric, HP recommends that you use the same building mode (dynamic or static) for all switches in the fabric and then perform the following configurations depending on your building mode. Building a fabric statically Tasks at a glance (Required.) Configuring VFC interfaces and FIP (Required.) Enabling or disabling the fabric configuration function (Required.) Setting a fabric name (Optional.) Configuring the allowed domain (Required.) Configuring a domain ID for a switch (Optional.) Configuring the mapping between the N_Port WWN and the FC address Remarks N/A To statically build a fabric, you must disable the fabric configuration function. When statically building a fabric, you must manually configure the fabric name, and make sure all switches in the fabric are configured with the same fabric name. N/A When statically building a fabric, you must manually configure a domain ID for each switch. N/A 21

28 Tasks at a glance (Optional.) Configuring the fabric timers Remarks N/A Building a fabric dynamically Tasks at a glance (Required.) Configuring VFC interfaces and FIP (Required.) Enabling or disabling the fabric configuration function (Optional.) Setting the switch priority (Optional.) Configuring the allowed domain (Optional.) Configuring a domain ID for a switch (Optional.) Configuring the mapping between the N_Port WWN and the FC address (Optional.) Configuring the fabric timers (Optional.) Configuring the fabric reconfiguration (Optional.) Configuring a VFC interface to reject incoming RCF requests Remarks N/A To dynamically build a fabric, you must enable the fabric configuration function. Principal switch selection relies on the switch priority. N/A When dynamically building a fabric, you can configure desired domain IDs for switches. N/A N/A N/A N/A Enabling or disabling the fabric configuration function After being enabled with the fabric configuration function, FCF switches exchange messages to select the principal switch. Then the principal switch dynamically assigns domain IDs to all switches in the fabric. Therefore, to dynamically build a fabric, you must enable the fabric configuration function on switches. To statically set up a fabric, you must disable the fabric configuration function on switches and manually configure domain IDs for the switches. To enable or disable the fabric configuration function: Step Command Remarks 1. Enter system view. system-view N/A 2. Enter VSAN view. vsan vsan-id N/A 3. Enable the fabric configuration function. 4. Disable the fabric configuration function. domain configure enable undo domain configure enable Enable or disable the function for all switches in the VSAN as required. By default, the fabric configuration function is enabled. 22

29 Setting a fabric name The fabric name configured takes effect only on a statically-built fabric. You must configure the same fabric name for all switches in a VSAN. To set a fabric name: Step Command Remarks 1. Enter system view. system-view N/A 2. Enter VSAN view. vsan vsan-id N/A 3. Configure a fabric name. fabric-name name By default, the fabric name is null. If the user does not configure a fabric name, the switch WWN is used as the fabric name after FCoE is enabled. Setting the switch priority The priority value for FCF switches is in the range of 1 to 254. The smaller the value, the higher the priority. The FCF switch with the highest priority will be elected as the principal switch. The priority is configured on a per-vsan basis, and one FCF switch may have different priorities in different VSANs. In a stable fabric, the configured priority does not take effect immediately. Therefore, the running priority of a switch might be different from the configured priority. To validate the configured priority, use the domain restart disruptive command to perform a disruptive reconfiguration. After a disruptive reconfiguration, the running priority could still be different from the configured priority. See the following possibilities on the principal and a non-principal switch, depending on the configured priority value, as shown in Table 2. Table 2 Configured priority and running priority mappings Configured priority 2 > 2 Running priority Principal switch Same as the configured priority. Non-principal switch 3. Principal switch 2. Non-principal switch Same as the configured priority. To set the switch priority: Step Command Remarks 1. Enter system view. system-view N/A 2. Enter VSAN view. vsan vsan-id N/A 3. Configure a priority value for the switch. priority value By default, the priority value of a switch is

30 Configuring the allowed domain ID list Configuring the allowed domain ID list has an effect on switches as follows: Principal switch Can only assign domains IDs within the allowed domain ID list. If the allowed domain ID list configured does not include any of the already assigned domain IDs or manually configured domain IDs, the configuration will fail. Non-principal switch The manually configured domain ID must be within the allowed domain ID list. Otherwise, the configuration will fail. The principal switch must assign the switch a domain ID within the allowed domain ID list. Otherwise, the switch refuses the assigned domain ID and isolates its interface connected to the principal switch. If the runtime domain ID for a switch is beyond the new allowed ID list, the configuration will also fail. HP recommends that you specify the same allowed domain ID list for the member switches of a VSAN. To configure the allowed domain IDs for a switch: Step Command Remarks 1. Enter system view. system-view N/A 2. Enter VSAN view. vsan vsan-id N/A 3. Configure the allowed domain IDs for the switch. allowed-domain-id domain-id-list By default, the allowed domain IDs are 1 to 239. Configuring a domain ID for a switch In different scenarios, the configured domain ID has different meanings. In a statically built fabric, the configured domain ID is the actual domain ID of the switch. In a dynamically built fabric, the configured domain ID is desired by the switch but may not be the actual domain ID. To statically build a fabric, you must manually configure a domain ID for each switch. To dynamically build a fabric, you can configure a desired domain ID for a switch, but the domain ID assigned to the switch may not be the desired one. The configured domain ID can be of a static or preferred type. In a statically built fabric, the two types make no difference. In a dynamically built fabric, when a non-principal switch fails to obtain the desired domain ID from the principal switch, the non-principal switch can use another domain ID assigned by the principal switch if the preferred type is configured. The non-principal switch will isolate the upstream link and refuse other domain IDs assigned by the principal switch if the static type is configured. HP recommends that you configure domain IDs of the same type for all switches in a VSAN. To configure a domain ID for a switch: Step Command Remarks 1. Enter system view. system-view N/A 2. Enter VSAN view. vsan vsan-id N/A 24