QLogic SRP Module on Linux for OpenFabrics and InfiniPath Version Table of Contents

Transcription

1 QLogic SRP Module on Linux for OpenFabrics and InfiniPath Version This software license applies only to QLogic customers. QLogic Corporation. All rights reserved. 1. Introduction 2. OS Support 3. Important Files 3. Package Installation 4. OFED Package Requirements 5. Sample Configuration File 6. Contacting Support Table of Contents 1. Introduction The the QLogic SRP module on Linux for OpenFabrics and InfiniPath, version provides the following capabilities: Specify individual adapters to which you wish to connect. Specify multiple connections(session) per adapter. Configure adapter to use multiple connections as either a failover or roundrobin (load balancing) mechanism. Hot swap Fibre Channel bridge modules. 2. OS Support Supported OS Distributions: Red Hat Enterprise Linux 4 Update 4/5/6 Red Hat Enterprise Linux 5/5.1 SuSE Linux Enterprise 10/SuSE Linux Enterprise 10 SP1 Supported Open Fabrics Enterprise Distributions, version 1.3 Supported InfiniPath Releases, version Important Files Configuration file: /etc/sysconfig/qlgc_srp.cfg Tools: qlgc_srptools is a package of tools for discovering and connecting to SRP targets on InfiniBand fabrics. qlgc_srptools contains following executables and scripts. ib_qlgc_srp_query ib_qlgc_srp_cfg ib_qlgc_srp_update (background daemon for FibreChannel bridge module hotswap) ib_qlgc_srp_build_cfg ib_qlgc_srp_stats ib_qlgc_srp_control To run ib_qlgc_srp_query, you must load the ib_umad module, as well as an appropriate low-level driver for the installed IB hardware ( for example, ib_mthca for a Mellanox HCA, ib_ipath for a QLogic HCA, or mlx4 for ConnectX.) Using ib_qlgc_srp_query: ib_qlgc_srp_query [-v #] QLogic SRP Module on Linux for OpenFabrics and InfiniPath Readme File 1 of 6

2 With no command line parameters, ib_qlgc_srp_query displays information about SRP targets in humanreadable form: # ib_qlgc_srp_query QLogic Corporation. Virtual HBA (SRP) SCSI Query Application, version IB Host Channel Adapter present in system. HCA Card 1 : 0x0002c Port 1 GUID : 0x0002c Port 2 GUID : 0x0002c a SRP Targets : SRP IOC Profile : FVIC in Chassis 0x00066a00db00001e, Slot 4, Ioc 1 SRP IOC GUID : 0x00066a01dd SRP IU SIZE : 320 SRP IU SG SIZE: 15 SRP IO CLASS: 0xff00 service 0 : name SRP.T10: id 0x service 1 : name SRP.T10: id 0x service 2 : name SRP.T10: id 0x service 3 : name SRP.T10: id 0x HCA 0 Port 1 0x0002c > Target Port GID 0xfe a11dd SRP IOC Profile : FVIC in Chassis 0x00066a00db00001e, Slot 4, Ioc 2 SRP IOC GUID : 0x00066a02dd SRP IU SIZE : 320 SRP IU SG SIZE: 15 SRP IO CLASS: 0xff00 service 0 : name SRP.T10: id 0x service 1 : name SRP.T10: id 0x service 2 : name SRP.T10: id 0x service 3 : name SRP.T10: id 0x HCA 0 Port 1 0x0002c > Target Port GID 0xfe a21dd SRP IOC Profile : Mellanox OFED SRP target SRP IOC GUID : 0x00066a SRP IU SIZE : 996 SRP IU SG SIZE: 58 SRP IO CLASS: 0x0100 service 0 : name SRP.T10:00066a id 0x00066a HCA 0 Port 1 0x0002c > Target Port GID 0xfe a00a SRP IOC Profile : Mellanox OFED SRP target SRP IOC GUID : 0x00066a SRP IU SIZE : 996 SRP IU SG SIZE: 58 SRP IO CLASS: 0x0100 service 0 : name SRP.T10:00066a id 0x00066a HCA 0 Port 1 0x0002c > Target Port GID 0xfe a01a NOTE: The output has the following minor differences as compared to the InfiniServ SRP /proc/driver/ ics_srp/driver - ib_qlgc_srp_query now reports the SRP IO CLASS. - Under Target Path(s) ib_qlgc_srp_query now reports Target Port GID instead of Target Port GUID. Using ib_qlgc_srp_cfg: ib_qlgc_srp_cfg [-v #] [-f <srp_cfg_file_name.cfg>] Used by ib_qlgc_srp_build_cfg Using ib_qlgc_srp_build_cfg: ib_qlgc_srp_build_cfg [option flags] ib_qlgc_srp_build_cfg creates a configuration file based on discovered target devices. By default, the information is send to stdout. In order to create a configuration file, output should be redirected to a disk file. Type "ib_qlgc_srp_build_cfg -h" for a list and description of the option flags. QLogic SRP Module on Linux for OpenFabrics and InfiniPath Readme File 2 of 6

3 The default configuration generated by ib_qlgc_srp_build_cfg for OFED is similar to the one generated for the QuickSilver Stack with the following differences: - For OFED, the configuration automatically includes IOClass. - For OFED, the configuration automatically includes SID. - For OFED, the configuration provides targetportgid instead of targetportguid. - For OFED, the configuration automatically includes targetiocprofileidstring. Displaying configured SRP targets and statistics: ib_qlgc_srp_stats [/sys/class/qlogic_infiniband_srp/<name of adapter>] The program by default will display all know configured adapters along with each configured session. By specifying an adapter name, the display will be restricted to only that individual adapter. Displaying statistics and controlling SRP targets: ib_qlgc_srp_control [-l [-i]] [-a <Adapter Name> [-i] [-x] ] [-a <Adapter Name> -s <Session Name> <[-i] [-x] [-u] [-d] [-r]>] [-h] -l - list all QLogic SRP Virtual Adapters and Sessions -i - include stats output for Adapters and Sessions -a - Specify SRP Virtual Adapter Name -s - Specify a SRP Virtual Adapter's Session Name -x - Remove SRP Virtual Adapter or Session -u - Start a SRP Virtual Adapter's Session -d - Stop a SRP Virtual Adapter's Session -r - Restart a SRP Virtual Adapter's Session -h - display this help This program by default is interactive and initially displays all known configured adapters. You can enter commands at this point to drill down and control or query an adapter or one of it's configured sessions. You can also use the switches above to control or query an individual adapter or one of it's sessions noninteractively from the command line. 4. Package Installation NOTE: When installing for InfiniPath, you must export the following environment variable: export IPATH_DISTRO=<as entered to install InfiniPath> As an example, for Red Hat Enterprise Linux 4 UD 4: export IPATH_DISTRO=2.6.9_U4 Installing the SRP driver, tools, and scripts To install the driver, tools, and scripts, you must be the root user. As the root user in the QLGC_SRP directory, enter the following command:./install.sh Install will ask you if you wish to start the driver at boot. The default is no. If you supply the -a option./ install.sh -a), the driver will be set to automatically load at system boot and you will not be prompted about load at system boot. Starting the driver manually when not loaded during system boot To start the driver, enter the following command: /etc/init.d/qlgc_srp start or service qlgc_srp start To restart the driver, enter the following command: /etc/init.d/qlgc_srp restart or service qlgc_srp restart In order to stop the driver, enter the following command: /etc/init.d/qlgc_srp stop or service qlgc_srp stop To uninstall the driver, tools and scripts you must be the root user. As the root user in the QLGC_SRP directory, enter the following command:./uninstall.sh QLogic SRP Module on Linux for OpenFabrics and InfiniPath Readme File 3 of 6

4 5. OFED Package Requirements Choose the appropriate package for the installed HCA ipath HCAs: libipathverbs libipathverbs-devel Mellanox HCAs: libmthca libmthca-devel Mellanox ConnectX HCAs: libmlx4 libmlx4-devel Other required packages: ib_verbs kernel-ib kernel-ib-devel libibcm libibcm-devel libibcommon libibcommon-devel libibmad libibmad-devel libibumad libibumad-devel libibverbs libibverbs-devel libibverbs-utils libopensm libopensm-devel libosmcomp libosmcomp-devel libosmvendor libosmvendor-devel librdmacm librdmacm-devel librdmacm-utils mstflint openib-diags perftest tvflash ofed-docs ofed-scripts ibutils NOTE: Hot swapping a Fibre Channel bridge card requires exchanging I/Os in order for the QLogic SRP package to detect that the bridge card has been swapped. (This is identical to the QuickSilver SRP driver's functionality.) 6. Sample Configuration File # qlgc_srp.cfg file generated by /usr/sbin/ib_qlgc_srp_build_cfg, version , on Tue Apr 15 10:49:09 EDT 2008 registeradaptersinorder: ON # ============================================================================ # IOC Name: FVIC in Chassis 0x00066a00db00001e, Slot 4, Ioc 1 # IOC GUID: 0x00066a01dd SRP IU SIZE : 320 # service 0 : name SRP.T10: id 0x session card: 0 port: 1 #portguid: 0x0002c initiatorextension: 1 targetiocguid: 0x00066a01dd targetiocprofileidstring: "FVIC in Chassis 0x00066a00db00001e, Slot 4, Ioc 1" targetportgid: 0xfe a11dd targetextension: 0x SID: 0x IOClass: 0xff00 IUSize: 320 IUSGTableSize: 15 QLogic SRP Module on Linux for OpenFabrics and InfiniPath Readme File 4 of 6

5 end adapter adapteriodepth: 1000 luniodepth: 16 adaptermaxio: 128 adaptermaxluns: 512 adapternoconnecttimeout: 60 adapterdevicerequesttimeout: 2 # set to 1 if you do not want target connectivity verification noverify: 0 description: "SRP Virtual HBA 1" end # ============================================================================ # IOC Name: Mellanox OFED SRP target # IOC GUID: 0x00066a SRP IU SIZE : 996 # service 0 : name SRP.T10:00066a id 0x00066a session card: 0 port: 1 #portguid: 0x0002c initiatorextension: 1 targetiocguid: 0x00066a targetiocprofileidstring: "Mellanox OFED SRP target" targetportgid: 0xfe a00a targetextension: 0x00066a SID: 0x00066a IOClass: 0x0100 IUSize: 996 IUSGTableSize: 58 end adapter adapteriodepth: 1000 luniodepth: 16 adaptermaxio: 128 adaptermaxluns: 512 adapternoconnecttimeout: 60 adapterdevicerequesttimeout: 2 # set to 1 if you do not want target connectivity verification noverify: 0 description: "SRP Virtual HBA 2" end Sample ib_qlgc_srp_stats output: SCSI Host # : 54 Mode : FAILOVER Trgt Adapter Depth : 768 Verify Target : Yes Rqst Adapter Depth : 1000 Rqst LUN Depth : 16 Tot Adapter Depth : 768 Tot LUN Depth : 16 Act Adapter Depth : 766 Act LUN Depth : 16 Max LUNs Scanned : 512 Max IO : (128 KB) Max Sectors : 256 Max SG Depth : 32 Session Count : 1 No Connect T/O : 60 Second(s) Register In Order : ON Dev Reqst T/O : 2 Second(s) Description : SRP Virtual HBA 1 Session : Session 1 State : Active Source GID : 0xfe c Destination GID : 0xfe a11dd SRP IOC Profile : FVIC in Chassis 0x00066a00db00001e, Slot 4, Ioc 1 SRP Target IOClass : 0xFF00 SRP Target SID : 0x SRP IPI Guid : 0x0002c SRP IPI Extnsn : 0x SRP TPI Guid : 0x00066a01dd SRP TPI Extnsn : 0x Source LID : 0x001a Dest LID : 0x0009 Completed Sends : 0x Send Errors : 0x Completed Receives : 0x Receive Errors : 0x Connect Attempts : 0x Test Attempts : 0x Total SWUs : 0x Available SWUs : 0x Busy SWUs : 0x SRP Req Limit : 0x SRP Max ITIU : 0x SRP Max TIIU : 0x QLogic SRP Module on Linux for OpenFabrics and InfiniPath Readme File 5 of 6

6 SRP Max Send IU : 0x SRP Max Cmd : 0x Req IU SG Depth : 0x f Act IU SG Depth : 0x f Max SG Used : 0x f Host Busys : 0x SCSI Host # : 55 Mode : FAILOVER Trgt Adapter Depth : 128 Verify Target : Yes Rqst Adapter Depth : 1000 Rqst LUN Depth : 16 Tot Adapter Depth : 128 Tot LUN Depth : 16 Act Adapter Depth : 126 Act LUN Depth : 16 Max LUNs Scanned : 512 Max IO : (128 KB) Max Sectors : 256 Max SG Depth : 32 Session Count : 1 No Connect T/O : 60 Second(s) Register In Order : ON Dev Reqst T/O : 2 Second(s) Description : SRP Virtual HBA 2 Session : Session 2 State : Active Source GID : 0xfe c Destination GID : 0xfe a00a SRP IOC Profile : Mellanox OFED SRP target SRP Target IOClass : 0x0100 SRP Target SID : 0x00066a SRP IPI Guid : 0x0002c SRP IPI Extnsn : 0x SRP TPI Guid : 0x00066a SRP TPI Extnsn : 0x00066a Source LID : 0x001a Dest LID : 0x000d Completed Sends : 0x f6b2 Send Errors : 0x Completed Receives : 0x f6b2 Receive Errors : 0x Connect Attempts : 0x Test Attempts : 0x Total SWUs : 0x Available SWUs : 0x Busy SWUs : 0x SRP Req Limit : 0x f SRP Max ITIU : 0x e4 SRP Max TIIU : 0x e4 SRP Max Send IU : 0x SRP Max Cmd : 0x Req IU SG Depth : 0x a Act IU SG Depth : 0x Max SG Used : 0x Host Busys : 0x Contacting Support Please feel free to contact your QLogic approved reseller or QLogic Technical Support at any phase of integration for assistance. QLogic Technical Support can be reached by the following methods: Web: North America Contact Information [email protected] Phone: (952) Support contact information for other regions of the world is available at the QLogic website: Go to Top Copyright All rights reserved worldwide. QLogic, the QLogic logo, and the Powered by QLogic logo are registered trademarks of QLogic Corporation. All other brand and product names are trademarks or registered trademarks of their respective owners. QLogic SRP Module on Linux for OpenFabrics and InfiniPath Readme File 6 of 6