HP Enterprise Data Warehouse Appliance Firmware and Drivers for MRA v0.0.0.0 User Guide Abstract This document provides information on updating the firmware and drivers for the HP Enterprise Data Warehouse Appliance. Part Number: 656398-001 March 2011 Edition: 1
Copyright 2011 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. The only warranties for HP products and services are set forth in the express warranty statements accompanying such products and services. Nothing herein should be construed as constituting an additional warranty. HP shall not be liable for technical or editorial errors or omissions contained herein. Confidential computer software. Valid license from HP required for possession, use or copying. Consistent with FAR 12.211 and 12.212, Commercial Computer Software, Computer Software Documentation, and Technical Data for Commercial Items are licensed to the U.S. Government under vendor s standard commercial license. Microsoft, Windows, and Windows Server are U.S. registered trademarks of Microsoft Corporation. Intel is a registered trademark of Intel Corporation or its subsidiaries in the United States and other countries.
Contents Introduction... 4 Overview... 4 HP EDW servicing model... 4 Firmware... 6 MRAv1.0 firmware and drivers... 6 SAN and Switch devices... 6 Device drivers... 8 Required drivers per node... 8 Firmware and driver update options... 9 Firmware deployment options... 9 HP SUM... 9 Enterprise Data Warehouse updating process... 11 Supported scenarios... 11 Individual note update example... 11 Updating the firmware... 13 Appliance update using EDW update script... 13 Collecting firmware and driver versions using EDW versions script... 16 Checking firmware and driver versions against MRA configuration... 18 Firmware and driver updates using HP Smart Update... 20 Fiber Channel HBA firmware upgrade... 26 InfiniBand 4xDDR DP HCA firmware upgrade... 30 HP P2000G3 FC Controller firmware upgrade... 31 QLogic IB 9040 and 9024 Switch firmware upgrade... 37 Fiber Channel SAN Switch firmware upgrade... 43 ProCurve 2810-24/48 G Switch firmware upgrade... 45 Software Depot EDW firmware/driver ISO image... 47 Default IP naming schema... 48 Device interface names... 48 External device names... 48 Storage device names... 49 Switch device names... 49 Node device names... 49 Data rack... 50 IP address single data rack... 50 Firmware versions... 52 MRA v1.0 firmware versions by node type... 52 Acronyms and abbreviations... 54 Contents 3
Introduction Overview Updating firmware and drivers are critical to maintaining the health of servers, storage arrays and switches. When considering the process for updating firmware in a multi-server, MPP architecture appliance such as the HP Enterprise Data Warehouse (EDW), powered by SQL Server 2008 R2 Parallel Data Warehouse Edition, other key considerations come into play. In addition to requiring a specific, tested firmware/driver "recipe," the appliance also requires that the firmware and driver versions be consistent across all like nodes and subsystems throughout the appliance. The updating process requires careful coordination with Microsoft support. This document is part of a Data ZIP file (which can also be burned to a data DVD) called the HP EDW Support Pack. For those familiar with the ProLiant Support Pack, the EDW Support Pack performs similar functions and includes similar contents. This document lists the firmware and driver versions supported in the HP EDW appliance MRA v0.0.0.0, as well as the servicing process for updating an HP EDW Appliance. As this is a scale-out, shared nothing appliance which operates as a single entity, the approach to servicing the unit (hardware replacement, driver and firmware updates) vary from the standard processes for that of a scale-up, SMP server. HP EDW servicing model The following aspects of servicing are unique to EDW: 1. With few exceptions, all servicing events require coordination with Microsoft support to bring nodes offline and back online. The coordination effort book-ends the standard servicing procedures in place today for updating firmware, drivers, or servicing/troubleshooting hardware failures. 2. The HP EDW is a multi-node, scale-out appliance. A Master Reference Architecture (MRA) pertains to a number of supported hardware topologies with a prescriptive, locked-down "recipe" of drivers, firmware and software versions called a Reference ID. a. The customer must work with Microsoft Support to identify the current MRA Reference ID version prior to implementing any updates b. Individual drivers and firmware cannot be updated without ensuring they comply with a tested MRA Reference ID. c. All drivers and firmware pertaining to a given MRA Reference ID must be updated together no mix and match. d. The exception is when a customer outage demands implementing a driver or firmware to restore service, in which case a new MRA Reference ID will be formalized after the event. 3. When a driver or firmware is updated on one device within an EDW, the change must be propagated to all related devices in the appliance. The actual process for implementing driver and firmware updates leverage the standard means provided by HP (such as HP Smart Update Manager, or HP SUM) or the third party vendor (QLogic, Brocade, Mellanox). Introduction 4
4. When a servicing event applies to more than one node (almost all driver and firmware updates), it requires a scheduled outage of the appliance (coordinated through Microsoft Support and/or the Customer). Introduction 5
Firmware MRAv1.0 firmware and drivers The tables below list the supported firmware and drivers for MRA v0.0.0.0. The HP Smart Update Manager is used to install a majority of the firmware and drivers required for devices in the server nodes as noted in the Installed using HP SUM column. Server Device Version Date Filename MAD CTL CMP LZ BU Installed using HP SUM SA 410i controller 3.52 cp013900.exe X X X X X X 10/13/2010 NC382i Boot Code 5.2.2 cp013823.exe X X X X X iscsi 4.2.10 NCSI 2.0.6 NC375i 4.0.539 sp50017.exe X X X X X 12/15/2010 ilo3 1.16 cp014253.exe X X 1/1/2011 ilo2 2.05 cp014258.exe X X 1/4/2011 SAS Expander card 2.06 cp014139.exe X X X 5/27/2010 DL360G7 BIOS P68 12/1/2010 cp014315.exe X X X DL380G7 BIOS P67 12/1/2010 DL370G6 BIOS P63 12/1/2010 82E DP FC HBA 1.00A12 1/30/2009 IB 4x DDR PCIe DP HCA cp014312.exe X X X cp014293.exe X X* X ud100a12.all X X X 5.03.0000 fw-25218-5_3_0 00-483513-B21_ A1.bin *The Test/Dev topology uses the DL370G6 server for the backup node. SAN and Switch devices X X X X X Device Version Date Filename HP P2000 G3 FC Controllers HP 8/8 16 port SAN Switch HP 8/40 24 port SAN Switch ProCurve Switch 2810-24 G TS201R015 11/22/2010 6.2.2d 10/5/2010 6.2.2d 10/5/2010 N.11.15 10/21/2008 TS201R015.bin cp014120.exe V6.2.2d.zip V6.2.2d.zip N_11_15.swi Firmware 6
Device Version Date Filename Qlogic IB DDR 9040 Switch 4.2.5.1.1 infinio9000.t3.pkg Qlogic IB DDR 9024 Switch 4.2.5.1.1 infinio9000.t3.pkg Firmware 7
Device drivers Required drivers per node Driver Version File Name MAD CTL CMP LZ BU Installed using HP SUM ilo 3 management 3.1.0.0 cp012623.exe X X X X X controller driver (all but LZ) Intel Chipset Driver (all 7.2.1.0 cp011867.exe X X X X X* X servers) Storport 3.14.0.0 cp012913.exe X X X X` Driver for HP Emulex HBA (all fiber attached servers) Lights-Out Management 1.15.0.0 cp011817.exe X X* X Interface Driver (LZ) ilo 2 Management 1.13.0.0 cp011799.exe X X* X Controller Driver (LZ) SAS Controller Driver 6.20.0.64 cp011328.exe X X X X X X HP NC-Series QLogic 4.0.534.0 cp013190.exe X X nx3 Multifunction Driver NC-Series Braodcom 5.2.22.0 cp012445.exe X X X X X 1Gb network driver Insight Management Agents 8.50.0.0 cp012575.exe X X X X X X Mellanox IB driver** 2.0.5_433 5 MLNX_WinOF_ 2_0_5_wlh_x64 _fre_2_0_5_43 35.msi *The Test/Dev topology uses the DL370G6 server for the backup node. **Must be installed through MS services instructions. X X X X X Device drivers 8
Firmware and driver update options Firmware deployment options Four key terms are used when describing typical firmware update use case scenarios; local, remote, online, and offline. Local installation the installation tool runs on the physical hardware being updated. An example is a utility that is executed on a server to update its system ROM. Remote installation the installation tool runs on one system but updates other physical targets. The physical tool runs on a workstation or system like MAD01, but remotely updates the firmware. Online installation the installation takes place while the host processor is running in the customer s normal environment. For instance, if the customer runs Microsoft Windows Server 2003, the update takes place under this environment and does not require the user to boot to a special environment to update the firmware. An online installation does not mean that a reboot may not be necessary for the firmware to be activated. Offline installation the installation takes place while the server is booted to a special environment that prevents the user from doing normal operations until the firmware update operations are completed and server is rebooted. An example is when a customer boots to the Firmware Maintenance CD to deploy Emulex, QLogic, and Brocade Fibre Channel HBA firmware updates. These updates require that no I/O operations are done through the adapters while they are being updated, and the only way to ensure this is to boot to a special environment. HP SUM These terms can be used together to designate the type of environment required for firmware updates to occur, such as local-online or remote-online. Specifically for EDW, the firmware update process is designed as a remote-online process, driven by a number of scripts. HP Smart Update Manager (HP SUM) is the primary deployment tool used to update firmware on a majority of the EDW components. HP SUM is designed for maximum flexibility and is shipped within the HP ProLiant Support Pack, Firmware Maintenance CD, HP BladeSystem Firmware Deployment Tool, and HP EDW Support Pack (downloadable from Software Depot). HP SUM provides a Graphical User Interface (GUI) and a command-line, scriptable interface for deployment of firmware for single or one-to-many servers and network-based targets such as ilos, OAs, and Virtual Connect Ethernet and Fibre Channel modules. HP SUM has an integrated hardware and software discovery engine that discovers the installed hardware, current versions of firmware in use on a target, and software versions on target servers. This prevents extraneous network traffic by only sending the components to a target host that are needed. HP SUM also has logic to install updates in the correct order and ensure all dependencies are met before deployment of a firmware update. It also contains logic to prevent version-based dependencies from derailing an installation and ensures firmware updates are handled in a manner that reduces any downtime required for the firmware update process. HP SUM does not require an agent for remote installations as it copies a small, secure SOAP server to the target server for the duration of the installation. After the installation is complete, the SOAP server and all Firmware and driver update options 9
remote files associated with the installation except installation log files are removed. HP SUM copies the log files from the remote targets back to the system where HP SUM is executed. The key features of HP SUM include: Dependency checking, which ensures appropriate install order and dependency checking between components. Intelligent deployment deploys only required updates. Simultaneous firmware and software deployment. Improved deployment performance. Local or remote (one-to-many) online deployment. Local offline deployments with the HP Firmware Deployment Tool and Firmware Maintenance CD. Remote offline deployment when used with the SmartStart Scripting Toolkit or ilo Virtual Media. GUI or CLI/scriptable with extensive logging. Remote command-line deployment. HP SUM provides updates for the following firmware types: Firmware types System ROM ilo2 / ilo3 firmware Broadcom NIC firmware Intel NIC firmware QLogic NIC firmware Smart Array controller firmware SAS and SATA hard drive firmware behind Smart Array controllers Emulex, QLogic, and Brocade Fibre Channel Host Bus Adapter firmware SAS Expander QLogic InfiniBand HCA firmware Brocade Fibre Channel switch firmware QLogic InfiniBand switch firmware ProCurve Ethernet switch firmware MSA P2000G3 FC Storage array firmware Supported by HP SUM Yes Yes, both server-based and directly to the ilo Management port Yes Yes, after firmware is available No Yes Yes, some older SATA hard drives require offline mode Yes, in offline mode only because of HBA firmware limitations Not as of 12/10 (in process) No No No No No For the non-supported components, the firmware update tools are included on the EDW Support Pack ISO. Firmware and driver update options 10
Enterprise Data Warehouse updating process Supported scenarios The EDW Support Pack contains all of the firmware, drivers, and tools necessary to update your EDW topology to a given MRA Reference ID. New Reference IDs for a given topology will be released to Software Depot over time as new driver/firmware combinations are tested and validated. The following scenarios are supported from the EDW Support Pack: Type Approach Script Full Appliance Update Use the EDW_update.cmd script to update all of the EDW_update.cmd firmware and drivers to comply with a given MRA Reference ID. Use this when updating to a new MRA Reference ID. This approach requires that the EDW Appliance be brought offline for servicing. Coordinate this effort with Microsoft Support. Individual node update Use the individual scripts and/or smart components to Various (see below) perform a local update on the node Use this when replacing a node within the appliance. This likely uses a local offline approach, and requires the node be brought offline; the rest of the appliance may still be operational. Coordinate this effort with Microsoft Support. Individual component update Use the script or smart component associated with the Various (see below) specific firmware or driver requiring updating. Use this when only a specific component requires updating to support a new Reference ID. This uses the remote-offline approach, and requires the appliance to be brought offline for servicing. Coordinate this effort with Microsoft Support. Individual note update example In the case of a failed compute node (ProLiant DL360 G7), follow the instructions below to update the firmware and drivers prior to running the node recovery process through the SQL Server PDW management console: 1. Download the appropriate EDW Support Pack from HP software depot (https://h20392.www2.hp.com/portal/swdepot/displayproductinfo.do?productnumber=edw-mr A). 2. With the server separately physically configured outside of the EDW appliance, login to ilo3. 3. Start the integrated remote console using a user that has domain administrator privilege and mount the HP OEM version of Windows Server 2008 SP2 included with the HP EDW appliance to load Windows on the server. 4. Unzip the EDW Support Pack ZIP file to C:\. This step is necessary in order to speed up deployment. Enterprise Data Warehouse updating process 11
5. Run the following Smart Components and executables from the MRA folder: Component Version Executable SA 410i controller 3.52 10/13/2010 NC382i Boot Code 5.2.2 iscsi 4.2.10 NCSI 2.0.6 ilo3 1.16 1/11/2011 DL360G7 BIOS P68 12/11/2010 82E DP FC HBA 1.00A12 1/30/2009 cp013900.exe cp013823.exe cp014253.exe cp014315.exe ud100a12.all IB 4x DDR PCIe DP HCA 5.03.0000 fw-25218-5_3_000-483513-b21_a1.bin ilo 3 management controller 3.1.0.0 cp012623.exe driver Intel Chipset Driver 7.2.1.0 cp011867.exe Storport 3.14.0.0 cp012913.exe Driver for HP Emulex HBA SAS Controller Driver 6.20.0.64 cp011328.exe HP NC-Series QLogic nx3 4.0.534.0 cp013190.exe Multifunction Driver NC-Series Broadcom 1Gb 5.2.22.0 cp012445.exe network driver Insight Management Agents 8.50.0.0 cp012575.exe Mellanox IB driver 2.0.5_4335 MLNX_WinOF_2_0_5_wlh_x64_fre_2_0_5_433 5.msi 6. The server is now compliant with the existing MRA. Contact Microsoft Support to start the node replacement process. Enterprise Data Warehouse updating process 12
Updating the firmware Appliance update using EDW update script To update the EDW appliance to the latest MRA version: 1. Download the appropriate MRA EDW Support Pack from HP software depot (https://h20392.www2.hp.com/portal/swdepot/displayproductinfo.do?productnumber=edw-mr A). 2. Copy EDW Support Pack to the management node <appliance name>-mad01. 3. Unzip EDW Support Pack to C:\. This step is necessary in order to speed up deployment. 4. Open c:\mra v0.0.0.0\edw_update\edw_update.cmd in Notepad and change the default username/passwords to match your environment for P2000 and FC/IB/Ethernet switches. 5. Open a command windows and start c:\mra v0.0.0.0\edw_update\edw_update.cmd. The script cycles automatically through all the steps described in this document. In case you need to override or downgrade firmware/driver versions you can use the force option, for example EDW_update.cmd force. Updating the firmware 13
The first step runs updates using HP SUM which require some user intervention in order to select the nodes impacted. The rest of the steps do not require user intervention. The deployment time varies based on the topology and number of data racks. After the update is complete the script reboots the EDW appliance, first MAD01 node and then all other nodes. Once MAD01 is up again you can check to see when all nodes are operational using HPC Cluster Manager. You can find the link to it at Start>Programs>Microsoft HPC Pack>HPC Cluster Manager. Select Node Management on the left side if not already selected and wait for all nodes to show OK health status. Updating the firmware 14
6. Once all nodes are operational, run C:\MRA v0.0.0.0\edw_update\edw_align.cmd which verifies that all cluster nodes have the correct primary host selected before you attempt to start the PDW services with DwConfig. Updating the firmware 15
Collecting firmware and driver versions using EDW versions script To update the EDW appliance to the latest MRA version: 1. Download the appropriate EDW Support Pack from HP software depot (https://h20392.www2.hp.com/portal/swdepot/displayproductinfo.do?productnumber=edw-mr A). 2. Copy EDW Support Pack to the management node <appliance name>-mad01. 3. Unzip EDW Support Pack to C:\. This step is necessary in order to speed up deployment. Updating the firmware 16
4. Open c:\mra v0.0.0.0\edw Update\Version\EDW_versions.cmd in Notepad and change the default username/passwords to match your environment for P2000 and FC/IB/Ethernet switches. 5. Open a command windows and start c:\mra v0.0.0.0\edw Update\Version\EDW_versions.cmd The script cycles automatically through all the firmware and drivers discover steps. The version scan time varies based on the topology and number of data racks. As a guideline, for one standard data rack topology it takes about three minutes to scan. Updating the firmware 17
After the discovery is complete, you can view the collected versioning information in C:\MRA v0.0.0.0\edw Update\Version\versions.out. Checking firmware and driver versions against MRA configuration Follow these steps in order to update the EDW appliance to the latest MRA version: 1. Follow the instructions from "Collecting Firmware and Driver versions using EDW versions script" in order to collect versioning information. 2. Open a command windows and start c:\mra v0.0.0.0\edw Update\Version\MRA_check.cmd. Updating the firmware 18
The script will check your configuration against the MRA configuration. The MRA configuration is defined in MRA.cfg and it looks like this: Updating the firmware 19
The output will highlight all the components that have the wrong version and provide information about the expected version. Firmware and driver updates using HP Smart Update The EDW appliance device firmware and drivers listed in Table 1 and Table 3 above and marked as Installed using HP SUM will be upgraded if needed through the HPSUM_update.cmd script. Applying firmware and driver updates using HP SUM on EDW: 1. Download the appropriate EDW Support Pack from HP software depot (https://h20392.www2.hp.com/portal/swdepot/displayproductinfo.do?productnumber=edw-mr A). 2. Copy EDW Support Pack to the management node <appliance name>-mad01. 3. Unzip EDW Support Pack to C:\. This step is necessary to speed up deployment. 4. Start c:\mra v0.0.0.0\edw Update\HPSUM\HPSUM_update.cmd. This starts HP SUM with appropriate targets and bundle. The default behavior updates only the drivers/firmware that need updating. If you need to reflash or downgrade drivers/firmware to the MRA version contained on this disc you can start the script with the Updating the firmware 20
force parameter. For example: c:\mra v0.0.0.0\edw. Update\HPSUM\HPSUM_update.cmd force. The script updates all the remote nodes first in parallel (twenty servers can be in parallel at one time) and then it updates the local node separately in order to avoid issues related to upgrading local network drivers/firmware while having connections opened to the remote nodes. Updating the firmware 21
5. Click Start Inventory. Updating the firmware 22
The list of remote nodes are preselected for you and the script proceeds to authenticate using current credentials against all of the remote nodes. This might take a few minutes depending on the number of nodes in the appliance as HP SUM has copied a temporary local version to each node. Updating the firmware 23
This screen shows the progress of the discovery process on all remote nodes. This process identifies firmware and drivers versions existent on the system. 6. A list of all the remote nodes that were identified is presented and the updates will be preselected to run on all of the nodes. Click Install to start installing the updates. Updating the firmware 24
Deselect nodes if you need to update only specific nodes. Selecting a node from the list and clicking View host also provides additional information on existing firmware/driver levels and what components are preselected to be installed on the host. Updating the firmware 25
7. This screen shows the update progress across remote nodes. 8. The final screen shows the upgrade status across all nodes. Select individual nodes, and then click View Host to see additional information about each update step. Click Exit to finish HP SUM update for remote nodes. If you encounter errors deploying on some nodes you can re-run the EDWupdate.cmd and it will try again only on the nodes and components that were not upgraded. HP SUM restarts automatically and it repeats steps 4 to 8 on the local node. Fiber Channel HBA firmware upgrade Use the following scripts to upgrade the FC HBA firmware on the EDW appliance: 1. Download the appropriate EDW Support Pack from HP software depot (https://h20392.www2.hp.com/portal/swdepot/displayproductinfo.do?productnumber=edw-mr A). 2. Copy EDW Support Pack to the management node <appliance name>-mad01. 3. Unzip EDW Support Pack to c:\. This step is necessary to speed up deployment. 4. Run c:\mra v0.0.0.0\edw Update\FC_HBA\FC_HBA_update.cmd. This copies the proper firmware update tool to all the nodes and run the update process. Updating the firmware 26
The default behavior updates only the drivers/firmware that needs updating. In case you need to reflash or downgrade drivers/firmware to the MRA version contained on this disc, start the script with the force parameter. For example: c:\mra v0.0.0.0\edw. Update\FC_HBA\FC_HBA_update.cmd force. Example command prompt output. To update the firmware on a single FC HBA adapter, follow the steps below. The appliance uses the HP 82E DP FC HBA in the Control, Backup, and Compute nodes. Update these to FW v1.00a12 using the following instructions: 1. Download the appropriate EDW Support Pack from HP software depot (https://h20392.www2.hp.com/portal/swdepot/displayproductinfo.do?productnumber=edw-mr A). 2. Copy EDW Support Pack to the management node <appliance name>-mad01. 3. Unzip EDW Support Pack to C:\. This step is necessary to speed up deployment. 4. Open a command prompt and navigate to C:\MRA v0.0.0.0\edw Update\FC_HBA. 5. Install HBAnyware on all Control, Backup, and Compute nodes. (cp011160.exe) Updating the firmware 27
6. Select All Programs > Emulex > HBAnywhere to start the HBAnywhere software from one of the Control nodes. Updating the firmware 28
7. Select Batch > Download Firmware to open the HBAnywhere Batch Firmware Download dialog window. 8. Browse to the firmware file, <ISO Image drive>:\edw MRAv1.0\FC_HBA\ud100a12.all. 9. Verify all controllers and ports are selected. Updating the firmware 29
10. Click Start Download. 11. Follow the same steps on the Backup node and on one of the Compute nodes. InfiniBand 4xDDR DP HCA firmware upgrade Use the following custom scripts to upgrade the IB HCA firmware on the EDW appliance: 1. Download the appropriate EDW Support Pack from HP software depot (https://h20392.www2.hp.com/portal/swdepot/displayproductinfo.do?productnumber=edw-mr A). 2. Copy EDW Support Pack to the management node <appliance name>-mad01. 3. Unzip EDW Support Pack to C:\. This step is necessary to speed up deployment. Updating the firmware 30
4. Run C:\MRA v0.0.0.0\edw Update\IB_HCA\IB_HCA_update.cmd. This copies the proper firmware update tool to all the nodes and run the update process. The default behavior updates only the drivers/firmware that needs updating. In case you need to reflash or downgrade drivers/firmware to the MRA version contained on this disc, start the script with the force parameter. For example: C:\MRA v0.0.0.0\edw. Update\IB_HCA\IB_HCA_update.cmd force. Example command prompt output. HP P2000G3 FC Controller firmware upgrade Use the following custom scripts to upgrade the HP P2000 G3 FC array firmware on the EDW appliance: 1. Download the appropriate EDW Support Pack from HP software depot (https://h20392.www2.hp.com/portal/swdepot/displayproductinfo.do?productnumber=edw-mr A). 2. Copy EDW Support Pack to the management node <appliance name>-mad01. 3. Unzip EDW Support Pack to C:\. This step is necessary to speed up deployment. 4. Run C:\MRA v0.0.0.0\edw Update\P2000\P2000_update.cmd. This updates the firmware on all the P2000 storage controllers. The script requires you to provide the username and password for P2000 array and expects all P2000 arrays to use the same credentials. If you encounter a situation where a P2000 G3 array controller shows as failed after the firmware update, restart the storage controller and then restart the management controller on the failed P2000 controller. Updating the firmware 31
Example command prompt output. This script expects the FTP service to be enabled on all P2000 as per default configuration. If the scripts fails to update, check and make sure all P2000 have the FTP service enabled. To update the firmware on a single P2000 G3 FC array follow the steps below. The appliance uses the HP P2000 G3 FC arrays for the Control, Backup, and Compute node storage. The controllers in the arrays should be updated to firmware version TS201R015, using the following instructions: 1. Download the appropriate EDW Support Pack from HP software depot (https://h20392.www2.hp.com/portal/swdepot/displayproductinfo.do?productnumber=edw-mr A). 2. Copy EDW Support Pack to the management node <appliance name>-mad01. 3. Unzip EDW Support Pack to C:\. This step is necessary to speed up deployment. 4. Run C:\MRA v1.0\p2000\cp014120.exe. Updating the firmware 32
5. Click Install. 6. Enter the IP Address, User Name, and Password of the P2000 G3 FC array. The storage array IP addresses should be 172.16.2.x, Compute node storage arrays x = 1, 2,, Control node storage arrays x = 240, 241,, Backup node storage arrays x = 220, 221,. (for example, Compute node CMP01-172.16.2.1, manage, HPinvent1). This is the default password. Updating the firmware 33
7. Click Continue. 8. Click Continue. Updating the firmware 34
The storage array Discovery Process begins. 9. The current version is displayed. If the firmware version is not up to date, click Install to continue. Updating the firmware 35
Controller (B) firmware is flashed. Controller (A) firmware is flashed. 10. Upon successful completion, click Close. 11. This process needs to be repeated for all P2000 G3 FC storage arrays in the EDW appliance. (i.e. The HP DL360G7 1 Data Rack High Performance EDW topology has 10 Compute node storage arrays IP addresses 172.16.2.[1 10], 1 Control node storage array IP address 172.16.2.240, and 2 Backup storage arrays IP addresses 172.16.2.[220-221]) Updating the firmware 36
QLogic IB 9040 and 9024 Switch firmware upgrade Use the following custom scripts to upgrade the QLogic 9040 and 9024 switch firmware on the EDW appliance: 1. Download the appropriate EDW Support Pack from HP software depot (https://h20392.www2.hp.com/portal/swdepot/displayproductinfo.do?productnumber=edw-mr A). 2. Copy EDW Support Pack to the management node <appliance name>-mad01. 3. Unzip EDW Support Pack to C:\. This step is necessary to speed up deployment. 4. Run C:\MRA v0.0.0.0\edw Update\IB_Switch\IB_Switch_update.cmd. This updates the firmware on all the QLogic IB switches. Command Syntex: IB_Switch_update <user> <password>. Start the command and make sure that you provide the username and password. All switches are expected to have the same credentials. Example command prompt output. To update the firmware on each switch separately: The firmware version should be 4.2.5.1.1 on all three switches. If the firmware version needs to be updated it is easiest to log on to each switch through the assigned IP address. QLogic SilverStorm 9040 Switch 1. Log on to the QLogic 9040 switch via a browser on the 172.16.4.101. Updating the firmware 37
2. Login information needs to supplied by the customer. The factory default at time of shipping should be (admin/hpinvent1) the manufacturer default user/password is (admin/adminpass). 3. Click on the outer gray chassis as outlined in the above picture to get to the Chassis Details menu. Updating the firmware 38
4. Click Maintenance, and then click Firmware Update. 5. Select the Spine 1 checkbox, and then select the Boot? checkbox. 6. Browse to the Firmware Update Package file InfiIO9000.t3.pkg. (C:\MRA v0.0.0.0\edw Update\IB_Switch\InfiIO9000.t3.pkg) 7. Click Update. 8. Once the Completion State is 100% complete, click Close. QLogic SilverStorm 9024 Switches 1. Log on to the QLogic 9024 switch via a browser on the 172.16.4.[102,103 ] The number of 9024 switches depends on the EDW topology. Update the firmware on all of the 9024 switches. Updating the firmware 39
2. Login information is supplied by the customer. The factory default at time of shipping should be (admin/hpinvent1) the manufacturer default user/password is (admin/adminpass). Updating the firmware 40
3. Select Maintenance, and then Update Firmware. 4. Browse to the Firmware Update Package file InfiIO9000.t3.pkg. (C:\MRA v0.0.0.0\edw Update\IB_Switch\InfiIO9000.t3.pkg) Updating the firmware 41
5. Click Update Firmware. Updating the firmware 42
Allow the firmware update to 100% complete. 6. Once completed, click Close. Fiber Channel SAN Switch firmware upgrade The appliance uses HP 8/8 and 8/40 SAN Switches. The SAN switches should be updated to FW version 6.2.2d. Use the following custom scripts to upgrade the Brocade FC switch firmware on the EDW appliance: 1. Download the appropriate EDW Support Pack from HP software depot (https://h20392.www2.hp.com/portal/swdepot/displayproductinfo.do?productnumber=edw-mr A). 2. Copy EDW Support Pack to the management node <appliance name>-mad01. 3. Unzip EDW Support Pack to C:\. This step is necessary to speed up deployment. 4. Run C:\MRA v0.0.0.0\edw Update\FC_Switch\FC_Switch_update.cmd. This updates the firmware on all the Brocade FC switches. For example: FC_Switch_update <user> <password>. Start the command and make sure you provide the username and password. All switches are expected to have the same credentials. Updating the firmware 43
Example command prompt output. To update the firmware on each switch separately follow the steps below. During a firmware download, the switch reboots and the browser temporarily loses connection with the switch. When the connection is restored, the version of the software running in the browser is different from the new software version that has been installed and activated on the switch. Close all of the Web Tools windows and log in again to avoid a firmware version mismatch. You might get pop-up messages that imply the loss of connection is temporary and will soon be resolved. You still need to close all windows and re-log in. 1. Open a web browser to the FC switch IP address (172.16.4.[51,52, ]. 2. Login information will need to supplied by the customer. The factory default at time of shipping should be (admin/hpinvent1) the manufacturer default user/password is (admin/admin). 3. Open the Switch Administration window. 4. Click the Firmware Download tab. Updating the firmware 44
5. Choose the option for downloading the firmware. 6. Select network as download source. 7. Type the host name or IP address, user name, password, and fully qualified path to the file release.plist. The path name should follow the structure: //<directory>/<fos_version_directory>/release.plist where the <directory> is the path up to the entry point of <fos_version_directory> and <fos_version_directory> is the location of the unzipped version of Fabric OS. For example: //v6.2.2d/release.plist. Under v6.2.2d there is no release.plist file, but several SWBDXX folders for each switch type supported by this firmware upgrade. 8. Select the FTP protocol type in the Protocol Type field. 9. Click Download. The firmware download begins. You can monitor the progress by looking at the Firmware Download progress bar. About halfway through the download process, after the firmware key is downloaded to the switch, connection to the switch is lost and Web Tools invalidates the current session. Web Tools invalidates all windows if upfront login is enabled. 10. Close all Web Tools windows and log in again. If the firmware download is in progress when you log in, you can continue to monitor its progress. ProCurve 2810-24/48 G Switch firmware upgrade Use the following custom scripts to upgrade the ProCurve 2810 G switch firmware on all of the ProCurve 2810 switches in the EDW appliance: 1. Download the appropriate EDW Support Pack from HP software depot (https://h20392.www2.hp.com/portal/swdepot/displayproductinfo.do?productnumber=edw-mr A). 2. Copy EDW Support Pack to the management node <appliance name>-mad01. 3. Unzip EDW Support Pack to C:\. This step is necessary to speed up deployment. 4. Run C:\MRA v0.0.0.0\edw Upadte\ETH_Switch\ETH_Switch_update.cmd. This flashes the firmware on all the ProCurve 2810 switches in the appliance. Example command prompt output. Updating the firmware 45
To update the firmware on a single switch: 1. Log on to the ProCurve 2810-24/48 G switch via a browser on the 172.16.4.[1,2 ]. The number of 2810-24/48 G switches depends on the EDW appliance topology. Update the firmware on all of the 2810 24/48 G switches to version N.11.15. 2. Login information is supplied by the customer. The factory default at time of shipping should be (admin/hpinvent1) the manufacturer default user/password is (blank/blank). 3. Select the Configuration tab. 4. Click Upload/Download. 5. Select upload a Software Image for Destination Primary. 6. Browse to the firmware image (C:\MRA v0.0.0.0\edw Update\Eth_Switch\N_11_15.swi). Updating the firmware 46
7. Click Upload. 8. Wait for the Upload to complete. You might need to log on again during the upload, since the firmware has been reset. Software Depot EDW firmware/driver ISO image The EDW Support Pack ZIP file can be downloaded from HP software depot (https://h20392.www2.hp.com/portal/swdepot/displayproductinfo.do?productnumber=edw-mra). The ZIP file contains the following: PDW Setup/Installation drivers HP specific drivers installed during the factory PDW setup. These files are required on the management node in this directory structure in addition to the PDW installation files during the setup procedure for an initial PDW installation or a reconfiguration of the appliance at the customer site. ZIP file location - \External\ EDW Firmware and driver upgrade Installation of the EDW appliance firmware and drivers as outlined in this document. ZIP filelocation - \EDW update\ Documentation A copy of this document HP EDW Firmware and Drivers for MRA v1.0 and additional material. ZIP file location - \Documents Updating the firmware 47
Default IP naming schema Device interface names The first two octets can change if the site survey indicates there is a conflict. Device BMC IP ilo IP External Ethernet Interface Ethernet Interface 1 - Private Ethernet Interface 2 - Private0 Infiniband Interface 1 - Application Infiniband Interface 1 - Application0 Failover Cluster Prefix (Appliance Name) <appliance name> <appliance name> DNS Name BMC[CTL,CMP, MAD,LZ,BU][num ] RAC[CTL,CMP, MAD,LZ,BU][num ] IP Address/ Range N/A Not used DNS [site generated] Netmask 172.16.5.x 255.255.12 8.0 172.15.6.x 255.255.12 8.0 [site generated] N/A Not used DNS 172.16.0.x 255.255.12 8.0 N/A Not used DNS 172.16.1.x 255.255.12 8.0 <appliance name> <appliance name> <appliance name> Virtual Node <appliance name> SQL Instance (external Ethernet) Not used DNS [CTL,CMP,LZ,BU, MAD][num] [CTL,CMP,LZ,BU, MAD][num] WFO[CTL,CMP] [num] SQL[CTL,CMP,LZ, BU][num] Not used DNS 172.16.128. x 172.16.129. x 172.16.121. x 172.16.130. x [site generated] 255.255.12 8.0 255.255.12 8.0 255.255.12 8.0 255.255.12 8.0 [site generated] Example Name HPMTP-BMCCT L02 HPMTP-RACCT L01 Example IP Misc information 172.16.5.2 4 172.16.6.0 N/A TBD Eth 3 = External N/A 172.16.0.1 2 N/A 172.16.1.1 2 HPMTP-CMP01 172.16.12 8.1 HPMTP-CMP01 172.16.12 9.1 HPMTP-WFOC MP01 HPMTP-AQLC MP12 172.16.13 1.1 172.16.13 0.12 Eth 1 = Heartbeat Network 1 Eth 2 = Heartbeat Network 2 IB 1 = Application Network 1 IB 2 = Application Network 2 Cluster Virtual Network Virtual Application Network N/A TBD Eth 3 = External External device names The external IP naming schema are used temporarily for setup purposes. Name IP address Netmask <appliance name>mad01 10.10.10.250 255.255.255.0 <appliance name>mad02 10.10.10.251 255.255.255.0 Default IP naming schema 48
<appliance name>bu01 10.10.10.220 255.255.255.0 <appliance name>lz01 10.10.10.230 255.255.255.0 <appliance name>ctl01 10.10.10.240 255.255.255.0 <appliance name>ctl02 10.10.10.241 255.255.255.0 <appliance name>sqlctl01 10.10.10.242 255.255.255.0 Storage device names Device Prefix DNS name IP address Netmask Example name Example IP Storage (HP or EMC) SP-A IP address <appliance name> SA1[CTL,CMP, BU][num] 172.16.2.x 255.255.128.0 HPMTP-SA1CTL01 172.16.2.x Storage (HP or EMC) SP-B IP address <appliance name> Switch device names SB1[CTL,CMP, BU][num] 172.16.3.x 255.255.128.0 HPMTP-SB1CTL01 172.16.3.x Device Prefix DNS name IP address Netmask Example name Example IP Ethernet IP Fibre IP IB IP PDU IP <appliance name> <appliance name> <appliance name> <appliance name> ETHSW[num] 172.16.4.1-50 255.255.128.0 HPMTP-ETHSW01 172.16.12.1 FBRSW[num] 172.16.4.51-100 255.255.128.0 HPMTP-FBRSW02 172.16.12.52 IBSW[num] 172.16.4.101-15 0 PDU[num] 172.16.4.151-20 0 255.255.128.0 HPMTP-IBSW01 172.16.12.101 255.255.128.0 HPMTP-PDU01 172.16.12.151 Node device names Device Prefix DNS name IP address Misc information Active Directory Server <appliance name> MAD01-04 x.x.x[250-253] For internal only devices Control Node - All <appliance name> CTL01-10 x.x.x[240-249] gateway: 172.16.0.250 Compute Node - Active <appliance name> CMP01-89 x.x.x[1-89] dns1: 172.16.0.250 Compute Node - Passive <appliance name> CMP91-99 x.x.x[91-99] dns2: 172.16.0.251 Landingzone Node <appliance name> LZ01-10 x.x.x[230-239] Backup Node <appliance name> BU01-10 x.x.x[220-229] DOMAIN: naming <appliance name>.dwpu.local (example: HPMTP.DWPU.LOCAL) Default IP naming schema 49
Data rack IP address single data rack Name IP Address ilo Address <app name>-ibswcore 172.16.4.101 <app name>-msa2bu01 A: 172.16.2.221 B: 172.16.3.221 <app name>-msa1bu01 A: 172.16.2.220 B: 172.16.3.220 <app name>-bu01 172.16.4.220 <app name>-swf01 172.16.4.51 <app name>-swf02 172.16.4.52 <app name>-swf03 172.16.4.53 <app name>-swf04 172.16.4.54 <app name>-swe01 172.16.4.1 <app name>-swe02 172.16.4.2 <app name>-ctl1 172.16.6.240 <app name>-ctl2 172.16.5.241 <app name>-msactl01 A: 172.16.2.240 B: 172.16.3.240 <app name>-cmp01 172.16.6.1 <app name>-cmp02 172.16.6.2 <app name>-cmp03 172.16.6.3 <app name>-cmp04 172.16.6.4 <app name>-cmp05 172.16.6.5 <app name>-cmp06 172.16.6.6 <app name>-cmp07 172.16.6.7 <app name>-cmp08 172.16.6.8 <app name>-cmp09 172.16.6.9 <app name>-cmp10 172.16.6.10 <app name>-cmp11 172.16.6.91 <app name>-swe03 172.16.4.3 <app name>-swe04 172.16.4.4 <app name>-msa01 A: 172.16.2.1 B: 172.16.3.1 <app name>-msa02 A: 172.16.2.2 B: 172.16.3.2 <app name>-msa03 A: 172.16.2.3 B: 172.16.3.3 <app name>-msa04 A: 172.16.2.4 B: 172.16.3.4 <app name>-msa05 A: 172.16.2.5 B: 172.16.3.5 Data rack 50
Name IP Address ilo Address <app name>-msa06 A: 172.16.2.6 B: 172.16.3.6 <app name>-msa07 A: 172.16.2.7 B: 172.16.3.7 <app name>-msa08 A: 172.16.2.8 B: 172.16.3.8 <app name>-msa09 A: 172.16.2.9 B: 172.16.3.9 <app name>-msa10 A: 172.16.2.10 B: 172.16.3.10 Data rack 51
Firmware versions MRA v1.0 firmware versions by node type Control Rack Component RTM Landing Zone Server (DL370 G6) BIOS System ROM P63 12/01/2010 ilo 2 2.05 P410i 3.52 NC375i 4.0.539 HP SAS Expander Card 2.06 HP IB 4X DDR PCI-e DUAL PORT 0 mem HCA 5.03.0000 Control Node Server 1 (DL380 G7) BIOS System ROM P67 12/01/2010 ilo3 1.16 P410i 3.52 NC382i 5.2.2 BootCode HP IB 4X DDR PCI-e DUAL PORT 0 mem HCA 5.03.0000 HP82E 8GB Dual-port PCI-e FCHBA 1.00A12 HP SAS Expander Card 2.06 Control Node Server 2 (DL380 G7) BIOS System ROM P67 12/01/2010 ilo3 1.16 P410i 3.52 NC382i 5.2.2 BootCode HP IB 4X DDR PCI-e DUAL PORT 0 mem HCA 5.03.0000 HP82E 8GB Dual-port PCI-e FCHBA 1.00A12 HP SAS Expander Card 2.06 Management Server 1 'MAD01' (DL60 G7) BIOS System ROM P68 12/01/2010 ilo3 1.16 P410i 3.52 NC382i 5.2.2 BootCode HP IB 4X DDR PCI-e DUAL PORT 0 mem HCA 5.03.0000 Management Server 2 (DL60 G7) BIOS System ROM P68 12/01/2010 ilo3 1.16 P410i 3.52 NC382i 5.2.2 BootCode Firmware versions 52
Control Rack Component RTM HP IB 4X DDR PCI-e DUAL PORT 0 mem HCA 5.03.0000 Backup Server (DL380 G7) BIOS - System ROM P67 12/01/2010 ilo3 1.16 P410i 3.52 NC382i 5.2.2 BootCode HP IB 4X DDR PCI-e DUAL PORT 0 mem HCA 5.03.0000 HP82E 8Gb Dual-port PCI-e FCHBA 1.00A12 Storage Arrays HP P2000 G3 MSA FC Dual Cntrl LFF Array TS201R015 Switches HP 8/8 (8) ports Enabled SAN Switch 6.2.2d ProCurve Switch 2810-24 G N.11.15, ROM N.10.01 QLogic 9040 IB Fabric Director Kit 4.2.5.1.1 Data Rack Component RTM Compute Node (DL360 G7) BIOSSystem ROM P68 12/01/2010 ilo 3 1.16 P410i 3.52 HP 82E 8Gb Dual-port PCI-e FCHBA 1.00A12 NC382i 5.2.2 BootCode HP IB 4X DDR PCI-e DUAL PORT 0 mem HCA 5.03.0000 Storage Arrays HP P2000 G3 MSA FC Dual Cntrl LFF Array TS201R015 HP P2000 G3 MSA FC Dual Cntrl SFF Array TS201R015 Switches HP 8/40 Base 24-ports Enabled SAN Switch 6.2.2d ProCurve Switch 2810-24 G N.11.15, ROM N.10.01 QLogic 9024 InfiniBand DDR Switch 4.2.5.1.1 Firmware versions 53
Acronyms and abbreviations BIOS Basic Input/Output System CLI Command Line Interface DDR double data rate EDW enterprise data warehouse FC Fibre Channel GUI graphical user interface HBA host bus adapter HCA Host Channel Adapter HP SUM HP Smart Update Manager IB InfiniBand ilo Integrated Lights-Out IP Internet Protocol Acronyms and abbreviations 54
iscsi Internet Small Computer System Interface ISO International Organization for Standardization MRA master reference architecture OA Onboard Administrator PCI peripheral component interface PDW parallel data warehouse SA Smart Array SAN storage area network SAS serial attached SCSI SATA serial ATA SMP Server Migration Pack SOAP Simple Object Access Protocol SQL structured query language Acronyms and abbreviations 55