Technical white paper Proof Point: Example Clustered Microsoft SQL Configuration on the HP ProLiant DL980 Table of contents Introduction 2 Server Configuration 3 Microsoft SQL Server Overview 3 Network Layout 4 Storage Layout 5 Versions Used 8 BIOS 9 For More Information 13 Documentation Feedback 13
Introduction The 8-socket scalable HP ProLiant DL980 is an excellent fit for customers to handle their most demanding Microsoft SQL workloads or to consolidate their Microsoft SQL environments. This paper demonstrates an example HP ProLiant DL980 configuration in a clustered SQL environment. The objective is to offer proof points illustrating that key pieces of the stack have been tested together on the HP ProLiant DL980 to provide customers confidence and guidance when deploying a similar solution. An important part of HP testing is to exercise a common software stack with our scalable HP ProLiant DL980 in a realistic customer scenario. This configuration runs Microsoft Windows Server and Microsoft SQL Server on the HP ProLiant DL980, using high availability redundancy features such as clustering, multipath I/O and NIC teaming. This paper covers the tested set of software, hardware, and firmware versions and settings that HP used in this configuration. Note that this is not a benchmarking configuration. The goal is not to push the highest levels of performance but rather to validate that the whole solution stack runs together. The logical configuration of the test case is shown in the following figure. This configuration has two 8- socket HP ProLiant DL980s, clustered together with Microsoft (MS) Failover Clustering under Windows Server 2008 R2. For redundancy, NIC teaming and multipath I/O are used in addition to MS Failover Clustering. Figure 1 Logical Configuration of Test Case 2
Server Configuration In this configuration, each node is a ProLiant DL980 containing eight 2.40 GHz Xeon E7-2870 processors (10 cores each), and 64 x 8 GB 2Rx4 PC3 10600R RAM HP P/N 500205-071 (512 GB total). During testing, each node is taken down in turn, to ascertain that the SQL instance fails over properly. The failover is expected to transfer all shared storage to the new active node, but is not expected to maintain client connections. The SQL application is expected to reconnect and retry any transactions that fail during the switchover. There is no criterion in this test for the failover time, but the database tends to become available again in just a few minutes. Since Microsoft SQL Server 2012 is used, a more seamless transition is possible through the new SQL Server 2012 Always On Availability Group feature. However, Always On Availability Group is not part of this test case, and is beyond the scope of this paper. For more information on the Microsoft SQL Server 2012 s Always On features, refer to the Microsoft SQL website: http://www.microsoft.com/sqlserver/en/us/default.aspx. Microsoft SQL Server Overview For this configuration, Microsoft SQL Server 2012 is used, with an OLTP-type load on a 513 GB database with 300,000 customers. The available storage in this configuration would certainly support a larger database, and the DL980 routinely handles far larger workloads. However, the database size is intentionally kept small here for testing agility. The intention is to have SQL running continuously, recovering automatically from various types of failure. In general, the default settings for SQL are used, then the HP Enterprise SQL Optimizer (ESO) is run, and all recommendations are taken. ESO is a program available at no charge from HP that compares settings on the DL980 with established best practices, and offers to optimize these settings if desired. For more information on HP ESO, refer to the HP ESO User Guide at: http://www.hp.com/go/proliant-dl980-docs. 3
Network Layout The following figure depicts the network topology of this test case. Figure 2 Test Case Network Topology 4
The main public LAN connection for each cluster node consists of two NIC ports teamed together, one from port 2 of an NC360T in slot 1, and the other from port 2 of an NC360T in slot 7. Slot 1 is a PCI-e 1.0 slot in the DL980, but still has ample bandwidth to keep up with a dual gigabit ethernet adapter. Port 1 on each NIC card is not used for the purposes of testing. Two distinct cards are used, rather than two ports on one card, to provide redundancy against the theoretical possibility of one slot failing. During testing, one or the other LAN cable (but not both at once) of the main public LAN is periodically disconnected and reconnected, to verify that the NIC teaming tolerates the fault and recovers properly. The HP Network Configuration Utility is used for the teaming. Default settings are used, except that the Team Type Selection is Switch-assisted Load Balancing with Fault Tolerance (SLB). Embedded NC375i port 1 on each node is used for the cluster heartbeat. This connection is not teamed, as failure of the heartbeat would not cause catastrophic failure of the cluster. The heartbeat is on its own subnet, distinct from the subnet of the main public LAN. However, both subnets are physically connected to the same network switch, an HP ProCurve 2900-48G. There is only one network switch, as network switch redundancy is not part of this test case. Embedded NIC ports 2, 3, and 4 are not used. Storage Layout The operating system boots from two 146 GB disks in the embedded drive bays of the DL980. These disks form a RAID-1 volume through the internal p410i Smart Array. Note that the Microsoft SQL Server 2012 executable also resides on this volume. All cluster shared storage volumes are on HP p2000 storage enclosures, connected through fibre channel. Figure 3 Storage Layout - User Database Raid 5 The first P2000 enclosure contains two arrays. The array that holds the User Database volume is RAID-5 with a hot spare disk, so that it can recover automatically from a disk failure. In fact, this functionality is verified during testing. The second array holds backups of the user database. Each time the test is run, the database is restored from backup, to insure a consistent known state. As it takes much longer to recreate the database from scratch, the backup volume is valuable, and protected with RAID-5. Each array also contains an additional small volume for test purposes. These small volumes are slightly different in size, for easy identification. Figure 4 Storage Layout - TempDB Raid 10 5
On the second P2000 enclosure, RAID-10 is used for TempDB. This gives a boost in speed, although it makes little difference for this particular test. The User Log, Master Log, Master Database, Cluster Quorum, and DTC all reside on one RAID-5 array. Of these, the User Log is the only volume that should experience any significant traffic. Once again, small test volumes are embedded in each array. This configuration takes advantage of the native multipath IO (MPIO) feature included with Windows Server 2008 R2. Figure 5 shows the two previously mentioned P2000 s at the top. Below these are two HP StorageWorks 8/20q Fibre Channel Switches. At the bottom are the two DL980 s serving as nodes of the cluster. For simplicity, each storage enclosure presents all LUNs on all ports, and each Fibre Channel switch has only one zone encompassing all ports. MPIO is permitted to discover all paths automatically. On more complex configurations, it might be desirable to mask or selectively present LUNs by various means, but that is unnecessary for the small number of paths in this configuration, and beyond the scope of this paper. During the test, each fibre link is disconnected for about a minute then reconnected. SQL is expected to execute transactions seamlessly while this is being done. In addition, each of the fibre channel switches is powered down in turn, to verify that SQL continues seamlessly. Additional AJ764A host bus adapters are installed in slots 2 and 9 on each node, but not used for the purposes of this particular test. 6
Figure 5 Test Case Storage Topology 7
Versions Used The table below lists the versions used in this test configuration. Please note that although firmware and driver versions are mentioned in this paper, they are shown only to establish an example of a baseline set that worked together when the test was run. In general, HP recommends using the most recent drivers, firmware, and operating system patches published on hp.com. Table 1 Versions Used in Test Configuration Item Version Window Driver Version HP ProLiant DL980 G7 P4101 Embedded Smart Array 5.06 6.22.0.64 NC375i Embedded Quad Gigabit Ethernet 4.0.579 4.4.8.812 NC360T Dual Gigabit Ethernet n/a 9.13.41.0 AJ76A Dual 8 Gb Fibre Channel Host Adapter 5.06.50 9.1.9.45 ilo 1.28 BIOS P66 05/01/2012 cpld 0x8c_0x8dCPU_042611 FMP 2.0-0 FPGA Power Management 2.0-0 Controller 1.7 Operating System Microsoft Windows Server 2008 R2 SP1 Operating System Patches Per HP x64 QFE CD 9.10 HP System Providers 9.0.5.0 HP Insight WBEM Providers 9.0.0.0 HP Network Configuration Utility 10.50.0.32 Database Microsoft SQL 2012 8
BIOS The HP ProLiant DL980 BIOS version is P66 05/01/2012 for this configuration. All BIOS default settings are taken, except: 44-bit address mode is enabled. All PCI-e card ROMs are disabled. HP Power Profile is set to Maximum Performance. The tables below summarize the BIOS settings used for this test case configuration. Table 2 System Options Serial Port Options Embedded Serial Port COM 1: IRQ4: IO: 3F8h-3FFh Virtual Serial Port COM 2: IRQ3: IO: 2F8h-2FFh Embedded NICs Embedded NIC Boot Options Network Boot Advanced Memory Advanced ECC Support Protection USB Options USB Control USB USB 2.0 Controller Removable Flash Media Boot Sequence External DriveKeys First Processor Options No-Execute Memory Protection Intel Virtualization Technology Intel Hyperthreading Options Processor Core Disable All Cores Enhanced Processor Core Disable (Intel Core Select) 10 Intel Turbo Boost Technology Intel VT-d NUMLOCK Power-On State Off Table 3 Power Management Options HP Power Profile Maximum Performance HP Power Regulator OS Control Redundant Power Supply Mode Balanced Mode Advanced Power Management Options Minimum Processor Idle Power Core State No C-states Minimum Processor Idle Power Package State No Package State Maximum Memory Bus Frequency Auto Memory Power Capping PCI Express Generation 2.0 Support Auto Dynamic Power Savings Mode Response Fast Collaborative Power Control DIMM Idle Power Saving Mode 9
Table 4 PCI IRQ Settings Embedded USB 1.1 Controller 1 5 Embedded USB 1.1 Controller 2 7 Embedded USB 1.1 Controller 3 10 Embedded USB 1.1 Controller 4 7 Embedded USB 2.0 Controller 5 Embedded Intel SATA Controller #1 11 Embedded ATI ES 1000 Video Controller 7 Embedded HP Integrated Lights-Out 3 Controller 10 Embedded HP Integrated Lights-Out 3 Processor 11 Embedded HP Integrated Lights-Out 3 UHCI Virtual Media 11 Embedded HP Smart Array P410i Controller 10 Embedded HP NC375i Integrated Quad Pot Gigabit Adapter 10 Embedded HP NC375i Integrated Quad Pot Gigabit Adapter 10 Embedded HP NC375i Integrated Quad Pot Gigabit Adapter 10 Embedded HP NC375i Integrated Quad Pot Gigabit Adapter 10 Slot 1: HP NC360T PCIe DP Gigabit Server Adapter 10 Slot 1: HP NC360T PCIe DP Gigabit Server Adapter 11 Slot 2: HP StorageWorks 82Q 8Gb PCI-e Dual Port FC HBA 10 Slot 2: HP StorageWorks 82Q 8Gb PCI-e Dual Port FC HBA 11 Slot 6: HP StorageWorks 82Q 8Gb PCI-e Dual Port FC HBA 10 Slot 6: HP StorageWorks 82Q 8Gb PCI-e Dual Port FC HBA 11 Slot 7: HP NC360T PCIe DP Gigabit Server Adapter 10 Slot 7: HP NC360T PCIe DP Gigabit Server Adapter 11 Slot 9: HP StorageWorks 82Q 8Gb PCI-e Dual Port FC HBA 10 Slot 9: HP StorageWorks 82Q 8Gb PCI-e Dual Port FC HBA 11 Slot 11: HP StorageWorks 82Q 8Gb PCI-e Dual Port FC HBA 10 Slot 11: HP StorageWorks 82Q 8Gb PCI-e Dual Port FC HBA 11 Table 5 PCI Device / Embedded Intel SATA Controller #1 Embedded HP Smart Array P410i Controller Embedded HP NC375i Integrated Quad Port Gigabit Adapter Slot 1: HP NC360T PCIe DP Gigabit Server Adapter Slot 2: HP StorageWorks 82Q 8Gb PCI-e Dual Port FC HBA Slot 6: HP StorageWorks 82Q 8Gb PCI-e Dual Port FC HBA Slot 7: HP NC360T PCIe DP Gigabit Server Adapter Slot 9: HP StorageWorks 82Q 8Gb PCI-e Dual Port FC HBA Slot 11: HP StorageWorks 82Q 8Gb PCI-e Dual Port FC HBA Table 6 PCI Option ROM / Slot 1: HP NC360T PCIe DP Gigabit Server Adapter Slot 2: HP StorageWorks 82Q 8Gb PCI-e Dual Port FC HBA Slot 6: HP StorageWorks 82Q 8Gb PCI-e Dual Port FC HBA Slot 7: HP NC360T PCIe DP Gigabit Server Adapter Slot 9: HP StorageWorks 82Q 8 Gb PCI-e Dual Port FC HBA Slot 11: HP StorageWorks 82Q 8Gb PCI-e Dual Port FC HBA 10
Table 7 Standard Boot Order (IPL) 1 CD-ROM 2 Floppy Drive (A:) 3 USB DriveKey (C:) 4 Hard Drive C: (See Boot Controller Order) 5 PCI Embedded HP NC375i Integrated Dual Port Gigabit Adapter 7 PCI Slot 7 HP NC360T PCIe DP Gigabit Server Adapter 8 PCI Slot 1 HP NC360T PCIe DP Gigabit Server Adapter 9 PCI Slot 1 HP NC360T PCIe DP Gigabit Server Adapter Table 8 Boot Controller Order 1 Embedded HP Smart Array P410i Controller 2 Embedded Intel SATA Controller #1 3 Slot 9: HP StorageWorks 82Q 8Gb PCI-e Dual Port FC HBA 4 Slot 9: HP StorageWorks 82Q 8Gb PCI-e Dual Port FC HBA 5 Slot 11: HP StorageWorks 82Q 8Gb PCI-e Dual Port FC HBA 6 Slot 11: HP StorageWorks 82Q 8Gb PCI-e Dual Port FC HBA 7 Slot 2: HP StorageWorks 82Q 8Gb PCI-e Dual Port FC HBA 8 Slot 2: HP StorageWorks 82Q 8Gb PCI-e Dual Port FC HBA 9 Slot 6: HP StorageWorks 82Q 8Gb PCI-e Dual Port FC HBA 10 Slot 6: HP StorageWorks 82Q 8Gb PCI-e Dual Port FC HBA Table 9 Server Availability ASR Status ASR Timeout Thermal Shutdown Wake-On LAN POST F1 Prompt Power Button Automatic Power-On Power-On Delay 10 Minutes Delayed No Delay Table 10 Server Security Set Power-On Password Set Admin Password Table 11 BIOS Serial Console & EMS BIOS Serial Console Port Auto BIOS Serial Console Baud Rate 9600 EMS Console BIOS Interface Mode Auto 11
Table 12 Server Asset Text Server Info Text Administrator Info Text Service Contact Text Custom POST Message Table 13 Advanced Options Advanced System ROM Options Option ROM Loading Sequence Load Embedded Devices First MPS Table Mode Full Table APIC ROM Selection Use Current ROM NMI Debug Button Virtual Install Disk PCI Bus Padding Options Memory Mapped I/O Options 20B of Memory Mapped I/O Address Mode 44-bit Power-On Logo Hide Option ROM Messages X2APIC Options Auto Video Options Optional Video Primary, Embedded Video Thermal Configuration Optimal Cooling Service Options Serial Number Product ID Advanced Performance Tuning HW Prefetcher Options Adjacent Sector Prefetch DCU Prefetcher Drive Write Cache Asset Tag Protection Unlocked Power Requirements Override Table 14 Misc Settings Time and Date System Default Options Utility Language (n/a) English 12
For More Information For an overview of the HP ProLiant DL980 server: http://www.hp.com/servers/dl980 For more information about the HP ProLiant DL980 G7 server with HP PREMA Architecture: http://h10010.www1.hp.com/wwpc/us/en/sm/wf25a/15351-15351-3328412-241644-4222584-4231377.html To download the recommended components described in this document, along with other drivers and software, visit the HP ProLiant DL980 G7 Support web page: http://www.hp.com/support/dl980g7 For a complete library of all documentation supporting the HP ProLiant DL980 G7 server: http://www.hp.com/servers/dl980 (click on the link, HP ProLiant DL980 G7 Server series ) For additional best practices whitepapers for the HP ProLiant DL980 : http://www.hp.com/go/proliant-dl980-docs Documentation Feedback HP welcomes your feedback. To make comments and suggestions about product documentation, send a message to: docsfeedback@hp.com. Include the document title, part number, and filename found at the end of the URL string (for example: c02861709.pdf). All submissions become the property of HP. Get connected hp.com/go/getconnected Current HP driver, support, and security alerts delivered directly to your desktop. Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. The only warranties for HP products and services are set forth in the express warranty statements accompanying such products and services. Nothing herein should be construed as constituting an additional warranty. HP shall not be liable for technical or editorial errors or omissions contained herein. Microsoft, Windows, and Windows Server are U.S. trademarks of Microsoft Corporation. Intel is a trademark of Intel Corporation 4AA1-1036ENW, Created July 2012 13