HP reference configuration for Microsoft Exchange 2013 on HP ConvergedSystem 700x

Transcription

1 Technical white paper HP reference configuration for Microsoft Exchange 2013 on HP ConvergedSystem 700x Microsoft Exchange Server 2013 with Hyper-V on HP ConvergedSystem 700x Table of contents Executive summary... 2 Introduction... 2 Overview... 2 Solution components... 5 Hardware components... 5 Software components Microsoft Windows Server 2012 Hyper-V Solution specific Hyper-V configurations Capacity and sizing HP hardware and Exchange Server 2013 recommended configuration Exchange solution capacity calculations and sizing Analysis and recommendations Consolidating applications on an HP ConvergedSystem 700x Planning tools Bill of materials Summary Implementing a proof-of-concept Appendix A: Optimization of Cluster Shared Volumes Appendix B: Windows PowerShell scripting Scripts Commands For more information... 43

2 Executive summary Electronic messaging is the most critical service and the primary consideration in most business virtual environments. functionality has the greatest impact on all organizational functions; in most businesses and corporations, is the primary means of communication and collaboration. Microsoft Exchange Server is one of the most popular messaging and collaboration software products in the IT industry. With the recent release of Microsoft Exchange Server 2013, many organizations are interested in integrating the new features into their existing messaging environment. This document identifies the configuration considerations and optimizations for running Microsoft Exchange Server 2013 on the HP ConvergedSystem 700x for Microsoft, which provides highly available, flexible and collaboration services to an organizational private cloud. The HP ConvergedSystem 700x for Microsoft saves a tremendous amount of time and administrative effort associated with such tasks as system planning, design and equipment procurement. However, organizations new to virtualization technologies, yet wanting to establish a private, public or hybrid cloud infrastructure, need to determine how to best deploy applications to reap the associated benefits and savings. Some HP customers, who have invested in an HP ConvergedSystem 700x, may benefit by making some changes to their administration and resource management policies. For example, an organization that prefers to manage applications by breaking them down and manage them into dedicated administrative sections when running multiple VMs on shared server resources could result in some conflicts regarding hardware assets. To avoid such management and control issues, customers can integrate the role of Virtualization Administrator (VA) into their IT management structures. The VA oversees and controls the distribution of hardware resources to each application or service. The VA also reduces some of the workload for the administrators of these virtualized applications. Instead of planning and procuring the server hardware and system resources, administrators can present their requests for resources to the VA, who makes them available. The details regarding the supporting hardware are transparent and irrelevant to the application administrator. While the main topic of this document is dedicated to Microsoft Exchange 2013 running on the HP ConvergedSystem 700x; the information within is beneficial to the Exchange administrator, VA and IT management to identify planning considerations for running and sharing multiple applications on the HP ConvergedSystem 700x. Target audience: The target audience for this document is technical decision-makers and solution architects. Document purpose: The purpose of this document is to describe a recommended architecture, highlighting recognizable benefits to technical audiences. Introduction Companies today are heavily investing in methods to establish continuous and flexible access to their business-critical data and applications, seeking to leverage the full potential of their IT hardware assets to maximize their hardware return on investments and minimize operational costs. Such business needs have led to an adoption of cloud computing, which is widely used in the IT industry and media today. With cloud computing, IT departments can provide critical business services to support their organizational users anywhere, anytime, breaking the bonds of traditional hardware requirements. The supporting hardware underneath these applications is fluid and abstracted away from the services they provide, as if they are floating in a cloud. Making cloud computing possible requires extensive knowledge of technologies and virtualization. For business organizations who desire to take advantage of the benefits of cloud computing but are limited in the time, expertise and available resources to adapt to a cloud methodology, HP offers the HP ConvergedSystem 700x. The HP ConvergedSystem 700x is a complete turnkey solution containing the necessary hardware and software components in a factory-assembled pre-racked and cabled configuration, especially designed to establish a total virtualized cloud environment ideal for supporting today s most popular business applications. Also, HP Consulting services are included to ensure an efficient on-site deployment of the software stack, including integrating Microsoft Windows Server 2012 and System Center 2012 into your existing IT infrastructure. Additionally, HP makes publications such as this white paper available to assist customers as they integrate mainstream business applications on an HP ConvergedSystem 700x, using associated hardware to establish an organizational virtual cloud. This document addresses the considerations for deploying Microsoft Exchange Server 2013, with an emphasis on virtualization, using Microsoft Windows Server 2012 with Hyper-V as featured in HP s release of the HP ConvergedSystem 700x for Microsoft. Overview The HP ConvergedSystem 700x supports Microsoft s Reference Architecture that is designed to conform to the Microsoft Private Cloud Fast Track reference architecture, reducing data center complexity, increasing agility and mitigating the risks that come with deployment of a virtualized environment. 2

3 As seen in Figure 1, the HP ConvergedSystem 700x for Microsoft solution is pre-built with HP s converged infrastructure consisting of HP ProLiant Gen8 servers, HP 3PAR StoreServ storage, HP networking, and Microsoft Windows Server 2012 with Hyper-V for server virtualization, as well as System Center 2012 SP1 for systems management integrated with HP Insight Control. The Windows Server 2012 Operating System and System Center 2012 enterprise licensing are managed and sold separately. The hardware components are factory assembled and racked. The system components can be tailor-fitted to individual customer environments. HP Consulting performs the initial configuration of storage, servers and the management stack as they integrate the solution into the customer s data center environment. HP ConvergedSystem 700x for Microsoft establishes the foundation of a cloud environment with the world's most self-sufficient servers and the industry's fastest growing hypervisor. Figure 1. HP ConvergedSystem 700x for Microsoft The HP ConvergedSystem 700x is an all-in-one solution containing all components needed for customers to get their business applications up and running and establish their own corporate cloud infrastructures. The HP ConvergedSystem 700x offers the flexibility to procure the HP ConvergedSystem 700x with the desired hardware, software, installation and configuration services to meet each customer s operational needs. HP understands that no two customer IT infrastructures are the same, and stands with customers to design a system that meets their unique operational requirements. The HP ConvergedSystem 700x offers the flexibility to change hardware options and procure technical services as needed. The HP ConvergedSystem 700x is delivered as one of three solution configurations: HP ConvergedSystem 700x ( B21) This base solution offers customers the ability to procure the HP ConvergedSystem 700x hardware components. It does not include OS integration or configuration preferences for a particular hypervisor, and offers no software licensing as a standard option. The HP ConvergedSystem 700x solution can be configured by the customer to accommodate non-virtualized environments or for virtualization, to establish the foundation of a cloud computing work environment. This model provides HP customers with the hardware components necessary to 3

4 deploy business applications. The HP ConvergedSystem 700x is well suited for an organization seeking the flexibility to acquire only the HP ConvergedSystem 700x hardware solution and desiring to cut some of the associated installation and configuration costs. The HP ConvergedSystem 700x solution is designed for customers who have the knowledge and expertise and are not in need of additional assistance to get the HP ConvergedSystem 700x operational. HP ConvergedSystem 700x for VMware ( B21) This solution is offered to HP customers who wish to use VMware on HP s factory assembled HP ConvergedSystem 700x hardware solution. It is specifically tailored to accommodate server workloads running on VMware vsphere 5.1 virtualization solutions. The HP ConvergedSystem 700x for VMware comes standard with HP Insight Control v7.2 software, the HP Insight Control plug-in for vcenter, VMware vsphere 5.1 Enterprise Plus and VMware vcenter Server Standard. HP Insight Control is installed and configured at the customer site. VMware software is factory installed for convenience, and licenses are included if purchased with the solution. The hardware and software in this solution is developed as a partner-integration joint effort, established on a close engineering partnership with VMware. The HP ConvergedSystem 700x for VMware offers the VMware vcloud Suite as the management tool for direct cloud management. HP ConvergedSystem 700x for Microsoft ( B21) This solution is the one that is covered in this document. It is offered to HP customers who prefer to take advantage of the many benefits of the HP ConvergedSystem 700x and base their cloud design on the HP ConvergedSystem 700x for Microsoft solution. This solution is best suited for server workloads running Microsoft applications and leveraging the combination of Microsoft Hyper-V and Microsoft System Center 2012 SP1 Datacenter, together with HP s market-leading Converged Infrastructure. The HP ConvergedSystem 700x for Microsoft comes standard with HP Insight Control v7.2 software and the HP ProLiant Server and BladeSystem Management Packs for Microsoft System Center Operations Manager. Both the Windows Server 2012 Datacenter and System Center 2012 Datacenter SP1 software licenses can be purchased from HP as part of the solution. The HP ConvergedSystem 700x for Microsoft is based on HP and Microsoft expertise in virtualization, and has been validated with the Microsoft Private Cloud Fast Track v3 program. It is designed to speed time to deployment, improve scalability and be highly available by default. HP ConvergedSystem 700x Services HP ConvergedSystem 700x solutions are intended to meet rapid response times and accommodate customers project schedules, from the time the platform is purchased through delivery and on-site setup. As part of the customer experience, several services are incorporated into the solution, including the following: On-site startup and integration services HP ConvergedSystem 700x solutions include factory integration services to rack and cable the hardware infrastructure, based in part on customer data collected with the Customer Intent Document (CID), to ensure the system is built and ready for data center integration. Installation and Startup Services Unpack system and inspect for damages Connection of HP ConvergedSystem 700x into customer network environment Basic configuration of the network uplinks to the customer environment and connection verification Review equipment and basic processes, including process for placing calls to request support HP ConvergedSystem 700x for Microsoft Deployment Accelerator Service Configuration of the HP 3PAR StoreServ storage array Setup and configuration of the compute cluster(s) Configuration of the System Center management servers, including installation of the following Microsoft System Center 2012 Datacenter SP1 components: SQL Server cluster System Center Virtual Machine Manager cluster System Center Operations Manager System Center Service Manager System Center Orchestrator System Center AppController Installation of HP Insight Control for System Center and HP management packs Basic validation of the Microsoft Private Cloud environment (for example, creation of new VMs from a service template) Delivery of As-Built documentation to the customer as part of the review and handover of the turnkey solution 4

5 Connecting the HP ConvergedSystem 700x solution in the HP customer data center Compute rack Each compute rack has two (2) HP 5920 top of rack switches with up to four (4) network uplinks per switch to connect to the data center as discussed here. Transceivers and appropriate cables need to be ordered or provided separately. Both 10Gb and 1Gb options are supported. For transceiver options, refer to the HP 5920 Network Switch QuickSpecs. Storage rack There are two connectivity requirements for the storage rack: connectivity of the Service Processor and connectivity of the management ports on the controllers. CAT5 Ethernet cables are included in the storage component for connectivity, which may be modified as needed. Note The focus and emphasis of this document is dedicated to the planning, configuration and deployment of Microsoft Exchange Server 2013 in a virtualized environment, based on the HP ConvergedSystem 700x for Microsoft featuring Microsoft Windows Server 2012 Hyper-V. The reader may find references to certain installation and configuration information that may be part of the installation and customer integration services procured as part of HP ConvergedSystem 700x and is the responsibility of HP Factory Express to preconfigure. These references have been added for the benefit of the virtualization administrator and Exchange administer, to provide additional insight into the functionality of the HP ConvergedSystem 700x and Exchange Server 2013 Solution components The HP ConvergedSystem 700x for Microsoft is an all-in-one solution containing all hardware and basic software components needed for customers to get their business applications up and running and establishes their own corporate cloud infrastructures. The HP ConvergedSystem 700x offers the flexibility to procure the HP ConvergedSystem 700x with the desired hardware, software, installation and configuration services to meet each customer s operational needs. HP understands that no two customer IT infrastructures are the same, and stands with customers to design a system that meets their unique operational requirements. The HP ConvergedSystem 700x offers the flexibility to change hardware options and procure technical services as needed. The following is a detailed description of the hardware and software components that are composed in this recommended configuration for the HP ConvergedSystem 700x for Microsoft to support an Exchange Server 2013 deployment. Hardware components In this section, we identify the components of the HP ConvergedSystem 700x. Additionally, to maintain continuity for this Exchange application design document with other HP documentation regarding other business application configurations running on the HP ConvergedSystem 700x, the HP ConvergedSystem 700x for Microsoft configuration has been selected. This is to maintain a common hardware foundation across all published HP recommended configurations of applications, such as Microsoft Lync or SharePoint, which can be deployed in tandem on the same HP ConvergedSystem 700x. The intent is to provide HP customers with a library of business application recommended configurations that can be deployed together on the same HP ConvergedSystem 700x virtual cloud or as individual virtual applications running on Microsoft Hyper-V. In this section, we identify the default components and configurations of the HP ConvergedSystem 700x for Microsoft. 5

6 Figure 2. HP ConvergedSystem 700x for Microsoft default options HP BladeSystem c7000 Platinum enclosure The BladeSystem c7000 Platinum enclosure provides all the power, cooling, and I/O infrastructure needed to support modular servers, and interconnects for the HP ConvergedSystem 700x. The enclosure is a 10U unit with the capacity to hold up to 16 servers and/or storage blades plus optional redundant network and storage interconnect modules. It includes a shared, 5-terabit per second high-speed Nonstop midplane for wire-once connectivity of server blades to network and shared storage. HP ProLiant server blades The HP ConvergedSystem 700x is intended to be scalable and modular in design with the addition of server building blocks. The server building blocks consist of HP ProLiant BL460c Gen8 server blades deployed within an HP BladeSystem c7000 Platinum enclosure ( B21) (with a minimum of 4 blades). This recommended configuration is based on a default single rack HP ConvergedSystem 700x system with a single fully populated HP BladeSystem c7000 Platinum enclosure and 16 HP ProLiant BL460c Gen8 server blades ( B21). These servers are intended to support all of the business virtual applications to support the entire enterprise on Exchange The following is a summary of the hardware features in the default ProLiant BL460c Gen8 blade: Two (2) Intel Xeon E v2 (3.0GHz/10-core/25MB /130W) ( L21 and B21) 256 GB RAM, Sixteen (16) HP 16GB 2Rx4 PC3L-12800R-11 Kit ( B21) One (1) HP FlexFabric 10Gb 2-port 554FLB Adapter ( B21) One (1) HP FlexFabric 10Gb 2P 554M FIO Adapter ( B21) Two (2) 300GB 15K SFF SAS hard drives ( B21) 6

7 In the HP ConvergedSystem 700x recommended configuration design, each BL460c server blade is running Microsoft Windows Server 2012 Datacenter edition with Hyper-V enabled. Microsoft Windows Server 2012 Datacenter edition offers the ability to operate an unlimited number of VMs on each server. This is optimal for running the HP ConvergedSystem 700x recommended configuration, given the anticipated number of VMs that can be consolidated onto this platform. Along with the Hyper-V role, Failover Clustering is also recommended on each server. Each additional rack then becomes its own Failover Cluster resource that can be managed as additional resources with the System Center Virtual Machine Manager environment. BladeSystem interconnect modules The BladeSystem enclosure allows the use of a variety of interconnect modules to match the design requirements of many solutions. The Virtual Connect FlexFabric modules ( B21) shown in Figure 3 are used in this design to provide converged Ethernet and Fibre Channel storage area networks into a single set of connections and a single point of management. Figure 3. HP Virtual Connect FlexFabric 10Gb/24-port Module UID S H A R E D : U P L I N K o r X - L I N K X1 X2 X3 X4 X5 X6 X7 X8 HP VC FlexFabric 10Gb/24-Port Module There are eight external SFP+ ports (shown above) and 16 internal ports in each of the two FlexFabric interconnect modules. The eight SFP+ connections are used to provide Ethernet or Fibre Channel access outside of the enclosure. The first four connections (x1 through x4) are capable of providing Ethernet or FC support while the remainder of the ports (x5 through x8) are Ethernet only. Note Ports x7 and x8 in FlexFabric modules are shared uplink or cross-link ports that connect with interconnect modules in the Fibre Channel Storage Area Network (SAN). The BladeSystem enclosure is equipped with two (2) redundant BladeSystem Onboard Administrator Modules ( B21) shown in Figure 4. The BladeSystem Onboard Administrator provides for HP Integrated Lights-Out (ilo) remote management, and HP Insight Control software offers complete management of the server blades and interconnect modules, regardless of the state of the server operating system. Figure 4. HP Onboard Administrator with KVM 7

8 The Fibre Channel Storage Area Network (SAN) configuration The SAN configuration used in this design combines two specific sets of components and utilizes the HP converged network capabilities of the HP FlexFabric 554FLB and 554M adapters to connect the server blades to the SAN using FCoE. (Figure 5). Figure 5. Supporting HP ConvergedSystem 700x SAN infrastructure design B-series 16Gb Fibre Channel (FC) switches The HP SN3000B FC Switch delivers affordable and flexible 16 Gbit/sec connectivity and can be incorporated into an extensive core fabric or placed as an edge switch for SAN environments. It delivers 12 or 24 high performance auto-sensing 4, 8 and 16 Gbit/sec Fibre Channel ports. With powerful yet flexible capabilities such as Ports On Demand scalability to 24 ports in a 12-port increment, the SN3000B FC Switch enables organizations to start small and grow their storage networks in a non-disruptive manner. A fully populated SN3000B FC Switch with 24 enabled ports provides 768 Gbit/sec switching capacity for congested, sustained, 16 Gbit/sec full duplex throughput. The following are the HP SN3000B FC Switch product highlights: Features Ports on Demand capability for fast, easy and cost-effective scalability from 12 to 24 ports in a 12-port increment. The SN3000B FC Switch with 24 enabled ports may be deployed as a full-fabric switch or in Access Gateway mode, which provides connectivity into any SAN (the default mode setting is a switch). Attaching through NPIV-enabled switches and directors, the SN300B FC Switch with 24 enabled ports in Access Gateway mode can connect to other SAN fabrics. Optional enhanced ISL Trunking combines up to eight ISLs between a pair of switches into a single, logical high-speed trunk capable of up to 128 Gbit/sec of throughput. Two HP SN3000B Fibre Channel switches support redundant connections to the management servers, the BladeSystem enclosure FlexFabric interconnect modules and the HP 3PAR StoreServ 7400 array (shown in Figure 5). The Fibre Channel connections differ between the rack mount servers and server blades. The rack mount servers use Fibre Channel HBAs which are connected, using optical cables and 8Gb SFP+ transceivers to two SN3000B FC switches. The server blades connect using Fibre Channel over Ethernet (FCoE) from the 554FLB FlexibleLOM and 554M adapters to the FlexFabric interconnect modules, and Fibre Channel is used for the connection from the FlexFabric interconnect module to the SN3000B FC switches. 8

9 HP ProLiant DL Gen8 servers In addition to the BL460c compute nodes, two HP ProLiant DL360p Gen8 (DL360) servers provide management of the HP ConvergedSystem 700x rack. HP also recommends clustering these servers and running Hyper-V with several VMs running the management software used in the environment. Each DL360 has 20 CPU cores, 256GB RAM and eight 300GB HDDs. Management software components The HP ConvergedSystem 700x provides the underlying infrastructure for running multiple workloads requiring high levels of performance and availability. To complete the solution, a software stack that can deliver the management, monitoring, operational control and lifecycle requirements are needed for a virtualized environment. To deliver these capabilities System Center 2012 with HP Insight Control for System Center is recommended for the HP ConvergedSystem 700x recommended configuration. HP 3PAR 7000 Service Processor The physical Service Processor (QR516B) is a dedicated storage appliance located within the storage rack providing close proximity to the HP 3PAR StoreServ 7000 Storage. The physical Service Processor is fully supported and maintained by HP Services. The physical Service Processor has a serial port connectivity that provides maintenance access for troubleshooting capabilities. If a VMware server is not available to run the virtual Service Processor, the physical Service Processor is the alternative for remote monitoring and remote service. HP 3PAR StoreServ 7000 Storage The HP ConvergedSystem 700x features the HP 3PAR StoreServ 7000 storage arrays (Figure 6) to meet all the storage needs for all applications within the virtual cloud environment. HP 3PAR StoreServ 7000 Storage provides high performance and guarantees to double virtual machine density. HP 3PAR StoreServ is also the first product family with a common architecture that meets small and medium sized businesses (SMB) needs and spans to the largest global enterprise, offering access to the same features as the world s largest service providers. It offers the ability to start small and grow without painful upgrades down the road. Figure 6. HP 3PAR StoreServ 7000 Storage Arrays HP 3PAR StoreServ 7200 Storage HP 3PAR StoreServ 7400 Storage For supporting this configuration, the HP 3PAR StoreServ node storage array (QR485A) is being incorporated connected to HP M inch 4U SAS Drive Enclosures (QR491A) (Figure 7). Figure 7. HP M6720 Drive Enclosure Drive Shelf M6720 Each drive enclosure includes twenty four (24) drive bays and two (2) IO modules. There is a maximum of eighteen (18) drive enclosures that can be attached to the HP 3PAR StoreServ node array; mixed enclosure combinations are supported (HP ConvergedSystem 700x multi-rack configuration options are available). Drive enclosures are connected in daisy chains from the SAS ports of the controller nodes. The best practice is to balance the drive enclosures across the SAS ports. 9

10 The best practice, when including LFF and SFF drive enclosures in the same array, is to arrange them in the rack so that all 2U enclosures belonging to one node pair are together and all 4U enclosures for that node pair are together. When connecting the backend SAS cables, intermix the M6710 and M6720 enclosures on each SAS port. With a four-node configuration, the best practice is to attach the same number of drive enclosures to each node pair. To preserve proper airflow, all empty drive bays should contain a drive blank. To support the Exchange 2013 storage requirements for this solution, large capacity near line storage hard drives (HP M6720 3TB 6G SAS 7.2K 3.5in NL HDD (QR500A)) are used to provide large compaction, at a more affordable price, taking advantage of Exchange 2013 disk I/O requirements. The four (4) HP M6720 drive enclosures offer the capability to host a total of 96 hard drives. The Exchange 2013 storage requirements are calculated to require 48 disks, which leaves 48 drives to be used for other applications. Supporting the system drive requirements for the ten (10) Hyper-V Exchange VMs allocating 120GB, RAID10 for each of the VMs requires eight (8) HP M GB 6G SAS 15K 2.5in HDD (QR492A). These drives offer the better performance needed to accommodate the server OS workloads. The solution provides a total of 48 SFF hard drives in the HP 3PAR StoreServ node Storage Base unit. This leaves a total of 40 hard disks that can be used for other applications support. HP Networking When designing Hyper-V host servers, there are a number of distinct networks required for the solution to be in accordance with accepted best practices and meet Microsoft Fast Track compliance. Thus, the following networking design is recommended for each server in the HP ConvergedSystem 700x. The HP Virtual Connect FlexFabric 10/24- Port Module reduces required cabling and provides valuable networking flexibility, with profiles that can be designed for the networking needs of each blade. Figure 8 show the HP ConvergedSystem 700x recommended configuration network design and BL460c connectivity configuration. Teamed production (Prod) networks Teamed management (Mgt) networks Live VM migration (LM) network Cluster Shared Volume (CSV) network Redundant FC/FCoE networks for Multipath I/O (MPIO) 10

11 Figure 8. HP ConvergedSystem 700x recommended configuration network layout The HP ConvergedSystem 700x recommended configuration is designed to follow the above recommendations, providing redundant connectivity to each of the major network segments used in a Microsoft Hyper-V deployment. The management and guest (production) networks use NIC teaming for improved redundancy, whereas the live migration and CSV networks are set up to fail over to each other in case of a failed connection. 5920AF-24XG Switches (5920) Redundant 10 Gb switches for the majority of the Ethernet traffic leaving the c7000 enclosure The 5920 switches provide added performance and redundancy with the Intelligent Resilient Framework (IRF), which allows the switches to appear as a single device. For example, in the HP ConvergedSystem 700x recommended configuration, the HP Virtual Connect FlexFabric 10Gb /24 Module in interconnect bay 1 of the c7000 enclosure connects to each 5920 switch. However, with IRF configured between the 5920 switches, they appear to be a single switch to the FlexFabric module, allowing the two connections to be joined in an LACP (802.3ad) NIC team. HP G EI Switch (5120) low-cost 1Gb switch for intelligent PDU management The 5120 switch provides the 1Gb ports needed to manage the HP Intelligent Power Distribution Units (ipdu), discussed in more detail in the following section. These connections are non-critical and do not require full redundancy, hence the costeffective switch used for those connections. 11

12 Power This solution incorporates HP Intelligent rack and power components (Figure 9), allowing for remote management and monitoring of power. The HP ConvergedSystem 700x for Microsoft utilizes the HP Intelligent Power Distribution Units (ipdu) which bring an increased level of precision, control and automation to power distribution. This design allows intelligent power monitoring and management of all servers, including both blades and DL servers. With the HP ipdus, administrators have the ability to get exact power consumption at a core, stick or outlet level, and get precise reporting of the power utilization and consumption requirements of the HP ConvergedSystem 700x recommended configuration environment. Additionally the HP ipdus can be configured to send out SNMP-based alerts to monitor the power consumption for the HP ConvergedSystem 700x from the Microsoft System Center Operations Manager (SCOM) console or from an HP Systems Insight Manager environment for the data center. Figure 9. HP Intelligent Power Distribution HP Intelligent Cable HP Intelligent PDU Display HP Managed Extension Bar HP Intelligent PDU 12

13 For fault tolerance, this HP ConvergedSystem 700x rack provides six 24A HP ipdus that are connected to the customer s utility supply via multiple feeds to sustain the loss of a single power feed in the data center (Figure 10). Components in the rack are connected in a redundant manner, such that devices with multiple power supplies may survive a power failure in one supply. Due to the many power options used per region, further details on the power configuration are left to the customer to design. Figure 10. HP ConvergedSystem 700x Power Configuration 13

14 Software components Management software components The HP ConvergedSystem 700x for Microsoft provides the underlying infrastructure for running multiple workloads requiring high levels of performance and availability. To complete the solution, a software stack that can deliver the management, monitoring, operational control and lifecycle requirements is needed for a virtualized environment. To deliver these capabilities, System Center 2012 with HP Insight Control for System Center software is considered as part of the HP ConvergedSystem 700x for Microsoft solution. However the required licensing for System Center 2012 and Windows Server 2012 Datacenter SP1 is not included as a component part of HP ConvergedSystem 700x for Microsoft. HP customers that have already purchased Enterprise License Agreements (ELA) are covered and do not need to purchase any additional licenses. HP customers requiring licenses may purchase them as an addendum to their HP ConvergedSystem 700x product order. Microsoft System Center 2012 System Center 2012 is a comprehensive management platform consisting of capabilities for infrastructure management, service delivery and automation, and application management and control. The System Center 2012 management stack can be installed on a core set of VMs running on the HP ConvergedSystem 700x management servers (HP ProLiant DL360p Gen8 servers), through which the services are delivered to the business users. The System Center 2012 product includes a number of core components that can be installed including the following. Microsoft System Center Operations Manager (SCOM) Microsoft System Center Operations Manager (SCOM) with Service Pack 1 (SP1) provides a single point of management for servers, devices, operating systems, services and applications to ensure the data center infrastructure is running and to notify administrators in the event a monitored component fails (Figure 11). The basic functionality of SCOM is expandable using Management Packs. The Management Packs are available from Microsoft or other vendors, including HP. The Management Packs provide application or device awareness that integrates into the SCOM product and offers a single point of management to provide increased value for an organization to ensure that applications remain available and are running at the agreed upon service levels. Figure 11. System Center Operations Manager Additional information on Microsoft System Center Operations Manager (SCOM) is available from Microsoft System Center Operations Manager. 14

15 Microsoft System Center Virtual Machine Manager (SCVMM) Microsoft System Center Virtual Machine Manager (SCVMM) with Service Pack 1 (SP1) provides a single point of management for virtualized infrastructure including Hyper-V host and guest servers, network and storage devices, operating systems, services and applications to assist in deploying and managing the virtual infrastructure. SCVMM integrates with SCOM to ensure that the virtual infrastructure is running properly and to notify administrators in the event of a VM component failure. Figure 12. Microsoft System Center Virtual Machine Manager (SCVMM) Additional information on Microsoft System Center Virtual Machine Manager (SCVMM) is available from Microsoft System Center Virtual Machine Manager. HP Insight Control for Microsoft System Center HP Insight Control for Microsoft System Center provides essential infrastructure management, making it easy to deploy, migrate, monitor, control and optimize an IT infrastructure, using a single, simple management console. Within the HP ConvergedSystem 700x, HP Insight Control for Microsoft System Center seamlessly integrates unique HP ProLiant and HP BladeSystem manageability features into System Center Operations Manager, exposing the native capabilities of servers and enclosures to System Center. This increased visibility makes it easier to perform root-cause analysis to the subsystem and component level. For the HP ConvergedSystem 700x, Gen8 servers and the integration of Insight Control with System Center provide the following capabilities: Proactive hardware monitoring, managing and alerting of hardware health and intelligently responding to hardware events on servers running Windows. The HP ProLiant Agentless Management Pack manages the health of ProLiant Gen8 servers without the need for loading OS-based SNMP agents or WBEM providers. The HP SCVMM 2012 Integration Kit provides HP WinPE and production drivers for ProLiant servers to assist with OS deployment via SCVMM The HP ProLiant Updates Catalog can be used by SCVMM 2012 to provide simplified Windows driver and firmware updates via a rotating, automated workflow for Hyper-V clusters. HP Storage Management Pack for Microsoft System Center HP Storage Management Pack for Microsoft System Center provides seamless integration with SCOM 2012 by integrating predefined discovery and state monitoring policies, event processing rules and tasks. This comprehensive integration solution complements HP Insight Control for Microsoft System Center, and allows administrators to proactively streamline 15

16 IT operations and increase systems availability by monitoring ProLiant server environments and HP Storage products through a common console. HP 3PAR Policy Manager Software HP 3PAR Policy Manager Software works to implement customer-configurable remote service access policies. Installed on a customer-provided host, Policy Manager Software provides the customer with ultimate flexibility and control to allow or deny outbound communication or remote service connections to and from an HP 3PAR StoreServ Storage system. Policy Manager also serves as the centralized point for collecting and storing audit logs of all diagnostic data transfers and authorized remote service connections to and from all configured HP 3PAR StoreServ Storage systems. HP 3PAR Policy Manager provides the following benefits: The customer has complete control over policy administration. A centralized policy administration for all HP 3PAR StoreServ Storage systems is provided. A centralized audit log to facilitate security audits is provided. HP 3PAR Policy Manager Software LTU TE400B Microsoft Windows Server 2012 Hyper-V The Windows Server 2012 operating system is installed and preconfigured on all HP ProLiant servers of the HP ConvergedSystem 700x and is scalable for small, medium and large enterprises. HP ConvergedSystem 700x offers the ability to take full advantage of the virtualization capabilities bundled with the Windows Server 2012 operating system through Hyper-V, a Type 1, Microkernalized hypervisor. Hyper-V interacts directly with hardware and is entirely independent of the operating system. In reality, there is not a host operating system; instead, the hypervisor is providing the operating platform for all operating systems that will run in the environment. Some may find this confusing and ask the question, If the Windows 2012 operating system has to be installed to enable Hyper-V, how can it be independent? Before Hyper-V can be installed, the hardware must support virtualization and have Virtualization Technology (VT-x) enabled for Intel in the BIOS. Once the hardware has been prepared for virtualization, it is ready for the hypervisor to be installed. The source files for Microsoft Hyper-V software are on the Windows Server 2012 media. Microsoft s unique Microkernalized hypervisor design of Hyper-V incorporates a shared virtualization stack and a distributed driver model that offers flexibility and security. This Hyper-V architecture allows individual virtual machines to be isolated from others and provides highly durable and stable platforms. The following are some of the major considerations HP customers should be aware of regarding the significant changes and their effects on virtualization using Windows Server Network virtualization Hyper-V virtual switches and virtual network adapters enable the isolation of network traffic on a shared infrastructure without using virtual local area networks (VLANs). Hyper-V Network Virtualization offers the capability to move virtual machines as needed within their virtual infrastructure while preserving their virtual network assignments and transparently integrating these private networks into a pre-existing infrastructure on another site, without changing the underlying IP addresses. Virtual Fibre Channel (FC) support Provisioning direct connections to FC host bus adapter (HBA) and SAN fabric from Hyper-V virtual machines for higher performance and availability. Live Storage Migration Supporting high availability while moving virtual hard disks between hosts. Non Uniformed Memory Access (NUMA) Virtual processors and guest memory are grouped into virtual NUMA nodes, and the virtual machine presents a guest topology based on the underlying physical topology of compute and memory resources. With guest NUMA, the guest workloads can use their knowledge of NUMA and be self-optimized on the basis of this data. NUMA ensures the guest operating system creates the most optimal mapping between virtual and physical resources, which in turn makes sure the guest operating systems and applications have the most efficient execution, best performance and most linear scale. Single Root-IO Virtualization (SR-IOV) Networking Introduces a new high-throughput, low latency networking solution that surfaces a portion (a virtual function) of a physical NIC within a virtual machine and delivers increased performance without sacrificing key mobility features such as live migration. The HP FlexFabric 554FLB and 554 M network adapters, installed in the HP ProLiant BL460c Gen8 server blades in the HP ConvergedSystem 700x, support industry standard SR- IOV. This feature requires firmware, software and OS support. Dynamic memory Allows for expanded memory configuration options. However, the use of dynamic memory is not supported for Microsoft Exchange Server Hyper-V replica Allows asynchronous replication to an additional Hyper-V server for disaster recovery. The use of Hyper-V replica is not an Exchange aware application and is not supported with Microsoft Exchange Server Live migration Expanded to enable migration to and from stand-alone (non-clustered) servers. Storage migration Allows for the movement of virtual hard disks while the virtual machine continues to run. File servers with Server Message Block (SMB) 3.0 Can be used to host the virtual machine storage. Virtual Hard Disk (VHDX) The VHDX format in Windows Server 2012 has been improved to allow expanded capabilities. 16

17 Combined, these new technologies help ensure that virtualized infrastructures support the configuration of large, highperforming virtual machines to sustain enterprise-level workloads that meet the scalability needs of all-sized organizations. Licensing General Windows Server 2012 changes the Microsoft licensing model, especially around Hyper-V virtualization. The Enterprise edition of Windows licensing no longer exists. Table 1, below, identifies the virtualization licensing options available in Windows Server Table 1. Microsoft Windows Server 2012 virtualization-licensing options Windows 2012 Edition Physical Processors Virtualization Instances Windows Features Foundation One No Virtual OS Instances Limited features Essentials Up to two per license No Virtual OS Instances Limited features Standard Up to two per license Two Virtual OS Instances All features Datacenter Up to two per license Unlimited virtual OS instances All features In addition to changes in the licensing of Windows Server 2012, significant advancements in the virtualization capabilities of Windows Server 2012 have led to an increase in the capabilities and use of available resources for Hyper-V. Table 2, below, identifies the improved capabilities of Windows Server 2012 in comparison with Windows Server 2008 R2. Table 2. Hyper-V Capability comparison on Windows Server 2008 R2 and Windows Server 2012 Resource Windows Server 2008 R2 Windows Server 2012 Host Logical Processor Host Physical Memory 1TB 4TB Host Virtual Processors per host 512 2,048 Host Cluster Nodes Host Cluster Virtual Machines 1,000 4,000 Max Virtual Processors per VM 4 64 Max Memory per VM 64GB 1TB Max Virtual Machine Disk Size 2TB 64TB VM Max VHD Size 2TB 64TB with VHDX Windows Server 2012 Cluster-Aware Updating (CAU) Cluster-Aware Updating (CAU) is a new feature available with Windows Server 2012, and when used with Hyper-V, automatically moves all VMs from a cluster node to be updated to another node. The cluster node to be updated is then taken offline and updated. The system is rebooted if needed. Once the updated node is back online, live migration moves the VMs back to the updated cluster node. The process is then repeated for the remainder of the nodes in the failover cluster. This feature ensures that all nodes in the failover cluster are maintained at a common level, to enable the best stability for cluster services. For additional information regarding Cluster-Aware Updating, please see Cluster-Aware Updating Overview. Cluster Shared Volumes (CSV) The Cluster Shared Volumes (CSV) offers an added feature to the failover cluster that enables multiple cluster nodes to concurrently access a volume with complete transparency and without any regards to the cluster node ownership. Windows Server 2012 provides the following improvements to CSV: New CSVFS file system BitLocker Support 17

18 The CSV namespace now enabled by default, identified as CSVFS Support for CSV no longer limited to Hyper-V. CSV now includes support for File Server role, Scale-out server and Microsoft SQL Servers CSV now uses standard mount points Improved backup and restore of the CSV supporting parallel (multiple backups conducted on the same CSV at the same time) and incremental backups CSV can be presented to up to 64 nodes in a Windows Server Cluster Windows Server 2012 BitLocker Windows Server 2012 BitLocker offers volume encryption and protection against data theft, and additional security for the deletion of data on decommissioned systems. BitLocker has been updated in Windows Server 2012 to provide: Option of full volume encryption (slow) or used space encryption (quick) Ability to encrypt Cluster Shared Volumes (CSV) Additional information on BitLocker is available from BitLocker Overview. Offloaded Data Transfer (ODX) Offloaded data transfer (ODX) works with Windows Server 2012 hosts and the HP 3PAR StoreServ 7400 storage to shift storage processing from the server to the storage array for files of 256 KB or larger. This allows the storage array to handle the bulk of the storage operations tasks and reduces CPU and network utilization on the requesting host server. The server initiates an operation such as a file copy, asks the StoreServ array to complete the file copy, and then reports back to the server that the operation has completed. This enables a significant increase in the file copy throughput since the StoreServ is handling the operation. The ASICs in the StoreServ array have zero-detect, speeding file transfers by copying only those data blocks containing data. Windows PowerShell Windows PowerShell is not new; however, it is becoming more prominent in daily administrative tasks and server configurations. PowerShell offers the virtualization administrator the advantage and the flexibility to leverage Hyper-V management capabilities to automate many of the administrative functions through the creation of PowerShell scripts. PowerShell makes it possible to rapidly implement changes to a large number of VMs with a single line of code. Appendix C contains a list of PowerShell scripts and cmdlets that customers can modify and use in their Hyper-V environments. Solution specific Hyper-V configurations This section is dedicated to assisting and educating the virtualization administrator about the installation and configuration details of Hyper-V to support the deployment of a virtual Exchange 2013 configuration. The installation of the Windows Server 2012 operating system and Hyper-V are preconfigured as part of the HP ConvergedSystem 700x for Microsoft solution. An additional benefit, HP ConvergedSystem 700x offers HP customers, is that the system arrives on site installed and preconfigured, freeing them from the complexities associated with architecture and designing of a complete and fully functional virtual computing infrastructure. The HP customer will have the convenience of having all configurations already completed and optimized, ready to join their existing environment. The following section contains some pertinent configuration information regarding this Exchange 2013 recommended configuration that will be completed by an HP Factory Express Level 5 professional technician. The following information is intended as information only, in order to benefit the virtual administrator and Exchange administrator as a technical reference guide identifying key design configurations that will be completed for them. Hyper-V installation Hyper-V hypervisor installation is achieved by adding Hyper-V as a Windows Server 2012 feature using Server Manager. The Add Roles and Features wizard simplifies the installation process and takes care of all role dependencies: 1. In Server Manager, click Manage, and select Add Roles and Features. 2. Verify the Destination Server on the Before You Begin page, and click Next to proceed. 3. On the Installation Type page, select Role-Based Or Feature-Based Installation, and click Next to proceed. 4. From the Server Selection page, select the server from the Server Pool, and then click Next. 5. Select Hyper-V from the Server Roles page and click Next. 6. The Hyper-V Management Tools are automatically selected on the Feature page. Click Next to proceed. 7. The Create Virtual Switches, Virtual Machine Migration and Default Stores pages may be modified to configure the host defaults. 18

19 Installation of Windows Server 2012 Failover Clustering The installation and configuration of Windows Server 2012 Failover Clustering is another preconfigured feature of the HP ConvergedSystem 700x for Microsoft. The installation procedure for Failover Clustering is conducted by adding it as a Windows Server 2012 feature. The Failover Clustering feature can be installed by using Server Manager or command line. Active Directory integration is a key to a stable and successful cluster implementation. Cluster nodes need to be members in the same Active Directory domain. It is recommended the Cluster Name Object (CNO) and cluster machines be placed in their own Organization Unit (OU) within Active Directory to preclude any Group Policies having undesirable effects on the cluster. Use the following procedures to install Failover Clustering. 1. Open the Server Manager dashboard, click Manage, and then click Add Roles and Features. 2. In the Add Roles Features Wizard, on the Before You Begin page, select Next. 3. On the Select Installation Type page, select Role-Based or Feature Based Installation, and then click Next. 4. On the Select Destination Server Page, click Next. 5. On the Select Server Roles page, click Next. 6. On the Select Features page, click Failover Clustering, and then click Next. 7. Click Add Features. 8. On the Confirm Installation Selections page, click Install. 9. On the Results page, click Close. To install Failover Clustering by using Windows PowerShell, run the following Server Manager PowerShell command: Install WindowsFeature Name Failover-Clustering IncludeManagementTools The following are the components of Windows Server 2012 Failover Clustering: Failover Clustering Feature Failover Cluster Management tools that consist of the Failover Cluster Manager and Cluster-Aware Updating UI Failover Cluster modules for PowerShell Failover Cluster Automation Server Failover Cluster command interface Failover Clustering validation Although Windows Server is a preconfigured component of the HP ConvergedSystem 700x for Microsoft, there may be occasions when the Virtualization Administrator (VA) needs to conduct a cluster validation. Microsoft recommends running the validation process prior to forming a cluster and after making any changes to the cluster configuration. Failover Clustering validation should be performed prior to forming a cluster within the Failover Cluster Manager console. This validation will ensure all hardware components function as required and the cluster operates as intended. The process will also ensure network connectivity between the cluster nodes and the connectivity, as well as functionality of the shared storage. The minimum number of nodes to run all tests is two. The cluster validation process can be a valuable diagnostic tool for the virtualization administrator to troubleshoot problems on already operational clusters. Use the following procedures to run validation testing for Failover Clustering: 1. Launch Server Manager. In the navigation pane, click Dashboard, Tools, and then Failover Cluster Manager. 2. In Failover Cluster Manager, click Validate Configuration in the Actions pane. 3. In the Validate A Configuration Wizard, select Next on the Before You Begin page. 4. On the Select Servers or a Cluster page, enter the Fully Qualified Domain Name (FQDN) of your targeted cluster nodes in the Enter Name field. Use semicolons between names to add nodes. 5. On the Testing Options page, click Run All Tests, and then click Next. 6. On the Confirmation page, review the test list, and then click Next. 7. On the SUM page, deselect the Create Cluster Now, and then click View Report. To run cluster validation using PowerShell, run the following Windows Failover Clustering PowerShell command: Test-Cluster Node Host1, Host2, Host3 Note Conducting a validation storage test against online and configured storage will result with a production outage throughout the duration of the test. 19

20 Create failover cluster In the eventuality of a situation requiring the virtualization administrator to reconfigure and recreate the failover cluster, use the following procedures form the failover cluster in Windows Server 2012: 1. Launch Server Manager. In the navigation pane, click Dashboard, Tools, and then Failover Cluster Manager. 2. In Failover Cluster Manager, click Create Cluster in the Actions pane to launch the Create Cluster Wizard. 3. On the Before You Begin page, click Next. 4. On the Select Servers Page, type the FQDN of your targeted cluster nodes in the Enter Name field. Use semicolons between names to add addition nodes, and then click Next. 5. On the Access Point For Administering, type Cluster1 on the Cluster page in the Cluster Name box. 6. In the Networks section, deselect any inappropriate networks that may be used for internal cluster communications, and select only the production-facing networks. Enter an assigned IP address in the appropriate network range, and then click Next. 7. On the Confirmation page, deselect the Add All Eligible Storage to cluster check box. Review the cluster information and click Next. 8. On the Summary page, click Finish. In Failover Cluster Manager, expand the cluster container in the navigation pane, and click Next. Verify the status of the configured cluster hosts. The cluster can be created using PowerShell by running the following command: New-Cluster Name Cluster1 Node Blade1, Blade2, Blade3, Blade4, Blade5, Blade6, Blade7 StaticAddress NoStorage Add cluster storage The following procedures are for provisioning storage to the cluster to accommodate services and applications. They add storage disks to the failover cluster in Windows Server Launch Server Manager. In the navigation pane, click Dashboard, Tools, and then Failover Cluster Manager. 2. In Failover Cluster Manager, expand the cluster container in the navigation pane, and click Storage. Right-click Disks and select Add Disk. 3. In the Add Disk To A Cluster interface, select the appropriate disks that are made available and provisioned from the HP 3PAR StoreServ 7400 storage array. 4. Verify the status of the added disks and that they are assigned to a server node online. Note that these disks are now Physical Disks Resources and not a Cluster Shared Volume. To add an available disk to the cluster by using PowerShell, run the following Windows Failover Clustering PowerShell command: Add-ClusterDisk "Cluster Disk 1" To get a list of available disks, use the following PowerShell command: Get-ClusterAvailableDisks fl Name Hyper-V Failover Clustering This virtual Exchange 2013 configuration is designed to provide high availability capabilities for the Exchange virtual machines through Windows Server 2012 failover clustering and Hyper-V live migration. After the initial installation and preparation of the Hyper-V host servers, the virtualization administrator will need to build the Hyper-V cluster. Hyper-V itself has no clustering capability and relies on the close integration with Windows Failover Clustering. This section provides insight to the VA for the installation and configuration of the Hyper-V cluster. A cluster is a group of servers and associated shared storage that work together as a single system. The individual server nodes within the cluster work as one to distribute workloads. A failure of one or more components within the cluster will result in the automatic recovery of services by other members of the cluster. When installing and configuring the failover clustering on the host operating system, the VA should take into consideration the following to maintain reliability, redundancy and connectivity amount the cluster nodes: The server is configured with more than one NIC or can have a Load Balancing/Failover (LBFO) team of NICs that are connected to multiple networks. Servers are connected to some form of shared storage. This configuration utilizes the storage provided by the HP 3PAR StoreServ node storage array and is made accessible through the HP FlexFabric 554FLB and HP 554M 10 Gbs network adapters by implementing the FCoE capabilities of the adapter and the integrated Windows Server 2012 MPIO features. Multipath I/O provides redundant failover and load balancing for multipath storage devices. 20

21 Two or more independent networks must connect the nodes in a cluster to avoid single point of failure. Cluster configurations of a single network are not recommended. The HP ConvergedSystem 700x for Microsoft is procured with default configurations, the components are shipped with a cluster network configuration, and a FlexFabric Server Profile is preconfigured. Each cluster network must be configured as a single IP subnet whose subnet number is distinct from other cluster networks. All network adapters used to connect server nodes should be configured to use identical settings such as speed, duplex mode, flow control, media type and hardware offloads. The storage array must support the SCSI-3 SPC3-compliant SCSI command for persistent reservation and release. The cluster validation process checks to ensure compatibility. The HP 3PAR StoreServ 7400 is fully SCSI-3 SPC3 compliant. All host bus adapters must use the storport/miniport driver model provided by Microsoft in order to be listed in the Windows Server Catalog. Clustered Shared Volumes The Exchange Server 2013 solution benefits considerably from the associated enhancements and capabilities offered through the application of Cluster Shared Volumes (CSV). Cluster Shared Volumes offers flexibility and reduces some of the complexity associated with Exchange Server high availability and database availability groups (DAGs). Exchange Server 2013 DAG integration is dependent on continuous replication, which automates the installation and configuration of Windows Server 2012 Failover Clustering. CSVs reduce more complexity of configuring and provisioning storage than previous methods used with cluster configuration. Additionally, a requirement for Exchange Server 2013 DAG configuration is that the absolute file path for database copies must be identical and consistent on DAG member servers. The CSV s universal name space is enabled by default and makes the DAG database configuration easier to manage and consistent. CSVs can make storage management easier for the VM, and complexities and allocation of storage are transparent to application administrators who just need to identify their requirements to the VM. CSV enables multiple nodes within the cluster, concurrent access and shared volumes with complete transparency regarding the cluster node ownership. CSV is used within the Exchange 2013 configuration and is depended upon to achieve the flexibility needed for the storage solution. CSV requires one of the nodes in the cluster to take responsibility for synchronizing access to the files on the shared volume. This node holds ownership of the cluster s physical disk resources associated with its LUN and is referred to as the coordinator node. Each LUN has its own coordinator node, and all nodes in the cluster are considered equal and capable of being a coordinator node. When the virtual machine is deployed and running on a CSV, all of the hypervisors access to the virtual hard disk is disk I/O, and the coordinator node is not involved in the process. This enables the virtual machine to have fast, direct access to the disk. The metadata changes to the volume are controlled and managed by the coordinator node. Changes in metadata from any node within the cluster are redirected over the network to the coordinator node. CSV is compatible with any block-level shared storage, which includes Fibre Channel, iscsi, and Serial Attached SCSI (SAS). CSV utilizes the Windows NTFS file system on basic disks formatted either with Master Boot Record (MBR) or with GUID Partition Table (GPT). CSV is being used in single shared LUNs or multi-site clusters where nodes in the same cluster reside in different subnets. CSV can take advantage of SMB3 features such as multichannel and RDMA. Only NTFS partitions can be used for CSV, and Client for Microsoft Network and File and Printer Sharing need to be enabled. To add a cluster disk to the CSV namespace, follow the following procedures: 1. Launch Failover Cluster Manager, expand the cluster node container, click the Storage node and then click the disk node. 2. Select a physical disk resource in the Disk view. Right-click and choose Add to Cluster Shared Volume. To verify the CSV configuration, run the following PowerShell command: Get-CLusterSharedVolume "Cluster Disk Name" fl CSV incorporates a single, consistent namespace across all the nodes in a Windows failover cluster. The files on the CSV have the same name and the same relative file path regardless of which node those files are viewed from. CSV volumes are exposed as directories and subdirectories under the %SystemDrive%\ClusterStorage directory. This also enables CSV to scale to large numbers of volumes without the limitation of running out of drive-letter identifiers. Each node in the cluster maps the identical namespace to each shared volume: C:\ClusterStorage\ExchageVM\ C:\ClusterStorage\Exchange Data\ 21

22 In Windows Server 2012, CSV volumes are standard mount points. When configuring Exchange Server 2013 databases that are members of a DAG, the relative file path and name of the database copies must be the same on both Exchange virtual servers. Using CSV Volumes for Exchange requires careful consideration for the Exchange file structure on the CSV for DAG implementation. The following are recommended best practices when implementing CSV in a Windows failover cluster: Defragment virtual hard disks before placing them on CSV storage. Distribute logical disk ownership across the nodes in the cluster. Route non-hyper-v I/O through the CSV owner (coordinator node). Distribute logical disk ownership to prevent any cluster hot spots. Implement at least one CSV volume per host to provide the best balance. Hyper-V live migration One of the important capabilities of this Exchange 2013 virtual configuration is the flexibility and agility of the supporting infrastructure. Hyper-V abstracts servers and the services away from the hardware by converting these machines into files and memory. These files are easy to be moved and memory copied. These capabilities are the foundation for live migration and presently make Windows Server 2012 Hyper-V one of the most flexible and agile virtualization hypervisors on the market today. Live migration was first introduced with Hyper-V Windows Server 2008 R2 and could move a virtual machine from one Hyper-V host cluster to another host in that cluster with no downtime to service availability. However, this earlier version of live migration had the following limitations: Could conduct only one live migration of a VM on a host at a time. Live migration could not take advantage of 10GbE or faster network capabilities. Did not offer zero-downtime for moving files and virtual hard disks. Live migration in Windows Server 2012 offers a number of enhancements that have dramatically expanded the flexibility of managing business application in a virtual cloud. The following are the enhancements capabilities and characteristics of live migration in Windows Server 2012: Live Storage Migration Live Storage Migration allows the relocation of virtual hard disks to another storage location while access to the disk continues during the migration process. This feature can be used whenever you want to keep a virtual machine running and need to perform maintenance on the underlying storage devices that the VM is using. The feature operates in a similar manner to VM live migration, since disk access continues throughout the storage migration process. This feature can place a large amount of data on the network during the migration process, so proper design of the storage network is necessary to ensure there is no contention for network resources. Note Microsoft Hotfix KB must be applied to enable support of ODX for Live Storage Migration of Cluster Shared Volumes (CSV) on the same array. The hotfix is available from: Live migration does not require a cluster Live migration can be used to move online VMs between clustered hosts, nonclustered hosts or a combination of both. Support virtual machines stored on shared folder Live migration can be used on VMs that are stored on Windows Server 2012 file shares. Shared-Nothing live migration Live migration does not require shared storage, such as a SAN or a shared folder. Bandwidth allocation Windows Server 2012 can better utilize large capacity networks such as 10GbE or 56Gbps InfiniBand. This becomes useful for live migration of VM copies, synchronizes memory of a VM over the network, and a large host can be drained more quickly (up to 4TB). Unlimited live migration Windows Server 2012 can perform multiple live migrations at once. There is not an arbitrary number of live migrations that can be conducted at one time; the limitations are dependent on processor workload and the capabilities of the Live Migration Network. There must be at least one (1) Gbps of bandwidth between the source and destination hosts. Additionally, the source and destination hosts must be from the same manufacturer and have the same instruction set. 22

23 The following are procedures to configure shared storage to support Live Storage Migration: 1. On one of the nodes of your cluster, launch Windows PowerShell. 2. Use the following PowerShell command to create a shared folder and set the correct share and NTFS permissions: MD '<LOCAL_DRIVE_PATH>\VM-Store' New-SmbShare -Name VM-Store -Path "<LOCAL_DRIVE_PATH>\VM-Store"' -FullAccess domain\administrator, \host1$, domain\host2$, domain\host3$(get-smbshare -Name VM-Store).PresetPathAcl Set-Acl Initiating Live Storage Migration Follow the following procedure to initiate a Live Storage Migration between different storage repositories: 1. In Hyper-V Manager, select the appropriate VM in the Virtual Machines list. In the Actions pane, click Move. 2. In the Move Machine Wizard, click Next on the Before You Begin page. 3. On the Choose Move Type page, select Move The Virtual Machine's Storage. Then click Next. 4. On the Choose Options For Moving Storage page, leave the default Move All Of The Virtual Machine's Data To A Single Location option selected, and click Next. 5. On the Choose A New Location For Virtual Machine page, type \\Server Name\VM-Store\VM Name in the Folder box, click Next, and then click Finish. To initiate Live Storage Migration by using PowerShell, run the following Hyper-V PowerShell command: Move-VMStorage -VMName Exch-VM1 -DestinationStoragePath "\\Server Name\VM-Store\ Exch-VM1" Capacity and sizing HP hardware and Exchange Server 2013 recommended configuration This section summarizes the details for the example recommended configuration of Exchange Server 2013 in a virtualized deployment to address the planning considerations for optimal performance and efficient server utilization. The configuration will accommodate a 10,000-user enterprise organization with an average mailbox size of 2GB. The individual user profile is based on a client workload of 200 messages sent and received per user each day, with an average message size of 75KB, producing the estimated workload of 0.13 IOPS per user on system storage. This recommended configuration includes high availability features with the adaptation of a two copy DAG designed to tolerate the loss of a single Exchange VM, and leverages the associated hardware of the HP ConvergedSystem 700x in a single rack design option. This configuration is based on an HP ConvergedSystem 700x procured with a fully provisioned HP BladeSystem c7000 Platinum blade enclosure, composed of 16 HP ProLiant BL460c Gen8 server blades, and an HP 3PAR StoreServ node storage array with four (4) HP M6720 LFF Drive Enclosures (see Figure 1). The configuration has been designed on the assumption that this HP ConvergedSystem 700x solution has been procured for operation in a facility that has already established a supporting Active Directory and DNS infrastructure. Establishing an efficient virtualization environment with Microsoft Hyper-V requires careful planning considerations for consolidating CPU, memory and network resources. To distribute the Exchange 2013 infrastructure to support 10,000 users, this recommended configuration incorporates ten (10) Exchange 2013 virtual machines hosted on 10 of the 16 ProLiant BL460c server blades within the c7000 enclosures (see Figure 2). Each of the 10 Exchange 2013 VMs will be configured to operate on one ProLiant BL460c server blade, and is assigned 8 processor cores of the 20 available on each server blade and 64GB of memory out of the 256GB memory available on each server blade. This configuration design provides a relatively simple Exchange 2013 topology to manage, while still offering the benefits associated with virtualization and high availability. Each VM utilizes only a portion of the overall CPU and memory resources on each server blade, leaving a sufficient amount of resources available (12 processor cores, 192GB RAM per server) to be allocated to any other application running on the HP ConvergedSystem 700x. The Exchange 2013 VMs will utilize an estimated 31% of processor resources and 15% of the total memory resources available in the HP ConvergedSystem 700x. Exchange solution capacity calculations and sizing This section identifies the calculations for determining hardware requirements to support the Exchange 2013 design for this recommended configuration. This Exchange 2013 solution design has been derived as a representation of an enterprise organization deploying Exchange 2013 in a Hyper-V virtual cloud environment hosted on the HP ConvergedSystem 700x for Microsoft. The Exchange 2013 configuration will accommodate 10,000 users with an average mailbox size of 2GB. The average Exchange user profile is determined to be a heavy Exchange user based on the generation of 200 messages a day 23

24 with an average message size of 75KB, generating an estimated workload of 0.13 IOPS on system storage. The solution incorporates high availability features for a two copy DAG with the tolerance of the loss of a single Exchange server. The solution is built on the assumption that the HP ConvergedSystem 700x has been procured with a fully provisioned HP BladeSystem c7000 Platinum blade enclosure composed of 16 HP ProLiant BL460c Gen8 server blades, an HP 3PAR StoreServ node storage array and 4 HP M6720 LFF drive enclosures. The solution utilizes an HP ConvergedSystem 700x solution that is assumed to be in an established data center supporting Active Directory infrastructure. The following is a detailed Exchange 2013 configuration layout and design, based on the results from the Exchange 2013 Server Role Requirements Calculator and the HP Sizer for Microsoft Exchange Server Table 3. Exchange database layout and design Configuration Parameters Values Total Number of users 10,000 Total Number of Exchange Servers 10 (Mailbox, CAS) 1 Exchange VM per blade Total users per database 111 Total number of active databases 90 Total number of databases per DAG 180 Number of databases per server Exchange Database Size Exchange log files 18 (9 active / 9 passive) 255GB 16GB Number database per LUN 1 Size of LUN 393GB Number of LUNs per server 18 Recovery Volume 1 Size of Recovery Volume Total Exchange storage per server Total Exchange Size 285GB 7,071GB 70,776GB The implementation of Microsoft Exchange Server 2013 requires increased demand on server CPU and memory resources than in previous versions of Exchange Server. The increase in resource requirements is largely due to changes in the Exchange 2013 supporting architecture. Customers can expect an increase of about 35% in memory requirements, in comparison to calculations for server requirements for Exchange Server Table 4, below, identifies the server requirements recommended to support this configuration, based on the results from the Exchange 2013 Server Role Requirements Calculator. Table 4. Exchange Server requirements Server Resource Processor Utilization Requirement 8 cores Processor Available Adjusted Megacycles 20,444 Server CPU Megacycle Requirements Memory Requirements Server System Drive Requirements 15,665 at 77 % processor utilization 64GB 79GB Min Required, 120GB Recommended 24

25 Note These calculations include virtual server hypervisor overhead. CPU sizing Each of the 10 virtual Exchange servers in this solution requires 8 cores on an Intel Xeon E v2 processor to accommodate the anticipated 10,000 users concurrent workload at an estimated 77% CPU workload. The CPU sizing for Exchange 2013 is based on the SPECInt 2006 values. These values can be obtained through the Microsoft Exchange Processor Query Tool or the Standard Performance Evaluation Corporation (SPEC) website. The Intel E v2 processor is a default option for the BL460c Gen8 servers in the HP ConvergedSystem 700x for Microsoft solution and contains 20 cores and a SPECInt 2006 score of 864. The Exchange 2013 Server Role Requirements Calculator reveals that distributing the 10,000 heavy profile users across 10 Exchange 2013 virtual machines, each utilizing 8 of 20 processor cores on the Intel E v2 CPU in a worse case failover scenario, would be at an estimated 77 % processor utilization. For planning purposes, it is recommended to plan for an 80% processor utilization and a Hyper-V hypervisor overhead of 10%. For this configuration solution, Hyper-Threading is not enabled on the Hyper-V host servers, based on Microsoft s Exchange 2013 performance recommendations. Hyper-Threading will improve CPU throughput for most applications; however, for Microsoft Exchange Server 2013 the benefits of Hyper-Threading do not outweigh the negative impacts to memory utilization on Exchange servers with Hyper-Threading enabled. This is attributed to the way the.net server garbage collector allocates heaps. The server garbage collector looks at the total number of logical processors when an application starts up and allocates a heap per logical processor. This results in memory utilization during the startup of services that utilize the server garbage collector to be close to double with Hyper-Threading enabled then it is with Hyper-Threading disabled. Additionally, the virtual CPUs that Hyper-Threading provides should not be considered an equivalent to the hardware physical cores of the CPU. With Hyper-Threading enabled, results can be misleading the 20 cores of the E v2 processors may be considered 40 as interpreted by the Windows Server 2012 operating system. The virtual CPUs provided by Hyper-Threading are not equivalent to the physical cores on the CPUs. Storage sizing The storage design for this recommended configuration was based on the results from the Microsoft Exchange Server Role Requirements Calculator and the HP Sizer for Microsoft Exchange Server Both of these tools provide detailed solutions based on Microsoft s best practice recommendations for Microsoft Exchange Server 2013, as outlined in article Sizing Exchange 2013 Deployments. Due to the nature of virtualization in environments with Microsoft Hyper-V, this solution requires unique considerations for provisioning storage from the HP 3PAR StoreServ 7400 Storage array. Each Exchange VM running requires access to VHDX stored on Cluster Shared Volumes (CSV). The Exchange VM accesses the VHDX disk through the Hyper-V host on the storage provisioned on the HP 3PAR StoreServ 7400 by the virtualization administrator. There are multiple storage options available for an Exchange 2013 VM running on an HP BL460c host. In this configuration, the following method is simplistic and efficient. In Figure 13, below, the virtual storage configuration for this Exchange solution is depicted and shows the distribution of Hyper-V host server nodes between clusters: cluster 1 (blades 1 through 5) and cluster 2 (blades 9 through 13). Each of the BL460c blades is configured as a Windows Server 2012 Hyper-V host server and is configured to support one Exchange 2013 VM server. 25

26 Figure 13. Virtual storage layout In each of the two clusters, all 5 cluster nodes running as Hyper-V hosts are provisioned storage from the HP 3PAR StoreServ 7400 Storage array for two cluster shared volumes (CSV). One CSV will be provisioned to support the virtual operating systems for all Exchange VMs in the cluster. This configuration recommends that the virtual machines be numbered one through ten. Odd numbered VMs are assigned to cluster 1 and even numbered VMs are assigned to cluster 2. This is beneficial for distributing database copies. The physical disks provisioned for this CSV on the HP 3PAR StoreServ 7400 should be assigned to a Common Provisioning Group (CPG) consisting of HP M GB 6G SAS 15K 2.5in Hard Drives and configured to use RAID 5. The other CSV will be provisioned for Exchange 2013 data storage. The physical disks provisioned for Exchange data should be assigned to a CPG consisting of HP M6720 3TB 6G SAS 7.2K 3.5in NL Hard Drives and configured for RAID 10. The Hyper-V host servers will present the 2 CSVs to the Exchange VM by configuring Hyper-V virtual storage and to the Exchange VM in the form of VHDX volumes. These CSV volumes will be accessible to all members of the cluster and will use a common CSV Namespace consistent across all nodes in the cluster used to configure the Exchange 2013 objects. Table 5, below, shows the storage requirements to support this configuration, based on the results from the Exchange 2013 Server Role Requirements Calculator. Table 5. HP 3PAR StoreServ 7400 Storage Requirements Designated Use Disk Type Configuration Details VM Operating System Exchange Server Data HP M GB 6G SAS 15K 2.5in Hard Drives HP M6720 3TB 6G SAS 7.2K 3.5in NL Hard Drives Min Number of Disks: 8 RAID 5 Capacity 1,200GB 120GB per VM / 79GB Min 600GB Per Volume (Cluster) 1.2 TB Total Min Number of Disks: 48 RAID 10 Capacity TB Storage Required Per Server 7,076GB (18 DB 9 Active / 9 Passive) Storage Required Per Volume (Cluster) 35.35TB / TB Total 26

27 A key consideration regarding this configuration is the distribution of the Exchange databases between all of the 10 Exchange VMs within the configuration. This configuration offers multiple levels of high availability incorporating Windows Server Failover Clustering by distributing the servers between two clusters for VM fault tolerance. The Exchange 2013 databases offer high availability capabilities with the implementation of a two copy DAG. Preventing the possibility of an active and passive copy of the databases being assigned to the same physical node of the cluster, and presenting a risk of losing all copies of a database, requires careful planning for database distribution. In the following example, we address how the distribution of databases and the complexities associated with Exchange virtualization can have significant impacts and require careful considerations. Figure 14 below, is an example of the distribution of Exchange databases that would prevent two database copies of the same database being moved to the same cluster node. The 90 active copies of the Exchange databases are distributed evenly between all nodes in both clusters (45 databases per cluster). The 90 passive copies of the Exchange databases are also distributed between all nodes in both clusters (45 databases per cluster), being careful not to have two database copies assigned to a node within the same cluster. In this configuration, all odd numbered VMs are members of cluster 1 and even numbers in cluster 2. The Exchange databases are distributed evenly, carefully assigning each database copy to an Exchange VM. In this example, Database 1 (active) is assigned to EXCH-VM1. Database 1 (passive) cannot be assigned to the same cluster node, so Database 1(passive) is created on EXCH-VM2 in cluster 2. Since both VMs are configured on nodes in a different cluster, it is impossible for both to be inadvertently assigned to the same node. This database layout is similar for all 90 databases; Database 2 (active) is created on EXCH-VM2, and Database 2 (passive) is created on EXCH-VM3, and so on. The resulting database distribution depicted in Figure 14 may seem to offer an evenly distributed and well-organized solution. Conveniently, for administrative management and organization purposes, all odd numbered primary database copies have been assigned to cluster 1, and all odd numbered passive copies are assigned to cluster 2. The reverse is true for the even numbered databases: even numbered active copies of the databases are assigned to cluster 2, and passive copies are assigned to cluster 1. Additionally, the last digit of the number of an active database copy correlates to the Exchange virtual machine assigned. Figure 14. Exchange Database example 1 Note that there is a serious design flaw within the database layout depicted in Figure 14 that is detrimental in the event of a server failure even though each Exchange virtual machine has an evenly distributed number of active (9) and passive (9) databases as specified in the Exchange 2013 Server Role Requirements Calculator. DAG design high availability requirements cannot safely be met, having the tolerance of a single server failure. The reason is, in the event of a server 27

28 failure, as depicted in Figure 14 of EXCH-VM1, the results of failure will cause all of the active copies (green) and passive copies (blue) to go offline. The DAG will then activate the database passive copies on EXCH-VM2, and the continuous replication of the passive copies of databases on EXCH-VM1 with the active databases on EXCH-VM10 will fail. All databases activated on EXCH-VM2 will result in the virtual machine CPU and memory resources being over-provisioned, as shown in Figure 14. Even though resources were provisioned based on the recommendations of the Exchange 2013 Server Role Requirements Calculator, there are not enough CPU and memory resources available for a DAG with 18 database copies on one server. This is because the Exchange 2013 Server Role Requirements Calculator accounts for a server failure in a DAG by evenly dividing the server workload between the remaining members of the DAG. The database distribution shown in Figure 14, results in the active databases on EXCH-VM1 being replicated to EXCH-VM2 instead of being distributed between all other members in the DAG. This is the case for every Exchange virtual machine and results in a total of 1,998 users distributed between 18 databases. Placing this information into the Exchange 2013 Server Role Requirements Calculator yields the result of 115% of CPU utilization at 10 cores and 128 GB of RAM. Figure 15 presents an improved distribution of the Exchange databases that mitigates the server overload issue. Figure 15. Exchange 2013 Database Distribution example 2 28 In Figure 15 above, the active database copies are still distributed evenly between the Exchange virtual machines, as explained in Example 1. However, the distributions of passive copies are carefully laid out so that they are evenly distributed between all DAG members in the opposing cluster of the active databases. Figure 15 is a depiction of the failure of EXCH- VM1 and, as in the Example 1 scenario, the active database copies go offline and continuous replication of the passive copies fails. However, in Figure 15, because of the better distribution of the database copies between members of the DAG, the workload on EXCH-VM2 is significantly reduced, and only 11 of the 18 databases are active. In this example, the server failure results in 1,221 users distributed across 11 databases. Placing this information into the Exchange 2013 Server Role Requirements Calculator yields the result of 70% of CPU utilization at 6 cores and 64 GB of RAM. The comparisons of these two examples show a significant reduction in the amount of resources consumed for a single server failure, and the importance of thorough planning for deploying Exchange 2013 in a virtual environment with the HP ConvergedSystem 700x. Exchange network configuration For the Exchange 2013 configuration to function properly, specific networking requirements must be met for each member of the DAG (Figure 16). The DAG consists of a single MAPI network, which is used by all DAG members to communicate with other servers (Exchange 2013 servers or AD servers) or mailbox users, and one Replication network dedicated to log shipping and seeding. Additionally, this configuration is provisioned for a single 1Gbs heartbeat maintenance network to

29 facilitate the Windows Server 2012 failover clustering on the Exchange VM supporting Exchange Continuous Replication. The networks are provisioned using Virtual Switches configured on the Hyper-V host servers and presented to the Exchange VMs. To implement this Exchange configuration, the DAG should have a minimum of two networks, one MAPI and one Replication network. This provides redundancy for the network and the network path and enables the system to distinguish between a server failure and a network failure. In the event of a MAPI network failure, a server failover will occur. In the event of a Replication network failure, if available, the MAPI network will be used for log shipping, and seeding operations. When Replication network connectivity is restored, it will need to be manually switched back to the Replication network. The following recommendations should be incorporated into the DAG design: Suspend and resume operations to avoid the brief outage caused by restarting the Microsoft Exchange Replication service. Each DAG member must have the same number of networks configured, consistent on all members of the DAG. Each DAG must have no more than one MAPI network. The MAPI network must provide connectivity to other Exchange servers and other services, such as Active Directory and DNS. Additional Replication networks can be added, or network adapter teaming configured, to prevent single point of failure. Each network in each DAG member server must be on its own network subnet. Each server in the DAG can be on a different subnet, but the MAPI and Replication networks must be routable and provide connectivity. Each member of the DAG must have a round trip network latency no greater than 500 milliseconds between each other member of the DAG. For environments with relatively high latency, values may require significant tuning of DAG and replication; consider increasing the number of databases or decreasing the number of mailboxes per database, to improve replication. Evaluate the total overall network workload, including client access, Active Directory, transport, continuous replication and other application traffic to determine the necessary network requirements for each environment. DAG networks support Internet Protocol version 4 (IPv4) and IPv6. IPv6 is supported only with IPv4 configured. Automatic Private IP Addressing (APIPA) (including manually assigned addresses from the APIPA address range) is not supported for use by DAGs or by Exchange Figure 16. Exchange Server 2013 virtual network infrastructure 29