Redpaper. IBM Reference Configuration for Microsoft Private Cloud: Deployment Guide. Front cover. ibm.com/redbooks

Transcription

1 Front cover IBM Reference Configuration for Microsoft Private Cloud: Deployment Guide Understand the components of the IBM Reference Configuration Configure an approved Microsoft Private Cloud Fast Track solution Follow preferred practices to create the solution Eric Johnson Peter Kisich Marco Rengan Scott Smith William Watson Michael Wilcox ibm.com/redbooks Redpaper

2

3 International Technical Support Organization IBM Reference Configuration for Microsoft Private Cloud: Deployment Guide December 2011 REDP

4 Preparation for use: Before using this information and the product it supports, read the information in Notices on page v. First Edition (December 2011) This edition applies to the IBM Reference Configuration for Microsoft Private Cloud. This document created or updated on December 22, Copyright International Business Machines Corporation All rights reserved. Note to U.S. Government Users Restricted Rights -- Use, duplication or disclosure restricted by GSA ADP Schedule Contract with IBM Corp.

5 Contents Notices v Trademarks vi Preface vii The team who wrote this paper vii Contacts viii Now you can become a published author, too! viii Comments welcome ix Stay connected to IBM Redbooks ix Chapter 1. Introduction Chapter 2. Architecture and component overview Architecture overview Component overview for the IBM Reference Configuration architecture Rack and power for IBM Reference Configuration architecture Chapter 3. Host compute and production nodes IBM System x3650 M Compute/production node cluster Management node for System Center Virtual Machine Manager IBM System x3550 M Active Directory server Chapter 4. Storage for the IBM Reference Configuration architecture Chapter 5. Networking, connectivity, and scaling out Networking overview Connectivity for the IBM Reference Configuration architecture Scale-out methodology Chapter 6. Operating system and management software Operating system software Windows Server Virtual machine guests with Hyper-V Private cloud operational software Management software IBM Hardware Management Pack for Microsoft System Center Operations Manager IBM System Storage (XIV) Management Pack for Microsoft System Center Operations Manager System Director agent Appendix A. System x rack and power options Appendix B. Parts lists Compute/production node parts list Management server parts list Active Directory server parts list Copyright IBM Corp All rights reserved. iii

6 Appendix C. Ordering options for the IBM XIV Storage System Capacity on demand (CoD) Sample XIV system parts lists Appendix D. Network ordering information Appendix E. Host operating software ordering and parts Windows Server 2008 R2 Datacenter Edition 2-CPU Management server parts list for Windows Server 2008 R2 Enterprise 10 CAL Active Directory server parts list for Windows Server 2008 R2 Standard Ordering information for additional Microsoft components Software ordering summary Related publications IBM Redbooks Online resources Help from IBM iv IBM Reference Configuration for Microsoft Private Cloud: Deployment Guide

7 Notices This information was developed for products and services offered in the U.S.A. IBM may not offer the products, services, or features discussed in this document in other countries. Consult your local IBM representative for information on the products and services currently available in your area. Any reference to an IBM product, program, or service is not intended to state or imply that only that IBM product, program, or service may be used. Any functionally equivalent product, program, or service that does not infringe any IBM intellectual property right may be used instead. However, it is the user's responsibility to evaluate and verify the operation of any non-ibm product, program, or service. IBM may have patents or pending patent applications covering subject matter described in this document. The furnishing of this document does not give you any license to these patents. You can send license inquiries, in writing, to: IBM Director of Licensing, IBM Corporation, North Castle Drive, Armonk, NY U.S.A. The following paragraph does not apply to the United Kingdom or any other country where such provisions are inconsistent with local law: INTERNATIONAL BUSINESS MACHINES CORPORATION PROVIDES THIS PUBLICATION "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESS OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF NON-INFRINGEMENT, MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. Some states do not allow disclaimer of express or implied warranties in certain transactions, therefore, this statement may not apply to you. This information could include technical inaccuracies or typographical errors. Changes are periodically made to the information herein; these changes will be incorporated in new editions of the publication. IBM may make improvements and/or changes in the product(s) and/or the program(s) described in this publication at any time without notice. Any references in this information to non-ibm websites are provided for convenience only and do not in any manner serve as an endorsement of those websites. The materials at those websites are not part of the materials for this IBM product and use of those websites is at your own risk. IBM may use or distribute any of the information you supply in any way it believes appropriate without incurring any obligation to you. Information concerning non-ibm products was obtained from the suppliers of those products, their published announcements or other publicly available sources. IBM has not tested those products and cannot confirm the accuracy of performance, compatibility or any other claims related to non-ibm products. Questions on the capabilities of non-ibm products should be addressed to the suppliers of those products. This information contains examples of data and reports used in daily business operations. To illustrate them as completely as possible, the examples include the names of individuals, companies, brands, and products. All of these names are fictitious and any similarity to the names and addresses used by an actual business enterprise is entirely coincidental. COPYRIGHT LICENSE: This information contains sample application programs in source language, which illustrate programming techniques on various operating platforms. You may copy, modify, and distribute these sample programs in any form without payment to IBM, for the purposes of developing, using, marketing or distributing application programs conforming to the application programming interface for the operating platform for which the sample programs are written. These examples have not been thoroughly tested under all conditions. IBM, therefore, cannot guarantee or imply reliability, serviceability, or function of these programs. Copyright IBM Corp All rights reserved. v

8 Trademarks IBM, the IBM logo, and ibm.com are trademarks or registered trademarks of International Business Machines Corporation in the United States, other countries, or both. These and other IBM trademarked terms are marked on their first occurrence in this information with the appropriate symbol ( or ), indicating US registered or common law trademarks owned by IBM at the time this information was published. Such trademarks may also be registered or common law trademarks in other countries. A current list of IBM trademarks is available on the Web at The following terms are trademarks of the International Business Machines Corporation in the United States, other countries, or both: BladeCenter BNT IBM Redbooks Redpapers Redbooks (logo) System x XIV The following terms are trademarks of other companies: Intel Xeon, Intel, Intel logo, Intel Inside logo, and Intel Centrino logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States and other countries. Microsoft, Windows, and the Windows logo are trademarks of Microsoft Corporation in the United States, other countries, or both. Other company, product, or service names may be trademarks or service marks of others. vi IBM Reference Configuration for Microsoft Private Cloud: Deployment Guide

9 Preface The IBM Reference Configuration for Microsoft Private Cloud is a comprehensive, ready-to-use, easy-to-manage, high-performance, and ultra-reliable computing solution using Microsoft Hyper-V. The offering consists of IBM approved and tested servers, storage, and networking components which provide best-of-breed technology from entry level to the largest private cloud infrastructures. This IBM Redpapers publication provides an overview of the Reference Configuration and a detailed parts list for IT teams responsible for integrating and deploying a private cloud infrastructure. It describes the process of setup, installation, and configuration to enable quick and easy deployments for a private cloud in a data center. This offering features an eight-node Microsoft Private Cloud Fast Track solution based on IBM System x3650 M3 servers with the IBM XIV Storage System. This paper is intended for clients who want to learn about the IBM reference architecture for the Microsoft Private Cloud Fast Track program and better understand how all of the components work together to form an end-to-end private cloud solution. Sample ordering configurations and ordering considerations are addressed. This paper is a partner to IBM Reference Configuration for Microsoft Private Cloud: Implementation Guide, REDP The team who wrote this paper This paper was written by a team of specialists from around the world. Eric Johnson Peter Kisich Marco Rengan Scott Smith William Watson Michael Wilcox Thanks to the following people for their contributions to this project: Karen Lawrence Linda Robinson David Watts IBM Redbooks David Hartman Jim Meyer IBM Bryan Hoke Steven Tong Brocade Copyright IBM Corp All rights reserved. vii

10 Special thanks to Avanade Inc. and especially Pat Cimprich for their contributions to the creation of this paper. Avanade spent many hours evaluating reference configuration components, running performance tests, assessing component sizings, and discovering opportunities for automation and integration within the Microsoft System Center Suite. Avanade is an IBM Business Partner and provides integration and customization services for cloud and virtualization projects. Contacts For more information or to place an order, contact your IBM sales or IBM Business Partner representative. Avanade is an IBM Business Partner and provides integration and customization services for cloud and virtualization projects. Its website is here: For additional information, send an to Avanade directly at to get started. Now you can become a published author, too! Here s an opportunity to spotlight your skills, grow your career, and become a published author all at the same time! Join an ITSO residency project and help write a book in your area of expertise, while honing your experience using leading-edge technologies. Your efforts will help to increase product acceptance and customer satisfaction, as you expand your network of technical contacts and relationships. Residencies run from two to six weeks in length, and you can participate either in person or as a remote resident working from your home base. Find out more about the residency program, browse the residency index, and apply online at: ibm.com/redbooks/residencies.html viii IBM Reference Configuration for Microsoft Private Cloud: Deployment Guide

11 Comments welcome Your comments are important to us! We want our papers to be as helpful as possible. Send us your comments about this paper or other IBM Redbooks publications in one of the following ways: Use the online Contact us review Redbooks form found at: ibm.com/redbooks Send your comments in an to: Mail your comments to: IBM Corporation, International Technical Support Organization Dept. HYTD Mail Station P South Road Poughkeepsie, NY Stay connected to IBM Redbooks Find us on Facebook: Follow us on Twitter: Look for us on LinkedIn: Explore new Redbooks publications, residencies, and workshops with the IBM Redbooks weekly newsletter: Stay current on recent Redbooks publications with RSS Feeds: Preface ix

12 x IBM Reference Configuration for Microsoft Private Cloud: Deployment Guide

13 1 Chapter 1. Introduction IBM offers a powerful yet affordable cloud computing platform that can be rapidly deployed into multiple cloud environments for a variety of on-demand use cases and applications. The components incorporated into the IBM Reference Configuration for Microsoft Private Cloud architecture use leading-edge hardware technology, including advanced servers and storage from IBM along with Brocade and IBM networking products. This hardware, coupled with the latest Microsoft Self-Service Portal and Microsoft Hyper-V software versions, continues the tradition of delivering simple, easy-to-use solutions, incorporating the IBM System Director platform to provide a single pane of glass (SPOG) for setup, provisioning, management, and administration. The IBM Reference Configuration architecture combines high-quality IBM System x servers and best-of-breed networking hardware with robust tools and integrated software to create an innovative, comprehensive, and simplified cloud-computing offering. IBM reference architecture for Hyper-V Cloud Fast Track maximizes performance per square foot of data center space, while lowering total cost of ownership for proposed cloud solutions. This ensures that both mid-market businesses and enterprise data centers can reap cloud-computing benefits without a large IT staff or initial investment barriers. The IBM Reference Configuration architecture is a pre-tested, end-to-end cloud solution that is ready to deploy. The following technologies are tested by IBM and approved as part of the Microsoft Private Cloud Fast Track program: IBM System x rack servers, combining high performance with reliability and including advanced predictive failure technology Microsoft Private Cloud Fast Track software, providing the latest Microsoft virtualization and cloud management software IBM Systems Director Upward Integration Modules (UIMs) and Performance and Resource Optimization Tips (PRO Tips), adding enhanced management when using Microsoft System Center Brocade top-of-rack switches and adapters, adding converged connectivity to storage and networking IBM XIV Storage System automated configuration, data protection, and upgradeability, providing greatly simplified management with tier one storage features and performance Copyright IBM Corp All rights reserved. 1

14 The IBM Reference Configuration architecture extends the inherent benefits of cloud infrastructures by further reducing cost of hardware, power, cooling, and space requirements while enhancing operational efficiencies. Important considerations such as capacity needs can be predefined within the integrated management tools, simplifying planning of the private cloud and VM infrastructure. Microsoft Self-Service Portal software is key to this rapid and automated VM deployment capability. Microsoft Hyper-V virtualization technology and flexible licensing helps the IBM Reference Configuration architecture to take advantage of the cost savings of virtualization through Windows Server 2008 R2 SP1 with Hyper-V. Features including Live Migration and Cluster Shared Volumes provide the features dynamic virtual infrastructures require to maximize performance and efficiency in the data center. 2 IBM Reference Configuration for Microsoft Private Cloud: Deployment Guide

15 2 Chapter 2. Architecture and component overview This chapter highlights the architecture of the IBM Reference Configuration for Microsoft Private Cloud solution. It includes the following sections: Architecture overview Component overview for the IBM Reference Configuration architecture Rack and power for IBM Reference Configuration architecture Copyright IBM Corp All rights reserved. 3

16 2.1 Architecture overview The IBM Reference Configuration for Microsoft Private Cloud offers an architecture that can address the data and compute needs of data centers ranging from the small branch office to the highly scalable enterprise. Each selected configuration is a predefined, robust solution which provides necessary compute and application resources available quickly and efficiently. Each IBM cloud reference configuration is a tested and Microsoft Private Cloud Fast Track certified architecture, an end-to-end solution combining all components configured in a modular, extensible, flexible, and scalable rack. This solution provides dynamically flexible compute resources and ease of deployment, including administration, support, expansion, and, most importantly, affordability. To support changing business needs, cloud solutions must have the ability to dynamically add or reduce resources such as memory and disk storage capacity. The IBM Reference Configuration architecture supports dynamic scaling of compute resources. Combined with the Microsoft Self-Service Portal, requests can be created autonomously. The following key components enable this flexibility: IBM System x3650 M3 management and host servers and a x3550 M3 active directory server are optimized to meet the virtualization management and advanced workload demands of private cloud data centers. The x3650 M3 servers are 2U servers and can be substituted with the x3550 M3 1U server to provide more density per rack. Brocade 10 Gbps converged networking components form the backbone for data and network connectivity. The combination of high-performance adapters and low-latency cut-through switches enables the high-speed infrastructure that is critical for resource utilization and load balancing within the cloud. The IBM XIV Storage System offers linear scalability, simplified configuration and management, high performance, and automation. The structural elements of the offering consist of a set of functional groups, as discussed in this paper. The IBM reference architecture for Microsoft Private Cloud Fast Track structural components has been selected primarily to contribute a combination of performance, reliability, and value. As consolidation of compute resources increases, reliability and high availability are paramount when offering cloud services. These components provide the up time and fault tolerance essential to any successful cloud deployment. Along with reliability, these components have been selected for businesses of all sizes, from mid-market to large enterprises. IBM is uniquely positioned to provide leading-edge technologies at a mainstream price point for the wide range of IBM Reference Configuration architecture deployments. 4 IBM Reference Configuration for Microsoft Private Cloud: Deployment Guide

17 2.2 Component overview for the IBM Reference Configuration architecture The IBM Reference Configuration uses Microsoft software, IBM servers and storage, and Brocade networking hardware, including the IBM System x3650 M3 for server production and management nodes, IBM System x3550 M3 for Active Directory server, and the XIV system. As shown in Figure 2-1, the following hardware building blocks make up the IBM Cloud Reference: Networking components Management nodes Compute/production nodes Storage system The software stack includes the following components: Microsoft Windows Server 2008 R2 SP1 Microsoft System Center Virtual Machine Manager (SCVMM) 2008 R2 SP1 Microsoft System Center Operations Manager (SCOM) 2007 R2 SP1 SQL Server 2008 x64 SP1 Standard Edition Microsoft Self Service Portal 2.0 (SSP) 2.0 Forefront Security Client IBM System Director Platform Agent IBM System Director System Center UIM IBM B32 10 Gb Converged Switch IBM B24Y 1 Gb Switch IBM x3550 M3 Active Dir IBM x3650 M3 - Management IBM x3650 M3 - Production IBM XIV Storage Platform Figure 2-1 Major components of the IBM Reference Configuration for Microsoft Private Cloud Fast Track Chapter 2. Architecture and component overview 5

18 Table 2-1 outlines the configuration required for IBM Reference for Microsoft Private Cloud. Table 2-1 IBM cloud reference configuration summary Feature Host server family Active Directory server IBM Reference Configuration Components x3650 M3 x3550 M3 (Optional) Host CPU sockets/server 2 Number of Management hosts 2 Number of Production hosts 8 Number of production cluster cores 96 SAN family XIV Range of VMs (estimated) Rack and power for IBM Reference Configuration architecture IBM rack and power infrastructure offerings are custom designed for IBM System x servers and provide the following benefits: Improved data center efficiency Increased power efficiency Increased space efficiency (avoiding over-design) Lower cost through better data center utilization Improved IT availability Improved uptime Opportunity to act before downtime impacts business Matching utilization, power resources, and capacity planning To provide availability demanded in consolidated virtual infrastructures, data centers require rack and power components that securely manage and control power resources, servers, and appliances, maintain the highest levels of IT availability, and drive operational efficiencies. IBM offers many such rack and power options: Three new 1200 mm deep equipment racks, including 47U tall racks and 42U dynamic (ship-loadable) racks. An IBM lineup of optional uninterruptible power supply (UPS) units, including rack-mounted and tower units supporting voltages and configurations not previously available, with new 1500, 2200, 3000, and 6000 volt-ampere (VA) units. A new line of 0U strip power distribution units (PDUs) designed for tool-less installation in the new racks. These PDUs have 24 outlets for today s server-dense rack installations. New Local and Global Console Managers that support unique cabling options ( conversion options ) to enable chaining up to 1024 managed devices from a single console. For cabling, power, and rack ordering information, see Appendix A, System x rack and power options on page IBM Reference Configuration for Microsoft Private Cloud: Deployment Guide

19 3 Chapter 3. Host compute and production nodes The IBM System x3650 M3 rack servers address current cloud business requirements while delivering the scalability to manage future growth. Key capabilities of these servers include performance, reliability, and simplified management. The compute/production nodes are the processing elements for the IBM Reference Configuration for Microsoft Private Cloud. Virtualization technology in the Hyper-V hypervisor compute node resources emulate dedicated infrastructure even though it is shared among other processes, applications and business functions. Copyright IBM Corp All rights reserved. 7

20 3.1 IBM System x3650 M3 Built on the latest six-core Intel Xeon processor 5600 series, the IBM System x3650 M3 two-socket server offers a highly available, high-performance 2U platform for Windows Server 2008 R2 with Hyper-V. With integrated 6 Gbps RAID adapters and double the I/O performance, the x3650 M3 provides a resilient architecture that is well-suited for virtualization environments. The Intel Xeon processor 5600 series expands the benefits of virtualization with innovations that boost performance, increase consolidation ratios, and enable servers of separate generations to be combined in the same virtualized server pool. The result offers more flexibility and a robust infrastructure with improved load-balancing, virtual machine failover, and disaster recovery capabilities. IBM System x3650 M3 servers help provide a strong foundation platform for virtualized environments with large memory and I/O capacity to satisfy the high-speed processing and reliability requirements of running software in a virtualized cloud environment. The following list provides a quick summary of the x3650 M3 features: Innovative, energy-smart 2U design that helps to lower operational costs. Up to two Intel Xeon 5600 series processors. Maximum of 18 DIMMs of DDR3 memory for up to 288 GB of memory. One 6 Gbps SAS RAID adapter. Support for four PCIe x8 2.0 adapter slots. An optional embedded hypervisor for managing virtual workloads is also supported. Up to 16 hot-swap HDDs with an internal storage capacity of 8.0 TB (using 2.5 inch hot-swap SAS or SATA drives. Alternatively, up to 16 solid-state drives (SSDs) are also available, offering up to 800 GB of storage. Enhanced systems management with an integrated management module and Unified Extensible Firmware Interface (UEFI), and software tools including ToolsCenter, IBM Systems Director, and Active Energy Manager. Best-in-Class RAS (highly redundant design, predictive failure analysis (PFA), light path diagnostics). Open architecture and common tools for quick and simple installation. Better serviceability with a highly scalable, flexible design that is easy to deploy, integrate, service, and manage. 8 IBM Reference Configuration for Microsoft Private Cloud: Deployment Guide

21 Figure 3-1 shows the front view of the IBM system x3650 M3 server. Power Button with sliding cover Video port USB 2.0 ports Pop-out light path diagnostics panel Hot-swap 2.5 HDDs Optical drive Status lights Light path diagnostics panel release Figure 3-1 The IBM System x3650 M3 Figure 3-2 shows the rear view of the x3650 M3 server. PCI Express adapter slots Hot-swap power supplies Ethernet port for IMM Gb Ethernet ports Optional 3rd and 4th Gb Ethernet ports Video port Serial port Status LEDs USB ports Figure 3-2 System x3650 M3 rear view For additional details about the x3650 M3, refer to the IBM System x3650 M3 Product Guide, available at the following address: Chapter 3. Host compute and production nodes 9

22 3.2 Compute/production node cluster The compute/production node cluster uses System x3650 M3 servers. The configuration uses eight nodes. Each node in the cluster uses the following components: IBM System x3650 M3 Two Intel Xeon Processor X5670 6C 2.93 GHz 12 MB Cache 1333 MHz 95w 96 GB RAM (made up of twelve 8 GB DIMMs) One ServeRAID M1015 SAS/SATA Controller (battery not included) Two IBM 73 GB 15K 6 Gbps SAS hot-swap disk drives Two Brocade 10 Gb Dual Port Converged Network Adapters The compute/production node cluster components can be ordered using the parts list in Appendix B, Parts lists on page Management node for System Center Virtual Machine Manager The management node for this configuration consists of a pair of IBM System x3650 M3 rack servers. The management node architecture provides redundancy and fault tolerance and is an essential component in managing the cloud and virtualization. It manages the complete IBM private cloud environment. This management cluster runs the following Microsoft applications: Windows Server 2008 R2 SP1 Enterprise Center System Center Operations Manager (SCOM) 2007 R2 SQL Server 2008 x64 Standard Edition System Center Self-Service Portal 2.0 SP1 System Center Forefront Security Client Of particular note is Microsoft System Center Virtual Machine Manager (SCVMM), which manages the overall virtualized infrastructure. It enables increased physical server utilization, centralized management, and rapid provisioning of new VMs. In addition, there is no need to replace current VMware investments. Microsoft SCVMM helps manage the Hyper-V, virtual server, and VMware environments, all from a centralized platform. This is the essence of single pane of glass (SPOG) management: A key ingredient to the IBM ease-of-use feature that simplifies management of the data center. Microsoft System Center Virtual Machine Manager supports: Live migration Clustered shared volumes Hot addition and removal of storage Host Maintenance mode Storage migration Support for third-party cluster file systems For Management Node Ordering Information, see Appendix B, Parts lists on page IBM Reference Configuration for Microsoft Private Cloud: Deployment Guide

23 3.4 IBM System x3550 M3 Built on the latest six-core Intel Xeon processor 5600 series, the IBM System x3550 M3 rack-mounted two-socket server offers a highly available, high-performance 1U platform for Windows Server 2008 R2. With integrated 6 Gbps RAID adapters and double the I/O performance, the x3550 M3 provides a resilient architecture. The x3550 M3 uses significantly less power than previous generations, with unified systems management tools, leadership reliability, availability, and serviceability features, and broad systems flexibility, housed in a compact 1U mechanical package. The following list provides a quick summary of the x3550 M3 features: Innovative, energy-smart 1U design helps to lower operational costs. Up to two Intel Xeon 5600 series processors. A maximum of 18 DIMMs of DDR3 memory for up to 288 GB of memory. Chipkill, ECC, and memory mirroring are also supported. One 6 Gbps SAS RAID adapter. Support for PCIe 2.0 x16 adapter slots. An optional embedded hypervisor for managing virtual workloads is also supported. Up to eight hot-swap HDDs with an internal storage capacity of up to 8.0 TB (using 2.5 inch hot-swap SAS or SATA drives. Alternatively, up to eight solid-state drives (SSDs) are also available, offering up to 400 GB of storage. Enhanced systems management with an integrated management module and Unified Extensible Firmware Interface (UEFI), and software tools including ToolsCenter, IBM Systems Director, and Active Energy Manager. Best-in-Class RAS (highly redundant design, predictive failure analysis (PFA), light path diagnostics). Open architecture and common tools for quick and simple installation. Better serviceability with a highly scalable, flexible design that is easy to deploy, integrate, service, and manage. Figure 3-3 shows the front view of the IBM system x3550 M3 server. Four hot-swap 2.5" bays Four hot-swap 2.5" bays or one DVD bay Two USB 2.0 ports Video port Pop-out light path diagnostics panel with power button and status LEDs Light path diagnostics panel eject button Figure 3-3 The IBM System x3550 M3 Chapter 3. Host compute and production nodes 11

24 Figure 3-4 shows the rear view of the x3550 M3 server. Ethernet port for IMM PCIe slot Video port PCIe slot Power supply socket Two optional Gb Ethernet ports Standard Gb Ethernet ports Serial port USB ports Hot-swap power supply bays Figure 3-4 System x3550 M3 rear view For additional details about the x3550 M3, refer to the IBM System x3550 M3 Product Guide, available at the following address: Active Directory server This optional server provides an Active Directory and DNS environment for the IBM Reference Configuration servers, allowing the servers to participate in a Microsoft cluster and to work with several pieces of the management solution. If an Active Directory server is already available in the environment, this server is not required. The Active Directory servers can be virtualized or stand-alone servers, depending on requirements. If required, each Active Directory server in the IBM Reference Configuration architecture is configured with the following specifications: IBM System x3550 M3, as shown in Figure 3-3 on page 11 Intel Xeon X5640 processors, 4C 2.66 GHz 12 MB Cache 1066 MHz CPUs 6 GB RAM (configured as one 4 GB DIMM) ServeRAID M1015 SAS/SATA controller Two IBM 146 GB 10k 6 Gbps SAS 2.5 inch SFF Slim-HS disk drives For Active Directory server parts list provides parts ordering information see Appendix B, Parts lists on page IBM Reference Configuration for Microsoft Private Cloud: Deployment Guide

25 4 Chapter 4. Storage for the IBM Reference Configuration architecture The IBM XIV Storage System used in the IBM Reference Configuration for Microsoft Private Cloud offers performance, automation, reliability, and flexibility. The XIV system includes the following key features: Continuous, predictable high performance without traditional complex tuning requirements No single point of failure Industry-leading rebuild times in the event of disk or module failures (less than 60 minutes for 2 TB drives) All-inclusive enterprise software including: Innovative space efficient snapshots Asynchronous and synchronous replication Thin provisioning Reporting and management Data migration Quality of Service (QoS) Non-disruptive maintenance and upgrades Scalability from the IBM XIV Storage System Gen2 model to the IBM XIV Storage System Gen3 model, which can provide up to four times the throughput (10 GBps) and up to three times better response time matching the demands of high transactional workloads Completely automated configuration, ensuring every disk is balanced for optimal performance Unparalleled ease-of-use through the innovative XIV system GUI Option for future upgradeability of the XIV system Gen3 model 114 to solid-state drive (SSD) caching for breakthrough SSD performance levels at a fraction of typical SSD storage costs (planned availability for Gen3 in the first half of 2012) Copyright IBM Corp All rights reserved. 13

26 The XIV system is shown in Figure 4-1. Host FC XIV Module Host iscsi XIV Module Host XIV Module InfiniBand Connectivity XIV Module Figure 4-1 IBM XIV Storage System 14 IBM Reference Configuration for Microsoft Private Cloud: Deployment Guide

27 The XIV system offers a dramatic reduction in operational storage expense while also providing low capital expense for cloud infrastructures. Through innovative automation and a simple GUI (shown in Figure 4-2), XIV system storage is so easy to manage that only a minimal storage skill level is needed. Figure 4-2 XIV system GUI overview and status Chapter 4. Storage for the IBM Reference Configuration architecture 15

28 The XIV system GUI can create multiple LUNs in one step as shown in Figure 4-3. All LUNs are automatically added to the storage configuration. Figure 4-3 Creating LUNs using the XIV system GUI The XIV system Gen3 model currently ships either with 7200 RPM 2 TB or 3 TB Serial Attached SCSI (SAS) disks in a highly scalable configuration which accommodates usable capacities of up to 243 TB. At a minimum, the storage system ships with six modules, each of which contains 12 spindles, for a total of 72 disks. For ordering information about the XIV system, see Appendix C, Ordering options for the IBM XIV Storage System on page 37. The storage subsystem used by the IBM Reference Configuration architecture needs to support the two-node management cluster and the Hyper-V compute/production cluster (eight-node cluster). Storage I/O can vary widely depending on the applications being run. It is a good practice to profile storage needs to ensure that adequate space and I/O capacity are available while meeting any bandwidth or latency requirements. Clustered Hyper-V servers can use storage in several ways: The primary use of storage is to present LUNs (disks) to the host servers through a Fibre Channel SAN. All VM configuration and virtual hard disk (VHD) files use storage on these shared LUNs. Clustered Shared Volumes (CSV) is the technology available to clustered Hyper-V servers to allow simultaneous access to LUNs from all servers in the cluster for the purpose of hosting VM files. CSV can greatly simplify the storage configuration by providing a common namespace for all Hyper-V servers in a cluster. VHD files are the virtual hard disks used by VMs to contain the operating system and data files under typical circumstances. Use of virtual hard disks allows maximum flexibility in terms of VM migration and backup options. These VHD files reside on an XIV system LUN that has been made available to the host server. Certain circumstances might require a VM to have direct access to a storage LUN. Examples of this are the use of guest clustering in the VM directly and disk I/O-intensive 16 IBM Reference Configuration for Microsoft Private Cloud: Deployment Guide

29 applications. Two approaches to this are the use of iscsi initiator software in Windows to connect to iscsi targets presented by the XIV system or the use of a pass-through disk that is passed from the host to the VM. Future versions of Hyper-V will allow the use of Fibre Channel versus ISCSI. Currently, clustered highly available (HA) virtual machines (VMs) can only be implemented on iscsi if using direct access I/O. Chapter 4. Storage for the IBM Reference Configuration architecture 17

30 18 IBM Reference Configuration for Microsoft Private Cloud: Deployment Guide

31 5 Chapter 5. Networking, connectivity, and scaling out This chapter highlights networking and connectivity of the IBM Reference Configuration architecture. It also addresses a scale-out methodology for the architecture. This chapter includes the following sections: Networking overview Connectivity for the IBM Reference Configuration architecture Scale-out methodology Copyright IBM Corp All rights reserved. 19

32 5.1 Networking overview Using a solution built around high-bandwidth 10 Gbps converged networking leverages the latest advances in networking and storage architectures. The core of the high bandwidth converged backbone is a pair of IBM B32 Converged Network top-of-rack switches. These B32 switches connect to the servers through Brocade Converged Network Adapters (CNAs), which provide four 10 Gb ports per adapter to build teamed fault-tolerant, high-performance storage and communication networks. As an option, a pair of IBM B24Y switches can be connected to the B32 switches and used for ISCSI connectivity to the IBM XIV Storage System. The IBM Converged Switch B32 enables access to LAN and SAN environments over a common server connection by utilizing Data Center Bridging (DCB) and Fibre Channel over Ethernet (FCoE) protocols. The B32 uses full Fibre Channel Forwarding (FCF) capabilities and connects to servers through Brocade Converged Network Adapters (CNAs). The switch offers Gbps DCB Ethernet ports which can transport both FCoE and regular Ethernet traffic. FCoE can be forwarded into a FC SAN network through the eight 8 Gbps FC ports on the B32. The consolidation of server SAN and LAN ports and corresponding cables simplifies configuration and cabling in server cabinets and reduces acquisition costs. With fewer components using power and cooling, organizations can save significant operating costs as well. For the IBM Reference Configuration for Cloud, the 10 Gbps DCB Ethernet ports provide connectivity to the storage and hosts and can also provide uplinks for LAN aggregation. The B32 Fibre Channel ports are used to connect to the IBM XIV system. Figure 5-1 shows the IBM B32 Converged Switch Figure 5-1 IBM B32 Converged Network Switch The IBM B32 switch provides outstanding performance with eight Fibre Channel ports concurrently active at 8 Gbps and 24 Converged Enhanced Ethernet (CEE) ports concurrently active at 10 Gbps link speeds. The switch offers the following features: Twenty-four 1 Gbps SFP+ ports plus eight 8 Gbps Fibre Channel uplink ports Converged Enhanced Ethernet ports capable of transporting both FCoE and traditional Ethernet protocols. High-performance, 304 Gbps line speed, full-duplex throughput with low latencies Dual redundant, hot-swappable power supplies and highly available fan unit Mature and consistent IronWare OS found across all IBM b-type system networking devices, providing a common, industry-standard command line interface (CLI) Upwards integration into Systems Director Network Control with inventory and monitoring capabilities 20 IBM Reference Configuration for Microsoft Private Cloud: Deployment Guide

33 Figure 5-2 shows the IBM Ethernet switch B24Y. Figure 5-2 IBM Ethernet switch B24Y The optional IBM B24Y Ethernet switch is a compact, high-performance, highly available switch providing both 1 Gb and 10 Gb connectivity. Features of the switch include: Twenty-four 1 GbE RJ-45 ports plus four 10 GbE SFP+ uplink ports Full Layer 2 and Layer 3 capabilities with jumbo frame support High-performance, 128 Gbps line speed, full-duplex throughput with low latencies Dual redundant, hot-swappable power supplies and highly available fan unit Mature and consistent IronWare OS found across all IBM b-type system networking devices, providing a common, industry-standard command line interface (CLI) Upwards integration into Systems Director Network Control with inventory and monitoring capabilities Two dual-port 10 GbE CNAs, which act as NICs, are used to provide a highly available, redundant communications pathway and to physically separate iscsi storage traffic from other traffic. Figure 5-3 shows the Brocade dual-port GbE CNA. Figure 5-3 Brocade dual-port 10 GbE CNA for IBM System x Chapter 5. Networking, connectivity, and scaling out 21

34 The Brocade CNA includes the following features: Eight PCIe Gen 2 host interface ports Industry-leading 500k IOPS performance per port NIC teaming and multi-vlan capabilities integrated in Windows NDIS stack Full Data Center Bridging (DCB) and Fibre Channel over Ethernet (FCoE) capabilities Jumbo frame support TCP Segmentation Offload (TSO), Large Send Offload (LSO) v1/v2, Header Data Split (HDS), and Receive Side Scaling (RSS) offload capabilities 5.2 Connectivity for the IBM Reference Configuration architecture This section focuses on the connectivity of the key components for the IBM Reference Configuration architecture. These components include: Compute/production node Management node Storage subsystem Networking The building block architecture of the IBM Reference Configuration for Microsoft Private Cloud Fast Track is facilitated by the use of a converged network that provides 10 Gb server connectivity and FCoE for SAN connectivity. A pair of redundant, high-speed IBM B32 10 Gb Converged Networking Switches create the backbone of this network. Each server contains two, two-port converged network adapters per server, providing a total of four 10 Gb ports. SAN traffic uses a FCoE multipath configuration to access storage, while fault tolerant network teams support the data network needs on isolated VLANs. 22 IBM Reference Configuration for Microsoft Private Cloud: Deployment Guide

35 Figure 5-4 shows a high-level illustration of the network topology. Zones 1&2 Zones 3&4 VLAN 20 VLAN 30 VLAN 40 VLAN 50 VLAN 60 VLAN 70 VLAN 80 VLAN 90 FC Storage FC Storage iscsi Storage Production Cluster Private / CSV Production Cluster Private / Live Migration Production Cluster Public / VM Communication Management Management Cluster Private Management Cluster Private Live Migration SQL Cluster Private IBM B32 Switch IBM B32 Switch The XIV storage is connected to the FC ports on the B32. The hosts connect to the 10 GB FCoE ports. The h osts use MPIO on the storage side, an d a FT teams of 10 Gbe devices carved into the required VLANs for data communication. VLAN IBM System x3650 M3 Windows Server 2008 R2 SP1 Datacenter running Hyper-V 1 FT Pair 10 Gb CNA FC Zones 1&2 vswitch team 1 FT Pair 10 Gb CNA FC Zone 3&4 VLAN 30 (Prod Cluster Priv) VLAN 40 (Prod Cluster LM) VLAN 60 (Mgmt Network) IMM Port to VLAN 100 Out of Band Management - IMM IBM System x3550 M3 2 1 GbE Management IMM Port to VLAN IBM System x3650 M3 Windows Server 2008 R2 SP1 Enterprise running Hyper-V SQL Server 2008 System Center Ops Mgr 2007 R2 System Center VMM 2008 R2 Self Service Portal FT Pair 10 Gb CNA FC Zone 1&2 vswitch team 1 FT Pair 10 Gb CNA FC Zones 3&4 VLAN 60 (Mgmt Cluster Public) VLAN 70 (Mgmt Cluster Private) VLAN 80 (Mgmt Cluster Live Migration) IMM Port to VLAN 100 IBM XIV Two stacked B24Y2C with 10 Gb uplink modules for 1 GB connections (iscsi, in band and out of ban d management) 10 GbE uplink are connected to B32 switches. Figure 5-4 High-level IBM Reference Configuration network topology For network ordering information, see Appendix D, Network ordering information on page Scale-out methodology The IBM Reference Configuration architecture accommodates growth. This growth most typically takes one or more of these forms: Using a pooled resource model for CPU, memory, and storage, VMs can be added quickly as new business needs arise. Examples are new business applications or departments. Additional servers or storage can be brought online to increase pool reserves as needed. In addition, isolating specific hardware elements from a group of users helps maximize and prioritize hardware utilization and minimize costs. Depending on the starting point, the scale-out approach uses one of the following approaches: Adding additional x3560 M3 servers to the production cluster (up to 16). Additional networking switches might also be needed to accommodate additional servers and prevent networking bottlenecks. Adding additional storage using additional drives or expansion trays in the XIV system. Chapter 5. Networking, connectivity, and scaling out 23

36 In either case, the architecture allows for the same management servers to administer the additional host clusters that are added while providing the required security or performance driven by customer or business needs. Scaling out of production clusters adds additional considerations. In general, when starting with Intel-based servers, continue with the Intel processor-based server family. Regarding the physical attributes of the rack and the number of ports in the top-of-rack switch, scaling out requires additional cables, switches, racks, and rack components when their initial capacity is exceeded. 24 IBM Reference Configuration for Microsoft Private Cloud: Deployment Guide

37 6 Chapter 6. Operating system and management software This chapter examines the software involved in the IBM Reference Configuration for Microsoft Private Cloud. It includes the following sections: Operating system software Management software Copyright IBM Corp All rights reserved. 25

38 6.1 Operating system software The host servers are one of the critical components of a dynamic, virtual infrastructure. The host servers, running Windows Server 2008 R2 SP1 with Hyper-V technology, provide the foundation for running VM guests and also provide the management interface between the guests and Microsoft System Center Virtual Machine Manager (SCVMM). This section describes a detailed host server design and sizing methodology and presents a set of reference server architectures. The reference server architectures are intended to be a starting point for the design process and provide a foundation for documenting the final design. In the IBM Reference Configuration for Microsoft Private Cloud, the server infrastructure is distributed among three groups: Compute/production (host) node Management node Directory node (Active Directory) The underlying operating system for each of these nodes is Microsoft Windows Server 2008 R2 SP1. The preferred edition (Standard Edition, Enterprise Edition, or Datacenter Edition) depends on which type of node is under consideration. Because the compute/production node hosts the virtual guests that reflect the users, a multiple guest license is often preferred. Because the Management node is the control point for the compute/production node, the best license depends on how many compute/production node servers (and CPUs) are in the environment. Finally, because the Directory node works fairly independently, Windows Standard Edition is suitable. The software components used in the IBM Reference Configuration architecture are as follows: Windows Server 2008 R2 SP1 with Hyper-V (x64) Microsoft System Center Virtual Machine Manager (SCVMM) 2008 R2 SP1 System Center Operations Manager (SCOM) 2007 R2 SQL Server 2008 x64 Standard Edition System Center Self-Service Portal 2.0 SP1 System Center Forefront Security Client IBM Systems Director Platform Agent IBM System Director Upward Integration Module and PROTip IBM XIV Storage System SCOM Management Pack Windows Server 2008 As mentioned, the various Editions (licenses) of Windows Server 2008 allow support for a varying number of users and provide optimum scalability of the infrastructure. There are three groupings of the OS, each of which allows for a specific number of Windows-based VMs: Standard edition: Includes one running VM Enterprise Edition: Includes up to four VMs Datacenter Edition: Includes unlimited virtualization rights Each of these groupings provides a tiered level of virtualization user rights. Windows clustering requires either Enterprise or Datacenter editions. The Enterprise Edition with four included licenses allows up to four Windows guest VMs and is a good choice for the management servers. 26 IBM Reference Configuration for Microsoft Private Cloud: Deployment Guide

39 The license option suitable for the Production Server in the IBM Reference Configuration architecture is Windows Server 2008 R2 SP1 Datacenter Edition because it includes unlimited virtualization use rights and allows as many guests as are needed on the physical server Virtual machine guests with Hyper-V Hyper-V allows a wide range of resources to be assigned to guest virtual machines to greatly increases the scalability and maximize the number of workloads that can be run on a given host: Up to 64 GB or RAM and 4 vcpu per VM Up to eight vcpu for each logical processor Fixed and dynamic disks available for each VM Dynamic memory allows a range of memory to be assigned to each VM and allows Hyper-V to balance the allocation per VM based on VM priority and server workload In general, each guest should be configured with no more than the resources needed. This nominal guidance applies to most resource allocation scenarios to ensure equitable distribution for other guests and future growth Private cloud operational software Microsoft System Center is the central operational and control element that completes the cloud and virtualization management. Microsoft Systems Center has two key components: Microsoft System Center Virtual Machine Manager (SCVMM) System Center Operations Manager (SCOM) For detailed guidance on how to install and configure System Center Virtual Machine Manager 2008 R2, refer to this source: For host server software ordering and parts lists, see Appendix E, Host operating software ordering and parts on page Management software The core management environment formed by Microsoft System Center Operations Manager (SCOM) and IBM integration packs is tightly linked with Microsoft System Center Virtual Machine Manager (SCVMM) to monitor and manage the health and performance of the physical and the virtual environment. Microsoft Self Service Portal 2.0 (SSP) extends Microsoft SCVMM to provide self-service provisioning for end customers for their allocated virtual resources. In addition, Microsoft System Center Opalis Server can be used to automate key IT tasks, and System Center Data Protection Manager can provide backup services to the platform. Chapter 6. Operating system and management software 27

40 6.2.1 IBM Hardware Management Pack for Microsoft System Center Operations Manager 2007 The IBM Hardware Management Pack for Microsoft System Center Operations Manager (MSCOM) 2007 is an upward integration module that plugs into Microsoft SCOM Server 2007 R2. The IBM Hardware Performance and Resource Optimization Kit is an integration module that plugs into Microsoft Virtual Machine Manager 2008 R2. Using these management packs enables enhanced features for Microsoft System Center Operations Manager and System Center Virtual Machine Manager to manage the health of the IBM hardware offering. The IBM Hardware Management Pack includes these key features: Extensive monitoring of the health of hardware components for IBM System x servers and BladeCenter x86/x64 blade servers running Microsoft Windows Comprehensive monitoring of the health of software stacks for managing IBM hardware Easy determination of system health problems by aggregating monitors for hardware health Detection of hardware errors and evacuation of virtual machines from affected nodes. To download the latest version of this software, visit the following URL (IBM ID required; follow the link on the page to register) and select the UIM as shown in Figure 6-1: ector_uim&s_tact=sms&=en_us&cp=utf-8 Figure 6-1 Downloading the IBM Hardware Management Pack IBM System Storage (XIV) Management Pack for Microsoft System Center Operations Manager 2007 The IBM Storage Management Pack for Microsoft System Center Operations Manager (SCOM) is a set of software plug-ins that can be individually imported into SCOM to provide storage-specific extensive monitoring and diagnostic capabilities. It includes management packs for a handful of IBM enterprise-class storage solutions, including the IBM XIV Storage System Gen3 model. Thus, IT administrators can customize Microsoft System Center environments by importing management packs that apply only to active data center storage systems. SCOM then uses various authentication protocols to communicate with the corresponding devices. 28 IBM Reference Configuration for Microsoft Private Cloud: Deployment Guide