Reference Architecture Cisco Solution for EMC VSPEX Server Virtualization Microsoft Hyper-V for 50 Virtual Machines Enabled by Cisco Unified Computing System, Cisco Nexus Switches, Microsoft Hyper-V, EMC VNXe3100, and EMC Next-Generation Backup EMC VSPEX April 2012
Copyright 2012 EMC Corporation. All rights reserved. Published in the USA. Published April, 2012 EMC believes the information in this publication is accurate as of its publication date. The information is subject to change without notice. The information in this publication is provided as is. EMC Corporation makes no representations or warranties of any kind with respect to the information in this publication, and specifically disclaims implied warranties of merchantability or fitness for a particular purpose. Use, copying, and distribution of any EMC software described in this publication requires an applicable software license. EMC2, EMC, and the EMC logo are registered trademarks or trademarks of EMC Corporation in the United States and other countries. All other trademarks used herein are the property of their respective owners. For the most up-to-date regulatory document for your product line, go to the technical documentation and advisories section on the EMC online support website. Part Number: H10709 Cisco Solution for VSPEX Server Virtualization 2
Table of contents Reference architecture overview...... 5 Document purpose...5 Solution purpose...5 The business challenge...5 The technology solution...6 Solution benefits...6 Key components......... 7 Introduction...7 Cisco UCS C-Series Servers...7 Cisco Nexus 3000 series Switches...7 Microsoft Hyper-V...7 Microsoft System Center Virtual Machine Manager...8 EMC Storage Integrator...8 EMC VNXe Series...8 Software suites available...8 Software packs available...9 EMC next-generation backup solution...9 Solution architecture......... 10 Logical architecture diagram... 10 Reference architecture overview... 10 Hardware resources... 11 Software resources... 12 Server and Network architecture...... 14 Topology... 14 Server details... 14 Network Layout... 14 Storage architecture......... 15 Storage layout... 15 Storage layout overview... 15 High availability and failover...... 16 Introduction... 16 Storage layer... 16 Connectivity layer... 16 Host layer... 16 VSPEX Server Virtualization Solution 3
Backup environment profile......... 17 Backup characteristics... 17 Backup layout... 17 Conclusion......... 18 References......... 19 Cisco documentation... 19 EMC documentation... 19 Other documentation... 19 Cisco Solution for VSPEX Server Virtualization 4
Reference architecture overview Document purpose This document describes the reference architecture of the Cisco Solution for EMC VSPEX server virtualization for 50 virtual machines. This document provides guidelines for building similar solutions. Solution purpose VSPEX pre-validated and modular architectures are built with proven best-of-breed technologies to create complete virtualization solutions that enable you to make an informed decision in the hypervisor, compute and networking layers. VSPEX eliminates server virtualization planning and configuration burdens. VSPEX infrastructures accelerate your IT Transformation by enabling faster deployments, greater flexibility of choice, efficiency, and lower risk. The Cisco solution for VSPEX features the enterprise-class Cisco Unified Computing System (UCS) Servers and Nexus switches delivering performance, versatility, and density without compromise. Cisco s portfolio of products and technologies offers a holistic approach to the design, operation, and delivery of IT services. This approach makes it easier for IT to integrate existing technology silos and: Respond more quickly to business demands Reduce capital expenditures and operating expenses Simplify IT operations These platforms optimized for Virtualization, manageability, performance and scalability are ideal for: Consolidation and virtualization Private cloud computing Application or infrastructure-as-a-service The business challenge Customers require a scalable, tiered, and highly available infrastructure on which to deploy their business and mission-critical applications. There are several new technologies available to assist them in consolidating and virtualizing their server infrastructure, but they need to know how to use these technologies to maximize their investment, support service-level agreements, and reduce their total cost of ownership (TCO). This solution addresses the following challenges: Availability: Standalone servers incur downtime for maintenance or unexpected failures. Clusters of redundant standalone nodes are an inefficient in use of CPU, disk and memory resource. Server management and maintenance: Individually maintained servers require significant repetitive activities for monitoring, problem resolution, patching, and other common activities. This is labor intensive, costly, error-prone, and inefficient. Security, downtime, and outage risks are elevated. Ease of solution deployment: While small and medium businesses must address the same IT challenges as larger enterprises, staffing levels, experience, and training are generally more limited. IT generalists are often responsible for managing the entire IT infrastructure, or reliance is placed on third-party sources for maintenance or other tasks. Perceived complexity of the IT function raises fear of risk and may block VSPEX Server Virtualization Solution 5
adoption of new technology. Therefore, simplicity of deployment and management are highly valued Storage efficiency: Storage added locally to physical servers or provisioned directly from a shared resource or array leads to over-provisioning and waste. Backup: Traditional backup approaches are slow and frequently unreliable. There tends to be inflection points (or plateaus) in the virtualization adoption curve when the number of virtual machines increases. With a few virtual machines, the situation can be manageable and most organizations can get by with existing tools and processes. However, when the virtual environment grows, the backup and recovery become the limiting factor in deployment. The technology solution This solution uses the Cisco UCS C220 M3 Server, Cisco Nexus 3048 switches, EMC VNXe3100 array and Microsoft Hyper-V to provide storage and server hardware consolidation. The new virtualized infrastructure is centrally managed, allowing efficient deployment and management of a scalable number of virtual machines and associated shared storage. Solution benefits This VSPEX solution allows the customer that is new to server virtualization to realize the following benefits: Application Availability Highly available, scalable infrastructure that makes most efficient use of CPU, memory, and storage resources and grows with the business Simplified and centralized management of server and storage deployment, monitoring, patching, and other normal activities, resulting in lower management overhead, reduced risk, and reduced downtime Simplicity of solution deployment and use, targeting the IT generalist in the small or medium business (SMB) and providing ease of use, integration points, and ability to mask complex tasks through the use of advanced wizards Storage Efficiency EMC VNXe array features optimized storage efficiency, reduced over-provisioning and simplified storage management Backup Integration with EMC s next-generation backup solutions offering virtual machine and application backup and recovery capabilities Cisco Solution for VSPEX Server Virtualization 6
Key components Introduction Cisco UCS C-Series Servers This section briefly describes the key components of this solution. Cisco UCS C-Series Servers Cisco Nexus 3000 series Switches Microsoft Hyper-V Microsoft System Center Virtual Machine Manager EMC Storage Integrator EMC VNXe Series EMC next-generation backup solution The Solution architecture on page 10 provides details on all the components that make up the reference architecture. Cisco Unified Computing System is the first truly unified data center platform that combines industry-standard, x86-architecture blade and rack servers with networking and storage access into a single system. Key innovations in the platform include a standards-based unified network fabric, Cisco Virtualized Interface Card (VIC) support, and Cisco Extended Memory Technology. The system uses a wire- once architecture with a self-aware, selfintegrating, intelligent infrastructure that eliminates the time-consuming, manual, error-prone assembly of components into systems. Cisco UCS C-Series Rack servers reduce total cost of ownership and increase business agility by extending Cisco Unified Computing System innovations to a rack-mount form factor. These servers: Offer a form-factor-agnostic entry point into the Cisco Unified Computing System, which is a single converged system with configuration automated through integrated, model-based management Simplify and speed deployment of applications Increase customer choice with unique benefits in a familiar rack package Offer investment protection through the capability to deploy them either as standalone servers or as part of the Cisco Unified Computing System. Cisco Nexus 3000 series Switches Cisco Nexus 3000 Series Switches, part of Cisco s Unified Fabric framework, help address business, application, and operational requirements of evolving data centers. These switches: Deliver low latency, high port density for in a compact form factor Support Layer 2/3 wire-rate switching plus advanced unicast and multicast routing. Simplify management with Python scripting, EEM, and XML management tools. Microsoft Hyper-V Microsoft Hyper-V transforms or virtualizes computer hardware resources, including CPU, RAM, hard disk, and network controller. This transformation creates fully functional virtual machines that run their own operating systems and applications just like physical computers. VSPEX Server Virtualization Solution 7
Microsoft System Center Virtual Machine Manager While the Hyper-V Manager console provides critical functionality for creating and running a Hyper-V cluster, Microsoft System Center Virtual Machine Manager (SCVMM) makes the job faster and simpler. Additionally, SCVMM provides monitoring, reporting, migration, and conversion services that are not available in the Hyper-V Manager. SCVMM or a similar tool should be considered to set up and manage the Hyper-V virtual infrastructure. Microsoft System Center Essentials provides many SCVMM capabilities and includes other features of System Center family at a lower price point, with some limitations in function and ease of use. EMC Storage Integrator EMC Storage Integrator (ESI) simplifies the provisioning of EMC block and file storage to Windows servers. Current supported capabilities include the following: Provision, format, and present drives to Windows servers Provision new cluster disks and add them to the cluster automatically Provision shared CIFS storage and mount it to Windows server Provision SharePoint storage, sites and databases in a single wizard. EMC VNXe Series The EMC VNX family is optimized for virtual applications delivering industry-leading innovation and enterprise capabilities for file, block, and object storage in a scalable, easy-touse solution. This next-generation storage platform combines powerful and flexible hardware with advanced efficiency, management, and protection software to meet the demanding needs of today s enterprises. The VNXe series is powered by Intel Xeon processor, for intelligent storage that automatically and efficiently scales in performance, while ensuring data integrity and security. The VNXe series is purpose-built for IT managers in smaller environments and the VNX series is designed to meet the high-performance, high-scalability requirements of midsize and large enterprises. Table 1. Feature VNXe customer benefits Next-generation unified storage, optimized for virtualized applications Capacity optimization features including compression, deduplication, thin provisioning, and application-centric copies High availability, designed to deliver five 9s availability Multiprotocol support for file and block Simplified management with EMC Unisphere for a single management interface for all network-attached storage (NAS), storage area network (SAN), and replication needs ü ü ü ü ü Note: VNXe does not support block compression. Software suites available Remote Protection Suite Protects data against localized failures, outages, and disasters. Cisco Solution for VSPEX Server Virtualization 8
Application Protection Suite Automates application copies and proves compliance. EMC next-generation eneration backup solution Security and Compliance Suite Keeps data safe from changes, deletions, and malicious activity. Software packs available Total Value Pack Includes all protection software suites and the Security and Compliance Suite EMC s NetWorker Fast Start coupled with Data Domain deduplication storage systems seamlessly integrate into virtual environments, providing rapid backup and restoration capabilities. Data Domain deduplication results in vastly less data traversing the network leveraging the Data Domain Boost, and greatly reduces the amount of data being backed up and stored translating into storage, bandwidth and operational savings. The following are two of the most common recovery requests made to backup administrators: File-level level recovery: Object-level recoveries account for the vast majority of user support requests. Common actions requiring file-level recovery are individual users deleting files, applications requiring recoveries, and batch process-related erasures. System recovery: Although complete system recovery requests are less frequent in number than those for file-level recovery, this bare metal restore capability is vital to the enterprise. Some common root causes for full system recovery requests are viral infestation, registry corruption, or unidentifiable unrecoverable issues. The NetWorker System State protection functionality adds backup and recovery capabilities in both of these scenarios. VSPEX Server Virtualization Solution 9
Solution architecture Logical architecture diagram The architecture diagram in this section characterizes the infrastructure validated for support of up to 50 virtual machines. Figure 1 shows the overall logical architecture of the solution. Figure 1. Logical architecture Reference architecture overview The architecture includes the following components: Cisco UCS C220 M3 Server - The Cisco UCS C220 M3 Rack-Mount Server is designed for performance and density over a wide range of business workloads from web serving to distributed database. Building on the success of the Cisco UCS C200 M2 Rack-Mount Server, the enterprise-class Cisco UCS C220 M3 Rack-Mount Server further extends the capabilities of Cisco s Unified Computing System portfolio in a 1U form factor with the addition of the Intel E5-2600 series product family CPUs that deliver significant performance and efficiency gains. In addition, the UCS C220 M3 server provides 16 DIMM slots, 8 drives and 2 x 1 GbE LOM ports delivering outstanding levels of density and performance in a compact package. This platform currently holds the world record for key industry standard benchmarks such as SPECint_rate_base2006, SPECjbb2005 and SPECfp_rate_base2006 against comparable platforms. Cisco Nexus 3048 Switches - The Cisco Nexus 3048 Switch is a line-rate Gigabit Ethernet top-of-rack (ToR) switch and is part of the Cisco Nexus 3000 Series Switches portfolio. The Cisco Nexus 3048 provides the following main benefits: * Wire-rate Layer 2 and 3 switching - Layer 2 and 3 switching of up to 176 Gigabit per second (Gbps) and more than 132 million packets per second (mpps) in a compact 1RU form-factor switch. Robust and purpose-built Cisco NX-OS operating system for end-to-end Cisco Nexus fabric. Integration with Cisco Data Center Network Manager (DCNM) and XML management tools. Comprehensive feature set and innovations for next-generation data centers Cisco Solution for VSPEX Server Virtualization 10
Virtual PortChannel (vpc) provides Layer 2 multipathing through the elimination of Spanning Tree Protocol and enables fully utilized bisectional bandwidth and simplified Layer 2 logical topologies without the need to change the existing management and deployment models. EtherAnalyzer is a built-in packet analyzer for monitoring and troubleshooting control-plane traffic and is based on the popular Wireshark open source network protocol analyzer. Full Layer 3 unicast and multicast routing protocol suites are supported, Microsoft Hyper-V Provides a common virtualization layer to host a server environment. The specifics of the validated environment are listed in Table 2. Hyper-V provides highly available infrastructure through features such as failover clustering and redundant hardware components. Microsoft System Center Virtual Machine Manager (SCVMM) SCVMM is not required for this solution. However, if deployed, it (or its corresponding function in Microsoft System Center Essentials) greatly simplifies provisioning, management, and monitoring of the Hyper-V environment. Microsoft SQL 2008 R2 Server SCVMM, if used, requires a SQL Server database instance to store configuration and monitoring details. SQL Express may be used with System Center Essentials and can run on the same virtual machine as Essentials. IP Network All network traffic is carried by standard Ethernet network with redundant cabling and switching. User and management traffic is carried over a public network while storage traffic is carried over a private, non-routable subnet. Shared Infrastructure Domain name system (DNS) and authentication/authorization services like Microsoft Active Directory can be provided through existing infrastructure or set up as part of the new virtual infrastructure. EMC VNXe3100 array Provides storage by presenting Internet Small Computer System Interface (iscsi) datastores to Hyper-V hosts. These datastores are created using application aware wizards that are included in the EMC Unisphere interface. Hardware resources Table 2 lists the hardware used in this solution. Table 2. Solution hardware Hardware Configuration Notes Hyper-V servers Memory: CPU: 2 GB RAM per virtual machine (100 GB RAM across all servers) 2 GB RAM reservation per Hyper-V host One vcpu per virtual machine 1 to 4 vcpus per physical core Network: Six 1 GbE NIC ports per server Configured as a single Hyper-V cluster. VSPEX Server Virtualization Solution 11
Hardware Configuration Notes Cisco Nexus 3048 Two Nexus 3048 switches with: 48 x 1GbE fixed Ethernet ports 4 x 10 GbE fixed Ethernet ports 1 x 1 GbE Management port Redundant LAN configuration EMC nextgenerationbackup Data Domain Qty-2 Data Domain DD620 Factory EMC VNXe3100 Two Storage Processors (active / active) Cisco UCS C220 M3 Two 1 GbE interfaces per storage processor Forty 300 GB 15k rpm 3.5-inch SAS disks (8 * 300 GB 4+1 R5 Performance Drive Packs) 3 x C220 M3 Servers each with: 2 x Intel Xeon E5-2650 processor 64 GB (8 x 8GB 1600 MHz) memory 2 x 600 GB SAS 15k RPM hard disk 1 x Broadcom 1 GbE Quad port PCIe card (for 1 GbE Network) 2 x 1 GbE LOM (onboard Ethernet) 1 x 10/100/1000 Mbps dedicated Management port This can include the initial disk pack on the VNXe. Software resources Table 3 lists the software used in this solution. Table 3. Solution software Software Cisco UCS Server Cisco UCS C220 M3 Configuration 1.4(4a).1 Network Cisco Nexus 3048 version 5.0(3)U2(2b) Next-generation eneration backup NetWorker Fast Start 7.6.1 Data Domain OS 5.1.0.9-282511 VNXe3100 Cisco Solution for VSPEX Server Virtualization 12
Software Configuration Software version 2.2.0.16150 Microsoft Hyper-V Operating system for Hyper-V hosts Microsoft SQL Server Windows Server 2008 R2 Enterprise Edition (Enterprise or Datacenter edition is necessary for failover functionality) Version 2008 R2 Standard Edition Virtual Machines Base operating system Microsoft Windows Server 2008 R2 VSPEX Server Virtualization Solution 13
Server and Network architecture Topology The architecture diagram in this section shows the server configuration and network topology (Figure 2). Figure 2. Server configuration and Network topology Server details The Server uses the following configuration: The server uses local boot with mirrored drives for operating system The VMs created on the server are stored in iscsi LUNs provisioned on the VNXe 3100 storage. Network Layout Network adapters are teamed for fault tolerance and throughput. The network layout is as follows: Management, Storage, Cluster and Production traffic are provisioned as separate VLANs using 1GE connectivity Virtual port channels are implemented on two Nexus switches for redundant loop-free topologies. Cisco Solution for VSPEX Server Virtualization 14
Storage architecture Storage layout The storage architecture diagram in this section shows the physical disk layout. Disk provisioning on the VNXe series is simplified through the use of wizards, so the administrator does not choose which disks belong to a given storage pool. The wizard may choose any available disk of the proper type, regardless of where the disk physically resides in the array. Figure 3 shows the overall storage layout of the solution. Figure 3. Storage layout Storage layout overview The reference architecture uses the following configuration: Forty 300 GB SAS disks are allocated to a single storage pool as eight 4+1 RAID 5 groups (sold as 5-disk packs). EMC recommends that at least one hot spare disk is allocated for each 30 disks of a given type. At least four iscsi LUNs are allocated to the Hyper-V cluster from the single storage pool to serve as datastores for the virtual servers. VSPEX Server Virtualization Solution 15
High availability and failover Introduction This VSPEX solution provides a highly available virtualized server and storage infrastructure. Each component is configured to provide a scalable, robust architecture for the host, connectivity, and storage layers. Storage layer The VNX family is designed for five 9s availability by using redundant components throughout the array. All of the array components are capable of continued operation in case of hardware failure. The RAID disk configuration on the array provides protection against data loss due to individual disk failures, and the available hot spare drives can be dynamically allocated to replace a failing disk. Connectivity layer Host layer The advanced networking features of the VNX family provide protection against network connection failures at the array. Each Hyper-V host has multiple connections to user and storage Ethernet networks to guard against link failures. These connections are spread across multiple Ethernet switches to guard against component failure in the network. The Hyper-V servers have redundant power supplies and network connections to reduce the impact of failure of these components. Failover Clustering is configured on the Hyper-V cluster to help recover virtual servers quickly in case of a complete host failure. Cisco Solution for VSPEX Server Virtualization 16
Backup environment profile Backup characteristics The solution was sized with the following application environment profile. Table 4. Profile characteristics Profile characteristic Value Number of users 500 Number of virtual machines Exchange data SharePoint data SQL server User data 50 (20% DB, 80% Unstructured) 0.5TB (1GB mail box per user) 0.25 TB 0.25 TB 2.5 TB (5.0 GB per user) Daily change rate for the applications Exchange data 10% SharePoint data 2% SQL server 5% User data 2% Retention per data types All DB data User data 14 Dailies 30 Dailies, 4 Weeklies, 1 Monthly Backup layout NetWorker Fast Start provides various deployment options depending on the specific use case and the recovery requirements. In this case, the solution is deployed with both NetWorker Fast Start and Data Domain managed as a single solution. This enables the unstructured user data to be backed up directly to the Data Domain system for simple file level recovery. The database is managed by the NetWorker Fast Start software, but it is directed to the Data Domain system with the embedded Boost client library. This backup solution unifies the backup process and achieves dramatically increased performance and efficiency. VSPEX Server Virtualization Solution 17
Conclusion VSPEX have been engineered to enable the small or medium business to simply, quickly, and reliably deploy a broad range of shared storage workloads. Setup wizards enable an IT generalist to provision Hyper-V storage with a short series of mouse clicks and recommended best practices applied automatically. In conjunction with Hyper-V, VNXe provides a proven and economical path to server virtualization for the small or medium business. EMC nextgeneration backup enables protection of this dynamic environment and allows for growth and flexibility. Table 5. Feature Solution benefits Cisco Unified Computing System Cisco Nexus series Switches EMC next-generation-backup EMC VNXe Series Microsoft Hyper-V Benefits First truly unified platform that combines industrystandard, x86-architecture blade and rack servers with networking and storage access into a single system. The system uses a wire- once architecture with a self-aware, self-integrating, intelligent infrastructure delivering dramatic improvement in TCO. Designed for a broad range of physical, virtual, storage access, and high-performance computing environments, thus giving customers the flexibility to meet and scale their IT requirements in a gradual manner and at a pace that aligns with their business objectives. Unifies the backup process with industry leading backup software and deduplication storage, and achieves dramatically increased performance and efficiency. Provides a robust, reliable, high-performance, common storage platform for many virtual servers. A single storage platform that is efficient, powerful, and built for the most demanding virtual environments. Provides a well-established virtualization platform for Windows server consolidation. Cisco Solution for VSPEX Server Virtualization 18
References Cisco documentation EMC documentation Other documentation For documentation on Cisco Unified Computing System, please refer to the website http://www.cisco.com/go/unifiedcomputing Cisco Nexus series Switches, please refer to the website http://www.cisco.com/go/nexus Cisco Validated Designs, please refer to the website http://www.cisco.com/go/dcdesignzone The following documents, available on the EMC Online Support website, provide additional and relevant information. If you do not have access to a document, contact your EMC representative: EMC VSPEX Server Virtualization Solution Microsoft Hyper-V for 100 Virtual Machines Enabled by Microsoft Hyper-V, VNXe3300, and EMC Next-Generation Backup Reference Architecture EMC VSPEX Server Virtualization Solution Enabled by Microsoft Hyper-V, VNXe3100, and EMC Next-Generation Backup Deployment Guide EMC VSPEX Server Virtualization Solution Microsoft Hyper-V for 0-100 Virtual Machines Sizing Guide For documentation on Microsoft Hyper-V, System Center Virtual Machine Manager, and System Center Essentials, please refer to the Microsoft website at http://www.microsoft.com. VSPEX Server Virtualization Solution 19