White Paper for Microsoft Private Cloud Fast Track on Huawei Server and Storage System

Transcription

1 White Paper for Microsoft Private Cloud Fast Track on Huawei Server and Storage System

2 Contents 1 Introduction Microsoft Private Cloud Fast Track Program Description The Program Business Benefits Technical Overview Private Cloud Architecture Principles Conceptual Architecture Servers Storage Networking Virtualization Reference Architrecture Workload Categories Logical Architecture Server Architecture Storage Architecture Network Architecture Virtualiztion Architecture Conclusion Terms and Abbreviations Ver 1.0 HUAWEI Technologies Co., Ltd 2

3 1 Introduction The Microsoft Private Cloud Fast Track Program is a joint effort between Microsoft and its partner original equipment manufacturers (OEMs) to help organizations quickly develop and implement private clouds, while reducing both the cost and the risk. It provides a reference architecture that combines Microsoft software, consolidated guidance, and validated configurations with partner technology-such as computing power, network and storage architectures, and value-added software components. Each Private Cloud Fast Track solution is an integrated solution from Microsoft and its OEM partner. This document describes the Microsoft implementation and does not include information about partner implementations. Instead, it is designed to provide an overview of Microsoft guiding principles and design criteria for this solution, and how our Hyper-V technology cloud solution conforms to these principles. This private cloud fast track solution combines Microsoft software, consolidated guidance, and validated configurations with HUAWEI computing power, network, and storage architectures: The joint solution for server, network, storage management, and out-of-band management in the Private Cloud; Our approach to automation, maintenance, and security; The storage area network (SAN) multipathing solution, Redundant Array of Independent Disks (RAID) array design, and performance characteristics The SAN s cloning capabilities, use of data de-duplication and thinprovisioning, and Hyper-V integration; The network infrastructure card (NIC) teaming solution, and the host adaptor s Fibre Channel (FC) and Internet Small Computer System Interface (iscsi) settings; The network and storage tiering strategies and recommendations; Ver 1.0 HUAWEI Technologies Co., Ltd 3

4 The private cloud model provides much of the efficiency and agility of cloud computing along with the increased control and customization achieved through dedicated private resources. With the Microsoft Private Cloud Fast Track program, Microsoft and its hardware partners provide organizations both the control and the flexibility required to reap the full benefits of the private cloud. Ver 1.0 HUAWEI Technologies Co., Ltd 4

5 2 Microsoft Private Cloud Fast Track Program Description 2.1 The Program The Microsoft Private Cloud Fast Track Program is a joint reference architecture for building private clouds that combines Microsoft software, consolidated guidance, and validated configurations with OEM partner technology, including computing power, network and storage architectures, and value-added software components. Microsoft Private Cloud Fast Track solutions provide a turnkey approach for delivering preconfigured and validated implementations of the private cloud. With local control over data and operations, IT professionals can dynamically pool, allocate, and manage resources for agile infrastructures as a service. In addition, business unit managers can deploy line-of-business applications with speed and consistency using self-provisioning (and decommissioning) and automated data center services in a virtualized environment. 2.2 Business Benefits The Microsoft Private Cloud Fast TrackProgram provides reference architecture for building private clouds on each organization s unique terms. Each fast-track solution helps organizations implement private clouds with increased ease and confidence. Among the benefits of the Microsoft Private Cloud Fast TrackProgram are faster deployment, reduced risk, and a lower cost of ownership Faster Deployment End-to-end architectural and deployment guidance. Streamlined infrastructure planning due to predefined capacity. Ver 1.0 HUAWEI Technologies Co., Ltd 5

6 2.2.2 Reduced Risk Enhanced functionality and automation through deep knowledge of infrastructure. Integrated management for virtual machine and infrastructure deployment. Simplify infrastructure and virtual machine deployment with integrated management. Rapidly grow the infrastructure to adapt to market pressures by leveraging the scalable design of the architecture. Tested end-to-end interoperability for compute, storage, and network. Deploy pre-defined out-of-box solutions based on a common cloud architecture that has already been tested and validated. High degree of service availability through automated load balancing Adapt to failures by utilizing automation that detects and reacts to events Lower Cost-of-Ownership Near-zero downtime with exceptional fault tolerance, providing high availability Dynamic pooling that can enhance the use of virtualization resources with Hyper-V and with supported storage and network devices Utilization of low-cost switches that consume less power and deliver high throughput for large bandwidth requirements Benefit from high performance and scalability with HUAWEI Tecal RH2288 V2 Server and T Series Unified Storage solutions along with the Microsoft Windows Server 2012 operating system and Microsoft Hyper-V technology. Ver 1.0 HUAWEI Technologies Co., Ltd 6

7 3 Technical Overview 3.1 Private Cloud Architecture Principles Resource Pooling Microsoft Private Cloud architecture principles conform to the cloud attributes outlined by the National Institute of Standards and Technology (NIST) definition of cloud computing version 15: on-demand self-service, broad network access, resource pooling, rapid elasticity, and measured service. Similarly, the Microsoft Private Cloud architecture is based on seven principles: resource pooling, elasticity and the perception of infinite capacity, perception of continuous availability, predictability, metering/chargeback, multitenancy, and security and identity. Resource optimization is a principle that drives efficiency and cost reduction. It is primarily achieved through resource pooling. Abstracting the platform from the physical infrastructure enables optimization of resources through shared use. Allowing multiple consumers to share resources results in higher resource utilization and a more efficient use of the infrastructure. Optimization through abstraction is a key factor behind many of the Private Cloud principles, ultimately helping to improve agility and drive down costs Elasticity and Perception of Infinite Capacity From a consumer s perspective, cloud services appear to have infinite capacity. Using the electric utility provider as a metaphor, consumers use as much or as little of the service as they need. This utility approach to computing requires that capacity planning be proactive so that requests can be satisfied on demand. Applying the principle of elasticity reactively and in isolation often leads to inefficient use of resources and unnecessary costs. But when an organization encourages desired consumer behavior, it can use this principle to balance the desire for agility with the cost of unused capacity. Ver 1.0 HUAWEI Technologies Co., Ltd 7

8 3.1.3 Perception of Continuous Availability Predictability From the consumer s perspective, cloud services always appear to be available when needed. The consumer should never experience an interruption of service, even if failures occur within the Private Cloud environment. To achieve this perception, organizations must take a mature service management approach that combines inherent application resiliency with infrastructure redundancies in a highly automated environment. As with the perception of infinite capacity, this principle can only be achieved in conjunction with the other Private Cloud principles. Whether you re a consumer or a provider, predictability is a fundamental cloud principle. From the vantage point of the consumer, cloud services should be consistent; they should have the same quality and functionality any time they are used. To achieve predictability, a provider must deliver an underlying infrastructure that assures a consistent experience to the hosted workloads. This consistency is achieved through the homogenization of underlying physical servers, network devices, and storage systems. From the provider s service management perspective, this predictability is driven through the standardization of service offerings and processes. The principle of predictability is needed to ensure service quality Metering and Chargeback (Service Providers Approach to Delivering IT) Multitenancy When IT professionals are asked to deliver a service to the business, they typically purchase the necessary components and then build an infrastructure specific to the service requirements. This typically results in longer time to market and increased costs due to duplicate infrastructure. In addition, the service often fails to meet business expectations of agility and cost control. The problem is often compounded when an existing service needs to be expanded or upgraded. Taking a service provider s perspective toward delivering infrastructure transforms the IT approach. If infrastructure is provided as a service, IT can use a shared resource model that makes it possible to achieve economies of scale. This, combined with the other principles, helps the organization to realize greater agility at lower cost. Multitenancy refers to the ability of the infrastructure to be logically subdivided and provisioned to different organizations or organizational units. The traditional example is a hosting company that provides servers to multiple customer Ver 1.0 HUAWEI Technologies Co., Ltd 8

9 organizations. Increasingly, this is also a model being utilized by a centralized IT organization that provides services to multiple business units within a single organization, treating each as a customer or tenant Security and Identity Security for the Private Cloud is founded on two paradigms: protected infrastructure and network access. Protected infrastructure takes advantage of security and identity technologies to ensure that hosts, information, and applications are secured across all scenarios in the data center, including the physical (on-premises) and virtual (on-premises and cloud) environments. Application access helps ensure that IT managers can extend vital applications to internal users as well as to important business partners and cloud users. Network access uses an identity-centric approach to ensure that users-whether they re based in the central office or in remote locations-have more secure access no matter what device they re using. This helps ensure that productivity is maintained and that business gets done the way it should. Most important from a security standpoint, the secure data center makes use of a common integrated technology to assist users in gaining simple access using a common identity. Management is integrated across physical, virtual, and cloud environments so that businesses can take advantage of all capabilities without the need for significant additional financial investments. 3.2 Conceptual Architecture One of the key drivers of the layered approach to infrastructure architecture presented in this white paper is to allow complex workflow and automation to be developed over time. This is accomplished by creating a collection of simple automation tasks, assembling them into procedures that are managed by the management layer, and then creating workflows and process automation that are controlled by the orchestration layer. In a modular architecture, the concept of a scale unit refers to the extent to which a module can scale before another module is required. For example, an individual server can be considered a scale unit. In this case, the single server can be expanded to a certain point in terms of central processing units (CPU) and random access memory (RAM). However, beyond these limits an additional server is required to continue scaling. Each scale unit also has associated amounts of labor, such as physical installation labor and configuration labor. With large scale units such as a preconfigured full rack of servers, the labor overhead can be minimized. It is critical to know the scale limits of all components, both hardware and Ver 1.0 HUAWEI Technologies Co., Ltd 9

10 software, to determine the optimum scale units for input to the overall architecture. Scale units enable the documentation of all the requirements (such as space, power, HVAC, and connectivity) that are needed for implementation. 3.3 Servers The hardware architecture choices that are available to data center architects are constantly evolving. Choices range from rack-mounted servers, to tightly integrated, highly redundant blade systems, to container models. The same spectrum exists for storage and networking equipment. Server scale limits are well published and include factors such as the number and speed of CPU cores, maximum amount and speed of RAM, and the number and type of expansion slots. Particularly important are the number and type of onboard input-output (I/O) ports, as well as the number and type of supported I/O cards. Both Ethernet and Fibre Channel expansion cards often provide multiport options where a single card can have four ports. Additionally, in blade server architectures, there are often limitations on the number of I/O card and supported combinations. It is important to be aware of these limitations, as well as the oversubscription ratio between blade I/O ports and any blade chassis switch modules. A single server is not typically a good scale unit for a Private Cloud solution, due to the amount of overhead and cost required to install and configure an individual server, as well as the lack of high availability. 3.4 Storage Storage architecture is a critical design consideration for Private Cloud solutions. The topic is challenging, because it is rapidly evolving in terms of new standards, protocols, and implementations. Storage and the support of storage networking are critical to the overall performance of the environment; however, they also can play a large role in the overall cost, because storage tends to be one of the more expensive items in a Private Cloud solution. Storage architectures today have several layers including the storage arrays, the storage network, the storage protocol, and for virtualization, the clustered volume manager that utilizes the physical storage. One of the primary objectives of the private cloud is to enable rapid provisioning and deprovisioning of VMs. Doing so at large scale requires tight integration with the storage architecture and robust automation. Provisioning a new VM on an already existing logical unit number (LUN) is a simple operation. However, provisioning a new LUN, adding it to a host cluster, and then provisioning a new VM on that LUN involves relatively complex tasks that also greatly benefit from Ver 1.0 HUAWEI Technologies Co., Ltd 10

11 automation. For example, the HUAWEI OceanStor T series unified storage system supports all major storage protocols (including Fibre Channel, FCoE, and iscsi). Additionally, all HUAWEI OceanStor T series storage systems use the same integrated storage management software (ISM). 3.5 Networking Many network architectures include a tiered design with three or more layers such as core, distribution, and access. Designs are driven by the port bandwidth and quantity required at the edge, as well as the ability of the distribution and core layers to provide higher-speed uplinks to aggregate traffic. Additional considerations include Ethernet broadcast boundaries and limitations, and spanning tree and or other loop avoidance technologies. A dedicated management network is a common feature of advanced data center virtualization solutions. Most virtualization vendors recommend that hosts be managed via a dedicated network to avoid competition with guest traffic needs and to provide a degree of separation for security and ease of management purposes. This typically implies dedicating one network interface card (NIC) per host and one port per network device to the management network. With advanced data center virtualization, a frequent use case is to provide isolated networks where different owners such as particular departments or applications are provided their own dedicated networks. Multitenant networking refers to the use of technologies such as virtual local area networks (VLANs) or Internet Protocol security (IPSec) isolation techniques to provide dedicated networks that utilize a single network infrastructure or wire. Managing the network environment in an advanced data center virtualization solution can present challenges that must be addressed. Ideally, network settings and policies are defined centrally and applied universally by the management solution. In the case of IPSec-based isolation, this can be accomplished using the Active Directory service and Group Policy to control firewall settings across the hosts and guest as well as the IPSec policies controlling network communication. For VLAN-based network segmentation, several components-including the host servers, host clusters, Microsoft System Center Virtual Machine Manager, and the network switches---must be configured correctly to enable both rapid provisioning and network segmentation. With Hyper-V and host clusters, identical virtual networks must be defined on all nodes so that a VM can fail over to any node and maintain its connection to the network. At large scale, this can be accomplished via scripting with the Windows PowerShell command-line interface. Ver 1.0 HUAWEI Technologies Co., Ltd 11

12 3.6 Virtualization The virtualization layer is one of the primary enablers in mature IT environments. The decoupling of hardware, operating systems, data, applications, and user state opens up a wide range of options for better management and distribution of workloads across the physical infrastructure. The ability of the virtualization layer to migrate running VMs from one server to another with zero downtime, as well as many other features that are provided by hypervisor-based virtualization technologies, provides a rich set of capabilities. These capabilities can be utilized by the automation, management, and orchestration layers to maintain desired states (such as load distribution) or to proactively address decaying hardware (such as prefailure detection) or other issues that would otherwise cause faults or service disruptions. As with the hardware layer, the virtualization layer must be able to be managed by the automation, management, and orchestration layers. The abstraction of software from hardware that virtualization provides moves the majority of management and automation into the software space, instead of requiring people to perform manual operations on physical hardware. Ver 1.0 HUAWEI Technologies Co., Ltd 12

13 4 Reference Architrecture The HUAWEI and Microsoft Window Server 2012 Hyper-V Private Cloud Fast Track solution provides reference architecture for building private clouds according to the customer s organization s unique requirements. This private cloud fast track solution combines Microsoft software, consolidated guidance, and validated configurations with HUAWEI computing power, network, and storage architectures and value-added software components. This solution integrates all the components necessary for small infrastructure implementation. It consists of Microsoft Windows Server 2012 Hyper-V, HUAWEI Tecal RH2288 V2 series rack servers (RH2288 V2 for short), HUAWEI Quidway S5700 series network switches (S5700 for short) and HUAWEI OceanStor S5000T series unified storage systems (S5000T for short). All components are configured for high availability and reliability. This solution provides a highly scalable and reliable platform and enables small businesses and branch offices to take advantage of the power of cloud computing without the complexity entailed in designing and implementing custom solutions. The reference architecture for HUWEI solution for Microsoft Private Cloud Fast Track is shown as follow. Ver 1.0 HUAWEI Technologies Co., Ltd 13

14 Figure 4-1 Reference Architecture for Microsoft Private Cloud Fast Track Internal Network Switch External Access Network Switch WS2012 Failover Cluster: 2 * HUAWEI RH2288 V2 Rack Server Data Center Switch Storage System: HUAWEI OceanStor S5500T Unified Storage System VLAN A --- iscsi VLAN B --- VMvSwitch VLAN C --- LiveMigration & Heartbeat Internal Management Network Dedicated Network for VM Dedicated Network for iscsi Dedicated Network for Live Migration & Heartbeat Internal Network Switch is used for management of servers and storage systems; External Access Netwok Switch is used for end users to connect to virtual machines; The Internal Network Switch and External Access Network Switch are not included in this Fast Track solution. 4.1 Workload Categories Server Virtualization and Consolidation Server virtualization is based on the abstraction of physical system resources so that multiple logical partitions can be created and can host a heterogeneous set of operating systems that run simultaneously on a single physical server. Rather than paying for many under-utilized servers, each dedicated to a specific workload, server virtualization allows those workloads to be consolidated onto a smaller number of more efficiently utilized physical systems. Server virtualization provides the following benefits: Consolidates multiple, under-utilized physical servers on a single host, running VMs; Ver 1.0 HUAWEI Technologies Co., Ltd 14

15 Reduces workforce, space, and kilowatts by taking advantage of virtualization for server consolidation and agility; Helps save money because less management, less space, and fewer kilowatt hours are needed. Virtualization can also help to simplify and accelerate provisioning, one of the tenets of the private cloud. The acquisition of workload resources and hardware can be decoupled. Adding the capability that is required for a particular business process (say, a web commerce engine) becomes streamlined and immediate. In a more advanced virtualized environment, workload requirements can be selfprovisioning, which results in dynamic resource allocation. Figure 4-2 Server-based Consolidation through Virtualization (Supplied by Microsoft) Although virtualization-based server consolidation can provide many benefits, it can also add complexity if the environment is not managed properly. The savings from hardware consolidation could be offset by increases to IT management overhead. Because creating VMs is so easy, an unintentional and unnecessary virtual sprawl can result that far exceeds physical server capacity and that outpaces the tools used to manage VMs. A properly managed virtual infrastructure, however, automatically determines which servers are the best candidates for virtualization, allowing administrators to initiate automated processes to convert those servers to VMs and provision them to the right hosts in minutes-compared to the weeks or months it would take to procure and configure physical servers manually Virtual Desktop Infrastructure Virtual desktop infrastructure (VDI) allows IT managers to deploy desktops in VMs on secure and centralized hardware. A centralized and optimized virtual desktop enables IT staff to build a more agile and efficient IT infrastructure Ver 1.0 HUAWEI Technologies Co., Ltd 15

16 while allowing users to access and run their desktop applications wherever they may be. Flexible desktop scenarios using the Windows operating system give organizations the ability to choose the client computing scenarios that best meet the unique needs of their business. When an organization manages its virtual infrastructure with the same tools it uses to manage its physical assets, system complexity can be reduced, and changes made to the overall infrastructure can be streamlined. By using some or all of these technologies together, organizations can provide very flexible solutions to support many user scenarios. Figure 4-3 Virtualization Desktop Infrastracture (Supplied by Microsoft) 4.2 Logical Architecture A private cloud is far more than a highly available infrastructure that provides computing resources to higher-level applications. With cloud computing, a fundamental shift is that of IT moving from server operator to service provider. This requires a set of services to accompany the infrastructure, such as reporting, usage metering, and self-service provisioning. If these services are unavailable, then the cloud service layer is unavailable, and IT is little more than a traditional data center. For this reason, high availability must also be provided to the management systems. Ver 1.0 HUAWEI Technologies Co., Ltd 16

17 Figure 4-4 Private Cloud Logical Architecture 4.3 Server Architecture The host server architecture is a critical component of the virtualized infrastructure, as well as a key variable in the consolidation ratio and cost analysis. The ability of the host server to handle the workload of a large number of consolidation candidates increases the consolidation ratio and helps provide the desired cost benefit. The system architecture of the host server refers to the general category of the server hardware itself. The primary factor to keep in mind when selecting system architectures is that each Hyper-V host will contain multiple guests with multiple workloads. Processor, RAM, storage, and network capacity are critical, as well as high I/O capacity and low latency. The host server must be able to provide the required capacity in each of these categories HUAWEI Tecal RH2288 V2 Series Rack Server The HUAWEI Tecal RH2288 V2 series (RH2288 V2 for short) is a new-generation 2U 2-socket rack server. Featuring two Intel Xeon E series processors, the RH2288 V2 provides a memory capacity of 768 GB and a storage capacity of 38 TB, making it an ideal product for critical services and cloud computing. Ver 1.0 HUAWEI Technologies Co., Ltd 17

18 Figure 4-1 HUAWEI Tecal RH2288 V2 Series Rack Server The detailed information for HUAWEI RH2288 V2 servers used in the HUAWEI and Microsoft Private Cloud Fast Track solution is shown in Table 4-1. Table 4-1 Detailed Information for HUAWEI Tecal RH2288 V2 Rack Server Hardware HUAWEI Tecal RH2288 V2 Rack Server Description CPU: 2 * Intel Xeon E Series with 8 cores and Hyper-Threading technology, 32 physic cores in total Memory: 8 * 16 GB or 16 * 16 GB DDRIII Hard Disk: 2 * 300 GB 2.5 inch or 3.5 inch SAS Disk Network Adapter: 4 * built-in Intel Gigabyte network adapter 4 * extended Intel Gigabyte network adapter 8 Intel 1Gps Network ports in total For more information about HUAWEI Tecal RH2288 V2 series rack server, please see: Server Design Principles The server design must provide a high level of availability throughout. This includes, for example, features like redundant power distribution units (PDUs), storage path, networking, and disks. To provide this level of availability, it is Ver 1.0 HUAWEI Technologies Co., Ltd 18

19 necessary to use two or more storage controllers to support multipathing on the I/O side. Use multiple network adapters, multi-port network adapters, or both on each host server. For converged designs, network technologies that provide teaming or virtual network interface cards (NICs) can be utilized, provided that two or more physical adapters can be teamed for redundancy and multiple virtual NICs and/or VLANs can be presented to the hosts for traffic segmentation and bandwidth control. The following network connections are required: One network dedicated to management purposes on the host machine; One network dedicated to the clustered shared volumes (CSV) and cluster communication network; One network dedicated to the live migration network; One or more networks dedicated to the guest VMs (use 10Gbps network adapters for highest consolidation); If using Internet Small Computer System Interface (iscsi), one network dedicated to iscsi with multipath I/O (MPIO). For the recommended configuration by quantity and type of NIC, see the Hyper-V: Live Migration Network Configuration Guide in the Microsoft TechNet Library. On each HUAWEI Tecal RH2288V2 server, eight 1Gbps Ethernet network ports are configured including 2 ports using as iscsi connection to storage system, 2 ports using as a nic team for management console, 2 ports using as virtual machine virtual switches and last 2 ports using as a nic team for virtual machine live migration and heartbeat. The detailed configuration information for HUAWEI Tecal RH2288 V2 is shown as follow. Figure 4-2 Networking Configuration for HUAWEI RH2288 V2 Servers As shown above, eight network ports are divided into four groups, and each two ports in a group are used as a team located on different network adapter, which can implement high availability for network access. This configuration can Ver 1.0 HUAWEI Technologies Co., Ltd 19

20 effectively avoid single point of failure even one network adapter is down. In addition to redundancy, you need a platform that has optimal virtualization support through hardware virtualization capabilities such as Intel virtualization (Intel-VT) as the baseline, along with the Second Level Address Translation (SLAT) capabilities of Windows Server 2012 Hyper-V to optimize for the best possible performance. All of these features are available with HUAWEI Tecal RH2288 V2 series rack server Server Storage Connectivity For HUAWEI OceanStor S5000T series storage systems, Fibre Channel, FCoE and iscsi could be used as the interconnection between server hosts and storage front-end. In this solution reference architecture, 1Gbps iscsi is used as the interconnection between servers and storage system. 4.4 Storage Architecture The storage design for any virtualization-based solution is a critical element that is typically responsible for a large percentage of the solution s overall cost, performance, and agility. HUAWEI OceanStor S5000T series unified storage system is a new-generation storage product for mid-range and high-end storage applications. It boasts integration of block-level and file-level data storage. Support for a variety of storage protocols, and GUI-based central storage management. Delivering leading performance, enhanced efficiency, maximized return on investment, and all-in-one solutions, and the S5000T series is ideally applicable to scenarios such as large-database OLTP/OLAP, high-performance computing, digital media, Internet applications, central storage, backup, disaster recovery, and data migration. Figure 4-3 HUAWEI OceanStor S5500T Unified Storage System Ver 1.0 HUAWEI Technologies Co., Ltd 20

21 The internal architecture of HUAWEI OceanStor S5500T unified storage system meets the principle of high availability. Full component redundancy prevents single points of failure. Data coffer and file system mirror improve system reliability. The dual Active/Active controller provides interruption-free redundancy to ensure operational reliability and availability. Figure 4-4 HUAWEI OceanStor S5500T Storage System inside architecture 1System enclosure 4Controller module 2Power module 3Fan-BBU module 5Disk module The detailed information for HUAWEI S5500T storage system used in the HUAWEI and Microsoft Private Cloud Fast Track solution is shown in Table 3-2; Table 4-2 HUAWEI S5500T Series Unified Storage System Configuration Hardware Description HUAWEI OceanStor S5500T Unified Storage System Dual controllers 8G/16GB/32GB cache memory Front-end Ports for each Controller: 4 * 8 Gbps Fibre Channel ports 4 * 1 Gbps iscsi ports or 2 * 10 Gbps iscsi ports 24 * 300GB 2.5 inch SAS HDDs For more information about HUAWEI S5000T series unified storage system, please see: htm Ver 1.0 HUAWEI Technologies Co., Ltd 21

22 Each HUAWEI OceanStor S5500T Storage System consists of two controllers, redundant power supplies and fans, shown as follow. Figure 4-5 HUAWEI OceanStor S5500T Backplane The interfaces on controller of HUAWEI OceanStor S5500T Unified Storage System are shown as follow. Figure 4-6 Interfaces on HUAWEI OceanStor S5500T Storage Contrllers Storage Options Although many storage options exist, organizations should choose their storage devices based on their specific data management needs. Storage devices are typically modular and flexible midrange and high-end SANs. Modular midrange SANs are procured independently and can be chained together to provide greater capacity. They are efficient, can grow with the environment as needed, and require less up-front investment than high-end SANs. Large enterprises may have larger storage demands and may need to serve a larger set of customers and workloads. In this case, high-end SANs can provide the highest performance and Ver 1.0 HUAWEI Technologies Co., Ltd 22

23 capacity. High-end SANs typically include more advanced features such as continuous data availability through technologies like dispersed cluster support. The HUAWEI OceanStor S5000T series Unified Storage System supports all the capabilities listed above even LUN Copy, Smart Tiering, Snapshots and etc SAN Storage Protocols 1. Comparing iscsi, FC, and FCoE Fibre Channel has historically been the storage protocol of choice for enterprise data centers for a variety of reasons, including good performance and low latency. Over the past several years, however, the advancing performance of Ethernet from 1Gbps to 10Gbps and beyond has led to great interest in storage protocols that make use of Ethernet transport-such as Internet Small Computer System Interface (iscsi) and, more recently, Fibre Channel over Ethernet (FCoE). A key advantage of the protocols that use the Ethernet transport is the ability to use a converged network architecture in which a single Ethernet infrastructure serves as the transport for both LAN and storage traffic. FCoE is an emerging technology that brings the benefits of using an Ethernet transport while retaining the advantages of the Fibre Channel protocol and the ability to use Fibre Channel storage arrays. Several enhancements to standard Ethernet are required for FCoE. This is commonly referred to as enhanced Ethernet or data center Ethernet. These enhancements require Ethernet switches that are capable of supporting enhanced Ethernet. A common practice in large-scale Hyper-V deployments is to use both Fibre Channel and iscsi. Fibre Channel and iscsi can provide the host storage connectivity. In contrast, iscsi is directly used by guests-for example, for the shared disks in a guest cluster. In this case, although Ethernet and some storage I/O will share the same pipe, segregation is achieved by VLANs, and quality of service can be further applied by the OEM s networking software. In this solution, iscsi is used as the interconnection protocol between Hyper-V hosts and storage system. Multiple 1Gbps network links and two HUAWEI S5700 switches are used to implement interconnection between HUAWEI RH2288 V2 and S5500T, shown as follow. Ver 1.0 HUAWEI Technologies Co., Ltd 23

24 Figure 4-7 iscsi Network Schema Configuration for S5500T Unified Storage System As shown in Figure 4-7, Each S5500T s controller is connected to two HUAWEI S5700 switch s VLAN A, which is dedicated for iscsi connection; even one of the netwok switch is down, the other switch can take over all the businesses, without single point of failure. Each S5700 switch is connected to both two controllers so that even one controller is down, the other controller can take over all the businesses, without single point of failure. 2. Storage Network Both iscsi and FCoE utilize an Ethernet transport for storage networking. This provides another architecture choice in terms of whether to use a dedicated Ethernet network with separate switches, cables, paths, and other infrastructure, or instead to use a converged network where multiple traffic types are run over the same cabling and infrastructure. The storage solution must provide logical or physical isolation between storage and Ethernet I/O. If it s a converged network, QoS must be provided to guarantee storage performance. The storage solution must provide iscsi connectivity for guest clustering, and there must be fully redundant, independent paths for storage I/O. Standards-based converged network adapters, switches, and Fibre Channel storage arrays should be used for FCoE. The selected storage arrays should also provide iscsi connectivity over standard Ethernet so that Hyper-V guest clusters can be used. If iscsi or Fibre Channel is used, make sure that there are dedicated network adapters, switches, and paths for the storage traffic. Ver 1.0 HUAWEI Technologies Co., Ltd 24

25 Figure 4-8 illustrates the differences between a traditional architecture (left) with separate Ethernet and Fibre Channel switches, each with redundant paths, compared to a converged architecture (right) in which both Ethernet and Fibre Channel (via FCoE) utilize the same set of cables while still providing redundant paths. The converged architecture requires fewer switches and cables. In the converged architecture, the switches must be capable of supporting enhanced Ethernet. Figure 4-8 A Traditional Architecture compared to a Converaged Architecture ( Supplied by Microsoft ) Clust Shared Volumes Cluster Shared Volumes (CSVs) in a Windows Server 2012 failover cluster allow multiple nodes in the cluster to simultaneously have read-write access to the same LUN (disk) that is provisioned as an NTFS volume. With CSVs, clustered roles can fail over quickly from one node to another node without requiring a change in drive ownership, or dismounting and remounting a volume. CSVs also help simplify managing a potentially large number of LUNs in a failover cluster. CSV provides not only shared access to the disk, but also storage path I/O faulttolerance (dynamic I/O redirection). In the event that the storage path on one node becomes unavailable, the I/O for that node is rerouted via Server Message Block (SMB) through another node. CSV maintains metadata information about the volume access and requires that some I/O operations take place over the cluster communications network. One node in the cluster is designated as the coordinator node and is responsible for these disk operations. However, all nodes in the cluster can read/write directly and concurrently to the same volume (not the same file) through the dedicated storage paths for disk I/O, unless a failure scenario occurs as described above. 1. CSV Characteristics Table 4-3 below shows the characteristics that are defined by the New Technology File System (NTFS) and are inherited by CSV. Ver 1.0 HUAWEI Technologies Co., Ltd 25

26 Table 4-3 CSV Parameter Characteristics CSV Parameter Max Volume Size Characteristics 256TB Max Partitions 128 Directory Structure Max Files per CSV Max VMs per CSV Unrestricted 4+ Billion Unlimited 2. CSV Colume Sizing Because all cluster nodes can access all CSV volumes simultaneously, IT managers can now use standard LUN allocation methodologies based on performance and capacity requirements of the expected workloads. Generally speaking, isolating the VM operating system I/O from the application data I/O is a good start. In addition, implement application-specific considerations such as segregating the database I/O from the logging I/O and creating SAN volumes and storage pools that factor in the I/O profile itself (that is random read and write operations versus sequential write operations). The architecture of CSV differs from traditional clustered file systems, which frees it from common scalability limitations. As a result, there is no special guidance for scaling the number of Hyper-V nodes or VMs on a CSV volume. The important thing to keep in mind is that all VM virtual disks running on a particular CSV will contend for storage I/O. Also worth noting is that individual SAN LUNs do not necessarily equate to dedicated disk spindles. A SAN storage pool or RAID array might contain many LUNs. A LUN is simply a logical representation of a disk provisioned from a pool of disks. Therefore, if an enterprise application requires specific storage I/O operations per second (IOPS) or disk response times, IT managers must consider all the LUNs in use on that storage pool. An application that requires dedicated physical disks if it is not virtualized might require dedicated storage pools and CSV volumes running within a VM. Consider the following when setting up your CSV infrastructure: At least four CSVs per host cluster are recommended for segregating operating system I/O, random R/W I/O, sequential I/O, and other VM-specific data. Create a standard size and IOPS profile for each type of CSV LUN to use for capacity planning. When additional capacity is needed, provision additional standard CSV LUNs. Ver 1.0 HUAWEI Technologies Co., Ltd 26

27 Consider prioritizing the network used for CSV traffic. For more information, refer to Designating a Preferred Network for Cluster Shared Volumes Communication in the Microsoft TechNet Library. Figure 4-9 Example of a common CSV design for a large Hyper-V cluster SAN Design A highly available SAN design should have no single points of failure, including: Redundant power from independent power distribution units (PDUs) Redundant storage controllers Redundant storage paths (supported, for example, by redundant target ports of NICs per controller, redundant FC or IP network switches and redundant cabling) Data storage redundancy such as that that occurs with volume mirroring, or synchronous or asynchronous replication Address the following elements when designing or modifying your SAN as the basis of your Private Cloud storage infrastructure: Performance Drive types Multipathing Fibre Channel SAN iscsi SAN Thin provisioning Ver 1.0 HUAWEI Technologies Co., Ltd 27

28 Volume cloning Volume snapshots 1. Performance Storage performance is a complex mix of drive, interface, controller, cache, protocol, SAN, HBA, driver, and operating system considerations. The overall performance of the storage architecture is typically measured in terms of maximum throughput, maximum I/O operations per second (IOPS), and latency or response time. Although each of these performance measurements is important, IOPS and latency are the most relevant to server virtualization. Most modern SANs use a combination of high-speed disks, slower-speed disks, and large memory caches. A storage controller cache can improve performance during burst transfers or when the same data is accessed frequently by storing it in the cache memory, which is typically several orders of magnitude faster than the physical disk I/O. However, it is not a substitute for adequate disk spindles because caches are ineffective during heavy write operations. For HUAWEI OceanStor S5500T storage system, with HUAWEI SmartTier storage feature, it is easy combining solid state disk, traditional SAS and SATA or NL-SAS disk to implement high performance using Smart Tier. Solid state disks producted by HUAWEI can be used for Tier 0 as the high performance layer, SAS disks are used as the performance layer, and the NL-SAS disks are used as capacity layer. Figure 4-10 SmartTier Feature for HUAWEI OceanStor S5000T Storage System Using SmartTier feature, hotspot data is migrated from SAS or NL-SAS/SATA disk to solid state disk automatically, which means that the hotspot data would be on the high performance layer and it is easy to implement high IO Ver 1.0 HUAWEI Technologies Co., Ltd 28

29 performance. 2. Drive Types The type of hard drive utilized in the host server or the storage array has the most significant impact on the overall storage architecture performance. As with the storage connectivity, high IOPS and low latency are more critical than maximum sustained throughput when it comes to host server sizing and guest performance. When selecting drives, this translates into selecting those with the highest rotational speed and lowest latency possible. Utilizing 15K RPM drives over 10K RPM drives can result in up to 35 percent more IOPS per drive. HUWEI OceanStor S5000T series Storage System supports SAS, NL SAS, and SATA, FC and SSD disks, fitting into various scenarios. For high performance it is recommended to choose solid state disk (SSD), for both performance and capacity it is recommended to choose SSD and SAS disks, for capacity NL-SAS and SATA disks are recommended. For this solution, SSD and traditional 2.5 inch 10K RPM SAS disks are used for both performance and capacity. 3. Multipathing In all cases, multipathing should be used. Generally, storage vendors build a device-specific module (DSM) on top of Windows Server 2012 multipath I/O (MPIO) software. Each DSM and HBA has its own unique multipathing options, recommended number of connections, and other particulars. In this private cloud solution, Microsoft Multipath I/O and Microsoft DSM are used as default. 4. Fibre Channel SAN Fibre Channel is an option, because it is a supported storage connection protocol. 5. iscsi SAN As with a Fibre Channel-connected SAN, which is naturally on its own isolated network, the iscsi SAN must be on an isolated network, both for security and performance. Any networking standard practice method for achieving this goal is acceptable, including a physically separate, dedicated storage network and a physically shared network with the iscsi SAN running on a private VLAN. The switch hardware must provide class of service (CoS) or quality of service (QoS) guarantees for the private VLAN. Encryption and authentication. If multiple clusters or systems are used on the same SAN, proper segregation or device isolation must be provided. In other words, the storage used by cluster A must be visible only to cluster A, and not to any other cluster, nor to a node from a different cluster. We Ver 1.0 HUAWEI Technologies Co., Ltd 29

30 recommend the use of a session authentication protocol such as Challenge Handshake Authentication Protocol (CHAP). This provides a degree of security as well as segregation. Mutual CHAP or Internet Protocol Security (IPSec) can also be used. Jumbo frames. If supported at all points in the entire path of the iscsi network, jumbo frames can increase throughput by up to 20 percent. Jumbo frames are supported in Hyper-V at the host and guest levels. 6. Thin Provisioning Particularly in virtualization environments, thin provisioning is a common practice. This allows efficient use of the available storage capacity. The LUN and corresponding CSV can grow as needed, typically in an automated fashion, to provide availability of the LUN (autogrow). However, storage can become overprovisioned in this scenario, so careful management and capacity planning are critical. HUAWEI OceanStor S5000T series Storage System allows on-demand allocation of storage space rather than the traditional method of pre-allocating all storage space at the initial stage. It is more efficient because the amount of resources used is close to the amount of resources allocated. In this way, the initial purchase cost and total cost of ownership are reduced. The thin provisioning feature for HUAWEI OceanStor S5000T Storage is called SmartThin, shown as follow. Figure 4-11 Smart Thin Feature for HUAWEI OceanStor S5000T Storage System 7. Volume Cloning Volume cloning is another common practice in virtualization environments. This can be used for both host and VM volumes to dramatically decrease host Ver 1.0 HUAWEI Technologies Co., Ltd 30

31 installation times and VM provisioning times. Using volume cloning feature on HUAWEI OceanStor S5000T series storage system, it is easy to implement LUN replication and backup, and data recovery from replica. 8. Volume Snapshots SAN volume snapshots are a common method of providing a point-in-time, instantaneous backup of a SAN volume or LUN. These snapshots are typically block-level and only utilize storage capacity as blocks change on the originating volume. Some SANs provide tight integration with Hyper-V, integrating both the Hyper-V VSS Writer on hosts and volume snapshots on the SAN. This integration provides a comprehensive and high-performing backup and recovery solution. The volume snapshots feature for HUAWEI OceanStor S5000T series storage system is shown as follow. Figure 4-12 Snapshots Feature for HUAWEI OceanStor S5000T Storage System Using Snapshots feature on Huawei OceanStor S5000T storage system, it is easy to implement LUN backup and recovery for Specific point-in-time Storage Automation One of the objectives of the Microsoft private cloud solution is to enable rapid provisioning and deprovisioning of VMs. Doing so on a large scale requires tight integration with the storage architecture as well as robust automation. Provisioning a new VM on an already existing LUN is a simple operation. However, provisioning a new CSV LUN and adding it to a host cluster are relatively complicated tasks that should be automated. Ver 1.0 HUAWEI Technologies Co., Ltd 31

32 4.5 Network Architecture Many network architectures include a tiered design with three or more tiers such as Core, Distribution, and Access. Designs are driven by the port bandwidth and quantity required at the edge, as well as the ability of the Distribution and Core tiers to provide higher-speed uplinks to aggregate traffic. Additional considerations include Ethernet broadcast boundaries and limitations, and spanning tree and/or other loop avoidance technologies. In this solution, to implement high availability, two HUAWEI Quidway S5700 switches are used with three VLANs on each switch, including VLAN A for iscsi, VLAN B for virtual machines and VLAN C for server host management. Figure 4-13 Network Fabric Interconnections As shown in Figure 4-12, the HUAWEI RH2288 V2 servers can connect to the existing liver migration and heartbeat LAN, iscsi SAN and virtual machine vswitch WAN network. 4.6 Virtualiztion Architecture Windows Server 2012 and Hyper-V Design 1. Operating System Configuration The following list outlines the general considerations for the Hyper-V host operating system. Note that these are not meant to be installation instructions but rather the process requirements. Ver 1.0 HUAWEI Technologies Co., Ltd 32

33 Use Windows Server 2012 Datacenter Edition, with either the full or server core installation option. Use the latest hardware device drivers. The Hyper-V parent partition operating system should be domain-joined. Hyper-V server role and failover clustering features are required. Apply relevant Windows updates, and all relevant out-of-band updates that are not offered on Microsoft Update. For more information, see Hyper-V Update List for Windows Server 2012 in the Microsoft TechNet Library. All nodes, networks, and storage must pass the Cluster Validation Wizard in Windows Server Performance Settings Test the following Windows Server 2012 Hyper-V network performance improvements and consider them for production use: TCP checksum offload benefits both CPU and overall network throughput performance, and is fully supported by live migration. Jumbo frames capability is extended to VMS with Hyper-V in Windows Server Just as in physical network scenarios, jumbo frames add the same basic performance enhancements to virtual networking. That includes up to six times larger payloads per packet, which improves overall throughput and also reduces CPU utilization for large file transfers. Virtual machine queue (VMQ) allows the host s single NIC card to appear as multiple NICs to the VMs by allowing the host s NIC to direct memory access (DMA) packets directly into individual VM memory stacks. Each VM device buffer is assigned a VMQ, which avoids needless packet copies and route lookups in the virtual switch. The result is less data in the host s buffers and an overall performance improvement to I/O operations. 3. IP Networks Configuration Ensure that the following rules are followed when setting up the IP network: The cluster heartbeat network must be on a distinctly separate subnet from the host management network. The VM network adapter should not be shared with the host operating system and therefore should not have the TCP/IP protocols (IPv4 and IPv6) bound to it. The iscsi network must be on a distinctly separate and isolated network, with a dedicated IP range used only for storage. 4. MPIO Configuration Ver 1.0 HUAWEI Technologies Co., Ltd 33

34 Microsoft MPIO architecture supports iscsi, Fibre Channel, and Serial Attached Storage SAN connectivity by establishing multiple sessions or connections to the storage array. Multipathing solutions use redundant physical path components-adapters, cables, and switches-to create logical paths between the server and the storage device. In the event that one or more of these components fails, causing the path to fail, multipathing logic uses an alternate path for I/O so that applications can still access their data. Each network interface card (in the iscsi case) or HBA should be connected using redundant switch infrastructures to provide continued access to storage in the event of a failure in a storage fabric component. Failover times vary by storage vendor, and can be configured using timers in the Microsoft iscsi Software Initiator driver, or by modifying the Fibre Channel host bus adapter driver parameter settings. Microsoft Multipath I/O (MPIO) must be used by all storage adapters, both iscsi and Fibre Channel. Follow MPIO best practices as documented in Appendix B of the MPIO white paper Windows Server High Availability with Microsoft MPIO. In this solution, the Microsoft MPIO is used for iscsi interconnection as default option. It is recommended to use HUAWEI UltraPath in Windows Server 2012 as the default multipath software. 5. NIC Teaming Configuration NIC teaming or link aggregation (IEEE 802.3ad) enables network maintenance to occur at all points within the data center network topology without affecting applications. This technology bonds physical NICs together to form one or more logical network team that sends traffic to all NICs in the team. This allows a single NIC, cable, or switch to sustain a planned or unplanned outage without disrupting the host s Ethernet traffic. The NIC manufacturer is also the software provider for the NIC teaming software. Each NIC teaming software application has its own unique set of requirements, features, teaming modes, and configuration recommendations. NIC teaming should be used to provide high availability to the VM networks. In this solution, NIC teaming is used for VM virtual switch, live migration, heartbeat traffic, and management console. Detailed NIC teaming information is shown as follow. Ver 1.0 HUAWEI Technologies Co., Ltd 34

35 Figure 4-14 NIC Teaming for HUAWEI and Microsoft Fast Track Solution on Server Hosts For detailed information, please see Server Design Principles Hyper-V Host Cluster Design A failover cluster is a group of independent computers that work together to increase the availability and scalability of clustered roles (formerly called clustered applications and services). The clustered servers (called nodes) are connected by physical cables and by software. If one or more of the cluster nodes fail, other nodes begin to provide service (a process known as failover). In addition, the clustered roles are proactively monitored to verify that they are working properly. If they are not working, they are restarted or moved to another node. Failover clusters also provide Cluster Shared Volume (CSV) functionality that provides a consistent, distributed namespace that clustered roles can use to access shared storage from all nodes. With the Failover Clustering feature, users experience a minimum of disruptions in service. Failover clusters in Windows Server 2012 are managed by using the Failover Cluster Manager snap-in and the Failover Clustering Windows PowerShell cmdlets. File shares on file server clusters can additionally be managed by using the tools in File and Storage Services. The host servers are one of the critical components of a dynamic virtual infrastructure. Consolidation of multiple workloads onto the host servers requires that those servers be highly available. Windows Server 2012 provides advances in failover clustering that enable high availability and live migration of VMs between physical nodes. 1. Server Topology Ver 1.0 HUAWEI Technologies Co., Ltd 35