How Microsoft IT Planned and Architected an Enterprise Private Cloud Infrastructure Published: July 2012 The following content may no longer reflect Microsoft s current position or infrastructure. This content should be viewed as reference documentation only, to inform IT business decisions within your own company or organization. Learn how Microsoft IT envisioned an enterprise private cloud using System Center and Windows Server technologies. This article discusses the planning and architecting by Microsoft IT in preparation for moving applications and services to a private cloud providing increased agility, improved economics and innovative technology to the Microsoft internal organization. Intended Audience CIO CTO Technical Decision Maker Products & Technologies Microsoft System Center 2012 Microsoft Windows Server 2012 Introduction The Microsoft Information Technology (MSIT) organization recognizes that as the company grows and evolves and business units and product groups mature, the current infrastructure will be tested. MSIT is consistently working to determine the best ways to balance the need for increased technological capabilities while simplifying and improving processes and reducing overall cost. Situation The critical challenge of IT is meeting the needs and pressures of a continually evolving, expanding, and robust customer base that has high expectations and demands of the IT infrastructure. At Microsoft, it has become increasingly clear that the traditional server environment was not consistently adapting the way that it needed to. Incremental changes and improvements did not resolve the issues that application customers were encountering and did not increase the efficiencies of infrastructure operations. Even as Microsoft IT made improvements with consolidation and virtualization, many problems remained: Increased Capacity Demands: Microsoft internal customers often reported that capacity was not available when it was needed. If they could plan for demand months in advance, they were able to incorporate the standard lead time for hardware procurement and the configuration process into their business planning. However, if their plan changed or the project needed to be expedited, that lead time could create a serious barrier to the success of the project. Tool, Process, and Infrastructure Inconsistencies: Tools, processes and infrastructure were not consistent across environments. Hardware and data centers were rolled out over time resulting in variation in functions, capabilities, and tools. The customer might develop on one environment and deploy to production on another, creating uncertainty regarding the application performance and functionality. As the first enterprise customer of Microsoft products, Microsoft IT is an excellent test case for moving to the cloud. Because of the size and complexity of the organization, they face
many challenges throughout the transition to the cloud that other companies will also encounter. Microsoft will be able to provide real-life solutions, guidance, and best practices to companies who decide to follow Microsoft's lead. Solution Microsoft business leaders are committed to a comprehensive cloud solution Steve Ballmer was quoted as saying, "We're all in " back in 2010 when referring to Microsoft's commitment to cloud computing, and he continues to emphasize that message. Microsoft's CFO, COO, and CIO have also made commitments to leading cloud technology adoption and development. Microsoft IT's focus has been on finding what one IT Professional called the "sweet spot" where technology, infrastructure, and application architecture come together in the perfect storm creating the ideal time to move towards the goal of "making the cloud the center of our IT development and planning process." Microsoft determined that the solution should provide the following characteristics and features that are fundamental to achieving cloud benefits: Pooled Resources: It is no longer necessary to treat the environment as though it is limited to the constraints of a server, a set of disks, or a network path. Now there is a pool of compute, storage, and network resources that can be carved up logically and assigned as needed. Elasticity: By looking at the environment as pooled capacity, resources can be added or removed in both a physical and logical way without affecting service. Self-Service: As System Center is used to automate workflows and runbooks, those automated processes can be delegated directly to the customer to initiate at their convenience. Hardware Abstraction: The application owner or customer is able to convey their requirements of availability, capacity, and performance in generic terms of compute, storage, and network. They no longer need to understand the specific architectural and operational attributes of the hardware platform. Usage-Based Model: Efficiencies gained by pooling of resources and elasticity of the environment allows for finer-grained usage and billing models. Where customers previously requested and paid for capacity whether they actually used it or not, with cloud computing they pay only for what they use. Additionally, MSIT has design and implementation goals that further increase the value of the private cloud: End-to-end control of the environment Ability to customize the infrastructure and cloud services Isolation for workloads and data where required Common capabilities, tools, and process with Microsoft Windows Azure public cloud solutions where it is possible Throughout the planning and architecting process, MSIT has continued to look at the opportunities these various benefits can provide now and in the future. Other companies can learn from this process as they also begin to move to cloud-based IT solutions. They must understand their own current needs and try to predict their future requirements and Page 2
capabilities. Because progress and change are a fundamental part of this process, planning for future developments is critical. Internally, Microsoft IT works to engage with other teams, as early as possible, to help them with their IT requirements. An early partnership with product groups as they develop and test new products and services helps remove barriers to deployment. This also gives MSIT a view into the future of a product or service and allows them to anticipate potential problems as well as new opportunities. Expected Benefits/Anecdotal Evidence As businesses evolve and as processes change and improve, the number of applications, devices, and systems is escalating. This increases IT complexity, requiring a better way to plan for and manage the exponential growth. By moving to a cloud-based infrastructure, MSIT is proactively addressing this challenge and is already recognizing these preliminary benefits: On-demand services, delivered when they are needed Scalable capacity, as much as you need for your applications Developers can concentrate on improving and optimizing applications without needing to be concerned with the details of the infrastructure layer Increased focus on innovation and improvement, as opposed to the majority of effort going into sustaining maintenance Data center administration is simplified and predictable Although Microsoft IT had virtualized most of their server environment using Windows Server, Hyper-V, and System Center products prior to the private cloud efforts, achieving even the preliminary benefits described previously required a new set of products and approach. The specific design of the infrastructure is being optimized, but general design principles are in place. New Microsoft and partner products have been chosen. There is the potential to move from 60% virtualized to upwards of 90% of the total server workload population, while also adding process automation improving our effectiveness with critical applications. Changes in design and adoption of capabilities in new products include, but are not limited to, the following: Hardware Infrastructure o Four processor socket many core systems o Very large memory (512 GB or more) systems o Increased per-system I/O for network and storage o Resiliency and redundancy built-in for high availability o Defined fault and mobility domains o Scalable building block approach Windows Server 2012 with Hyper-V o Support for very large virtual machines 32 cores and 1 TB memory o Improved migration technology Concurrent live migrations Inter or non-cluster migration o Inbox network LBFO (Load Balancing and Failover) Also known as NIC teaming Virtual FC (Fibre Channel) for VMs o Page 3
o Numerous virtual switching improvements Quality of Service Access Control Lists Network Virtualization Extensibility o Failover replica virtual machines System Center 2012 o Creation of cloud fabrics using Microsoft Virtual Machine Manager (VMM) o Dynamic host and power optimization using VMM o General automation and integration using Microsoft System Center Orchestrator o Comprehensive monitoring of the infrastructure, including the network, using Microsoft Operations Manager o Deep application insight and service-centric monitoring using Operations Manager Application Performance Monitoring (APM) o Service catalog and process management using Service Manager o Self-service enablement using App Controller, Service Manager, and Virtual Machine Manager o Data backup/restore and disaster recovery capability using Data Protection Manager Microsoft IT's Journey to Implementation Throughout the planning process, one of the principal questions has been how to implement this new infrastructure without limiting its potential. Understanding how to preserve the best of traditional IT while embracing the advantages of the new thinking related to the private and public cloud is challenging. MSIT is addressing this challenge by carefully identifying and recognizing possible pitfalls as well as appreciating the potential benefits and capabilities that are inherent to each of these environments. MSIT developed their plan for an enterprise private cloud using this information, creating the best possible solution to fit the current organization. Microsoft IT is also striving to balance private cloud investment with very aggressive efforts to adopt and migrate to Microsoft public cloud services, such as Microsoft Office 365 and Windows Azure. Enterprise server workloads at various stages of their software life cycle already occupy the spectrum of cloud solutions, making the concept of hybrid cloud a reality for MSIT. Understanding the varying characteristics and capabilities of the public cloud services and then measuring them against application portfolio requirements allows MSIT to determine what to include and how to invest in developing a private cloud solution. These characteristics and capabilities are also used as criteria for directing workloads to the most suitable cloud when they are ready to adopt or migrate. Additionally, MSIT uses this information to select applications that will get the largest benefit from moving to each cloud solution. In an environment with tens of thousands of potential cloud candidates, it is important to prioritize applications that will receive the greatest and most immediate value from the cloud, rather than simply using a first come, first served adoption. Page 4
Figure 1 represents MSIT s anticipated cloud migration patterns, methods, and subjective criteria mentioned previously. As cloud technologies and business priorities evolve, these patterns, methods, and criteria will change over time. The goal is to stay on top of this evolution and to adjust to the changes in a way that moves Microsoft most quickly out of "Traditional IT" and into the right cloud environments. Figure 1. Anticipated Cloud Migration Patterns Conclusion As part of a long-term IT strategy, creating a private cloud infrastructure is an effective way for Microsoft IT to support, manage, and adapt to the growing and changing needs of the internal organization. By utilizing best-in-class technologies such as System Center 2012 and Windows Server 2012, there is strong, preliminary evidence that Microsoft IT's continued transition from a traditional IT environment to a cloud environment will deliver many advantages and value to MSIT's internal customers as well as external Microsoft customers. Page 5
For More Information For more information about Microsoft products or services, call the Microsoft Sales Information Center at (800) 426-9400. In Canada, call the Microsoft Canada information Centre at (800) 563-9048. Outside the 50 United States and Canada, please contact your local Microsoft subsidiary. To access information through the World Wide Web, go to: http://www.microsoft.com http://www.microsoft.com/technet/itshowcase 2012 Microsoft Corporation. All rights reserved. This document is for informational purposes only. MICROSOFT MAKES NO WARRANTIES, EXPRESS OR IMPLIED, IN THIS SUMMARY. Microsoft, Hyper-V, Windows Azure, and Windows Server are either registered trademarks or trademarks of Microsoft Corporation in the United States and/or other countries. The names of actual companies and products mentioned herein may be the trademarks of their respective owners. Page 6