Next-generation data center infrastructure

Transcription

1 ericsson White paper Uen February 2015 Next-generation data center infrastructure MAKING HYPERSCALE AVAILABLE In the Networked Society, enterprises will need 10 times their current IT capacity but without 10 times the budget. Leading cloud providers have already changed the game by developing their own hyperscale computing approaches, and operators and enterprises can also adopt hyperscale infrastructure that enables a lower total cost of ownership. This paper discusses the opportunity for a competitive economic, operational and technical solution to deliver on the future needs of enterprises. This paper was developed in collaboration between Ericsson and Intel.

2 Introduction The current rate of technical change is exponential, not linear. As the book Exponential Organizations by Salim Ismail explains: Together, all indications are that we are shifting to an information-based paradigm when you shift to an information-based environment, the pace of development jumps onto an exponential growth path and performance/price doubles every year or two [1]. This means that when a technical challenge is 1 percent accomplished, such as mapping the human genome, it is actually halfway to completion. This also explains why the amount of change in one year is always overestimated, but the amount of change in 10 years is always underestimated. In the Networked Society, everything that benefits from being connected will be connected. As a result, in the near future the vast majority of end points on networks are going to be machines independent of humans, rather than humans holding machines. New use cases will include data collection and control signaling, alongside voice, media and messaging. And in terms of traffic flow, the new use cases will be predominantly upload (data collection), not download. We do not know exactly what these use cases will require from the underlying compute, storage and network infrastructures, but early indications are that today s design assumptions will be challenged, and that the future will require a new hyperscale approach. NEXT-GENERATION DATA CENTER INFRASTRUCTURE INTRODUCTION 2

3 A different model All businesses are becoming both software companies and information enabled. In the near future, companies will be dependent on 10 times the IT capacity of today, yet they will not have 10 times the budget to deliver. Their approach must therefore change. The companies that have already accomplished this transformation are the public cloud providers (Google, Facebook and Amazon, among others), which have developed their own hyperscale computing approaches, and changed the rules of the game in data center design, construction and management. They have moved on from traditional IT practice around data center design, hardware procurement and life cycle management and business operation. To deliver hyperscale computing, they no longer buy from traditional IT vendors but instead have created their own technology and operations in-house. By having discipline and focus on issues of power, cooling, server, storage, network, automation and governance, these leading cloud providers have reached new levels of efficiency, performance and agility to realize what the industry is starting to call web-scale IT. This approach enables them to rapidly scale up or down, to adapt and deliver highly focused and resilient customer-centric solutions, while dramatically minimizing resource wastage and operational costs. The problem for the rest of the world is that the leaders in this space are not vendors, while the traditional IT vendors have not stepped up to meet the new demand. Most enterprises do not have access to the resources needed to develop their own hyperscale architecture. Instead, they must rely on equipment from traditional IT vendors or custom management solutions with little control or vendor support. This puts them at risk of getting left behind economically. Meanwhile, the traditional IT vendors are using a model that ensures margin maintenance for the vendor, but at the expense of the customer that must compete against hyperscale cloud performance and characteristics. This means that most companies have a bad choice: either use the public hyperscale cloud providers and be constrained to their business mandates, or carry on buying from traditional IT vendors and risk not being competitive in their market. NEXT-GENERATION DATA CENTER INFRASTRUCTURE A DIFFERENT MODEL 3

4 Disaggregated hardware architectures As the massive growth of information technology services places increasing demand on the data center, it is important to re-architect the underlying infrastructure, allowing companies and users to benefit from an increasingly services-oriented world. The core of a disaggregated architecture is a range of compute, fabric, storage and management modules that work together to build a wide range of virtual systems. This flexibility can be utilized differently by different solution stacks. In such a system, the following four pillars of capability are essential: > > manager for multi-rack management: this includes hardware, firmware and software application programming interfaces (APIs) that enable the management of resources and policies across total operation and expose a standard interface to both hardware below it and the orchestration layer above it. > > pooled system: this enables the creation of a system using pooled resources that include compute, network and storage based on workload requirements. > > scalable multi-rack storage: this includes Ethernet connected storage that can be loaded with storage algorithms to support a range of uses. Different configurations of storage hardware and compute nodes with local storage, as well as multi-rack resources, should be made available. > > efficient configurable network fabric: the networking hardware, interconnect (cables, backplane) and management that support a wide range of cost-effective network topologies. Designs should include current top of rack switch designs but also extend to designs that utilize distributed switches in the platforms, which remove levels of the switch hierarchy. Combining a disaggregated hardware architecture with optical interconnect removes the traditional distance and capacity limitations of electrical connections. It enables efficient pooling of resources while removing connectivity bottlenecks, which is critical for real-time services, for example. As data center demand grows, high-speed optical connectivity helps organizations to scale out by enabling software to span across large hardware pools that are geographically distant and do not simply scale up with bigger and faster hardware locally. NEXT-GENERATION DATA CENTER INFRASTRUCTURE DISAGGREGATED HARDWARE ARCHITECTURES 4

5 Hyperscale performance In an exponentially changing world, hyperscale performance is not a static state, and in the real world, it has to start from where an organization finds itself today. One methodology to achieve continuously improving hyperscale performance comes from adoption and continuous iteration through the industrialization cycle. This drives organizations to have one approach to all they do, since this is where best performance has the optimum chance of being found. Adoption of the approach enables disruptive economics, uses catalytic modernization capabilities, and depends on having an infrastructure approach that can break the traditional refresh cycle. The ability to answer Govern seemingly basic questions such as: How many servers are in operation?, Where are they?, and What are they running? highlights the maturity of any organization in its industrialization journey. The public cloud providers have this information in real time as a side effect of how they approach their total business and infrastructure management. The industrialization cycle shown in Figure 1 represents an operational improvement model continuously driven by business case argumentation. Automate STANDARDIZE The first step to modernization is to standardize one set of technologies. Google is a great example of a company that has standardized on a single hardware, software, operational and economic strategy all on an end-to-end basis. The key benefit of standardization is increased efficiency Figure 1: The industrialization cycle. the continuous focus on standardization enables continuous improvement. Relevant questions that organizations should ask here include: > > To what extent has your organization standardized along the dimensions of facilities, hardware, software, operations and economics? > > How effectively is your organization reaping the benefits of continuous improvement from standardization? Data center and central office Abstract Standardize Combine COMBINE Organizations need a combine and consolidate strategy. Consolidate means taking everything and bringing it onto one platform, and combine means leaving selected legacy systems where they are. It is critical to recognize that some things sit perfectly fine on a legacy mainframe, and should remain there. However, data and capabilities need to be combined so that they are exposed through a single platform. Relevant questions that organizations should ask here include: > > As an organization, where are you selectively consolidating systems? > > Where and how are you pursuing a combination strategy? > > Are data center economics a problem? How are you achieving high occupancy rates, so that your data center economics are not dominated by real estate costs? NEXT-GENERATION DATA CENTER INFRASTRUCTURE HYPERSCALE PERFORMANCE 5

6 ABSTRACT Organizations need to create proper levels of abstraction to expose cloud functionality. A key advantage of cloud systems is that they are not a black box; they are in fact highly accessible. Relevant questions that organizations should ask here include: > > How have you approached the abstraction of your cloud-based systems? > > At what layers in the cloud stack have you modularized and abstracted? How well is this working for you? > > Are there functions or capabilities where complete programmability is lacking? AUTOMATE The process of automation delivers benefits in terms of cost savings and agility. Taking out the human element provides opex benefits and facilitates responsiveness. However, it also introduces the question of security and control automation and governance really go hand-in-hand. Relevant questions that organizations should ask here include: > > To what extent have you been able to reduce the human element in your cloud system? Where would you like to push harder to derive additional opex benefits? > > How much of your budget is spent on maintaining what you currently have? > > What needs to happen for your organization to accelerate its automation journey? > > How would you prioritize the opportunities for automation and the level of effort required? GOVERN Normally, security is managed by limiting access. But as already noted, clouds are highly accessible. Think about the public cloud a situation where unknown people with unknown credentials are handling unknown workloads and have complete access to hardware. This is the very definition of compromise. Security is the biggest inhibitor to cloud adoption, and there is no question that robust security models have to complete the industrialization cycle. Relevant questions that organizations should ask here include: > > Security: How secure would you say your key cloud implementations are? Why? How are you approaching cloud security today? > > Governance: What trade-offs are you making in exchange for governance (for example, speed or agility)? > > Governance: How much time does it take for a new policy to be enacted across all your data centers? For example, would you be able to change all passwords with a single command? If not, how distant is this from your reality today? > > Compliance: Do you have processes for procedures that come up regularly, such as the disposal of a server? > > Compliance: At any point in time, have you risked breaking the law because of jurisdiction or data location issues? Short-term vendor management on lowest price wins will not translate to business success unless placed in the context of true economic cycle improvement and thus continuous business improvement. If there is no continuous focus on standardizing facility, hardware, software, operations and economic strategy, there will not be continuous improvement. And if there is no continuous combine/consolidate strategy, then there will not be capex savings. Similarly, if there is no focus on abstracting underlying complexity into simple APIs, then programmability will not be possible and automation will not happen, translating to no opex savings or agility since humans are still required. And if there is no focus on governance, then all decisions taken may work or may not, may cause liability or may not. It will simply not be known until the first problems arise. NEXT-GENERATION DATA CENTER INFRASTRUCTURE HYPERSCALE PERFORMANCE 6

7 Disruptive Economics The effects of exponential technology change are all around us. Digital infrastructure is no different and defines the future market opportunity but delivering it requires adoption of best practice hyperscale approaches. Two backend engineers can now scale a system to support over 30 million active users [2]. Web-scale IT is a pattern of global-class computing that delivers the capabilities of large cloud service providers within an enterprise IT setting by rethinking positions across several dimensions. By 2017, web-scale IT will be an architectural approach found operating in 50 percent of global enterprises, up from less than 10 percent in 2013 [3]. The benefits of adoption of hyperscale hardware include: > > New levels of capex and opex savings while decreasing time to market. > > New levels of utilization and operating resource efficiencies. > > On-demand infrastructure is treated as a single system with one focus on building, managing and operating from an application delivery cost perspective. It is application-defined infrastructure. > > Lower total cost of ownership (TCO) results from full awareness of hardware infrastructure and system workloads as well as process change. In addition to designing hyperscale data centers, telecom operators have a particular opportunity to utilize under-occupied central office real estate to install modern data center infrastructure as a complement to (or replacement for) traditional IT equipment. Done the right way, this will result in huge increases in data center capacity and performance, as well as significant improvements in TCO and utilization per square meter of real estate. However, such an undertaking will require careful planning of the data center environment when it comes to, for example, power supply and cooling, which might look very different compared with central office usage. Designing for high-energy performance targeting reduced energy consumption is one of the key requirements in the development of hyperscale infrastructure. Traditionally, energy efficiency in data centers has primarily, or even exclusively, focused on the efficiency of the site equipment. Hyperscale infrastructure takes energy performance one step further by focusing on the network functionality platform itself. There are multiple reasons for the increased attention given to high network energy performance. Energy-related opex is a significant cost for data center operations. In addition, there is also an increasing awareness of and concrete requests for resource efficient and sustainable solutions among operators. Transparent asset management and server usage across the data center combined with software-defined infrastructure can take server utilization to the next step, beyond what has already been achieved with virtualization. This enables a more efficient packing of resources to fewer hardware units during low traffic periods. High server utilization, in combination with the ability to set unused servers in various types of sleep modes (or even to turn them off completely), has the potential to reduce power consumption and thereby energy-related opex significantly. This is enabled by modern server components, including the central processing unit (CPU), with state of the art energy saving features that enable significant reduction of energy consumption, at small loads in particular. Furthermore, if all intra-data center transmission solutions are optical, this excludes more energy consuming electrical transmission. This not only enables high performance but also minimizes energy consumption for extensive intra-data center transmissions. Although high-energy performance in hyperscale infrastructure starts with decreased energy consumption at the node and platform level, the site aspects remain crucial and must also be adequately addressed. Modern data center building practices for cooling and high voltage power feeds have in many places already realized significant savings in energy and cost. Modern cooling NEXT-GENERATION DATA CENTER INFRASTRUCTURE DISRUPTIVE ECONOMICS 7

8 techniques involving the separation of hot and cold air, using free-cooling, cooling close to the source and sometimes liquid, can often reduce energy (and CO2 emissions) by a factor of nearly 50 percent for the complete data center compared with older techniques. This gives it significant potential in modernizing older central office building practices. For operators, this further supports industry initiatives around software-defined networking and Network Functions Virtualization. These initiatives enable a transformation to a unified hyperscale architecture, which brings significant benefits. Chief among these is greater agility in operations, application creation and delivery, and network provisioning and adaptation. TCO is greatly reduced when resources can be rapidly and dynamically shared across multiple processes, applications and customers. In summary, hyperscale infrastructure has a positive effect on the environment as well as on operational cost. NEXT-GENERATION DATA CENTER INFRASTRUCTURE DISRUPTIVE ECONOMICS 8

9 Perpetual refresh Hyperscale infrastructure enables life cycle management and TCO to move from unit-based servers to individual components. The introduction of hardware disaggregation breaks the threeto-five-year refresh cycle, enables the replacement of components that benefit most from refresh, and eliminates forced replacement of entire systems. Compute and memory have the shortest refresh rates (one to two years) while the chassis has the longest (10 plus years). Disaggregation enables scale up or down at the component level, as well as the customization of hardware resources for unique application requirements. The customization can be defined per workload on an on-demand basis. Disaggregation also enables the use of the most up-todate components, which drive continuous best in class return on investment and capabilities, as well as a higher degree of flexibility. For example, disaggregation makes it possible to run a data center in a multi-vendor environment in which components can be procured from the best source. Such a scenario requires advanced infrastructure management tools, both for managing the use of the resources and to automate processes. A unified infrastructure management architecture can improve utilization by enabling the allocation of shared resources when a new application or workload is introduced, rather than requiring additional data center hardware. NEXT-GENERATION DATA CENTER INFRASTRUCTURE PERPETUAL REFRESH 9

10 Catalytic modernization Not all infrastructure can be replaced at the same time. However, through innovative equipment management software, the old and new hardware can be brought under one intelligence, governance and life cycle model. For example, it should be possible to perform natural language search for every component in all infrastructure. It should be possible to interrogate the BIOS version across the total installation of, say, 10,000 machines and replace a vulnerable version with one that is secured in one action. This should be possible across any vendor and any device, all managed from one single interface. To run high performance operations at hyperscale, it is important to enable analysis of all state information in real time. There is a need for comprehensive intelligence on performance, state and reliability. And to enable simple scale actions as referred to above, there is a need for simple, scalable workflows for driving complex, massive-scale configurations that can enable deployment of new machines in minutes, not months. When evaluating catalytic modernization, it is important to understand the six axes of scalability in hardware design. These represent independent levers that directly influence and address a variety of workloads, as shown in Figure 2. The six axes are: 1. CPU core count 2. dynamic random access memory (DRAM) size (memory available to applications) 3. storage capacity 4. storage performance (read/write performance) 5. network capacity (bandwidth available) 6. network performance (speed and latency of the network). The differential rates of change in these six components drive the need to consider them separately as part of overall data center life cycle management. Together with the disaggregated hardware approach, this enables organizations to implement a framework that systematically addresses all of the major workloads using a single system, as shown in Figure 3. The benefits extend beyond the positive capex and depreciation impacts that come from only replacing what needs to be replaced. The six axes of scalability in combination with the governance, automation, introspection and control of an DRAM size CPU core count Storage capacity Axes of scalability Storage performance Figure 2: The six axes of scalability. Standard workloads Large memory footprint Maximize throughput across six axes of scalability Network capacity Network performance Intensive CPU needs Single system architecture CPU core count DRAM size Storage performance Storage capacity Network capacity Network performance Bandwidth intensive Standard High memory High CPU High network I/O Figure 3: A single system architecture addresses all major workloads. Directly impact the number of applications that the system can support Impacts the system disk I/O capabilities Impacts the amount of data the system can store Impact the inter and intra system messaging, communications and throughput Large storage, low I/O High storage Fast disk read/write High disk I/O NEXT-GENERATION DATA CENTER INFRASTRUCTURE CATALYTIC MODERNIZATION 10

11 infrastructure manager enable organizations to gain additional operational agility from largescale resource orchestration and automation. They also help ensure optimal service experience by tailoring resources to meet specific workloads. Furthermore, they improve operational cost efficiency, with market-leading cloud providers having used these methodologies to move from the industry standard of 1:300 (system administrators-to-server ratio) to a new industry benchmark of 1:25,000. This kind of efficiency should be enabled by any new data center hardware. NEXT-GENERATION DATA CENTER INFRASTRUCTURE CATALYTIC MODERNIZATION 11

12 Conclusion The rate of technical change is exponential, not linear. In the future, companies will be dependent on 10 times their current IT capacity, yet they will not have 10 times the budget to deliver. The leading public cloud providers (Amazon, Facebook and Google, among others) have developed their own hyperscale computing approaches and have changed the rules of the game in data center design, construction and management. They have moved on from traditional IT practice around data center design, hardware procurement and life cycle management, and business operation. The need for new infrastructure will enable life cycle management and TCO to move from unit-based servers to individual components. The introduction of hardware disaggregation breaks the three-to-five-year refresh cycle, enables the replacement of components that benefit most from refresh, and eliminates forced replacement of entire systems. It will also make it possible to perform natural language search for every component in all infrastructure, and to manage any vendor and any device from a single interface. The approach outlined in this paper creates the opportunity for a competitive economic, operational and technical solution to deliver on the future needs of hyperscale infrastructure. One aspect of this is improved resource utilization, leading to lower energy consumption and a positive environmental effect. The concluding industrialization cycle represents a continuous operational improvement model that, if followed, creates total improvement rather than shortterm strategy. The future will combine the roads to 5G and the next generation cloud [4], and will give organizations the platform they need to transform from a sense of deploy and hope to one of trust and security. NEXT-GENERATION DATA CENTER INFRASTRUCTURE CONCLUSION 12

13 GLOSSARY API CPU DRAM I/O TCO application programming interface central processing unit dynamic random access memory input/output total cost of ownership NEXT-GENERATION DATA CENTER INFRASTRUCTURE GLOSSARY 13

14 References [1] Salim Ismail, 2014, Exponential Organizations: Why New Organizations Are Ten Times Better, Faster, and Cheaper than Yours (and What to Do About It), Diversion Books. [2] The Economic Times, May 2012, Cloud computing: How tech giants like Google, Facebook, Amazon store the world s data, available at: com/ /news/ _1_instagram-largest-online-retailer-users [3] Gartner, March 2014, Gartner Says By 2017 Web-Scale IT Will Be an Architectural Approach Found Operating in 50 Percent of Global Enterprises, available at: [4] Ericsson, February 2015, White Paper: 5G systems enabling industry and society transformation, available at: papers_ _c 2015 Ericsson AB All rights reserved NEXT-GENERATION DATA CENTER INFRASTRUCTURE REFERENCES 14