RE-DEFINING THE GREEN DATA CENTER

RE-DEFINING THE GREEN DATA CENTER A Dell Technical White Paper MANAGING THE DATA CENTER BY EFFICIENT USE OF IT RESOURCES By John Pflueger, Ph. D. Dell Data Center Infrastructure www.dell.com/hiddendatacenter

CONTENTS EXECUTIVE SUMMARY 2 INTRODUCTION 2 POWER AND CONSUMPTION AND ENERGY EFFICIENCY IN THE DATA CENTER 2 FRAMING THE PROBLEM 4 MANAGING DATA CENTERS FOR PRODUCTIVITY 6 CONCLUSION 13 FIGURES FIGURE 1: PROJECTED DATA CENTER ENERGY USE UNDER FIVE SCENARIOS (U.S. EPA, 2007) 3 FIGURE 2: DELL HISTORICAL SYSTEM PERFORMANCE (CFP2000rates) 5 FIGURE 3: FACTORS IN IMPROVING FACILITY PRODUCTIVITY 6 FIGURE 4: ESTIMATE OF HISTORICAL DATA CENTER PRODUCTIVITY METRICS FOR A SAMPLE DATA CENTER 8 FIGURE 5: COMPARISON OF ESTIMATED AND ADJUSTED ENERGY CONSUMPTION FOR VOLUME SERVERS (2000-2008) 8 FIGURE 6: DM1 STARTING SERVER POPULATION 9 FIGURE 7: PARETO CHART OF DM1 SERVER POPULATION USEFUL Work 9 FIGURE 8: ESTIMATED RELATIVE WORK PER RACK BY DATA CENTER ROW (DM1) 10 FIGURE 9: EVOLUTION OF A DATA CENTER S EXPECTED PRODUCTIVITY OVER TIME 10 FIGURE 10: DATA CENTER IT UTILIZATION ESTIMATES 11 FIGURE 11: ESTIMATED DATA CENTER POWER CONSUMPTION AS A FUNCTION OF UTILIZATION POLICIES 12

Executive Summary Server and data center power consumption are concerns for data center owners and managers and for policy-makers. Additionally, societal trends and interest in Green and Sustainable technologies have increased focus on data centers. The IT industry, policy-makers and regulators have been working this issue, focusing on improvements to infrastructure efficiency. While great strides have been made in this area, the potential for further improvement is diminishing. At the same time, we have been learning more about how data centers use resources and the ways in which they convert resources into their end product: information. While the interest and attention on infrastructure efficiency has been warranted, the real issue with data center power consumption has been a general lack of control over how data centers utilize IT assets. If IT utilization rates had been held constant over the past eight years, it is possible that the aggregate power consumption of volume servers in the U.S. would have held close to constant as well. Modeling work at Dell has shown that technologies and policies available to IT managers today have the potential to reverse the trend of declining IT utilization with resultant positive effects on data center power consumption. The Greenest data center is thus, not the one with the best efficiency with respect to delivering power to the IT equipment, but, instead, the one that manages to make the best use of the IT equipment it has commissioned. Introduction Most of us are aware these days of increasing concerns about the cost and availability of energy. We re also aware that we are now living in the Connected Era. Compute power, and our ability to leverage that power, has driven significant productivity and economic growth. It is not unexpected, then, that these two trends are converging. An increasing demand for computation has led to increasing industry power consumption. This in turn, has led to concerns about the sustainability of this computing engine: Can we support our needs for computation with the resources required by our servers and data centers? Industry action to-date has been substantial, but with a strong focus on garnering benefits from improving data center infrastructure efficiency minimizing losses in power distribution and improving facility cooling efficiency. Today, however, available power and cooling products and emergent power and cooling architectures are quickly reaching economic limits on efficiency / performance. In addition, an analysis of Data Center Productivity shows that these benefits, while very important, are only a stepping stone to more significant strategies with a more profound effect on data center performance. The much greater problems are the mechanisms our industry has incorporated into its IT architectures and policies that have led to chronic IT waste. In the future the greenest data center will not necessarily be the one with the most efficient power distribution architecture or the one that best leverages the local environment to minimize power consumption related to cooling equipment. Instead, Green will be defined by the efficiency with which a data center converts its resources into computation. Management of these facilities will focus on minimizing IT waste - solving the problems of poor utilization of IT assets and allocation of resources to relatively less productive equipment. Modern manufacturing facilities go to great lengths to ensure that capital equipment is properly utilized. Why should data centers be any less disciplined? 2 Power and Consumption and Energy Efficiency in the Data Center While data center owners and managers are concerned about the longevity of their data centers and their future ability to implement applications and commission equipment, aggregate server and data center power consumption has drawn the attention of individuals and organizations developing and driving energy policy worldwide. By now, many, if not most, of us are aware of the findings of the EPA s report on Server and Data Center Power Consumption from August of 2007 (U.S. EPA, 2007). One of the conclusions of this report was that, in the U.S. in 2011, servers and data centers could consume 2.5% of all purchased electricity. The figures for server and data center power consumption are staggering. They demand action. In order to determine the correct action, however, we must look at the root cause behind these numbers. In a precursor to the EPA s report in August of 2007, Dr. Jon Koomey published a detailed analysis of aggregate server power consumption (Koomey, 2007). In his executive summary, Dr. Koomey attributes most of the increase in server power consumption to growth in the number of least expensive servers, with only a small part in the growth in aggregate power consumption attributed to increases in power

consumption per unit. The implication is that the demand for computation is the root cause of this issue, not inherent runaway issues with individual server power or data center infrastructure waste. This need for computation is, in large part, a reflection of our transition to an economy based on information and computation. In February of this year, the ACEEE released its report on the effect of IT and Communications Technologies on Economic Growth and Economic Energy Efficiency(Laitner & Ehrhardt-Martinez, 2008). Among their findings were a 10:1 savings of energy invested in IT - one kw consumed in IT displaces 10 kw of power consumption elsewhere in the economy. In addition, the ACEEE found that the energy required to produce a dollar s worth of economic output today is less than half of that required in 1970. Figure 1: Projected Data Center Energy Use under Five Scenarios (U.S. EPA, 2007) Clearly, the watchword for handling issues around server and data center energy consumption is caution. Our data centers must be able to support our organizations increasing needs for computation. At the same time, public policy must be careful not to place a bottleneck on one of the most important engines for economic growth present in our society and one of the most effective tools in our arsenal for managing society s overall energy consumption. Current Industry and Governmental Efforts These server and data center power consumption issues have triggered strong responses from both industry stakeholders and governments worldwide. Industry efforts have focused on pulling various stakeholders together to identify systemic issues and develop an orchestrated approach to addressing them. Government efforts have focused on communicating high-level guidance to data center owners and operators, but also creating broad incentives (labeling programs) and identifying important overall goals. Both industry and government entities have agreed that both parties need better information on data center power consumption. Industry Efforts Over the past few years, the industry has produced a number of highly visible initiatives aimed at improving server and data center energy consumption. Examples are The Green Grid, the Climate Savers Computing Initiative and 80Plus. 80Plus is an electric-utility incentive program focusing on improving the efficiency of computer power supplies. Climate Savers targets power consumption characteristics and power management features of computing devices, focusing on individual IT components. The Green Grid targets improvements in data center energy efficiency focusing more on the data center as a system. Government and Policy Initiatives Policy-making organizations across the world are driving a number of initiatives in this space as well copying programs demonstrating success in other industries into this space. The EPA s Energy Star program has been focused on both servers and data centers. Similarly, the U.S. Department of Energy is extending its Save Energy Now program into the space as well. Across the Atlantic, the European Commission is developing Codes of Conduct for data centers. We also see significant efforts in China, Japan, and Australia. 3 1 http://www.energystar.gov/ia/partners/prod_development/downloads/epa_datacenter_report_congress_final1.pdf

Industry and Government Common Cause In order to provide guidance to data center owners and operators, however, both industry groups and policy-making organizations need means for evaluating these facilities. These organizations are looking for metrics that would enable data center operators to evaluate themselves both against industry averages and other facilities. Frequently, the miles-per-gallon metric applied to motor vehicles is held up as an example of that which is needed for data centers. The industry has developed metrics for evaluating the efficiency with which a data center delivers power to data center equipment. These metrics are very helpful in providing guidance with respect to power consumption, but are only useful for making decisions about facility infrastructure specifically, power distribution and air conditioning equipment and architectures. Focusing on Efficiency Having available metrics on data center infrastructure efficiency (The Green Grid, 2007) has been very helpful to the industry and to policy-makers looking to manage data center power consumption. The consequences are, however, that proposed policies and initiatives are focused on data center infrastructure efficiency. These do help address some of the energy consumption issues in data centers, but over-reliance on these metrics can potentially prevent focus on those areas where the most significant opportunities lay. The most visible goals set for data centers have come from U.S. policy-makers. The Department of Energy, through their Save Energy Now program, has set a goal to improve the data center energy efficiency of 1500 mid-tier and enterprise-class data centers an average of 25% by the beginning of 2011, with 200 enterprise-class data centers improving their data center energy efficiency by 50% (Scheihing, 2008). The proposed EC Code of Conduct for Data Centers (Data Centre Code of Conduct Working Group, 2008) specifically calls out the data center infrastructure efficiency metrics promoted by The Green Grid. A number of the commitments spelled out in this document refer directly to power calculations around the infrastructure metrics. In addition, however, the EC acknowledges that additional metrics will be developed that relate to the efficiency with which IT equipment converts power into computation ( asset efficiency ) and that future versions of the Code of Conduct will be more specific in this area. Framing the Problem The focus on efficiency has been important. There is, and there will continue to be for some time, opportunities to improve data center energy efficiency in updating, and more efficiently operating, power and cooling equipment. The most significant areas for improvement, however, lie within the IT realm. There is far more opportunity in management of IT assets than there is in infrastructure. Power and Cooling: Past, Present, and Future As the industry has only recently begun collecting and analyzing data on data center power consumption, it is difficult to document historical trends. A white paper initially published by APC in 2006 stated that 70% of the input power into a typical data center is either required by cooling equipment or lost in power distribution or (Rasmussen, 2006). Jon Koomey s paper on server and data center power consumption (Koomey, 2007) estimated that half of the power delivered to the facility was lost in power distribution or consumed in cooling. There is strong anecdotal evidence, however, that industry interest in power and cooling has resulted in improvements both to specific data centers and to the industry as a whole. At the component level, there is strong evidence that power distribution component efficiencies have significantly improved (The Green Grid, 2008). At the data center level, the evidence is at least as strong. For example, Microsoft and Google have both been very proactive with respect to managing their data centers. Microsoft has recently reported an annual Power Usage Effectiveness (PUE) of 1.22 for their Chicago facility (Manos, 2008), and Google has recently reported a PUE of 1.15 (Google, 2008). With best-in-class PUEs moving from > 2.0 to 1.15 to 1.25, there is little headroom left for additional improvements. A perfect PUE would be 1.0. This implies that a data center s power distribution network sees zero losses and that no energy is required to cool the equipment in the data center. Clearly, 2 http://www.thegreengrid.org 3 http://www.climatesaverscomputing.org 4 http://www.80plus.org 4

there is a limit to the opportunity for power and cooling components and architectures to improve data center energy efficiency. If we want data centers to continue to improve, we must look elsewhere. Information Technology Whereas power and cooling components and power and cooling architectures have physical performance limitations that are clearly in sight, the limits to Information Technology are not as well understood. Computer performance has been growing exponentially for decades. While Moore s law specifically calls out only the number of transistors that can be placed on an integrated circuit inexpensively, in practice, this has also meant that processor power has increased accordingly. Figure 2 shows the improvement of processor performance over time based on Dell-reported data for the CFP2000 component-level benchmark. Over the past seven years, our products have seen a 32-fold improvement in performance. While even Moore, himself, argues that there are limits to Moore s law (Gruene, 2007 ) (Gardiner, 2007), he also states that Moore s Law has always been believed to be coming to an end within two or three generations out from current manufacturing processes. At least for the foreseeable future, performance will continue to increase exponentially. Figure 2: Dell Historical System Performance (CFP2000rates) The Future for Facility Productivity The focus on the efficient delivery of power to IT equipment has been useful, but given that infrastructure efficiencies are flattening out and processing power continues to improve exponentially, any effort to manage the energy efficiency of the data center must start seriously focusing on IT efficiency as well. Figure 3 shows the expected relationship between Data Center Infrastructure Efficiency (DCiE), IT Productivity and Facility Productivity. As mentioned earlier, the best that can be achieved, with respect to power and cooling efficiency is 100%. This represents a power distribution architecture exhibiting no losses and a cooling architecture that is entirely passive (i.e., no active components requiring power). As the industry approaches this point, power and cooling capital costs can be expected to rise. At some point, there may be little to no economic benefit to further improvement. IT Productivity, on the other hand, should continue to improve as long as the industry s products continue to follow Moore s Law. The conclusion is that facility productivity will quickly start following IT productivity, as opposed to power and cooling efficiency. The Green data center must manage both infrastructure efficiency and IT productivity. Managing for infrastructure efficiency, however, will quickly become focused on meeting a set target one that will not improve greatly over time. Once the equipment is in place and the policies have been set, there will be little change other than operational and environmental maintenance. Managing for IT productivity, however, will mean tracking to a moving target, constantly being aware of technological improvements 5 PUE is Power Usage Effectiveness, defined as Power Delivered to the Facility / Power Delivered to the IT Equipment (The Green Grid, 2007). 5

and treating the IT core of the data center as a dynamic entity. The benefits, however, are clear. The data center that manages IT productivity successfully may be significantly more productive, with respect to resources consumed, than a data center that focuses solely on infrastructure efficiency. Managing Data Centers for Productivity While the infrastructure metrics mentioned above are extremely useful to provide insight on, and help manage, facility infrastructure, they provide no guidance as to how to manage the IT equipment within the data center. Understanding the limitations of existing metrics, policy-makers and industry stakeholders alike have been looking for means to measure the output of these facilities metrics that establish the Useful Work produced by the data center. Figure 3: Factors in Improving Facility Productivity Metrics for Data Center Productivity IT productivity should focus on two issues: Utilization of existing IT resources and expected performance per watt of IT assets. These two issues are separate, but related. It is possible to utilize existing assets well but have poor IT productivity. This is common in some older facilities where IT assets are upgraded infrequently. Similarly, a data center that commissions the most up-to-date IT equipment, but does a poor job of utilizing that equipment, leaves a substantial amount of potential productivity on the table. Accordingly, the industry should focus on two metrics to better manage issues around IT productivity: Data Center IT Utilization (DCIU) and Data Center Performance per Watt (DCPpW). Data Center IT Utilization (DCIU) The first of these new metrics, Data Center IT Utilization (DCIU), is meant to represent how much of IT equipments potential is currently being utilized. There are very few systems commissioned in data centers that are running 100% of the time at 100% utilization. DCIU is a measure of how effectively an organization uses those capital assets it has acquired for its data centers. DCIU should not simply be an average of utilization rates for existing equipment, but should, instead, be represented simply as the ratio of the amount of Useful Work produced by the data center and the maximum amount of work that could be produced if the data center were able to maximize all of its components utilization rates in other words: Data Center IT Utilization = (Total Useful Work) (Total Compute Capacity) While Total Useful Work represents the aggregate computation performed in the data center, Total Compute Capacity represents the potential amount of computation available if all compute resources 6 6 IT Productivity is defined by the amount of processing or computation completed compared to the resources consumed by the IT equipment. This document will also define, and refer to this quantity as, Data Center Performance per Watt. 7 Aggregation of Dell-reported benchmark data (CFP2000rates) to SPEC.org (January 2002 to January 2007). The CFP2000 is a component-level benchmark suite for SPEC CPU2000 that measures and compares compute-intensive floating point performance.

were being utilized to their full potential. While interesting, this metric remains academic unless there is some means for measuring or estimating Useful Work in the data center and a means for measuring Total Compute Capacity. In addition, for an aggregate IT utilization to have meaning, these two elements to the metric must use the same foundation. I.e., if an industry standard benchmark is used for estimating Total Compute Capacity, that benchmark must be reflected in the approach taken to measure or estimate Useful Work. At Dell, we have been investigating this issue and have developed an initial approach that shows promise. With this approach in hand, more opportunities arise for managing the data center. We can now investigate how IT deployment and operations policies affect the ROI or ROA for Data Center capital expenditures. Data Center Performance per Watt Data Center Performance per Watt works in a manner similar to Data Center IT Utilization. It is not meant to be an average of Performance per Watt measurements or individual components, but must be an aggregate calculation Total Useful Work divided by Facility Power Consumption. Data Center Performance per Watt = (Total Useful Work) (Facility Power Consumption) This is similar to the Data Center Energy Productivity (DCeP) metric proposed by The Green Grid (The Green Grid, 2008) and CUPS, proposed by Emerson (Emerson Network Power, 2008). For purposes of this paper, this metric is referred to as DCPpW as the specifics for calculating this metric differ from the formulation proposed by The Green Grid and CUPS is a generic measurement that is specifically related only to the year in which a server was purchased. Over time, as the industry settles on a standard means for calculating the productivity of data centers, Dell will likely adopt the industry s recommendations. As with DCIU, in order to be meaningful, the DCPpW metric requires an approach to calculating the Useful Work being performed by a system in the data center. Dell has chosen an initial approach and some of the results of that work are presented later in this document. DCPpW also enables new opportunities or approaches with respect to data center management. This is a true productivity number work out divided by resources in. With a means for estimating this, Dell can investigate how different data center decisions affect the use of resources and improve the data center s ability to convert those resources into processing. This will be the core of the Green data center in the future. Where We ve Been Understanding the Past Issues and Real Opportunity Not only are these metrics useful for making day-to-day decisions in the data center, they also provide insights into broader issues associated with server and data center power consumption. Figure 4 shows the evolution of these metrics for a model 5000 ft2 data center, taken from Dell s Data Center Performance Evaluation Tool. The model suggests that, over the last eight years, DCIU for this sample data center has been cut in half. Despite the reduction in asset utilization, however, the data center is still estimated to be up to four times more productive in 2008 as it was in 2000. At the same time that the industry has seen this sort of decline in IT utilization, data center power consumption has gone up. According to the EPA s report on server and data center power consumption, volume servers saw an annual increase in power consumption of 17% between 2000 and 2006. Figure 5 compares the EPA s growth curve for volume server power consumption with an adjusted curve estimating the power that would have been required by volume servers assuming a constant 12% rate for server utilization. The modeling suggests that, had IT utilization remained flat over this time, a significant fraction of the increase described by the EPA would not have occurred. 8 Data from Dell s facility productivity model 9 The current approach scales a system s expected maximum performance (from published benchmark data), with current CPU utilization and current CPU clock speed (collected in real-time). 10 Power and energy are confusing quantities to many. By definition, power is the rate at which energy is produced or consumed with respect to time. To be meaningful, therefore, the numerator, Work (Performance), must also be a rate. Dell is defining Work as the rate at which operations are completed with respect to time. Using this approach DCPpW could also be defined as a number of operations completed per Joule. 11 Once again, maximum system performance is scaled by current CPU utilization and current CPU clock speed. 12 Internal Dell tool 7

Figure 4: Estimate of Historical Data Center Productivity Metrics for a Sample Data Center Clearly then, the data shows that, with respect to data center energy consumption, the issues in the past are the same as the drivers of the future. Issues with IT utilization have made data center energy consumption an issue and future facility productivity will be driven by the productivity of our IT equipment. The focus on power and cooling is important, but if IT waste is not managed, the core problem will be unresolved. Insight into the Data Center It is helpful that these metrics address energy consumption at the macro level, but in order to be useful, they must also provide direct guidance for individual facilities. Without mechanisms that enable data center professionals to manage for productivity, the argument is interesting, but academic. Fortunately, DCIU, DCPpW, and Dell s formulation for an estimate of data center useful work are also useful at the micro level. Figure 5: Comparison of Estimated and Adjusted Energy Consumption for Volume Servers (2000-2008) When we talk to our customers about the challenges they face with respect to energy consumption, we usually hear one or more of a number of separate, though related, issues. Data centers can face very pragmatic issues with the size of their utility bill. They frequently have environmental issues such as thermal hotspots or specific power distribution issues. Occasionally, their suppliers (local utilities) place hard limits on their power consumption. Above all of these, however, the main issue is of facility longevity. A new data center is a large capital expense and IT organizations prefer, whenever possible, to defray or avoid this entirely. In many cases, however, their data centers and the power, cooling and space provided within these data centers, do not seem to co-operate. Appearances, however, are frequently deceiving. Although many data centers appear to be at the limits 13 Calculated through the Dell Data Center Performance Estimation Tool (DC PET, internal Dell tool). 8 14 Baseline energy consumption data from EPA report on server and data center power consumption (U.S. EPA, 2007); adjusted data calculated from baseline data and DC PET historical server utilization estimates.

of their capacity; few data centers are at the limits of their potential. The new metrics for data center productivity will help find this potential both by pointing out areas of remediation in existing data centers and helping data center managers set on-going data center management policy decisions. Understanding an Existing Data Center As part of a larger effort to develop prescriptive responses for legacy data centers, earlier this year Dell created sample infrastructure and IT populations for a typical 5000 ft^2 data center initially constructed and commissioned in 2000 (we refer to this data center as DM1 ). The IT population model was developed by copying existing rack populations from Dell production data centers in Austin and identifying specific servers in the population in such a way as to refer back to existing Dell production equipment. Different sections of DM1 s IT population represent different periods of time during its life. The latest servers in this IT population are Dell 9th generation products, assumed to have been commissioned in 2008. Figure 6 shows the breakdown, by server generation, of DM1 s server population. Specific servers were chosen for the IT population and are identified by asset tag in order to guarantee that both real-time system performance data and server configuration information are available for all members of the population. The availability of server performance data, as well as server configuration data (model, processor type, processor clock, memory) enables the estimation of useful work for each server in the population. Figure 6: DM1 Starting Server Population Analysis of the data provides some interesting insights. First, the majority of work in the data center is performed by relatively few servers. Figure 7 shows a Pareto chart of the data. In this dataset, those servers in the top quartile are responsible for about 80% of the data center s processing output. Those servers in the next quartile are performing a little over 10% of the data center s processing. This means that those servers in the third and fourth quartiles are, together, providing less than 10% of the data center s output. Upon a review of this data, it becomes clear that this server population is a strong candidate for consolidation with the intent to improve asset productivity. Figure 7: Pareto Chart of DM1 Server Population Useful Work 9

Second, different sections of the data center exhibit significantly different productivity characteristics. Figure 8 provides an estimate of the relative productivity of each row in the data center. In our dataset, we assumed that, as the data center was commissioned, servers were installed starting in row 1. As a result, the lowest numbered rows in this data center were comprised of the oldest equipment. This is a deployment strategy typical of many data centers. The consequence of that strategy, however, is that most of the Useful Work in this data center is being performed in those rows where servers have most recently been commissioned. In this instance, the data suggests that these rows may be a natural target for an effort aimed at improving the overall productivity of this data center. 80.0 70.0 60.0 50.0 40.0 30.0 20.0 10.0 0.0 9G 8G 6G 4G 3G 1 2 3 4 5 6 7 8 9 10 11 12 13 14 Row Figure 8: Estimated Relative Work per Rack by Data Center Row (DM1) While these insights are focused on a specific server population, they highlight the importance both of having productivity-related metrics and managing to them. With this information, data center managers can design projects that improve the greenness of their data centers by improving the productivity of their server populations. Even though these changes would be applied within the IT population, issues around IT utilization are such that the magnitude of the potential improvement dwarfs the available opportunity in improving infrastructure performance. The Effect of Policy on Productivity The previous example looks at a specific server population at a particular point in time. To firmly grasp the potential impact of managing productivity by managing IT utilization, however, one must look into the future of the data center. The metrics described above provide a means for exploration. Using these metrics, we have developed a model that looks into a data center s future, estimating the processing required by the data center and the server population required to support that processing. Figure 9 shows some of the model s results: the expected change in a data center s productivity, over time, as a function of different operational policies pertaining to virtualization and the replacement of deployed IT hardware. With a re-definition of the Green data center to be one that makes the best use of available resources, as opposed to one that delivers power to IT equipment efficiently, this curve becomes a direct reflection of how Green a data center is at any point-in-time. Figure 9: Evolution of a Data Center s Expected Productivity over Time 15 This assumes equal performance on behalf of the infrastructure with respect to each row. 10

The first conclusion is that a data center is a dynamic facility that should constantly improve over time regardless of policy. This is a result of the continuous improvement in performance per watt of servers, generation over generation. The data center of tomorrow should be more efficient and more productive than the data center of today. That being said, however, policy choices do provide environments that enable relatively smaller or relatively larger improvements. A data center that adopts aggressive policies towards implementation of technologies that give it control over IT utilization and policies that guarantee prompt replacement of relatively poor performing equipment could see twice the productivity of a more conservatively managed facility by 2012. Figure 10: Data Center IT Utilization Estimates The Effect of Policy on Utilization The main driver for improved productivity is clear: improved IT utilization. Data center professionals have a number of means to exert active control over their utilization of IT equipment, with Virtualization being one of the most popular today. In any case, however, standalone applications running on individual servers frequently hold the data center back. Figure 10 shows the model s expectations with respect to Data Center IT Utilization (DCIU) with respect to different virtualization adoption and hardware refresh scenarios. Even with these policies, momentum in the deployment of equipment will result in a slight decline in IT utilization until 2010. As adoption of virtualization continues, however, and hardware refresh policies start to have affect, the IT utilization curve starts climbing again. It is only with an aggressive approach to virtualization, however, that IT utilization may exceed historical levels by 2012. In fact, even conservative approaches to virtualization only halt the decline in utilization as any pool of work commissioned in a one-application-per-server use model pulls down the aggregate Data Center IT Utilization. The Effect of Policy on Power Consumption Still, for many individuals, a Green data center is governed by that facilities use of resources. As our model allows us to predict server populations, with reasonable assumptions about future server power consumption and data center infrastructure efficiency, we can estimate a data center s power consumption over time. Figure 11 shows the model s results for our sample data center. The potential exists for this data center to cut its power consumption by more than half by 2012. In addition, there s a very clear difference in data center power consumption before and after adoption of virtualization. Prior to the adoption of this technology, control over IT asset utilization was difficult With this technology, however, it becomes possible to manage to IT utilization, instead of accepting it as a result of application commissioning. Once the data center can control IT utilization, it can leverage the improvements in performance that occur with new generations of servers, magnifying the benefits of refreshing legacy hardware. 16 Dell s Data Center Performance Estimation Tool (DC PET), internal Dell tool 17 Aggressive adoption of virtualization implies that, by 2012, 90% of all new work performed by the data center is either performed on virtual machines or represented by scalable applications that enable data center professionals to manage utilization. 11 18 Specifically, energy.

Figure 11: Estimated Data Center Power Consumption as a Function of Utilization Policies Future Areas of Investigation The initial areas of investigation of this work involved adoption of virtualization and the speed with which hardware is refreshed. It is the availability of a metric that estimates aggregate data center IT utilization that makes this possible. With this metric, however, there are a number of additional areas where further research would be valuable. Disaster Recovery Continuity of business being important for any organization s vitality, Disaster Recovery may be the most important operational concern of an IT organization. It may, however, significantly affect utilization of IT assets. Until now, we have not had a means for understanding how DR affects the productivity, efficiency or greenness of the data center. With DCIU we can now size the effect of DR decisions on these key data center performance characteristics. Future work in this area should look into a comparison of the protection offered by various DR options and the effects of deploying those options on energy consumption, data center size, server population and total cost of ownership. With a focus on aggregate IT utilization, future DR innovations may enable the same, or greater, level of protection as today while consuming fewer resources in the data center. Cloud Computing Cloud Computing is a hot topic within the industry today. It is a general concept that incorporates a number of technology trends, including Software as a Service (SaaS). One of the themes within Cloud Computing is the disaggregation of hardware and software the user of the software and the executor of the software may bear no relationship to each other. With this disaggregation, another theme arising from Cloud Computing is the notion of commoditization of computation. Maximizing the efficiency of a compute cloud at the node, rack, and facility level is a key source of business value and an important design consideration. (Pike, Schmitt, Frankovsky, & Brannon, February). Those organizations that have the most productive data centers will have a strong advantage over those with which they compete. Other organizations, those considering leveraging the cloud for their computation needs may have to compare the productivity of their own data centers to that of their potential compute suppliers. This is difficult without strong metrics in this area. DCIU and DCPpW may help answer the question as to whether or not the cloud is green. Total and Projected Cost of Ownership In the final analysis, productivity may be important, but many of the potential policy changes (e.g. more frequent hardware refresh) may be hard to justify unless the changes result in lower IT costs. Fortunately, the sorts of analyses that become possible with the metrics described in this document (DCIU, DCPpW) provide a means for estimating the cost consequences of deployment and operations policies. 12 19 High performance computing clusters are, occasionally, an exception to this.

In addition, projecting future needs of the data center and future IT costs is easier with a model for estimating the useful work performed by, or required by, the data center. With these types of models, we can look into a data center s future to understand how the policies we set today will affect tomorrow s costs of operations as well. Conclusion If you cannot measure a quantity, you cannot control it. While this maxim was originally tied to the science of physical measurements, this holds true for the data center as well. There are a number of metrics available today for data center professionals to use as they face difficult decisions pertaining to facility planning, equipment deployment and day-to-day operations. At the moment, however, these metrics are focused solely on facility infrastructure. To better control the energy consumption of the world s data centers Dell recommends supplementing the existing infrastructure metrics with new metrics focused on IT assets. The metrics mentioned in this document, Data Center IT Utilization (DCIU) and Data Center Performance per Watt (DCPpW), can provide critical guidance. DCIU tells how well IT equipment is being utilized. DCPpW tells about the overall productivity of the data center. With these metrics/tools in hand, the notion of the Green data center can be revisited. With Productivity as a guide, we find that, while power and cooling are still extremely important, the Greenest data centers may ultimately be the ones that have control over the utilization of their IT assets and are making the best use of the most up-to-date compute servers, storage equipment, and networking equipment. THIS WHITE PAPER IS FOR INFORMATIONAL PURPOSES ONLY, AND MAY CONTAIN TYPOGRAPHICAL ERRORS AND TECHNICAL INACCURACIES. THE CONTENT IS PROVIDED AS IS, WITHOUT EXPRESS OR IMPLIED WARRANTIES OF ANY KIND. Works Cited Data Centre Code of Conduct Working Group. (2008). Code of Conduct on Data Centres Version 0.8. Retrieved October 28, 2008, from End-use Efficiency Activities at European Commission: http:// sunbird.jrc.it/energyefficiency/pdf/meeting%20data%20centers%20coc%209%20april%202008/ CoC%20DC-v0%208-WORKING-DRAFT.pdf Emerson Network Power. (2008, November). Energy Logic: Calculating and Prioritizing Your Data Center IT Efficiency Actions. Retrieved December 1, 2008, from Emerson Network Power: http:// www.liebert.com/common/viewdocument.aspx?id=1226 Gardiner, B. (2007, September 18). IDF: Gordon Moore Predicts End of Moore s Law (Again). Retrieved November 4, 2008, from Epicenter: http://blog.wired.com/business/2007/09/idf-gordon-mo-1.html Google. (2008). Data Center Efficiency Measurements. Retrieved November 4, 2008, from Google: http://www.google.com/corporate/datacenters/measuring.html Gruene, W. (2007, September 18). Gordon Moore says Moore s Law will hit fundamental barrier in 10 to 15 years. Retrieved November 7, 2008, from TGDaily: http://www.tgdaily.com/content/ view/33924/135/ Koomey, J. G. (2007, February 15). Estimating Total Power Consumption by Servers in the U.S. and the World. Retrieved October 31, 2008, from AMD Business: http://enterprise.amd.com/downloads/ svrpwrusecompletefinal.pdf Laitner, J. A., & Ehrhardt-Martinez, K. (2008, February). Information and Communication Technologies: The Power of Productivity. Retrieved October 31, 2008, from American Council for an Energy-Efficienct Technology: http://www.aceee.org/pubs/e081.htm Manos, M. (2008, October 20). Out of the Box Paradox. (M. Manos, Ed.) Retrieved November 4, 2008, from LooseBolts: http://loosebolts.wordpress.com/2008/10/20/out-of-the-box-paradox-manifestedaka-chicago-area-data-center-begins-its-journey/ 13

Pike, J., Schmitt, T., Frankovsky, F., & Brannon, T. (February, 2008). Creating a Hyper-Efficient Hyper- Scale Data Center. Dell Power Solutions, pp. 66-69. Rasmussen, N. (2006). Electrical Efficiency Modeling for Data Centers - APC White Paper #113. Retrieved October 30, 2008, from APC Product Information: http://www.apcmedia.com/salestools/ NRAN-66CK3D_R1_EN.pdf Scheihing, P. (2008). DOE Data Center Energy Efficiency Program. Retrieved October 28, 2008, from U.S. DOE Energy Efficiency and Renewable Energy (EERE): http://www1.eere.energy.gov/industry/ saveenergynow/pdfs/doe_data_centers_presentation.pdf The Green Grid. (2008, May 9). A Framework for Data Center Energy Productivity. Retrieved November 7, 2008, from The Green Grid: http://www.thegreengrid.org/gg_content/white_paper_13_-_ Framework_for_Data_Center_Energy_Productivity5.9.08.pdf The Green Grid. (2007). The Green Grid Data Center Power Efficiency Metrics: PUE and DCiE. Retrieved October 28, 2008, from The Green Grid: http://www.thegreengrid.org/gg_content/tgg_data_center_ Power_Efficiency_Metrics_PUE_and_DCiE.pdf The Green Grid. (2008). The Green Grid Peer Review of DC Power for Improved Data Center Efficiency by Lawrence Berkeley National Laboratory. Retrieved November 4, 2008, from The Green Grid: http:// www.thegreengrid.org/gg_content/white_paper_12_-_lbnl_peer_review05.09.08.pdf U.S. EPA. (2007, August 2). Report to Congress on Server and Data Center Energy Efficiency - Public Law 109-431. Retrieved October 31, 2008, from ENERGY STAR: http://www.energystar.gov/ia/ partners/prod_development/downloads/epa_datacenter_report_congress_final1.pdf 14