Letter from Ed Komoski

Transcription

1 volume 2 issue 1 Letter from Ed Komoski 02 How IT Management Can Green the Data Center 05 The Data Center as a Living Organism: Why History Is Not a Good Guide to the Future 07 The Green Imperative 09 Taming the Power-Hungry Distribution System Ed Komoski Vice President and General Manager Eaton Power Quality Eaton s commitment to green IT In power quality solutions, being green means being lean. It means delivering more real output for every utility kilowatt in, and consuming as little power as possible to run IT systems. It means efficiency. Eaton s dedicated business division for data center solutions has taken two approaches in its focus on efficiency-education and product development. We provide substantive information on how data centers can reduce energy consumption, and we deliver products that achieve it. For instance, in 2006, Eaton introduced the new Powerware BladeUPS for high-density server environments, which operates at an industryleading 97 percent efficiency in normal operation. Even at <30 percent load, where you would expect efficiency to be much lower, this UPS is more efficient than competitors modular products at full load. Register to receive the quarterly electronic version of Data Center Forum: This 6U unit expandable to 60 kw N+1 in a single 19 rack delivers twice the backup power of competitors modular solutions, while dissipating only one-third the heat. The higher efficiency and cooler operation of this UPS translate into substantial energy savings. Fewer kilowatts consumed from the electric grid, fewer kilowatts to pay for. For example, assuming a utility rate of $.10 per kwh, a 60 kw N+1 redundant configuration would save more than $30,000 in utility bills over five years. In 2007, Eaton introduced the Power Xpert Architecture to manage power system components from multiple vendors and the Powerware Energy Management System for data centers, which monitors power down to the branch circuit level. With detailed insights from these systems, data center managers can more effectively optimize energy consumption. Eaton also introduced the new Powerware 9395 UPS from 225 to 550 kva for data center applications. In 2008, the Powerware 9395 will expand to 1100 kva with greatly improved efficiency ratings. Eaton has also been active in promoting awareness of the critical need for energy efficiency in modern data centers. For example, over the last year, Eaton: Participated in IBM s Big Green Initiative and Pulse 2008 conference, which focused on the convergence of IT and facilities Has actively participated in The Green Grid, a global consortium dedicated to advancing energy efficiency in data centers and business computing environments ( Published the white paper, Top 10 Ways to Save Energy in Your Data Center, which takes a holistic view of efficiency Published the white paper, Energy-efficient Transformers Reduce Data Center Utility Costs, to highlight the benefits of using components that meet the Energy Policy Act of

2 Data Center Forum Launched this very publication, which features unabridged Gartner research and articles by Eaton experts, with a heavy focus on improving data center efficiency In fact, we ve devoted this entire issue to the subject. In How IT Management Can Green the Data Center, Rakesh Kumar and Simon Mingay of Gartner Research reveal large-scale inefficiencies that occur throughout the energy chain and how IT leaders can improve energy efficiency and develop a comprehensive green strategy. In The Data Center as a Living Organism, Rakesh Kumar and Philip Dawson of Gartner Research explain why legacy monolithic data center design principles are no longer appropriate. In a candid interview, Fred Miller of Eaton s Data Center Solutions discusses metrics and tools for power planning, along with the promise and limitations of new cooling technologies and alternative fuel sources for data center applications. Imran Ahmad, an Eaton product manager, describes practical and affordable ways to improve efficiency in the power distribution infrastructure, without an overhaul. We hope this issue of Data Center Forum will provide practical and achievable ways to reduce fuel for IT. Featured Research From the Gartner Files How IT Management Can Green the Data Center In data centers, large-scale inefficiencies (in areas such as distribution and power conversion) occur throughout the energy chain, from the potential of the fuel source to generation and distribution. IT management can help enterprises that run data centers improve their energy efficiency and develop a comprehensive green strategy. Key Findings Data centers tend to lose massive amounts of energy, but they can be outfitted and retrofitted for energy efficiency. Data center servers and information and communication technology (ICT) devices often have low utilization rates and extraneous software. IT managers should put together an environmental strategy for data centers that includes metrics, modeling, consolidation of equipment and machines, and possible decommissioning. Future data centers can be greened from the ground up, with energy efficiency and other environmental considerations integrated in site selection, structure design and construction, and choice of equipment. Recommendations Assess data center energy efficiency holistically, applying an end-to-end strategy that integrates the key components of the energy picture. Work closely with all the players along the energy chain. Establish goals, processes and responsibilities, targeting energy efficiency, as well as waste management, asset management, capacity management, support services and facilities management. Measure power consumption in the ICT infrastructure as granularly as possible, using metering tools at all levels (building, hardware and facilities components) to determine the amount of power going into the data center. ANALYSIS Data centers are energyintensive, so it follows that energy consumption is one of their biggest environmental issues. However, a green data center will broaden its environmental strategy beyond energy efficiency, gleaning the maximum amount of production from the minimum amount of materials and energy, without compromising performance, resilience and security. Such an approach requires an end-to-end integrated view that includes the configuration of the building, energy efficiency, waste management, asset management, capacity management, technology architecture, support services, energy sources and operations. This research provides IT managers with an outline of the trends affecting data centers and offers strategies with which to address these changes. Architectural Changes of Core Server and Storage Technologies More than 80% of server shipments are made up of x86 boxes; however, enterprise data center topologies vary. Most large clients maintain legacy and niche technologies, such as Alpha, Bull GCOS, Fujitsu and BS2000. Moving off these platforms is costly, because the application functionality has to be re-created on a new platform, which can be difficult and expensive. In some cases, users grow their legacy infrastructures. For example, many IBM mainframe customers have grown their installed MIPS during the past four years and will continue to do so. However, most new workloads are hosted in x86 high-density rack server environments, which are powerful enough for most current application workloads and will be used increasingly for database engines. Utilization of infrastructure remains low for most hardware platforms. A typical x86 server uses between 5% and 10% of its available capacity during a 24-hour period reduced instruction set computer (RISC) Unix systems are slightly better, at 10% to 20%. Emerging technologies such as server and storage virtualization, dynamic workload management, subprocessor partitioning and metering/monitoring tools will evolve rapidly during the next few years, resulting in increasingly real-time infrastructures. Enterprises planning data center space should factor in these changes, which means users will need to maintain legacy hardware environments or they ll be forced to move to newer platforms ext

3 Architecture Solutions for your Data Center The Energy Needed to Power and Cool ICT Infrastructures Is Likely to Increase During the next 10 years, the increased energy appetite of processors, servers, storage devices and network appliances will increase the energy required to power and cool the ICT hardware infrastructure. A shortage of prime quality data center space (especially in Europe) to host new infrastructure, as well as the rapidly increasing cost to build new data centers, will add to the problem. For many organizations, actual energy consumption will outpace this underlying trend for a period of time. Users will get onto a technology platform runway, and, by sticking to it and increasing the volume of hardware technology, they ll experience a sharp increase in energy consumption. However, the industry will see innovation at many levels (including servers, management software, liquid cooling and blowers), and, at some point, they ll switch to newer technologies and almost instantly reduce their energy consumption. Over a period of time, the curve will again go upward, and users will need another correction. This saw tooth type of curve will, however, always trend upward. Users should adopt a philosophical, technical and financial posture that takes advantage of this scenario. Managing the Evolving Energy Picture In a data center, massive inefficiencies occur throughout the energy chain, from the potential of the original fuel source through generation and distribution. Many of these inefficiencies lie outside the control of IT departments, resulting from how power is generated (different sorts of power stations) and distributed in countries national electricity grids. Once the electricity reaches the data center, cooling, uninterruptible power source (UPS) and other nonproductive uses expend resources. This is compounded when energy gets to the server and IT devices, which often have low utilization rates and extraneous software. Although data center energy efficiency is not about improving cooling, power management and the power source, these areas are a good place to start the greening process. The IT management team should work on driving energy efficiency across the software, technology infrastructure, architecture and design. This endeavor requires: Close cooperation with architects, software engineers and data center operations, as well as collaboration with the facilities and real estate teams. Sourcing decisions based on coefficiency, including shared service models, such as software as a service (SaaS), that have the potential to be energyand eco-efficient. Business process and application architects and designers need to take heed, because these decisions will affect service levels, technology choices and implementation all of which will influence energy efficiency. Models that provide a granular picture of energy costs, floor space and infrastructure topology of data centers and related offices. These models will be important to developing projections based on the growth of IT equipment and changes to the data center or office layout, and will enable the consideration of different scenarios of equipment deployment and internal financial management (such as chargeback). For example, a model using a computational fluid dynamics (CFD) analysis can determine how many more servers can go into a data center, where they should be located and whether any changes need to be made to an airconditioning system. A systematic program for consolidating machines and workloads to use available spare capacity and maximize return on investment (ROI) in server technology. This will delay the need to purchase more of the newer, highdensity hardware and could push back the manifestation of the energy problems by months, if not years. Procurement of new servers that run at approximately a 60% to 70% utilization rate. This will ensure that they better manage power and cooling issues at the initiation of hardware deployments. Increasing ICT equipment utilization levels from less than 20% has dramatic and multiplying benefits for a data center s energy consumption. Use of software tools to achieve optimal utilization rates. These tools include virtualization software from such companies as Microsoft and VMware, as well as better workload applications. However, these tools are of little use if the organizations don t change their operational processes so they benefit from the software. For example, multiple virtual machines in a single server should ensure that production partitions can be run next to test partitions in the same box. A rigorous decommissioning process that physically removes equipment once it becomes redundant. The IT organization must also identify low-utilization devices and consider consolidating and decommissioning them. Examine devices that are 3

4 Data Center Forum plugged in and drawing power for no purpose. Disposal strategies. Organizations should comply with directives and legislation Waste Electrical and Electronic Equipment (WEEE) and Restriction of Hazardous Substances (RoHS) as well as go further by developing processes for disposal/break-up/smelting and the recovery of metals. Measurements of power consumption at the power distribution unit level to gain a higher threshold of energy for core computing equipment. As with any transition, start by measuring key metrics to get a status report. The Green Grid and a number of others have proposed two useful data center metrics: power usage effectiveness (PUE) and data center efficiency (DCE). These measure the proportion of power consumed doing productive work vs. energy used doing nonproductive activities, such as cooling. Both metrics provide insights and can highlight areas for improvements in most data centers. This assumes that the power consumed by the IT equipment is productive. Often, it isn t: A server farm run 24/7, with 15% utilization rate during office hours, would not show up in these metrics. Herein lines the challenge for the industry and each individual enterprise: How do you measure what comes out of the data center? At the moment, there is no objective metric of real data center efficiency. An individual data center might be able to create one for internal purposes, depending on the nature of the work undertaken by the infrastructure, such as the total power consumed by ICT equipment/general ledger transactions. Such a measure captures an element of business activity, but it s imprecise. However, in the long term, it requires an energy management architecture. Work with the enterprise to encourage combining heat and power (CHP). The practical and economic availability of efficient power generation and the ability to apply sourcing strategies vary significantly, depending on the technology, location, availability of grants, planning permission, the local utility providers and so on. However, CHP offers high levels of efficiency, reliability and a positive ROI in most circumstances. It also offers increased security of supply when backed up by the grid. Photovoltaics offer an expensive, partial opportunity for some, but they are a long way from being a practical and economic solution for most. Google is running a pilot in California with a 1.6MW array, with the intention of expanding it considerably. However, for piloting and public relations reasons, it s too early for much more than that. Wind is economic and scalable, but intermittent, and it s not practical in many areas. Geothermal offers potential, but is not economical for most at the moment. Fuel cells are experimental at this stage T- Systems is running a 250 kw pilot in a Munich data center but they do offer potential for the future. Although these are examples of emerging technologies, users should consider all options. Point to the benefits of renewable energy. Off the grid, it s expensive and increasingly scarce as many enterprises look for quick and easy fixes, but renewable energy can be purchased for other parts of the operation where their application is less expensive. Conclusion: Data Centers Tend to Lose Energy, but They Can Improve Data centers of the future have the advantage of being groomed green from the ground up. Designers can choose a site based on energy security, cost and source. They can develop that site, probably on a modular basis, building the physical structure its fixtures and fittings to be energy-efficient and use relatively low-impact materials and construction techniques. They can outfit the center with chillers, as well as heating, ventilation and air conditioning systems, and introduce recycling and alternative resources, such as solar, wind, geothermal, hydro, fuel cell and CHP. They can monitor tools and manage server efficiencies to measure and manage the environmental footprint. Gartner s advice is to build all new large facilities with the chilled fluid plumbing at the outset. Established data centers have no such advantages, and enterprises will have to contend with many challenges. Massive inefficiencies can occur along the energy chain, from the original fuel source to generation and distribution. Once the electricity reaches the data center, cooling, UPS and other nonproductive uses expend resources. This is compounded when energy gets to the server and to ICT devices, which often have low utilization rates and extraneous software. IT managers can ameliorate this tendency by helping data centers to improve these conditions. For example, they could: Select and design support services that meet cost, service levels and performance requirements and operate in a coefficient manner. Select ICT equipment according to an assessment of lifetime environmental impact. Encourage the use of distributed (local) power generation using CHP, which can offer efficiencies and a positive ROI ext

5 Architecture Solutions for your Data Center Conduct a study of PUE to provide insights into the energy efficiency of the supporting data center infrastructure. Consider the energy efficiency of the workload through server utilization tests and other methods. Set goals for energy efficiency, waste, asset and capacity management, support services and facilities management. Measure power consumption in the ICT infrastructure and be as granular as possible, using submetering at the very least, measure the power going into the data center. Gartner RAS Core Research Note G , Rakesh Kumar, Simon Mingay, 22 January Featured Research From the Gartner Files The Data Center as a Living Organism: Why History Is Not a Good Guide to the Future Legacy monolithic data center design principles are no longer appropriate for new modular data designs. We outline the differences and suggest that new data centers should be viewed more as flexible, evolving living organisms in a cellular structure. Key Findings The first two generations of data center designs are no longer appropriate for current and future needs. New data center designs need to be based on flexibility and high levels of monitoring, and incorporate a mixture of power and cooling technologies alongside virtualization and management tools. New data centers should be perceived less as a static structure and more as an agile, living organism that evolves as the server and storage infrastructure changes. Recommendations For new data center designs, start with the topological layout of the hardware and incorporate the needs of the facility around that layout. Include flexible, modular, virtualized design principles in new data center designs. Ensure that linking building information systems to enterprise management tools is a high priority for new data centers. ANALYSIS The role of centralized enterprise data centers will evolve to become a more critical part of the IT services delivery chain during the next five years. Global economic changes will force large organizations to focus on business continuity, governance and regulation issues. At the same time, technology changes, such as virtualization, service-oriented architecture and hardware topology, will continue the development of large megadata centers. Although outsourcing and software as a service/hosting and other alternative delivery models will grow in importance, the data center will become even more critical on account of these changes. It will increasingly act as an aggregator/controller of this off-site computing functionality and in-house capabilities. Another critical factor at play here is the confluence of power, cooling and floor space constraints felt by many IT departments. Most data centers will need to find more floor space to house the increasing volume of servers, storage and networking equipment. They will also need to modify their facilities components to address increased energy needs that are resulting from changes in hardware design. The net result is that most organizations will need larger data centers. Thus, data centers will become more critical to the business than ever before. However, the manner in which new data centers need to be financed, designed, built and managed is significantly different from what has generally been accepted during the past 30 or more years. In this research, we present the reasons for the change and encourage users to apply a different set of principles when considering the role of their data centers. To understand the changes, Gartner has segmented the evolution of data centers into three stages (see Figure 1). Generation 1: Mainframe Principles and Mainframe Hardware Data centers have evolved as large buildings designed to house mainframe systems. In this sense, all the building design criteria were based on the characteristics of the mainframe. The philosophy of data center design at this stage was based on a static structure that was the main (and, in many cases, the only) IT processing capability of the business. Once the data center building was built and commissioned, there likely weren t any structural changes for the life of that building (typically around 20 years). Once capacity was reached, another data center would be built. This line of thought regarding a single structure and a single type of equipment drove many decisions. This was a reasonable and practical way to build data centers. For example, the specific need to have a raised floor was essential to house the huge amount of cables that were required to connect the different parts of the hardware systems (for example, front-end processors, storage controllers and networking boxes) and to deal with water cooling. At the same time, the temperature at which the equipment needed to be run was determined by the characteristics of the mainframe and all the facility s components were designed likewise. Also, because there were multiple single points of failure (the building and the hardware 5

6 Data Center Forum Figure 1. Discontinuities in the Data Center Design Model Client/server, Unix-based systems: Required a smaller envelope Created a static data center design envelope CRAC HVAC PDU UPS components), the design principles focused on overengineering the facility s components to create redundancy. The strategic and philosophical nature of data centers drove the financial and operational thinking. For example, the capital costs would be written off during that long period of time (20 or more years). Because data centers were seen as being so important to a business, justifying the financial case was relatively straightforward. Operating these data centers was perceived as being critical, and documented processes were strictly adhered to. Generation 2: Mainframe Principles and Client/Server Hardware In the 1990s, the notion of client/server computing and open systems became fashionable, and the strategic importance of the data center Generation 1 Mainframe Principles Mainframe Hardware Generation 2 Mainframe Principles Client/Server Hardware Generation 3 New Principles Mixed Hardware computer room air conditioner heating, ventilation and air conditioning power distribution unit uninterrupted power supply The data center is a living organism. Mainframe characteristics determined the data center's design envelope: Temperature range 68F to 77F Humidity 50% Raised floor PDU, CRAC, HVAC, UPS and generators High-density equipment means that the data center needs: A bigger design envelope A flexible (time, cost and topology) design envelope Modular build-out and integration of competencies declined. Many organizations developed decentralized processing capabilities using the Unix operating system and tools. Although the philosophical view of data centers was changing, new data centers were still being built. However, the design specifications for the builds were not changing a lot and were still based mostly on mainframe characteristics. Thus, although water cooling was generally no longer required, the level of cooling and the temperature of the machine room remained the same as it was for mainframes. Perhaps most important of all, the view of the data center as a static, hugely expensive and inflexible mechanism through which IT services were delivered was enhanced by the perceived flexibility of client/server computing. Toward the end of the 1990s, many organizations realized that running distributed IT systems was operationally Source: Gartner (December 2007) more difficult and more expensive than centralized processing. Moreover, consolidation and rationalization of systems were driven by cost cuts, and the strategic importance of data centers was once again being accepted. Many people placed their client/server hardware back into data centers. Because these systems required less energy than mainframes, the belief was that the data center design specifications were appropriate (if not a bit overengineered). The important thing to realize is that throughout these changes in the perceived value of data centers, their design and structure changed little. The principles of designing the shell of the building and the core facilities based on outdated mainframe specifications have generally remained intact until today. Generation 3. New Principles and Mixed Hardware The arrival of high-density x86-based hardware has resulted in a significant power, cooling and floor space problem for a large number of data centers. Moreover, the need to consolidate hardware and data center locations has exacerbated the problem. As a result, organizations need to rethink the philosophical, strategic and technical principles for new data centers. The cost of building new data centers has risen sharply during the past few years, especially in Tier 1 cities around the world where realestate prices have gone up. The options of using a hosting provider or an outsourcer are more viable than ever before. As a result, many organizations are questioning their need to build and own a new data center one that offers little flexibility as an asset for the future. Thus, organizations looking to build new facilities are considering options such as joint build-outs with partners. The key driver here is asset flexibility. Incorporate flexibility into the design envelope. The basis of a new design needs to be from the inside out rather than from the outside (building) in (the infrastructure that resides in the building). With this in mind, the topological layout of the hardware will drive a lot of design principles. For example, the energy needed to power and cool a rack of x86 servers is much higher than that needed for a reduced instruction set computer/unix server ext

7 Architecture Solutions for your Data Center Building a whole data center to the highest energy point is expensive and a needless overengineering exercise. Moreover, the type and level of cooling needed for these two platforms are different. Thus, new data center design principles need to start with a hardware topological layout (as much as possible) and then design the facilities components around that layout. Flexibility is also important when considering the size of the facility. The design principle in Generations 1 and 2 was around a single static structure. New designs need to be modular, with built-in expansion capabilities. For example, total design for a 30,000 square foot data center with a life expectancy of 15 years could be built in three modules of 10,000 square feet each. Each module may have a life expectancy of approximately five years. By doing this, users will be able to adapt future builds to the technology available during the next few years and limit their initial capital requirements. The third generation of data centers will need to have a high level of monitoring and modeling capabilities built in. Linking the building information systems to enterprise management tools such as Tivoli or OpenView will be critical. In essence, this new generation of data centers needs to be flexible, highly monitored and able to adapt to the changing environment much like a living organism. The role of data centers is becoming more critical to delivering sustainable IT services to users. The cost of building new facilities is also increasing rapidly. Ensuring that the next generation of data centers is future proofed as much as possible is very important. Designs need to start with the topological layout of the hardware. Facilities need to be designed around that layout. Incorporate flexible modular design principles into the layout for new data centers and link building information systems to enterprise management tools. Gartner RAS Core Research Note G , Rakesh Kumar, Philip Dawson, 7 December Eaton Technology Briefing: The Green Imperative: Power and Cooling in the Data Center Data Center Forum interviewed Fred Miller, Product Line Manager, Eaton Data Center Solutions. Miller discusses metrics and tools for power planning, and the promise and limitations of new cooling technologies and alternative fuel sources for data center applications. To increase efficiency while reducing power consumption and cost, data centers must plan ahead for energy management. What tools are available to address this issue? There are two key aspects to this issue. First, you must have reliable and meaningful data about present-day power consumption and quality not just a snapshot, but a performance record. Second, you need the ability to analyze, troubleshoot and assess trends over time. The good news is that even a small data center can afford the tools to address both these aspects, proactively manage energy consumption, prevent overload conditions, improve capacity planning and optimize power distribution. With new energy management systems, new or existing electrical infrastructures can easily be equipped for 24/7 monitoring, all the way to the branch circuit or receptacle level. New, high-speed power meters can accurately detect even the most fleeting anomalies in power quality on critical loads. Compact environmental monitors fit into unused side or rear channels in racks to monitor temperature and humidity. At a central vantage point, or from anywhere, software systems aggregate all this data, present it in easy-tograsp terms (or visuals) and support in-depth analysis. With these technologies, data center managers can get visibility into their power systems at multiple levels down to the individual branch circuit or receptacle, or summarized across loads or for the whole data center. Are more data centers using real-time automated monitoring systems to analyze power usage and help identify hot spots and cooling problems? Absolutely, and this has been driven not only by rising energy costs but by recent technology advances. For example, in the past, if you wanted high-speed sampling of power quality, you would have to call in a consultant with a $20,000 portable power meter. Even then, you would get a snapshot view of power consumption and quality. You could miss fleeting anomalies or seasonal variations. Now, this kind of high-end monitoring is affordable and practical for data centers to own and use all the time, and you do not have to be a power guru to understand the information it delivers. These technologies are now so readily available, and so easy to use, that we are definitely seeing an increase in interest from data center customers. The first step in getting a grasp of power usage would be an energy assessment. What recommendations do you have for data center managers to begin this process? Many data center managers do not know the efficiency of their IT equipment or site 7

8 Data Center Forum infrastructure or have a clear path in mind for maintaining and improving that efficiency. There is a lot of low-hanging fruit being overlooked, easy opportunities to reduce energy costs and become greener in the process. So, a good first step is simply to determine how much of the data center power budget goes to IT systems and how much goes to support systems, such as power distribution and cooling. For every kilowatthour of power being fed to IT systems, how much real IT output do you get, in terms of Web pages served, transactions processed or network traffic handled? The Green Grid has a variant on this metric called the Power Usage Effectiveness (PUE) ratio a ratio of total facility power to the power drawn by all IT equipment. After applying this calculation to several data centers, The Green Grid recommends an ideal PUE of 1.6 and a realistic goal PUE of 2 for a well-designed and operated data center. It estimates that most U.S. data centers have a PUE of 3.0, with an average of 2.4. If PUE is poor, data center managers can investigate operational changes, such as virtualization or consolidation; configuration changes, such as alternating hot and cool aisles; and new technologies, such as high-efficiency UPSs and power supplies. In this process, data center managers can turn to outside consultants to conduct an energy audit and make recommendations. If time and manpower are in short supply, this is an attractive option. But with new monitoring and management tools available, data center managers can be more self-sufficient than ever in assessing their power systems. Compared to diesel generators and lead acid batteries, fuel cells provide better efficiency, reliability, service life, maintenance cost and emissions, while running quietly. What research have you done on fuel cells, and do you foresee this technology playing a key role in the sector in the near future? The biggest barrier for fuel cell adoption today is the very high initial cost, compared to traditional backup energy sources. Most organizations have a hard time making a business case for the premium price, especially when existing solutions are still satisfactory. However, we will see changes in the next five years or so, due to several forces. Increasing concern about environmental issues puts a harsh light on the dark side of traditional technologies: dependence on fossil fuels, emissions from diesel generators and disposal of lead-based batteries. When we start to see greater tax benefits for green operations (and lower prices for maturing fuel cell technology), the technology will have to grow. Companies that seek the environmentally conscious path will drive adoption of fuel cells, more than the experimental level of today. At Eaton, we believe fuel cells represent an exciting potential for our customers. We continue to track trends in the industry, and we are working with a number of technology providers with an eye toward offering a commercially viable fuel cell solution as soon as practicable. Do you see data centers accelerating their technology refresh cycles to get the advantages of virtualization sooner? What is the best strategy to plan for this change? There is no doubt that virtualization can be a good deal. Some analysts predict that virtualization will improve server utilization for a typical x86 machine from 10 to 20 percent to at least 50 to 60 percent in the next three to five years. Eaton deployed energy-efficient blade servers and virtualization software and now saves more than $1.5 million annually in power and lease costs. The prospect of such savings will drive more data centers to adopt virtualization strategies to one degree or another and that would translate into an accelerated pace of server replacement. But virtualization will not be the salvation for everyone. Many data centers have to be designed to accommodate periodic peak loads far above daily loads, such as seasonal peak volumes for a retailer. In that case, having underutilized or idle hardware sitting around is just par for the course. Virtualization wouldn't buy this data center much, if anything. While virtualization may first appear like a silver bullet, one must also be aware of a phenomenon known as virtualization sprawl, which can be almost as detrimental as server sprawl. Virtualization sprawl involves the proliferation of virtual machines without the ability of IT to accurately track or control the virtual machines. This can lead to excessive man-hours attempting to place some reasonable controls on the IT infrastructure. The best strategy is therefore not a prescription but a process: stay abreast of new technology developments and regularly assess the ROI potential of replacing old technology. The business case will differ for every customer, geography and infrastructure. With power and heat rising higher in server racks from 3 kw in 2000 to 14 kw and even 24 kw today what are your thoughts about high-density cooling systems that provide up to 30 kw cooling per rack? As much as 30 to 60 percent of the data center utility bill goes to support cooling systems. If that figure seems too high, it is. As data center managers struggle to reduce this burden, they have to weigh several variables. Are you better off paying the higher initial cost of highdensity cooling systems (and enjoying lower energy costs over time) or spreading the IT load so average power level is less than 10 kw per rack, which can be supported with conventional raised floor cooling? As with so many issues in data center design, the answer is, It depends. Where existing air cooling seems to be ineffective, we are often finding that the cooling system has been inefficiently deployed, or racks are set up in a way that produces hot spots. These issues can be minimized by best practices, such as distributing servers across more racks, alternating hot and cool racks or aisles, and carefully regulating air flow across the data center. However, in areas of the country where energy and ext

9 Architecture Solutions for your Data Center real estate costs are high such as Manhattan or California a data center could more easily cost-justify high-density cooling. The higher initial investment would be repaid in lower utility bills for cooling systems and lower real estate costs, thanks to the ability to deploy more IT equipment in the same square footage. Power cost and energy usage can be reduced with simple solutions, such as only buying IT equipment with internal power supplies that are at least 80 percent efficient. How prevalent is this practice in the data center community? Even a small data center can save thousands of dollars simply by making wise purchasing decisions for IT hardware. A mid-sized data center with 1,500 servers could save tens of thousands of dollars, while reducing carbon footprint. For example, if you compare a server with the same performance characteristics, but one states it is 84 percent efficient and the other is only 75 percent efficient, the more efficient system will save you $38 per year in energy costs (calculated at $.10 per kwh, and 300 watts IT demand). Now that may not seem like much, but factor in 1,500 of these, so that total annual savings goes to $57,000, and then consider cooling savings could be another $50,000 to $60,000 in the average data center and the total yearly savings is now over $100,000. Unfortunately, this issue isn't getting serious attention yet. When selecting new hardware, buyers still tend to look more closely at initial purchase price than long-term operating cost. Now that high-profile organizations such as Google have announced energyconscious choices in their data centers, we are seeing more interest. We hope power issues do not have to reach crisis proportions before energy efficiency moves to the forefront of every planner's agenda. Eaton Technology Briefing: Taming the Power-Hungry Distribution System By Imran Ahmad, Eaton Product Manager As power density in modern data centers increases, more focus has been placed on improving efficiency in the power distribution infrastructure. Dramatic changes, such as switching to 600 Volt DC distribution systems, have been explored, but there are some practical and affordable options to significantly improve efficiency without making major changes to the existing power delivery infrastructure. Recent advancements in power distribution products can reduce energy, cabling and cooling costs. As an added bonus, new power distribution schemes and products also make the data center more adaptable and easier to manage. Let s take a look at how any data center can increase efficiency and adaptability simply by making smart choices in the major building blocks of power distribution: Uninterruptible power systems (UPSs) Power distribution units (PDUs) Power distribution racks (PDRs) Enclosure power distribution units (epdus) The UPS: From behind the scenes to the data center floor In traditional data center designs, a large, three-phase UPS stood alone in a separate room, providing conditioned power and battery backup for the whole data center, perhaps even the entire building. These UPSs fed PDUs on the data center floor. Advances in UPS technologies have greatly improved the efficiency of these large UPS systems. In the 1980s, a state-of-the-art UPS was 75 to 80 percent efficient at best. With the advent of faster switching devices in the 1990s, efficiency jumped to 85 to 90 percent and later to 90 to greater than 94 percent. Even higher efficiency is now possible. In 2006, Eaton introduced the Powerware BladeUPS for high-density computing environments. This modular UPS operates at an industry-leading 97 percent efficiency in normal operation. Even at <30 percent load, where you would expect much lower efficiency, this UPS is more efficient than others modular products at full load. Even small increases in UPS efficiency can quickly translate into tens of thousands of dollars. The savings compound with data center size. High UPS efficiency also extends battery runtimes and produces cooler operating conditions, extending the product s operational life. PDU The backbone of efficient distribution Eaton PDUs provide efficient and flexible distribution for today s data centers. These PDUs promote efficiency in two key ways: Reduce the tangle of power drops to racks. In a traditional distribution scheme, the PDU feeds a panelboard, and separate branch circuits deliver power to racks (Figure 1). Excessive cabling can restrict airflow under the raised floor. Eaton PDUs can dramatically streamline those cabling requirements by using subfeed breaker distribution rather than panelboard/branch circuit distribution. The subfeed distribution wiring can be connected to PDRs that sit at the end of rows (Figure 2), closer to rack loads. This arrangement has the added benefit of being able to handle moves, adds or changes (MACs) more easily, since branch circuit cables only have to be managed up to the PDR, not all the way back to the PDU. In addition to the obvious cost savings of less cabling, cooling efficiency is also increased by reducing airflow restrictions under the raised floor. Some data centers have 9

10 Data Center Forum Figure 1. An inefficient cable strategy has branch circuits from PDU to each rack. Figure 2. Simplify wiring to the row using subfeed distribution and a PDR. been designed with four-foot (or taller) raised floors to compensate for excessive wiring. These higher raised floors add cost and compromise the structural integrity of the floor. Use high-efficiency transformers. Eaton PDUs can use energy-efficient TP-1- compliant transformers that meet the requirements of the Energy Policy Act of The TP-1 standard calls for distribution transformers to be one to two percent more efficient at their typical loading level (30 to 50 percent). Although TP-1 transformers are more expensive, they pay for themselves in five to six years by significantly reducing energy costs and they continue to deliver cost savings for the remainder of their 20- to 30-year lifespan. High-density PDRs Optimizing three-phase distribution to the rack Further simplification can be achieved by using highdensity power distribution racks (HD-PDRs), which can be optimized for pure threephase distribution to the rack. The HD-PDR can contain up to eight panelboards, ranging from 12 to 42 pole positions. These panelboards are rated from 225A up to 800A to support high-power threephase distribution from the panel. Three-phase distribution allows more power to the racks with less wiring. The smaller 12 and 24 pole panelboards that are available allow more effective use of space in the unit for installation wiring and operational maintenance. Distributing three-phase power to the racks enables power changes at the rack level, rather than the PDU level, allowing the IT staff to make power changes for additions of new IT equipment to the rack. All Eaton HD-PDRs can support separate input sources to feed dual-corded loads or rack-mounted transfer switches. This can reduce the amount of cabling required. Comparing Figure 3 and Figure 4, you can see how cabling for dual-corded loads was reduced by approximately 25 percent Figure 3. Typical wiring strategy for redundant power supply loads. Figure 4. Branch circuit wiring reduced by 25 percent V V V V 550 kva UPS 550 kva UPS 550 kva UPS 550 kva UPS A B A B 300 kva PDU A 300 kva PDU B DATA CENTER 300 kva PDU A 300 kva PDU B DATA CENTER Subfeed Distribution Subfeed Distribution Branch C ircuit Wiring R ack Loads PDR PDR A B HD-PDR HD-PDR A & B A & B R a ck Loa ds Branch C ircuit Wiring ext

11 Architecture Solutions for your Data Center using dual-source PDRs. Minimizing cabling along the rows and to the racks improves airflow and therefore thermal efficiency of the room and racks. epdus Power distribution for the high-density rack In high power density environments, three-phase, rack-mounted epdus (enclosure-based PDUs) are quickly being recognized as the optimal solution for rack power distribution. These epdus are the perfect complement to the PDR (although epdus are certainly compatible with standard remote power panels and can even be directly connected all the way back to the PDU). There are many advantages of using three-phase, 208V power down to the epdu level, as opposed to just using a plethora of single-phase 120V rack power strips: Three-phase distribution can transfer almost twice as much power (1.73 x) as equivalent 208V, singlephase circuits over the same size conductors and three times the power as 120V single-phase circuits When switching from single-phase 208V to three-phase 208V distribution, only one extra wire is run in each power drop, which is a 25 or 33 percent increase in copper, compared to a 73 percent increase in power delivery Using higher voltage (and power) epdus in the rack reduces the number of cables that need to be brought in and managed A few high-power epdus replace an unwieldy web of cabling and a mass of lowpower plug strips. The result is greatly simplified cable management, more space available in the rack and improved air flow and thermal efficiency. The complete solution A simple case study Let s look at two strategies for distributing three-phase power to your racks one with a large, stand-alone UPS in a separate room and one using a rack-based, modular UPS, such as a Powerware BladeUPS system. In our example, we will consider a row of eight highdensity racks, each with three full HP C-class Blade Servers, requiring about 15 kw of power per rack. You ll notice that we have specified highdensity epdus for both configurations. Here s why: If we used traditional singlephase power strips Each 208V, 20A rack power strip would provide approximately 5 kw of power. Each rack would require at least three power strips for a single feed configuration and six power strips for a dual feed (redundant power supply) deployment. This requires a total of 24 (single feed) or 48 cables to be run to our eight IT racks. These cables would have to be routed from the PDR or PDU to the rack, which results in a large cable bundle that could impede cooling air flow. If we used high-density epdus instead The same amount of power can be delivered with one power feed per cabinet, requiring only eight cables for single feed deployment or 16 cables for a redundant configuration. Using these higher powered epdus also reduces the number of breaker poles used by 50 percent. Another alternative to reduce power wiring even more would be to use ultra high density power distribution epdus, which could save another 25 to 33 percent in number of cables required. The sample configurations on the following pages show how Eaton s high-density equipment reduces cabling and simplifies the power infrastructure, making the most efficient use of the data center s power resources V 550 kva UPS HD- PDR 300 kva PDU V to 208/120V 400A S ubfeed Distribution In the following three configurations, Eaton PDRs and epdus simplify threephase distribution to high density racks. Eaton PDUs use energy-efficient transformers and have the smallest footprint in the industry. Figures 6 and 7 take advantage of Powerware BladeUPS, which offers a higher efficiency, smaller UPS footprint and more reliable UPS architecture than other modular three-phase UPS systems available on the market. The sample configurations demonstrate how you can configure a lean power distribution infrastructure that improves efficiency (and adaptability) without making drastic changes in the data center. Configuration 1. UPS PDU PDR epdu This traditional data center configuration (Figure 5) is one of the most common in the industry. The easiest way to connect high-power epdus to your distribution system is to simply wire them directly to the PDR at the end or center of your aisle. An Eaton high-density PDR is a perfect fit for this application, since it is designed for these higher power applications and optimized for three-phase circuit breakers. Another common configuration gaining popularity is using modular, rack-mounted UPSs in an end or middle of row configuration. Three-phase 208V Bran ch circuit wirin g Figure 5. Traditional data center configuration. 17 kva 208V Three-phase epdu (0 R U) DATA CENTER 15 kw per R ack Loads 11

12 Data Center Forum Configuration 2. PDU BladeUPS RPM In Figure 6, power delivery to the loads can be accomplished using the Powerware 3U Rack Power Modules (RPMs), which are easily plugged into the output of each modular system, (Powerware BladeUPS). This integrates 12 kw power distribution blocks into the areas the power is needed, with a variety of receptacles that can deliver power directly to the servers or to epdus to be further distributed. Figure 6. Modular UPS with modular power distribution. Configuration 3. PDU BladeUPS PDR epdu In the application displayed in Figure 7, two 60 kw BladeUPS systems are hardwired to a dual-input PDR to give another flexible platform for distributing up to 120 kw of redundant UPS power to the critical load. Figure 7. Modular UPS with centralized power distribution. B u ild in g Feed - 480V Buildin g Feed - 480V 300kVA PDU V to 208/120V 300 kva PDU V to 208/120 V 12 kw DATA CENTER 225A S ubfeed D istribution 17 kva 208V Three-phase DATA CENTER 225A S ubfeed D istribution T hree-phase epdu (3 R U) epdu (0 R U) Blade UPS 60 kw Rack 3ph 208V Blade UPS 60 kw Rack Blade UPS 60 kw Rack Dual In put PDR 3ph 208V Blade UPS 60 kw Rack 12 kw per R ack Loads Hardwire Connection s 15 kw per R ack Loads About Eaton Global leader in industrial, automotive and electrical manufacturing Eaton s electrical business is a global leader in electrical control, power distribution, uninterruptible power supply and industrial automation products and services. Eaton s global electrical brands, including Cutler-Hammer, MGE Office Protection Systems, Powerware, Holec, MEM, Santak and Moeller, provide customer-driven PowerChain Management solutions to serve the power system needs of the industrial, institutional, government, utility, commercial, residential, IT, mission critical and OEM markets worldwide. Eaton Corporation is a diversified power management company with 2007 sales of $13 billion. Eaton is a global technology leader in electrical systems for power quality, distribution and control; hydraulics components, systems and services for industrial and mobile equipment; aerospace fuel, hydraulics and pneumatic systems for commercial and military use; and truck and automotive drivetrain and powertrain systems for performance, fuel economy and safety. Eaton has 81,000 employees and sells products to customers in more than 150 countries. For more information, visit In November 2007, Eaton acquired MGE Office Protection Systems. MGE Office Protection Systems designs and manufactures secured power products and solutions for enterprises, small business and homes. Eaton is now the world s second largest supplier of power protection solutions, including single-phase UPS, surge suppression and power distribution, among other products and services. For more information, visit Powerware, BladeUPS, Integrated Facility Systems, PowerChain Management, and Power Xpert are trademarks, trade names, and/or service marks of Eaton Corporation. All other trademarks are property of their respective owners. Data Center Forum is published by Eaton. Editorial supplied by Eaton is independent of Gartner analysis. All Gartner research is 2008 by Gartner, Inc. and/or its Affiliates. All rights reserved. All Gartner materials are used with Gartner s permission and in no way does the use or publication of Gartner research indicate Gartner s endorsement of Eaton s products and/or strategies. Reproduction and distribution of this publication in any form without prior written permission is forbidden. The information contained herein has been obtained from sources believed to be reliable. Gartner disclaims all warranties as to the accuracy, completeness or adequacy of such information. Gartner shall have no liability for errors, omissions or inadequacies in the information contained herein or for interpretations thereof. The reader assumes sole responsibility for the selection of these materials to achieve its intended results. The opinions expressed herein are subject to change without notice. SC ext