Raising Inlet Temperatures in the Data Centre

Transcription

1 Raising Inlet Temperatures in the Data Centre White paper of a round table debate by industry experts on this key topic. Hosted by Keysource with participants including the Uptime Institute. Sophistication Through Simplicity

2 Raising Inlet Temperatures in the Data Centre The white paper shares the findings of a round table consisting of the industry s leading experts and commentators. Representatives from the Uptime Institute, Operational Intelligence, Colt, Fujitsu, Vocalink, Norland Managed Services and The 451 Group joined Keysource for the round table. The event was designed to generate healthy debate based on real experiences on one of the industry s key topics. This is one of two white papers produced from the round table discussion. Executive Summary Much is talked about in the press and the industry in general about raising server inlet temperatures. However, often this is not referenced in the context of a live data centre. Consequently many operators run the risk of implementing strategies or changes to their facilities without understanding the implications of those changes and without monitoring or management tools to assess the impact of those changes. An education piece is required focussed on raising inlet temperatures addressing the questions: What does ASHRAE really refer to? How do you measure it? What is the real impact of changing set points? Which temperatures are being talked about? With the majority of current IT equipment the energy efficiency savings will not be achieved at the margins; so it is not about pushing inlet temperatures above 27 degrees consistently, but perhaps moving from 22 to 24 degrees. The greatest savings will potentially be made by making smaller adjustments within the recommended ranges, combined with the right design for the cooling. The data centre industry is waiting for the IT sector to catch up in order to take advantage of what can be achieved with just a few more degrees on the allowable range and a leniency on warranties. It is not simply about running all the time with increased inlet temperatures, but rather dynamically control set points to deal with higher external ambient temperatures, where a few more degrees of free cooling could be available. Training, processes and procedures are even more important than expected moving forward. Even when considering a design that has the most efficient solution, the challenges faced in the on-going operational management have to be recognised and how the IT load is deployed to deliver on those design efficiencies. The focus moving forward is not necessarily the quick wins, but about operating in a much more structured and controlled way as part of an overall management system for the data centre, including a joined up strategy between FM and IT. 2 Raising Inlet Temperatures in the Data Centre

3 Key Areas of Discussion The topic under discussion was Does increasing inlet temperatures put availability at risk? The temperatures in question are the ASHRAE recommended range, which all equipment had conformed to over the last 15 to 20 years. The range is from 18 to 27 degrees centigrade, although typically most data centres are supplying at between 20 to 24 degrees and not at the upper limit of the recommendations. Understanding the numbers to use There was general agreement that often there is confusion amongst data centre operators about which numbers to use and how to set the systems to run, many mixing up the temperature of the return air into the cooling unit and the supply air from the unit and the server inlet temperature - which is the temperature of the air going into the IT equipment. Note: for the purpose of this white paper, when the inlet temperature is referred to this is the temperature of the air at the front of the server racks. Despite ASHRAE clearly defining what the inlet temperature is (the air that is being supplied to the front of the server) there is still much confusion that causes a large increase in additional risk and disruption, often leading to poor performance with many systems being run too cold and inefficiently. Many existing facilities are controlled on return air, which causes a lot of the confusion, but the industry has now moved on and is starting to become far more sophisticated with new designs and airflow strategies. This focus on a very specific temperature target has been further compounded by the fact that the colocation market had very specific Service Level Agreements (SLA) which were often not clear about temperatures they referred to and did not give any room for flexibility. With no air management and lots of mixing taking place, it was virtually impossible for operators to control inlet temperatures so they focused on the Computer Room Air Conditioning (CRAC) set points for SLAs. The overall agreement was that it is not possible to look at raising temperatures until there is control over the entire space and the range of temperature onto the servers has been addressed and is consistent. Therefore to start debating this topic there needs to be an assumption; that there is good air management within the facility with little, if any mixing and so there is only a small, if any difference, between the air off the Air Handing Unit (AHU) and the server inlet temperature. Raising temperatures, raising risk Whilst defining the exact increase in risk was thought to be challenging, there was general agreement that, assuming there is control of the airflow and the inlet temperature is correct, going above 27 degrees for an extended period would probably impact availability. This is mainly due to different equipment reacting in different ways. For much equipment if temperatures increase it will complain a bit, but not much will happen, but with some it will simply shut down. Therefore, the closer the temperature is to the maximum recommended, the less comfort zone there is and the less flexibility there is before something happens. The rate of change also affects the performance and failure rate of equipment and this also needs to be evaluated. There was agreement however that everyone should work towards a consistent level and that if ASHRAE recommend 27 degrees then it should be possible to go up to this temperature on the inlet side. An interesting example which was given was of an international financial institution which ran an experiment whereby they turned off a number of the chillers and the temperature was allowed to rise up to 40 degrees. They measured no significant failures of any kind across the whole data centre. In addition, the organisation also measured which fans in the servers started or ramped up, as well as the energy consumption of all the servers, and there was no significant rise in the power consumption. One participant explained that this contradicted what is stated by ASHRAE, that by raising temperatures it is conceivably possible to consume more power because of the affect of fans ramping up. However when this does occur it can vary dramatically dependent on the equipment. For some manufacturers this point will be at 26 to 27 degrees, but tests on some blade servers have shown that the fans still run at minimum speed all the way up to 35 degrees. Therefore it is important to have the correct monitoring systems in place to track the impact and record the actual results of any changes. One barrier that was suggested to raising temperatures was that of warranties issued by equipment manufacturers. Typically if a data centre operator goes outside the ASHRAE recommended temperature it will invalidate the warranty on the equipment. So whether it is more efficient or not there is often a reluctance to raise temperatures to protect the warranty. 3 Raising Inlet Temperatures in the Data Centre

4 Raising to the recommended When considering whether to run a facility at the inlet temperatures of 27 degrees, the general view was that it would be possible. However it depends on whether this translates into a more efficient situation i.e. additional free cooling and good air management with dynamic response that could recover from a failure event. A key issue for many participants was the lack of management tools and expertise amongst many data centre operators to deal with this type of dynamic operation. A lot was said to be down to the level of education and training needed to understand the impact of certain actions. In fact for contained systems there are new parameters that cannot just be blindly applied, so it s necessary to really think about what is being done and consider why and how this impacts the specific site conditions and design. Hype versus reality The point was raised that a lot of statements suggest universal benefits of increasing inlet temperatures, which only encourage operators to act without fully understanding those changes in a dynamic environment. There may be an argument for accepting a greater variation within recommended limits, for short periods of time, and this is possibly the next stage to go to. But there are some fairly substantial barriers to widely increasing inlet temperatures. How high can you go? There is a clear and direct correlation between increasing inlet temperatures and the subsequent increase in the air off temperatures at the back of the server racks. The general view was that with servers with a high delta T, such as blade servers, air management is key as it is possible to have temperatures in excess of 50 degrees. Whilst this will often mean much more efficient cooling the impact on health and safety and working practices needs to be properly considered. In addition to the health and safety aspect, is the impact on other infrastructure such as power bars, cables or data cables, which need to be considered during the design and before making changes. One effective and increasingly adopted approach to deal with this issue was to contain the exhaust air at each rack (because it would not be possible to have a space for people to walk through or work in for any period of time). This solution of using ducted racks is being adopted more widely with this type of design and helps to further improve efficiency. There are two things to be said about air management in terms of energy efficiency: the ability to raise set points, where it is possible to save on compressor power. the fan power of the cooling units, which is a major consumer of energy and probably the most significant after the refrigeration system. There is a massive opportunity to save energy through reducing fan power with better control, but this comes down to air management. 4 Raising Inlet Temperatures in the Data Centre

5 Getting the design right The question of whether increasing inlet temperatures was actually necessary in a well-designed data centre was discussed. It was felt that this partly came down to finding the optimum level to achieve the greatest levels of free cooling and reduced annual costs, by measuring the total power of a facility and not just PUE*. However, this needs to be worked out against any increase in the probability of failure. A common theme emerged suggesting there is an argument that the design concepts should be developed around not controlling to a set point, but controlling to a range of temperatures and humidity. This would still be within the ASHRAE recommended, but enable temperatures to rise for a short number of hours during some hot periods. This too provides the opportunity to start to consider removing the chiller, when they are only needed for these few hot periods. It is also important to operate servers at a temperature that does not result in added energy consumption to the total facility, due to increase demand from the server fans. However, if the server fans typically ramp up between 26 and 30 degrees, the optimal level is unlikely to be as high as this. It may be possible to run a facility at around 24 degrees and only increase the supply air temperature where the trade-off between additional fan power versus mechanical cooling power crosses over. The problem with this is that it is not necessarily known where this crossover point is, because the manufacturers don t publish this data, and it s likely to be different for a range of equipment. The overall consensus was that it is important to have a strategy for increasing inlet temperatures as part of a coordinated approach to lowering energy use and improving efficiency. This, combined with improved cooling technologies and properly managed airflow, will deliver the most effective solutions and biggest wins. Everything must be undertaken with the appropriate levels of management and monitoring to ensure there is no compromise on availability or operational performance. If you would to discuss any of the points raised in this white paper please contact marketing at Keysource and we will be happy to put you in touch with the right specialist. You may also be interested in the first white paper from the round table The Use of Fresh Air in the Data Centre. We would like to thank: Phil Collerton - Managing Director EMEA at the Uptime Institute, Alfonso Aranda - Consultant at the Uptime Institute, Luke Neville - Senior Technical Lead, Design Authority, Data Centre Services (DCS) at Colt, Andy Lawrence - Research Director at The 451 Group, Jim McGregor - Head of Engineering and Data Centre Management at Vocalink and Mike West - Managing Director of Keysource, as well as representatives from Operational Intelligence, Norland Managed Services and Fujitsu for joining us and sharing their expertise. * Note: Power usage effectiveness (PUE) is the industry-recognised metric for infrastructure energy efficiency for data centres, developed by The Green Grid Association and is the ratio of total facilities energy to IT equipment energy. Keysource frequently use the PUEL2,YC measure for client facilities. This is a Category 2 measurement of PUE (as defined by the Green Grid and a 12 month total kwh consumption). 5 Raising Inlet Temperatures in the Data Centre

6 Keysource focuses on the full cycle of infrastructure technology in the data centre through consultation, design, build, ongoing management and optimisation. When high-performing resilient environments are needed to meet the latest demands of high-density IT and cloud technology, Keysource is trusted for simplifying the delivery of innovative data centre and integrated management solutions. We deploy the latest IT technology and reduce power consumption and operating costs for our customers through our own high performance cooling solution, ecofris. T: +44 (0) E: W: Keysource has delivered award winning data centres for some of the most established and innovative brands and organisations in the UK and globally. Our openness and honesty and vendor neutrality mean you ll get a straight, uncomplicated answer and the right solution for your business. The Raising Inlet Temperatures in the Data Centre white paper is published by Keysource (Copyright Keysource Ltd). All rights reserved. No part of this publication may be reproduced, copied or transmitted in any form or by any means, or stored in a retrieval system, without the prior permission of Keysource except as permitted by the provisions of the Copyright, Designs and Patents Act 1988 and related legislation. Application for permission to reproduce all or part of the Copyright material should be made to Keysource Ltd, North Heath Estate, Horsham, West Sussex, RH12, 5QE. Although the greatest care has been taken in the preparation and compilation of Raising Inlet Temperatures in the Data Centre white paper, no liability or responsibility of any kind (to extent permitted by law), including responsibility for negligence is accepted by Keysource or its agents. Keysource has reported the views and opinions of participants of the round table and does not endorse these views unless specifically stated. All information is believed correct at time of publication but there is no assurance or guarantee with respect to its accuracy. 6 Raising Inlet Temperatures in the Data Centre

7 T: +44 (0) E: W: