Technical Systems in Data Centres Helen Bedford
Technical Systems in Data Centres Overview Cooling IT and Peripheral Equipment Security: Access and Control Building Management System (BMS) Data Centre infrastructure Management (DCIM) Overall Data Centre Modular Data Centre Solution Power: Provision i and Back-Up Fire Detection and Suppression Excellence in Market Intelligence 2
Recent Changes to Cooling Requirements Resulted in New Technology Adoption Power Usage Effectiveness is a measure used to understand the energy efficiency of Data Centres IT + DC Power PUE = IT Power At the moment the average PUEinmany Data Centres ranges from 1.8-2.5, however, the most recent deployments, such as Capgemini Data Centre in Swindon (Merlin) states PUE=1.08 As cooling can account as much as 30% of operating costs, considerations of the type of cooling systems installed can directly impact PUE and operating costs after project deployment. However, traditionally, the main priority in the critical managed facilities was security and stability of operations running with low downtime risk. As IT equipment was more likely to suffer from faults and failures at higher temperatures, the cooling systems in Data Centres in the UK provided temperatures at ~18-20 C, inlet temperature falling sometimes as low as 12 C The new temperature and humidity ranges were relaxed in the latest ASHRAE9.9 guidelines to reflect the technological changes in server equipment able to operate effectively at higher levels At the same time, the back-up options should be implemented to ensure that the data centres stay operational under all conditions, incl. cases of prolonged power cuts and main cooling systems failures (DX is the most common back-up option in the UK). Another consideration is that, if the air temperature is raised to 25 C, then servers will start to aggressively increase air flow to the servers, thus counteracting the energy savings achieved in cooling (through increased fan power consumption by servers). Hot and cold aisle containment is a popular solution, providing a cost effective way to increase energy efficiency by blocking the air streams through installation of air blankets. Hot aisle is more popular in New Data Centre Market applications while cold aisle containment is the most popular option in retrofit. Rising inlet and return temperatures is promoted as an effective way to increase PUE and allow for the direct free cooling method being successfully deployed in certain regions in the UK (mainly in the North). Excellence in Market Intelligence 3 ASHRAE 9.9 Guidelines expanded allowable humidity and temperature corridors for Data Centre Applications Source: HP 2012, ASHRAE 2011, Green Grid ASHRAE 2004 and 2008 Guidelines: Comparison Year 2004 2008 Guidelines Recommended Allowable Recommended Allowable Temperature 20 C 25 C 15 C 32 C 18 C 27 C 10 C 35 C Humidity 40 % - 55 % 20 % - 80 % 5.5 C 60 C 20 % - 80 % In 2011, recommended range stayed at 2008 level, with allowable temperatures for A1 class being 15 C 32 C and extended to lower classes to 5 C 45 C; allowable humidity range 8% - 90%
Most Popular Cooling Solutions in Data Centres in the UK As a rule of thumb, if the data centre has 50+ racks it is most likely to go down the chilled water route, whereas those with <10 racks tend mostly to use DX condensers to cool the cooling solutions. Thus, DX condensers are more popular for smaller Data Centres, while chillers are mostly suitable for large Data Centres due to their applicability and the levels of costs. DX condensers in smaller Data Centres are installed in double capacity to cater for redundancy. Water Cooling Towers are not commonly used in the Data Centres in the UK, mainly owing to the risk associated with legionella disease. It is estimated that 70% of IT equipment in the UK is now cooled using hot or cold aisle containment, with Hot Aisle Containment being used mainly in the New Build Data Centres market segment. Cold Aisle Containments is a very cost-effective option for brown-field Data Centres. Hot Aisle containment is a less preferable option for shared Data Centre Space. Air and pressure blankets allow for implementation of Aisle containment and are relatively cheap. Excellence in Market Intelligence Most Common Cooling Systems Overview System Type Heat Rejection Mechanical Cooling Equipment Chilled Water Direct Expansion (DX) Source: ASHRAE, 2011 Cooling Tower Chiller (air-cooled) Chiller (watercooled) Condenser CRAC C (DX) Dry Cooler Hot Aisle Containment Courtesy http://www.42u.com/cooling Condensing Unit CRAC (water/ glycol) Self-contained AC unit/ rooftop unit Terminal Equipment CRAH (chilled water) Air handler (DX) Cold Aisle Containment
Special Features: Cooling - Overview (1) CRAC (Computer Room Air Conditioning) Still the most common type of cooling system that provides cooling via raised floor or an overhead duct distribution system CRAC units contain a cooling coil, a supply air fan, return air filer; some might also have refrigerant compressors, a humidifier, a reheat coil and a free-cooling coil. Can be combined either with DX condensers or watercooled chillers The cost of CRAC units with fans, centralised ventilation/dehumidification system and separate Direct Expansion (DX) Condensing Units DX systems are ideal for small Data Centres or as a system providing for additional redundancy They perform better in harsh environmental conditions, so are more reliable Normally in the UK they accompany evaporative cooling as a back-up option in case of failure, both in traditional and modular design humidifiers make them one of the most cost effective solutions for a new Data Centre facility Adding Variable Controls and EC Fans will provide the necessary option for energy efficiency; and, alongside other possible measures, such as raising supply and return air, bring additional increase of PUE in a New Data Centre in a cost effective way European players: Emerson, Schneider, GEA Denco, Stulz Not applicable for large Data Centres due to impracticality Require increased level of maintenance due to the multiple pieces of equipment (fans, controls, compressors) Can have high levels of energy consumption Players: Gea Denco, Climaveneta, APC, EdenAir Source: Uniflair Source: Airdale Chillers High efficiency multi scroll with free cooling nibbling at Chillers account for a large portion of the overall cooling system cost in the new Data Centres, especially larger ones the lower end. Above 500-600kW screw and turbocor dominate. Adiabatic cooling more relevant to southern Europe, although rolled out in some projects in the UK, It is estimated that total value of chillers sold in the UK mainly in the educational sector data centre market is 9% of the total value of the UK chiller market. This amounts to 14m in 2011. Cooling towers. Cost of maintenance of cooling towers is reported to be too high and puts a lot of organisations The trend in the market is that free cooling is being made available in smaller machinery. You need to have off installing the equipment, as well as the old chestnut of legionella disease the right water temperature in order to make free cooling a viable option Main leaders: Airdale, Liebert Hiross, Climaveneta and Uniflair. Excellence in Market Intelligence 5
Special Features: Cooling Overview (2) Evaporative (or Adiabatic) Cooling Not a very new method but is gaining popularity Utilises evaporation method as a means of cooling down the air or water temperatures Can be direct and indirect be increased if water is sprayed directly into the air stream going into the Data Centre There are some concerns over the applicability and reliability of evaporative cooling in wider range of temperatures in the UK and potential concerns The application of evaporative cooling can have over legionella ll disease geographic limitations in the UK (especially in the Need for maintenance every 24 hours with double coastal areas with increased levels of humidity) units The humidity of the air inside the data centre could Water consumption can be high and expensive Direct Free Cooling provides for wider range of allowable Allows to capitalise the new ASHRAE temperatures and humidity temperatures guidelines If the temperature goes above 18 C, then DX CAPEX costs reduced compared to indirect condenser will work to bring the temperature cooling down. Lower energy consumption Believed to be acceptable in London due to Direct Free Cooling is allowing ambient air to enter climate, the room with heat exchangers folded away and Close Coupled (In-Rack and In-Row) Open-Loop solutions bringing the heat transfer closer to the equipment rack and not completely independent of the room and interacting with the air. Main types: In-Row, in-line air conditioners, Rear Door Heat Exchangers (fairly popular in the UK), Overhead Heat Exchangers (more rarely applied) Liquid Cooling Rather innovative and requires complete change of mind-set to be adopted, especially from the risk adverse IT managers; thus, highly unlikely to gain much traction in the near 3-5 years Mainly refers to submersion into the liquid gas Novec1230 Source: Stulz Closed-Loop solutions is mainly In-Rack solution that are independent of the air stream in the room. AC is applied to server rack and completely sealed. Allows for the direct cooling of servers and not the whole room Provides for scalability and modularity Players: Rittal, APC, Emerson, Hitachi Source: Hitachi (In-Row), USystems Marginal deployment in the UK, undergoing some trials and minor installations in the academic sector More likely to be accepted as a norm on a micro (semiconductor) level, if successfully developed and deployed There are also other types of liquid possible: either Delivers higher cooling capability and lowers PUE directly into water or mineral oil Source: Iceotope Excellence in Market Intelligence 6
Other Technical Systems in Data Centres in the UK Helen Bedford
Other Technical Systems: Total Market The estimated total value of M&E systems for New Data Centres in the UK (excluding DCIM applications) is around 105 million Estimated Market Size for M&E Systems in New Data Centres in the UK (addressable market) The efficiency of power systems is very BMS Security high and unlikely to be changed in a way Fire that dramatically affects the industry as 5 3.5 4 whole. Introduction of generators feeding into the grid, as well as heat recovery could increase the potential spending on M&E 27 systems in the Data Centre segment, however, overall it is expected to remain at the fairly conservative level of growth of around 3-4%. A marginal dip could be expected in 2013 due to delay of some projects further into 2014. Fire Detection is estimated to be worth around 1.7millionwiththefiresuppression to be nearly twice of thatt at 3.33 million Fire Suppression is likely to remain a cost saving option and unlikely to grow dramatically in value. Cooling 65 Power Power Segment represents the majority of value due to the large expense of generators, transformers and other power provision in the Data Centres to cope with the increased IT load requirements Cooling is another big segment estimated to be around 27 million in 2012 for the New Data Centres in the UK BMS sector might receive further expansion to incorporate energy efficiency monitoring and integration with DCIM systems. Excellence in Market Intelligence
Major Categories of Gas Fire Suppression in the Data Centres in the UK Heat Absorption Inert Gas Agent Oxygen Hypoxic Atmosphere Du Pont FM-200 is the most common Argonite (Chubb) is electrically non- This type of fire suppression has Fire Suppression System application. The main characteristics behind its popularity are: conductive and is similar to air in density. Main disadvantages: increased in usage in recent years, especially becoming more popular in modular Data Centres. It is relatively cheap to install and Relatively slow flame extinguishing It is based on the notion of providing carries a moderate recharge cost Widely available, however, even in period of supply shortage, sales were not greatly affected end users were not driven away. In contract, Novec1230 has a far longer shelf life, up to 30 years, but could not compete on a larger scale with FM-200, as it is more expensive to buy and recharge, and it is more complicated in handling and transportation. In a longer term, it might be the preferable option. High pressure storage Large volume of gas required, limiting space in the Technical Area Inergen releases the gas under high pressure adjusting the atmosphere, so that fire cannot longer survive. Main disadvantages: Cannot be modular in design High pressure method requires excessive storage, in addition to long and complicated high-pressure pipework oxygen below the level required for fire ignition but at the level safe for the staff operating at the Data Centre. The typical solution falling under the category of Oxygen removal is DeOx by DigiPlex. It is achieved by reducing the level of oxygen to 155 beyond the required 16.2% for fire to sustain itself. The low level e of oxygen still allows staff to carry out essential maintenance work, if needed, and is equivalent to the oxygen conditions in high mountain areas. Water mist and other types of fire suppression are less commonly used in the UK in comparison to the US, where water mist is highly utilised. However, some colos are using this method as well (Digital Realty) Excellence in Market Intelligence 9