Energy-efficient & scalable data center infrastructure design

Transcription

1 Seminar am 30. September 2010 in Wetzlar Nachhaltigkeit durch UPI (Unified Physical Infrastructure) Vortrag von Stefan Fammler: Energieeffiziente Strategien im Rechenzentrum Energy-efficient & scalable data center infrastructure design Lars-Hendrik Thom Stefan Fammler Regional Technical Support Manager D-A-CH Streategic & Benelux Account Manager 1

2 Yesterday s Physical Infrastructure Most businesses maintained the various system physical infrastructures in silos Power Communication Computing Security Control The Actual Trend of Infrastructure Systems Demand for IP Communications is leading to convergence of these systems Convergence requires physical and logical system integration, which affects system performance 2

3 The Physical Infrastructure Vision Align Merge Optimize Infrastructure Risk Management The complexities of convergence create risk in physical layer An IT Risk incident has the potential to produce substantial business consequences that touch a wide range of stakeholders. In short, IT risk matter now more than ever! IT Risk Turning Business Threats into Competitive Advantage Harvard Business Press 3

4 Integration & Interdependence Effective infrastructure management reduces risk throughout architecture As investment in integration technology increases, IT organisationswill continue to evolve their enterprise-wide integration infrastructure to handle user interaction, business process, applications and data. Colin White, BI Research IT risk matter now more than ever! Most of the businesses today need already a solid IT, but redundancy concepts allow to handle the risk of loosing data. IT infrastructure tomorrow is more all systems in a building will use it. Systems, which may have no possibility to get a backup It makes sense to have a closer look at the passive components and to invest now in clever solutions. 4

5 Physical IT Infrastructure Design should be Energy-efficient using intelligent cooling systems support the most effective usage of the cooling power Scalable and Reliable save investment react fast and easy on business and technology changes...the journey from the coal to the server 60% efficiency loss ~0,3 0,5 MW used for computing 5-10% loss for distribution and transformation 16 MW coal energy 15-30% loss of power supplies and devices ~5 MW usable energy 65% cooling and power transformation 5

6 Increasing Heat Load 2002: INTEL Pentium 3, 1 GHz max. power consumption 26W 2004: INTEL Pentium 4, 3 GHz max. power consumption 83W 2007: INTEL Xenon, 3,6 GHz max. power consumption 130W More speed and higher performance traditionally result in a higher energy consumption... How to fight temperature raises? Decrease the air temperature at the cold aisle to better cool the components > higher energy consumption of the CRAC Increase the pressure of the cold air inside the raised floor or speed up the cold airflow > higher energy consumption of the CRAC Invest in additional CRAC units > expensive invest AND additional energy consumption CRAC = Computer Room Air Condition 6

7 The better way to pay the bill... Total cost of ownership for a DC Energy; 40% i.e. ~ 3 Mio energy costs Energy costs Cooling; 40% Increasing the cooling efficiency could easy save several per year. ~ 1.2 Mio cooling energy costs Cooling efficiency used for cooling; 40% bypass airflow; 60% bypass airflow: short cuts between the cold and hot regions at a DC without cooling active components ~ loss per year! Source: Uptime Institute Energy-efficient design 1) Clear and consequent hot/cold aisle setup Orientation of cabinets front to front with no exception Single height cabinets and closed rows of cabinets to clearly separate hot and cold aisles Keep space for additional cooling options (if later necessary) Source: Emerson Liebert 7

8 Closed rows of cabinets Vertical blanking panels at the outside of the 19 frames Easy to install horizontal blanking panels Transformable cabinets to keep them in place if the usage changes (i.e. Network, SAN, Server etc.) Keep Space for additional cooling If the heat load exceeds react -even if the DC need to stay under full operation -with the installation of a i.e.: cold aisle containment system hot aisle containment system dedicated water based heat exchangers 8

9 Cold Aisle Containment Separates the hot air from the cold aisle and makes sure there s no mixture above the cabinets and at the end of the rows Some recommendations: Easy to install at cabinets in operation Wide to open & self closing doors Easy to open transparent top panel for access to cable paths above cabinets No loose parts and stable enough to handle the air pressure inside the cold aisle Hot Aisle Containment Separates the hot air from the cold aisle by channelizing the exhausted air from the components directly back to the CRAC unit. No mixture of cold and hot air results in higher possible cold aisle temperaturesup to about 25 C (depends on the component requirements) better operational efficiency of the CRACunit due to a higher temperature difference between cold and hot air 9

10 Partial Hot Aisle Containment If the room has just enough height already the add of a vertical exhaust channel results in a better cooling performance ideal solution for fast help if the heat load exceed at one single cabinet i.e. at the end of a row How does it work? The hot air is blown out higher into the room and the resulting sphere of hot air is staying more away from the cold air sphere. This reduces the mixture of cold and hot air. Additional cooling with passive heat exchanger If Water at the DC is not an issue for you prepare water pipesinside the raised floor at areas you plan to use for high performance components (blade server etc.) easy on demand installation of a passive water based heat exchangerto the rear side of a cabinet -connected to the low pressure chilled water circle of the CRAC unit. Up to 20kW additional cooling power at the single cabinet (about 25-30kW in total) Nomoving parts reduces costs for maintenance and additional energy consumption Should be easy to install in full operation of the cabinet 10

11 Energy-efficient design 2) Optimize the airflow within the DC (for new and existing ones) Use solutions to proper route the airflow inside the cabinets and racks Eliminate any blockings for airflow (Re-)calculate the number and the opening ratio of the perforated tiles the necessary amount Close all other openings to keep the air pressure inside the raised floor Limit the length of cabinet rows to get enough cold air even to the last cabinet (or if air comes from 2 sides at the middle) Source: Emerson Liebert From right -> left to front -> back Exhaust ducts to route the hot air of right to left blowing switches to the rear of a cabinet Cabinet without Exhaust duct NET-ACCESS Cabinet with exhaust duct 11

12 Rear side mounted ToR switches Use cold air inlet ducts to provide ToRswitches with enough cold air at their inlets Top of Rack Switch Temperature (C) Switch Air Te mpe ratur e No Duct Duct Front closed with blind panels Without inlet duct Hot exhaust air of the server is flowing even to the cold air inlets of the switch Server % 75% 100% Fan Spe ed With cold air inlet duct Different Inlet Air Duct Samples Air inlets at the side ofa Cisco 4948 Front to back airflow at a Cisco Nexus 2k air duct extension to the front of the cabinet Inlet and exhaust air ducts for Cisco s Nexus

13 Eleminate airflow blockings Airflow blockings occur: inside the raised floor due to pipes and cable mess behind servers due to flexible cable arms or even cable mess At the front of the components due to many connected patch cords -> the main reason for airflow blocking is often a weak cable management Design clear paths inside the raised floor The raised floor should be high enough to provide the necessary cold air pressure everywhere with the same level (recommended cm). If cables are routed inside the raised floor it should be done on dedicated path ways, which allow the airflow above and below. Under floor cable pathway systems should be easy to install under operation (with minimized opening of tiles) to react on upcoming business demands for additional connections 13

14 Overhead Cable Routing Systems To minimize the risk of air flow blocking inside the raised floor use for the majority of cables the space above the cabinets if possible. Separate physically the more sensitive fiber patch cords from the copper cabling Look for a system that provides continuous bend radius control -for copper too! Easy access grants later the usage and minimizes work arounds Cabel Management./. Air Exhaust Even a clean looking cable management can cause heat problems at server cabinets. Flexible cable arms to slide a server out of the rack under operation become more and more useless due to the growing server virtualisation Virtual machines can be moved from one physical machine to an other Not necessary anymore to slide out a physical server under operation to change or repair parts inside the chassis. 14

15 Cable Management at RU level Vertical cable management supported by a finger system at the side of the 19 frame allows to proper route cables and overlenghts away from the back side of the server > minimized blocking of the air exhaust Side mounted patch panel allow also the usage of equal long patch cords > less stock. Look for dedicated cable path ways inside the server cabinets to separate cables by function. (LAN A / LAN B / OBM / SAN) This minimizes cable mess and makes moves and changes easier. Cable Management at the front Eliminate horizontal cable manager with a vertical cable management. Fingers for each RU at the left and right side allow to handle overlengthat the open space beside the 19 frame Look for angled panels, which also guide people to route patch cords to both sides and not crossing the middle of the panel -> minimized cable mess, over length at side -> easy operation, easy MAC -> no air flow blocking at the cold air inlets 15

16 Energy-efficient design 2) Optimize the airflow within the DC Route airflow front to back inside the cabinets Eliminate any blockings for airflow Calculate the number and the opening ratio of the perforated tiles the necessary amount Close all other openings to keep the air pressure inside the raised floor Limit the length of cabinet rows to get enough cold air even to the last cabinet (or if air comes from 2 sides at the middle) Source: Emerson Liebert Perforated tiles A typical DC is providing the cold air trough the raised floor. To bring the cold air to the front of the cabinets and to the inlet openings of the components, perforated tiles are positioned in front of the cabinet. The correct amount and position should be calculated to compensate only the expected heat load per cabinet/row. To many openings result in a loss of cold air pressure inside the raised floor To compensate this loss you need to increase the pressure at the CRAC units > increased energy costs. 16

17 Close all other openings As mentioned with each open hole at the raised floor you reduce the air pressure and need to compensate this with higher energy consumption of the CRAC unit. Close all unwanted holes especially at cable throughputs the more airtight the better! Closed but still easy to operate...? Brush systemsto close cable opening are very popular because they are easy to installand allow a fast an easy add or remove of cables But brush locksstill let air flow trough the opening and are not able to keep the air pressure. An alternative solution are closed air bags, which can also be installed around existing cable bundles and closed the hole AND keep the air pressure! MACsare almost as easy as with brush locks. 17

18 The lenght of a row... The length of a row of cabinets depends on the capacity and position of the CRAC units The longer a row the better should be the air flow control to provide enough cold air at any RU. Summary An optimized air flow and energy-efficient design creates more options and headroom for the specific data center infrastructure. Energy-efficiency and airflow control help to save energy and to reduce costs. 18

19 Thank you very much for your attention! More detailed information at Do you have a question? 19