Data Center Thermal Management

Transcription

1 An Employee-Owned Corporation Data Center Thermal Management BICSI South Central December All rights reserved.

2 Two Strategies in One A high density cooling strategy An energy conservation and money saving strategy Containment isolation between cool supply air and warm return air

3 Hot Aisle/Cold Aisle Ideal Implementation HOT AISLE COLD AISLE HOT AISLE COLD AISLE Fronts face fronts; backs face backs Open floor tiles in cold aisle; no open floor tiles in hot aisle Seal all floor tile cable access cut-outs

4 Bypass Air Solutions Close all unnecessary access floor openings Areas around perimeter walls, columns, behind cooling units, behind PDU s. Ensure all sleeves and penetrations in walls are properly sealed Add 40% - 100% cooling capacity by reclaiming wasted bypass air

5 Bypass Air Solutions Enlarged Scale

6 Bypass Air Solutions Enlarged Scale

7 Hot Aisle/Cold Aisle Ideal Implementation HOT AISLE COLD AISLE HOT AISLE COLD AISLE Fronts face fronts; backs face backs Open floor tiles in cold aisle; no open floor tiles in hot aisle Seal all floor tile cable access cut-outs Air delivery satisfies air consumption

8 Cooling Capacity Planning Electrical load X 0.28 = Tons AC UPS kva X Power Factor 45 = Tons AC Space 500 = Tons AC Add for total cooling capacity requirement Add redundancy requirement Planning Procedure from Bick Group WARNING Whatever you do will be wrong!

9 Locating Cooling Units Cold Aisle C U Hot Aisle C U Cooling units or air handlers should be at ends of hot aisles

10 Improperly Located Cooling Units Enlarged Scale

11 Improperly Located Cooling Units Enlarged Scale

12 Locating Cooling Units

13 Locating Cooling Units 8 ft min C U Cold Aisle Hot Aisle C U Cooling units or air handlers should be at ends of hot aisles Don t locate open floor tiles too close to CU s

14 The Cause of Unbalanced Flow Distribution CRAC Velocity decreases, pressure increases

15 Effect of Plenum Height

16 Effect of Tile Open Area

17 Use of Variable Tile Open Area

18 Locating Cooling Units 8 ft. min C U Cold Aisle Hot Aisle C U Cooling units or air handlers should be at ends of hot aisles Don t locate open floor tiles too close to CU s Close openings in cabinet rows

19 Cabinets and Cable Pathway HOT AISLE COLD AISLE HOT AISLE COLD AISLE Cable Tray Electrical, etc. Cable Tray Electrical, etc. Data cable tray should run down hot aisle and be ¾ from bottom of floor tile Power cables and conduit should be below and in the cold aisle

20 High Density Air Cooling Paradigm Shift The cabinet is a box for housing servers NOT! The cabinet is an integral architectural feature of the data center space for securing the isolation between supply air and return air

21 Manage Air Inside the Cabinet Deliver chilled air to critical point(s) of use Prevent hot air re-circulation Use filler panels Seal off sides of equipment Use side panel(s) in bayed cabinets if required

22 Blocks Re-Circulation Through Cabinets Enlarged Scale

23 Manage Air Inside the Cabinet Deliver chilled air to critical point(s) of use Prevent hot air re-circulation Use filler panels Seal off sides of equipment Use side panel(s) in bayed cabinets if required Remove hot air Rely on server fans and high flow doors or no doors Avoid cable congestion Beware fans

24 Top Mounted Fans Cause Hot Spots Dell White Paper: Rack Impacts on Cooling for High Density Servers The addition of rack fans or fan trays is not recommended. In some cases, additional top mounted rack fans have actually impeded server thermal performance.

25 Top Mounted Cabinet Fans Cause Hot Spots Enlarged Scale

26 Top Mounted Cabinet Fans Cause Hot Spots Enlarged Scale

27 Top Mounted Fans Cause Hot Spots CFM: Typical cabinet roof fan 1350 CFM: Highest standard roof fan on the market CFM: Typical rear fan door ratings 1820 CFM: Four IBM eserver Blade Centers 2400 CFM: Six Dell PowerEdge 1955

28 Case 1: Temperature Distribution at Central Plane of Test Bed Enlarged Scale

29 Curing Hot Spots with Ducted Exhaust Enlarged Scale

30 Curing Hot Spots with Ducted Exhaust Enlarged Scale

31 Curing Hot Spots with Ducted Exhaust Enlarged Scale

32 Curing Hot Spots with Ducted Exhaust Enlarged Scale

33 Curing Hot Spots with Ducted Exhaust

34 Curing Hot Spots with ducted Exhaust Enlarged Scale

35 The Problem Heat densities are exceeding the levels that can be cooled by air delivered through a perforated floor tile in front of a server cabinet, according to conventional wisdom

36 Cabinet Cooling Requirements Increase Blade Server IBM eserver Blade Center 14 servers, dual-processors, 9 RMU Sun Blade servers, quad-processors, 19 RMU Dell PowerEdge servers, dual-processors, 7 RMU HP BladeSystem c-class 16 servers, dual-processor, 10 RMU 42U Qty. Watts BTU/h Tons A/C M 3 h 4 20,068 68, ,000 51, ~ 6 21,570 73, ,056 68, ~

37 High Density Application Hot air is physically segregated from cold air in return air duct Rear door is sealed to prevent exhaust air from leaking into room Air Dams in front of cabinet prevent air recirculation inside cabinet Room A/C is easy to manage, with little regard to concentrated heat loads Cabinets can be placed and oriented any way desired Cold air delivery can be shared throughout room with fewer zone issues

38 Isolate Hot Air In The Cabinet Exploit the laws of physics from the Bernoulli Equation Inverse relationship between velocity and pressure

39 Isolate Hot Air In The Cabinet Enlarged Scale

40 Data Source Third Party Test Servers (6) 7U IBM eserver BladeCenter chassis (14) dual-processor HS-20 blade servers (Intel Itanium 3.2 GHz) (4) 2000W (nameplate) power supplies (2) 350CFM nominal output variable-speed centrifugal blowers Power Source (2) 60A 208V 3-phase circuits Environment Active data center Average intake air temperature of 71 F (22 C)

41 Test Configuration INTERMEDIATE STRINGERS ADDED HOT AIR 22 1/2" x 22 1/2" Hot Air Duct V T dp 23" x 77" SOLID DOOR T T 24" x 48" Cabinet T T T T " " AISLE T P V TEMPERATURE STATIC PRESSURE VELOCITY HOT AIR T T T T T T COLD AIR COLD AIR COLD AIR T COLD AIR

42 Data Source Third Party Test Servers (6) 7U IBM eserver BladeCenter chassis (14) dual-processor HS-20 blade servers (Intel Itanium 3.2 GHz) (4) 2000W (nameplate) power supplies (2) 350CFM nominal output variable-speed centrifugal blowers Power Source (2) 60A 208V 3-phase circuits Environment Active data center Average intake air temperature of 71 F (22 C) 23kW Actual Measured Heat Load

43 Remove Hot Air From The Room Enlarged Scale

44 Standard Data Center Temperature Variation Enlarged Scale

45 No Temperature Variation Enlarged Scale

46 High Return Air Temperatures Are GOOD Cooling Unit Supply Air Temperature Return Air Temperature Cooling Capacity Liebert FH200C 60 F 70 F 7.8 tons Liebert FH200C 60 F 90 F 15.5 tons Liebert FH200C 60 F 105 F 20.7 tons Liebert FH600C 60 F 70 F 23.0 tons Liebert FH600C 60 F 90 F 46.0 tons Liebert FH600C 60 F 105 F 61.3 tons

47 Ducted Exhaust with Ceiling Return Plenum All hot spots eliminated Total temperature variation in room is 6.7 C, so supply air can be raised from 11 CF to 18 C

48 New Construction Energy Saving Strategy Case Study Data Center: 800 kw actual heat load Free space return air Liebert FH422 (2 ea) Liebert FH529 (7 ea) Liebert FH600 (2 ea) Liebert FH740 (1 ea) TOTAL Isolated return air path Liebert FH529 (3 ea) Liebert FH600 (3 ea) Liebert FH740 (1 ea) TOTAL 40 tons 210 tons 60 tons 40 tons 350 Tons 90 tons 90 tons 40 tons 220 tons

49 Effect of Complete Isolation Best Practice CRAC w/ Economizer Evaporative Air Economizer Delivered Supply Water Approach Free Hours???

50 New Construction Results 5 Megawatt data center in San Francisco Bay Area 9 megawatts One year cooling energy use Chilled water CRAC units with water economizer and hot and cold aisles ($669,778) 2 megawatts One year cooling energy use Chilled water air handler with evaporative air economizer and isolation ($155,965) - Analysis by McKinstry Company

51 Conclusion Isolate return air from supply air De-couple point of air delivery from heat load for high densities Exploit economizer benefits Do your homework

52 Homework Air-Cooled High-Performance Data Centers: Case Studies and Best Methods, Utilizing Economizers Effectively in the Data Center, Kyoto Cooling, Reducing Data Center Energy Consumption with Wet-Side Economizers,

53 An Employee-Owned Corporation Thank You Ian Seaton 2003 All rights reserved.