Georgia Institute of Technology. Yogendra Joshi G.W. Woodruff School of Mechanical Engineering

Transcription

1 Georgia Institute of Technology Yogendra Joshi G.W. Woodruff School of Mechanical Engineering

2 Site Personnel Site Director: Dr. Yogendra Joshi Faculty team: Dr. Satish Kumar Dr. Minami Yoda Researchers/Students: Vaibhav K. Arghode (Post Doc) Jayati Athavale (Ph.D.) Yunji Gu (M.S.)

3 Research Focus Areas Air flow imaging Air delivery from floor tiles measurements and modeling Server air flow, temperature, measurements and modeling Dynamic sensing and controls for energy usage optimization Reduced order modeling Liquid cooling Waste heat recovery Containerized data centers

4 Particle Image Velocimetry Perforated Tile Air Flow Rate = m 3 /s (496 CFM) A 20kW rack requires ~ 1m 3 /s (2,118 CFM) with a temperature difference of 20 C across the rack Inlet velocities ~O(0.3 (top)-1.2 (tile surface) m/s Y. Joshi and P. Kumar, Eds., Energy Efficient Thermal Management of Data s, Springer, 2012

5 Perforated Tile Air Flow Rate = m 3 /s (2594 CFM) Inlet velocities ~O(5.83 (top)-7.5 (tile surface) m/s Bottom servers up to the height of 500 mm from the floor do not receive cool air Air entrainment velocity in the cold aisle (shown white dotted) has increased from ~ 0.6m/s (case 3) to 1.8m/s (case 4) severely disrupting the air distribution to the opposite rack Increased severity of reversed flow in the servers located in the bottom of the rack (v~1.5 to 2.5 m/s). Reversed flow height increases to ~ 400 mm

6 Aisle Rack Data Lab 6 CFD/Reduced Order Model of Tiles 2500 CFM of air flow through both Tile and Rack Pressure outlet Aisle top Top (-46.2%) Top (-43.5%) Top (-0.8%) Pressure inlet (K=10) Pressure inlet Grill Tile Gap Fan Mass flow inlet Pressure inlet Symmetry Velocity (m/s) Tile Experiments (PIV) Tile Geometrical Resolution (GR) Tile Modified Body Force (MBF) Tile Porous Jump (PJ) MBF model able to capture prominent flow features from PIV and GR model [ref] Arghode, V. K., Joshi, Y., Room Level Modeling of Air Flow in a Contained Data Aisle, ASME Journal of Electronic Packaging, v 136, p , 2014.

7 Aisle Rack Data Lab 7 Air Delivery from Tile to Adjacent Rack (PIV) Top (67.7%) Top (100.1%) Top (64.6%) Top (72.5%) U mag /U in K = P/(0.5 U in2 ) Tile (K=9.7) Tile (K=38.3) Tile (K=9.2) Tile (K=9.7) Tile Flow = Rack Flow 36.7% 1/4 1,177 CFM Porosity 21.1% Pore size 1/8 Flow rate 1,766 CFM Lower porosity higher air by-pass Smaller pore size non-negligible effect on flow field Higher flow rate minimal effect [ref] Arghode, V. K., Joshi, Y., Experimental Investigation of Air Flow through Perforated Tile in a Raised Floor Data, (ISTP 2013) International Symposium on Transport Phenomena, November 2013, Yamaguchi, Japan.

8 Flow Rate (%) Rack1 Rack2 Rack3 Rack4 Rack5 Rack6 Rack7 Rack14 Rack13 Rack12 Rack11 Rack10 Rack9 Rack8 Data Lab Temperature ( C) 8 Cold Aisle Containment Studies EXP, open, 1 only Open Aisle Closed Aisle 1U Storage V15 V16 V17 V18 V19 V20 V21 Empty 1,14 2,13 3,12 4,11 5,10 6,9 7,8 EXP, closed, 1 only CRAC 1 - ON T14 T13 T12 T11 T10 T9 T8 T1 T2 T3 T4 T5 T6 T7 CRAC 3 - OFF Networking CRAC 2 - OFF V14 V13 V12 V11 V10 V9 V8 V1 V2 V3 V4 V5 V6 V7 1,14 2,13 3,12 4,11 5,10 6,9 7,8 Containing cold aisle improves the thermal field uniformity [ref] Arghode, V. K., Sundaralingam, V., Joshi, Y., Phelps, W., Thermal Characteristics of Open and Contained Data Cold Aisle, Journal of Heat Transfer, v 135, p , % 125% 100% 75% 50% Total Tile/Rack Air FLow Rate 89% 79% Open, 1 Only Closed, 1 Only

9 Data Laboratory at Georgia Tech

10 8.75m Power distribution unit D O O R Downflow CRAC Data Laboratory Layout APC In row cooler HP rack HP rack IBM rack rack Mixed HP/ IBM rack CEETHERM Data Layout Upflow CRAC APC In row cooler Mixed HP/ IBM rack HP rack IBM rack rack Server Simulator Downflow CRAC Upflow CRAC Dell Power edge 64 Node Rack 640 node rack Storage Storage Downflow CRAC Spare Network Downflow CRAC 6.4m 6.4m Experimental zone Computational a zone Details: 10 Racks 6 Servers / Rack 14 Blades / Server 4 Cores/ Blade Total: 3360 Cores Power distribution unit Power distribution unit D O O R Equipment Donations courtesy: Racks: APC Fan assist Chimney Racks: Wrightline Servers: Intel and OIT CRAC unit: Liebert In row coolers: APC (Critical Components) Server Simulator: APC Branch circuit power metering: PDI corp Remote KVM switch: Minicom and Digi

11 Experimental Facilities Fan speed and Heat setting dials 3-D PIV system 3-D Stereoscopic PIV (Particle Image Velocimetry system for room level air flow mapping. 25kW Server Simulator with adjustable fan and heater settings to simulate a variety of heat loads. Perforated tiles with adjustable dampers to control air discharge rate. Server Simulator Perforated floor tiles with dampers

12 Data Lab Particle Image Velocimetry System (PIV) Pulsed Laser (Nd:YAG) t Cylindrical lens Top Aisle Top Light Sheet Laser Cameras Rack Inlet Image map 1 pulse 1 Interrogation area Image map 2 pulse 2 CCD camera Flow seeded with particles x, y ( Particle Displacement) Traverse Aisle 1 2 Rack 6 ft 3 Aisle Camera 2 (Image map B) Camera 1 (Image map A) Measurement plane area (0.61x1.98) m 2 Cross correlation function Vector map Data Analysis Velocity Stream lines Vorticity Turbulence Noise filtering and random vector elimination Velocity vectors Δx Δy U V Δt Δt Vector Statistics Tile 2 ft Laser Traverse Y Light Sheet X Perforated floor tile Sever Rack PIV system One of the first PIV systems for rack level air flow mapping Customized to scan the desired plane and stitch vector maps together to get full flow field

13 T type thermocouples with wire gauge 32 Covers width of the cold/hot aisle and height of the server racks Data Lab 13 Grid based Temperature Measurement Cold Aisle (252 thermocouples) Hot Aisle (126 thermocouples)

14 Calorimetric Flow Rate (kg/s) Data Lab 14 Other Measurement Tools Thermistor + Anemometer (15 3) 1.2 Server Simulator Tile Air Flow Rate Measurement Measurement of dynamic pressure head to obtain velocity, compensation for flow resistance through the hood ±10% Anemometic Flow Rate (kg/s) Rack Air Flow Rate Measurement (Custom, Degree C) Measurement of velocity using thermal anemometer Plenum Pressure Measurement (Degree C) Mass flow measured based on differential temperature over an heated surface & correlated to differential pressure Compensation for connecting tube lengths

15 Data Lab 15 Cold Aisle Containment CRAC- 1-D Racks Return Racks Return CRAC- 3-U Deployable curtain at the top PDU-1 PDU-2 CRAC- 2-D Door at aisle entrance Open Aisle Fully Contained Aisle Blanking panel for front Physically separating the hot and cold aisle can result in uniform server inlet temperature Door at aisle entrance Network rack Racks with rear door heat exchangers

16 Data Lab Different Tile Designs Under Investigation TOP BOTTOM Fan Generic Chamfered Slotted Perforated tiles used to supply air from plenum to the room space Parameters porosity, pore size and shape, anterior dampers or fins

17 Power (W) Dynamic Events in Data s Fluctuating IT load VM Power Profile Liu et al., Phil. Trans. R. Soc. A Time (min) Courtesy Junwei Li, CERCS, GT Microsoft Live Messenger 17

18 Dynamic Resource Allocation Over-Provisioning Loss of cooling resources ( Lower CRAC set points than required) Armbrust et al., 2009, Report UCB/EECS

19 Power (W) Potential Solution VM Power Profile Fluctuating IT Load Control of CRAC Set Points Rapid Thermal Characterization Need for real-time datacenter thermal Time (min) characterization for better capacity planning. 19