Role of the PI System in Georgia Tech Data Center Laboratory

Transcription

1 Role of the PI System in Georgia Tech Data Center Laboratory Presented by Vaibhav K. Arghode Prepared by Vikneshan Sundaralingam

2 Outline Data center laboratory general information Current use of PI System Future use of PI System Copyri g h t 2012 OSIso f t, LLC. i

3 3 Data Center Georgia Tech A Platform for Transformative Research on Green Data Centers Director: Dr. Yogendra Joshi yogendra.joshi@me.gatech.edu

4 4 8.75m Power distribution unit D O O R Downflow CRAC APC In row cooler HP rack HP rack IBM rack Center rack Mixed HP/ IBM rack CEETHERM Data Center Layout Upflow CRAC APC In row cooler Mixed HP/ IBM rack HP rack IBM rack Center rack Server Simulator Downflow CRAC Upflow CRAC Dell Power edge 64 Node Rack 640 node rack Storage Center Center Center Center Storage Downflow CRAC Spare Center Center Center Center Center Center Network Downflow CRAC 6.4m 6.4m Experimental zone Computational a zone Center Details: 10 Racks 6 Servers / Rack 14 Blades / Server 4 Cores/ Blade Total: 3360 Cores Power distribution unit Power distribution unit Data Center Layout D O O R Equipment Donations courtesy: Data Acquisition: OSIsoft Building Management System: McKenney s Racks: APC Fan assist Chimney Racks: Wrightline Servers: Intel, Yahoo and OIT CRAC unit: Liebert Server Simulator: APC Branch circuit power metering: PDI corp Remote KVM switch: Minicom and Digi In row coolers: APC

5 CRAC (Down flow) CRAC (Down flow) CRAC (Up flow) CRAC (Up flow) CRAC (Down flow) CRAC (Down flow) 5 Dampers 6 inch thick Insulated collapsible partition Data Center- Air Distribution Cold air inlets Hot air exhaust Contained hot air roof return 2 ft. ducted supply Plenum 2 ft. ducted supply Plenum 3 ft Raised floor plenum Over head supply Under floor supply 2 ft. ducted supply Plenum 2 ft. ducted supply Plenum Economizer

6 6 Particle Image Velocimetry System (PIV) Fan speed and Heat setting dials PIV system One of the first PIV systems for rack level air flow mapping. 22 kw Server Simulator with adjustable fan speed and heater power settings to simulate a variety of air flow and heat loads. Server Simulator Perforated floor tiles with dampers

7 7 Particle Image Velocimetry Pulsed Laser t Cylindrical lens Measurement area and plane (0.60x1.9) m 2 Light Sheet Image map 1 pulse 1 Interrogation area Image map 2 pulse 2 Cross correlation function Vector map Data Analysis Velocity Stream lines Vorticity Turbulence CCD camera Flow seeded with particles x, y ( Particle Displacement) Noise filtering and random vector elimination x y u Velocity vectors Δx u Δt v Δy v Δt Vector Statistics Camera 2 (Image map B) Camera 1 (Image map A) Light Sheet Y X Perforated floor tile Rack (server simulator)

8 Height of the server simulator measured from the perforated floor tile (mm) 8 Tile/Rack Air Flow Studies using PIV Cold aisle Distance measured from the centre of cold aisle (mm) 5.5 kw/rack (625 CFM) Rack air inlet 625 CFM Velocity (m/s) Perforated tile 625 CFM Entrainment Cold aisle By-Pass Rack air inlet 2500 CFM 22 kw/rack (2500 CFM) Recirculation Perforated tile 2500 CFM [ref] Kumar, P., Joshi, Y., Experimental Investigations on the Effect of Perforated Tile Air Jet Velocity on Server Air Distribution in a High Density Data Center, Intersociety Conference on Thermal Phenomena (ITHERM), Jun 2-5, 2010, Las Vegas, USA.

9 Aisle center Rack Aisle center Rack Aisle center Rack Aisle center Rack 9 Pressure outlet Pressure inlet (K=10) Pressure inlet 22 kw/rack (2500 CFM) Aisle top Numerical Modeling Aisle top Aisle top Aisle top Grill Tile Gap Fan Mass flow inlet Pressure inlet Symmetry Body force 4.2million cells Velocity (m/s) Tile PIV (Experiments) Tile Geometrical Resolution Tile Modified Body Force (Developed) Modified body force with momentum source region of 22 width and 6 height suggests closer comparison with geometrical resolution and PIV results Tile Porous Jump (Generally Used) [mbf] Arghode, V. K., Kumar, P., Joshi, Y., Weiss, T., Meyer, G., Rack Level Modeling of Air Flow Through Perforated Tile in a Data Center, International Mechanical Engineering Congress and Exposition (IMECE), Nov 9-15, 2012, Houston, USA.

10 10 Grid Based Temperature Measurements T type thermocouples (wire gauge 28, mm) 252 thermocouples (width = 4ft, depth = 2ft, height = 6ft 6inch) 3D mapping of the cold aisle 6 planes along the height, 5 planes along the depth and 10 planes along the width

11 11 Rack Air Flow and Heat Load Measurement Air Inlet Air Outlet T in RACK Rack Load Tester T out Thermistor (T) Thermal Anemometer (V) Array (15 3) of temperature (thermistor) and velocity (thermal anemometer) sensors Mass flow rate calculated based on the measured velocity Rack heat load calculated based on mass weighted outlet temperature and inlet temperatures and the mass flow rate Air mass flow Heat load m= A V Measured Velocity Flow Area Air density Q=m C p (T out -T in ) Air specific heat Measured Temperatures

12 12 Containment System with Active Fan Tiles Top Curtain Fan Assisted Tiles

13 Outline Data center laboratory general information Current use of PI System Future use of PI system Copyri g h t 2012 OSIso f t, LLC. ii

14 14 Temperature Thermal diodes (CPU temperatures) Sensor Types Resistance temperature detectors, thermistors, and thermocouples (air temperature) Power Branch Circuit Monitoring System (at Power Distribution PDU level) Power Outlet Power Sensing (at Power Strips) Water Velocity/Flow : Dual turbine flow meter (Rear Door Heat Exchanger) Magnetic flow meter (Computer Room Air Conditioning) Air Velocity/Flow: Constant temperature hotwire anemometer (point velocity measurement) Particle image velocimetry (PIV) (velocity field) Flow hood (tile flow rate) Pressure Sensors Humidity Sensors

15 15 Interfaces MODBUS TCP/IP Communication protocol published by Modicon in 1979 for use with its programmable logic controllers (PLCs). Currently used as the interface with the building management system. SNMP (Simple Network Management Protocol) Internet-standard protocol for managing devices on IP networks. Used in facility to communicate with power strips and blade centers. IPMI (Intelligent Platform Management Interface) Used by system administrators to manage computers and gather data for monitoring purposes. Used in facility to communicate with server to collect server health data

16 16 Data Acquisition and Archiving Building Management User Machine 1 (e. g. CRAC) MODBUS Web Server Machine 2 (e. g. Power Strip) PI Server Software: Analyze and Visualize (e.g. Matlab, Labview) Machine 3 (e. g. Server) IPMI Headnode with Linux Platform

17 17 Rack Level Monitoring Example (SNMP)

18 18 Control System Design and Implementation Building management system (BMS) uses Niagra by Tridium to regulate cooling MATLAB script or code communicates with PI System using PI OLEDB, which sends commands through output tags to BMS. MATLAB built-in functions and tools used (e.g. SIMULINK) to design and implement controller. Benefits: A comprehensive library of tools in MATLAB for control system design No need for actual controller board Fail safe: no changes made to existing controllers in data center Suited for research purposes because of its flexibility and fast turnaround times PI Server MATLAB Building Management System

19 19 Controller Implementation Example Regulating maximum CPU temperatures for a rack by varying CRAC supply air temperature CPU Max Temp. Setpoint Idle to full load Integral action controller as an add-on to existing CRAC controls. Sampling period : 2 minutes Only one rack turned active in experimental zone (preliminary testing) Test case: Step increase in compute load [ref] Sundaralingam, V., Joshi, Y, Controller to regulate maximum server CPU temperatures in a rack by varying CRAC supply air temperatures, International Mechanical Engineering Congress and Exposition (IMECE), Nov 9-15, 2012, Houston, USA.

20 Outline Data center laboratory general information Current use of PI system Future use of PI system Copyri g h t 2012 OSIso f t, LLC. iii

21 21 Future Work Webserver Development Integrate Sharepoint Server with the PI System tools: PI WebParts, PI ProcessBook and etc. Webserver function Facilitate data extraction Visualize data Disseminate live and archived data to the public (login credentials will be provided for approved requests Tentative beta site availability (April 2013) Development of logic/physics based controllers for air cooled data center Contained aisles Active fan tiles Advanced air delivery systems

22 Brought to you by Copyri g h t 2012 OSIso f t, LLC.