NetApp Data Center Design Presentation for SVLG Data Center Efficiency Summit November 5, 2014 Mark Skiff, PE, CEM, CFM Director, IT Data Center Solutions
Data Center Design Evolution Capacity Build Cost Configuration Density kw/rack mw $/kw year power & cooling >1 PUE Generation 1 Multiple Smaller 1-3 n/a Generation 2 3 1.8 Generation 3 25 2 / 8 Generation 4 25 (scalable) 2.6 Hot / Cold Aisle Prior to 2006 <4 1.65 Cold Aisle Containment 2006 Gen 2 w/ Economizer 2007 (Delivered 2009) Gen 3 Plus Refinement 2014 8 1.55 12 1.16 4 to 12 1.12 * Estimated annual 2
Data Center Design Evolution Capacity Build Cost Configuration Density kw/rack mw $/kw year power & cooling >1 PUE Generation 1 Multiple Smaller 1-3 n/a Generation 2 3 1.8 Generation 3 25 2 / 8 Generation 4 25 (scalable) 2.6 Hot / Cold Aisle Prior to 2006 <4 1.65 Cold Aisle Containment 2006 Gen 2 w/ Economizer 2007 (Delivered 2009) Gen 3 Plus Refinement 2014 8 1.55 12 1.16 4 to 12 1.12 * Estimated annual 3
Measuring Performance with PUE 2.0 1.5 1.0 0.5 Chiller Plant AHU UPS Humidity Control Transformer Lights IT Power Usage Effectiveness (PUE) ratio cited as the data center infrastructure efficiency metric Total power / IT equipment power Infrastructure systems account for half of total energy in data centers Typical data center PUE = 2.0 Lower the number the more efficient 0.0 Typical 2.0 PUE 4
Opportunity + Innovation 5
Typical Hot Aisle / Cold Aisle Configuration Supply Air Supply Air 6
Hot/Cold Aisle w/ VAV at 4 kw/rack (150 watts/square Foot) Control Damper Temp. Sensor C R A H R A C K Return Air 7
Contained Cold Aisle w/ Pressure Controlled Cooling C R A H Control Damper R A C K Press. Sensor Operational January 2007 360 racks @ 8kW / Rack (high density; 770 watts/sf) CRAHs Supply Constant Temp Neutral Cold Room Pressure Modulating Dampers Eliminates Air Mixing Uniform temperatures Infrared shows warm areas blanking gaps Return Air 8
Cold Room Cold Room Cold Room Cold Room Cold Room Cold Room Cold Room Cold Room Cold Room Cold Room Cold Room Cold Room Plan View & Cross Section AHU AHU AHU AHU AHU AHU AHU AHU AHU AHU AHU AHU Cold Room Hot Aisle 9
End of Aisle Air Handling and Center Aisle Air Handlers Main Aisle 10
Contained Cold Aisle (Cold Room) and Hot Aisle Cold Room Hot Aisle 11
NetApp Global Dynamic Lab (GDL) & IT Site 2 Opened March, 2009 2,166 racks, 12 kw per rack average load Predicted PUE = 1.2 (annualized) FIRST EVER ENERGY STAR RATED 12
Plan View 36 Cold Rooms x 60 cabinets Corporate IT 25 mw = 770 watts/sf R&D 13
GDL1 Outside Relief Dampers Outside Air Hot Air Return Mixing Dampers Cooling Coil Speed Control CRAH Fan 360 IT racks 6kW/rack Dual path UPS & generators 1800 Engineering racks 12 kw/rack Utility power only 30 Engineering Network racks 3 kw/rack Single path UPS & generator R A C K Pressure Sensor Cold Room Cold Room with Pressure Controlled Cooling Airside economizer Elevated Air and CHW Temps 14
Main Hall; 1 st Floor 15
Cold Room; 1 st Floor 16
2.0 1.5 1.0 0.5 0.0 2.0 Measuring Efficiency with PUE Typical 2.0 PUE Chiller Plant AHU UPS Humidity Control Transformer Lights IT Power Usage Effectiveness (PUE) ratio cited as the data center infrastructure efficiency metric Total power / IT equipment power Infrastructure systems account for half of total energy in data centers Typical data center PUE = 2.0 Lower the number the more efficient NetApp RTP Building 4, predicted PUE = 1.2 1.5 1.0 0.5 Chiller Plant AHU UPS Humidity Control Transformer Lights IT State-of-the-Art Data Center 2009 2010 2011 EPA assumed PUE ratio 1.67 1.56 1.45 0.0 1.2 PUE (Annualized) 17
Full FREE Cooling PUE = 1.07 Outside Air Temp = 73 o F (23 o C) Mixing Dampers Cooling Coil Speed Control Outside Relief Dampers 90% 90% CRAH 90% 10% 10% Fan Hot Air Return R A C K Return Air 91 o F (33 o C) R A C K Pressure Sensor Cold Room Cold Room 75 o F (24 o C) 18
Full PAY Cooling PUE = 1.30 Outside Air Temp > 91 O F (33 o C) Outside Relief Dampers 90% 100% 10% 90% Mixing Dampers Cooling Coil Speed Control CRAH Fan Hot Air Return Return Air 91 o (33 O C) R A C K Pressure Sensor Cold Room Cold Room 75 o F (24 o C) 19
PARTIAL Free Cooling PUE = 1.07 1.30 Outside Air Temp >75 o F (24 o C) to <91 o F (33 o C) Outside Relief Dampers 90% 100% 90% 100% 10% Mixing Dampers Cooling Coil Speed Control CRAH Fan Hot Air Return Return Air Pressure Sensor 91 o (33 O C) Cold Room 75 o F (24 o C) R A C K Cold Room 20 2014 NetApp, Inc. All rights reserved. NetApp Confidential Limited Use 20
Raleigh Weather Data TEMP (F) #HRS/YR % of YR <75 o (24 o C) 5481 62% 75-80 o 2043 23% 80-85 o 585 7% 85-90 o 426 5% >90 o (32 o C) 225 3% FREE PAY 21
Actual Performance: 2013 Free Cooling 22 2014 NetApp, Inc. All rights reserved. NetApp Proprietary Limited Use Only
Actual Performance: 4Year Average Free Cooling 23 2014 NetApp, Inc. All rights reserved. NetApp Proprietary Limited Use Only
Actual Performance: 4Year Monthly Free Cooling 24 2014 NetApp, Inc. All rights reserved. NetApp Proprietary Limited Use Only
Actual Performance: PUE Current PUE = 1.16 25 2014 NetApp, Inc. All rights reserved. NetApp Proprietary Limited Use Only
Lessons Learned and a Few Ah-Ha s PUE Range = 1.07 1.30 (average annual 1.16) 4 kw / rack average after 5 years Construction Costs 20-33% of Industry TCOE 1/3 Site 1, 1/5 HQ, 1/7 Colo 60 Degree Chilled Water Temp Rise 16 degrees 26 2014 NetApp, Inc. All rights reserved. NetApp Proprietary Limited Use Only
Global Dynamic Lab 2 Project Goals Bigger, Better, Cheaper Project Completion; May, 2014 Predicted PUE 1.12 27
Data Center Space 28
Center Aisle GDL2 29
Mechanical System Improvements Increased Air Handling Capacity Incorporated Washable Outside Air Pre-filters Improved Air Distribution Eliminated Cooling Systems for Electrical Distribution Rooms Eliminated Evaporative Humidification Systems Selected More Efficient Fan Wheel Changed to Operable Window System Vs. Louver/Damper System 30
Air Handling Capacity 31
Washable Pre-Filters 32
Actual Performance: 2013 Free Cooling 33 2014 NetApp, Inc. All rights reserved. NetApp Proprietary Limited Use Only
Improved Air Distribution 34
Air Flow Through End of Hot Aisles 35
Eliminated Electrical Room Air Conditioning 36
Operable Window System for Exhaust 37
Exterior View (open) 38
Interior View (closed) 39 3
Interior View (open) 40
Summary and Recommendations Containment w/ Pressure Control Allows for High Density (and low construction cost) Air Side Free Cooling Required for Low PUE Sustainable Data Centers Low Energy Use Recycled Materials Non Potable for Cooling Tower Make-up Low Construction Costs Supplier (A/E, GC) Selection is Critical Success Factor Phased Construction Provides Flexibilty (first cost and purchasing) 41
Thank you 42 2014 NetApp, Inc. All rights reserved. NetApp Confidential Limited Use