Dealing with Thermal Issues in Data Center Universal Aisle Containment Daniele Tordin BICSI RCDD Technical System Engineer - Panduit Europe Daniele.Tordin@Panduit.com
AGENDA Business Drivers Challenges Data Center Containment Universal Aisle Contaiment Case Study
Business Drivers in Data Center End User Demands Increased data requirements Applications / Storage Security / Disaster Recovery / Redundancy Regulation / Consolidation Higher Data Rates Efficiency/Cost improvements Smaller Footprint / Higher Utilization Agile Infrastructure Faster Deployments Optimized Solutions Op-Ex and Cap-Ex savings Innovative Partners
Challenges.+ Where did my availability, capacity, efficiency go? Server sprawl Lowering supply air setpoints to address hot-spots Inefficient physical infrastructure Overprovisioned Power, Cooling and Space Higher Costs (Energy & OpEx) Paying for space regardless of utilization Higher PUE Need More Capacity Additional IT CapEx Reduced IT ROI Less Agility / Availability
Challenges. TCO Ever Changing Why Multivendor design approach the need for cross party talks key Capacity utilization would like v actual Variable IT ever changing Siloed operations Visibility and lack of actual data capture and tracking
Capacity utilization would like v actual Do you know anyone who can play the perfect game of TETRIS? NEVER happens
Challenges - Energy Regulation European Code of Conduct for Energy Efficiency in Data Centre IT Hardware expected operating range of 10 to 35⁰C 20 to 80% humidity. Data Centres temperature intake air to IT of 10 to 35⁰C (ASHRAE Class A2) Containment Design Hot/Cold (over 1KW per ) Installation of blanking plates Installation of systems to cover ALL air leakage Equipment segregation Cool Boot
Data center layouts. In reality, each data center layouts vary from row to row and often have empty spaces where racks have been removed, varied heights of cabinets etc.
Containment A Strategy to Reclaim Capacity in Legacy Data Centers How much capacity can be reclaimed? What energy savings can be realized? How much can I increase rack density?
Panduit Energy Efficient DC Containment Allows higher data center set points and reduces cooling system energy consumption by up to 40% Seal every gap for complete separation of cooling and exhaust air Direct cold air to where it needs to go Contain cooling or exhaust air for maximum cooling capacity efficiency Monitor to optimize operational and energy efficiency Controlling Small Leaks and Maintaining Hot/Cold Air Separation Leads To Large Cooling Energy Savings 40% Refer to Panduit Whitepaper WP-20 Impact of Air Containment Systems
Thermal Issues Quick Remedies for Immediate Impact 11
Hot air recirculation through the cabinet 12
Recirculation Remedy prevent hot air leakage into the cold aisle to improve thermal efficiency Full Length Blanking Shade Tool-less blanking panels (also with pass-trough openings) 13
Cable Cut-Outs 14
Cool-Boot 15
Cool-Boot Return Of Investment Eliminates leakage around cable bundles or power cords passing thru raised floors 16
Bad Cable Management 17
Smart Cable Management Minimize Airflow Obstruction Minimize Cable Overlenghts Minimize Risk of Damages -> improved Availability Optimize Accessibility -> improved Maintenance Optimize Visibility -> improved MAC 18
Start thinking early Choosing a Cabinet for Optimal Thermals 19
Cabinets Cable Management Wider cabinets / Vertical patch panels Hot air re-circulation Vertical blanking panels CFD and Thermal Lab Testing In-Situ testing with new technologies Available accessories Switch Ducting Think about the future Maximize your investment! Add in containment when heat loads increase Depth is important 20
Direct cold air to where it needs to go Switches frequently have air inlets and exhausts on the side of the chassis Exhaust air fills the cabinet and is drawn back to the inlet - dramatically increasing switch temperatures Ducting separates the intake and exhaust airflow Engineered solutions designed for minimal airflow impedance Airflow separation also critical to enabling containment solutions Nexus 7018 Switch without Duct Nexus 7018 Switch with Duct
Thermal Management Two 6509 Switches at 6kW CFD analysis software image of front of cabinets: Reduced switch temp in NET-ACCESS cabinet with ducting Exhaust air prevented from re-circulating within cabinet 22 Cabinet with Duct Cabinet without Duct
Complete Ducting Solutions for High Performance Switches Inlet & exhaust ducting Retrofit or fix installed at cabinets 23
Air Flow Optimization Vertical Exhaust Ducting 24
Vertical Exhaust Ducting Vertical Exhaust Duct (VED) Cabinets (aka Chimney cabinets) are used to provide hot exhaust containment in data centers. Solid rear doors capture hot air and direct it through the VED. Heated air exhausts upward and away from the cabinet inlet, preventing recirculation. 25
Function of a VED system Passive system, no additional moving parts Directly vents up to 30kW+ of the exhaust from servers into return plenum of data center Increases cooling efficiency and significantly lowers operating expenses Above Ceiling Hot Air Return Plenum Cooling air directed to entire room Vertical Exhaust Duct Vertical Exhaust Duct Cooling air directed to entire room Slab Floor or Raised Floor Plenum 26
CFD Analysis of a VED Without VES Above Ceiling Hot Air Return Plenum Cold Aisle Hot Aisle Cold Aisle Hot Aisle Cold Aisle Under Floor Plenum With VES Above Ceiling Hot Air Return Plenum Cold Aisle Hot Aisle Cold Aisle Hot Aisle Cold Aisle 27 Under Floor Plenum
Air Flow Optimization Hot / Cold Aisle Containment 28
Panduit Net-Contain Aisle Containment Systems Hot Aisle Containment Cold Aisle Containment 29
Function of a Cold Aisle Containment System Prevents mixing of cold and hot air streams Eliminates recirculation of hot air to inlets Provides uniform temperature at the inlets of IT equipment Improves Cooling System Thermal Performance Allows raising the supply air set point temperature Reduces cooling energy costs up to 42% 30
CFD Analysis of a Cold Aisle Containment Without Containment Cold Aisle Hot Aisle Cold Aisle Hot Aisle Cold Aisle Under Floor Plenum With Containment Cold Aisle Hot Aisle Cold Aisle Hot Aisle Cold Aisle Under Floor Plenum 31
Choosing a Passive System Standard Hot / Cold Aisle Cold Aisle Containment Vertical Exhaust System CapEx: $ CapEx: $$ CapEx: $$ OpEx: $$$ OpEx: $ OpEx: $ No Height Restrictions Easy to add cabinets and scale data center Accommodates mix of equipment Use for 0-10kW Low density Open environment, warm DC Works in height restricted data centers Some downtime when adding cabinets Accommodates mix of equipment Use for 0-20+kW High density Contained Cold environment, hot DC Requires significant height above cabinet Easy to add cabinets and scale data center Optimal with uniform equipment Use for 0-30+kW High density Hot Air Containment, cold DC 32
The solution. Universal Aisle Containment System!
Universal Aisle Containment System! Retrofitting the Universal Aisle Containment System into your existing data center allows you to reclaim as much as 28% of your existing data center capacity. Containment system, reduces cooling energy consumption up to 40%. The flexibility to build a containment system and then add cabinets of varying sizes and design as your needs dictate reducing deployment time and capital investment.
Features & Attributes Floor Mounted, Free Structure Height Adjustable Frame Hot Aisle or Cold Aisle Compatible with Industry Leading Brand of Cabinets Full or Partial Pod Raised or Slab Floor Expandable Containment Length Tool-less Full or Top of Cabinet Blanking Panels Flexible Design
Application flexibility Partial Populated Hot Aisle Containment Day 1 Containment ready Partial Populated Cold Aisle Containment
Application Flexibility..2
Containment Impact Hot Air Recirculation Hot Spots in Cold Aisle Contain Cold Air Assess Current Condition Model End State Condition
Panduit Data Center Thermal Lab 150m 2 of raised floor space 200 ton chilled water plant 90 ton AHU (expandable to 2) 320 KW IT power simulation Cold air delivery via floor, overhead, or wall Fully instrumented
Reclaiming Lost Capacity Optimize utilization of rack space Increase power load / rack Group higher density applications in High Density Zones Maximize use of available cooling capacity Ensure cool air goes to where it is needed Optimize separation of hot and cool air Increase supply air temperature Manage power, cooling, and space capacity utilization Real time monitoring of power & environmental conditions Monitor PUE
Using PUE to Help Measure Efficiency Total Facility Power The PUE (Power Usage Effectiveness) is a simple ratio for assisting in measuring datacenter efficiency. It compares total electrical power at the intake to power delivered to IT equipment. IT Equipment Power PUE is a snapshot in time and is entirely influenced by a site s M&E configuration, environmental conditions and Tier rating, PUE cannot be readily used to compare one site with another but is used to track the efficacy
Case Study
Case Study Brown field site 2 Data Halls 28 CRACS per data hall Data Hall 1 393 Cabs Split into 3 DC Halls Data Hall 2 396 Cabs Split into 3 DC Halls
Case Study Continued CRAC units Dual Chilled Water coil with down flow units Units running at 88% Temp inlet at server 19C PUE 2.3 2009
A Balanced floor is Key to Operational Efficiency Visibility to the environmental conditions of the room enable sustained operation of an efficiently balanced raised floor Floor Pressure (target 0.03-0.04 ) BEFORE Unbalanced Floor Uneven Pressure - Fan Speeds @ 88% 1 Hour of Optimization of floor balancing for airflow 2 Hours Optimization of floor balancing for airflow 4 Hours Optimization of floor balancing for airflow Ramping Down Of 28 CRAH fans to 60% After Ramping Down Of 28 CRAH fans to 40%
Case Study: Cold Aisle Subfloor Pressure Floor Pressure (target.03 -.04 ) Floor Re- Balancing Pre Optimization Conditions Reducing Fan Speeds After Reducing Fan Speeds Re-balancing the raised floor using the (1) Rack Temperature and (2) Floor Pressure Sensors Increased floor pressure 2-3X Enabling Fan Speed Reduction on CRAHs from 88% to 60%
Cabinet Top Temps vs. Used Power ASHRAE recommendations.(18-27 C or 64.4 deg F through 80.6 deg F)
% diff between Top & Bottom Cabinet Temp Area of interest room for improvement
Next Steps Obtained using measurements a temp base line across the facility Migrated equipment to ensure an even distribution across the floor. Changed temperature settings to 20 and allowed to stabilise. Took a new baseline Changed temp settings to 22 allowed to settle Changed temp settings to 24 and finalised
Next Steps Looked to floor tile placement Blanking panels in cabs Floor pressure set to 3-4 Pascal's Retrofitted Cold Isle containment Savings of??$ million within the total DC since 2009 Annualised PUE of 1.45
Cost Model Universal Aisle Containment Hall 1, tot cost: 170.000 Universal Aisle Containment Hall 2, tot cost: 200.000 Monitoring Hardware Cost Hall 1 + 2: 380.000 Monitoring Software Cost: 63.000 4 Milion Savings in 5 years
Thank you