Data Center Retrofit Project: 90% Energy Use Reduction on a Budget

Transcription

1 Rocky Mountain ASHRAE 2013 Data Center Retrofit Project: 90% Energy Use Reduction on a Budget David Gallaher, National Snow and Ice Data Center (NSIDC) Otto Van Geet, National Renewable Energy Laboratory (NREL) Rick Osbaugh, RMH Group Presented by Design by Affiliations and Sponsorships

2 So why are we here Data centers can waste 90 percent or more of the electricity they pull off the grid. Worldwide, there are now more than 3,000,000 data centers that use about 30 billion watts of electricity; 30 nuclear power plants. They don t get a bonus for saving on the electric bill. They get a bonus for having the data center available percent of the time. It s just not sustainable.they re going to hit a brick wall. Mark Bramfitt, utility and IT consultant Source: New York Times 9/22/12

3 The National Snow and Ice Data Center Manages and distributes scientific data Provides tools for data access Researches the cryosphere and data science Supports local and traditional knowledge Supports data users Educates the public about the cryosphere

4 So what is this about? NSIDC is a center under the CIRES Institute at the University of Colorado, Boulder The NSIDC is a NASA-essential services facility and a NSF Arctic archive The NSIDC takes polar orbiting satellite data and processes, archives and distributes it worldwide The center contains petabyte-scale disk arrays The center uses a virtualized server environment and bare metal servers, totaling ~100 servers The 1600 sq ft data center consumed ½ the power of a 77,000 sq ft research lab (as of 2009)

5 Agenda Project Impetus Cooling Phase Design Specifics Construction Solar Phase Construction Results Summary Photo Courtesy Andy Mahoney, NSIDC

6 Objectives How retrofits can produce performance on par with new construction How inexpensive solutions can deliver low energy use Importance of clear goal setting by project management Benefits of open, collaborative design environment Photo Courtesy Robert Massom, NSIDC Photo Courtesy Ted Scambos and Rob Bauer, NSIDC

7 PROJECT IMPETUS

8 Original Plan: CRAC UP at NSIDC 2008 original project was to replace 20-ton CRAC ( the rusting hulk ) with 30 ton unit Reliability issues (cottonwood, over pressure) Cost estimate of $160,000 installed Crane and elevator follies

9 NSIDC Research Lab #2 at UC NSIDC 1600 sq ft datacenter consumed ½ of the total power for 77,000 sq ft building.

10 Could We Do Better? Irony of NSIDC research contributing to the problem we are studying. The data center used 96kW (51kW load for data center equipment, 45kW for cooling). Are there other options? Rather than cool the air in a closed system, couldn t we just dump it outside? Assumed initial budget of $200,000 Could it be built with off-the-shelf components? What was available and economically feasible?

11 Air Economizer Hours Could we use an air economizer? What about contamination? Source:

12 NSF and NASA Funded Project Installation of a replacement cooling system featuring air economizer and indirect evaporative cooling systems Consolidation of IT facilities in a single-server room Virtualization of servers and arrangement of systems in a hot/cold-aisle rack configuration Solar panels to generate electrical power A solar battery array, connected to the solar system, to substitute for the lack of a backup generator Upgrading internet connectivity to 1gb to better support the needs of NASA data archive users The systems will reduce the computer center s carbon footprint and support NSF s cyberinfrastructure goals

13 NSF/NASA Green Data Center Proposal: NSIDC Green Data Center Old Configuration New Configuration

14 Our Solution was Not Conventional In science, your client will not fund you if your proposal does not push the edge In engineering, your client will not accept your proposal if it has not been done somewhere else We were well aware that this was a new idea We needed to educate the mechanical engineers on our tolerance to risk (we had a backup 30-ton CRAC unit) We needed to educate the mechanical engineers what was possible with this new cooling technology

15 COOLING PHASE

16 Engineer s Original $375,000 Proposal Not off-the-shelf Maintenance and parts issues Roof unit 12 high and 30 long Requires visual screen and railing Subject to freezing Would require the CRAC 1000 hrs/year Way over budget

17 Red Rocks Data Center: a $27,000 Solution Housed in old satellite tracking facility School of Mines (initial design) Used unfiltered outside air Backed up by conventional A/C Not suitable for NSIDC, but interesting ideas

18 Perseverance for Performance: Investigate CRAC replacement options NSIDC desire for off-the-shelf system A Three-Year Odyssey NSF solicitation for facility upgrades NASA agrees to $200,000 budget NSIDC wins internal competition to submit to NSF $375,000 design proposed Tour $27,000 solution NREL consults with NSIDC and RMH OCT 2008 JAN 2009 MAR 2009 MAY JUN AUG 2009 SEP 2009 JAN 2010

19 Search for Out-of-the-Box Design We needed someone to work within our specs and budget. We needed a miracle!

20 RMH Conceptual Design

21 Perseverance for Performance: Investigate CRAC replacement options NSIDC desire for off-the-shelf system A Three Year Odyssey NSF solicitation for facility upgrades NASA agrees to $200,000 budget NSIDC wins internal competition to submit to NSF $375,000 design proposed Tour $27,000 solution NREL consults with NSIDC and RMH OCT 2008 JAN 2009 MAR 2009 MAY JUN AUG 2009 SEP 2009 JAN 2010 RMH proposes design with 95% reduction in cooling energy FEB 2010

22 COOLING DESIGN SPECIFICS

23

24 Cooling System Design Considerations Consolidated, reconfigured data center Favorable low-humidity climate Improvements in the design of alternative cooling systems New 2008 ASHRAE standards IT equipment that can tolerate higher temperatures Third floor (directly below the roof) Eliminate compressors, if possible Exceed reliability standards Move cooling units inside Decouple AHU and cooling system

25 2008 ASRAE Revised Standard

26 Psychrometric Bin Analysis Zone1: DX Cooling Only Zone2: Coolerado + DX (H80) Zone3: Coolerado Only Zone4: Evap. Cooler Only Zone5: Evap. Cooler + Economizer Zone6: Economizer Only Zone7: 100% Outside Air

27 Psychrometric Bin Analysis: Boulder, CO Design Conditions (0.4%): 91.2 db, 60.6 wb

28 How the Indirect Evaporative Units Work 1 Fresh air drawn in 2 Air is filtered 3 Heat and mass exchanger 4 Half of air saturated with water and returned to atmosphere carrying heat energy 5 Conditioned air to facility

29 Technology Description Innovative indirect evaporative cooling 22-stage cooling with wet-bulb effectiveness from 90% to 125% No moisture added to the product airstream Fan is the only major Temperature o C y u b e pe atu es (deg C) d ect S de Dry Bulb Temperatures (deg C) - Direct Side Multiple passes of indirect 200 energy input Y (mm) Temperature o C Y (mm)

30 System Advantages Cost-effective components Single air handling unit is simple, inexpensive Fan, filters and dampers sized for low pressure drop, low fan energy use The units are expandable and modular Precise control of temperature and humidity No compressors used in design = greater reliability Energy savings guaranteed; no equipment to run except fans; 24/7 operation; lower maintenance costs Connected HP for system is so low that it can be run from UPS (fan is always run from UPS). Low energy use > 90% less than existing system

31 Old System: Summer Operation Four ¾ HP Condenser Fans Refrigeration Pipe Two 25 HP Compressors Total system = 45 kw

32 Old System: Winter Operation Four ¾ HP Condenser Fans Refrigeration Pipe Two 25 HP Compressors Total system = 46 kw

33 New System: Summer Operation 1 Economizer Air Supply 2 Product Supply Air 3 Aisle Supply Air 4 Hot Aisle Return Air 5 Relief Air 6 Return Damper Air 7 Outside Air 8 Working Air Exhaust Total system = 6 kw

34 Cooling Units Product Air Side with Working Air Humidification Dampers

35 Making Use of a Waste Product Outside air humidity from the air-economizer can drop below 10% The units produce cool moist air as a waste product Installing dampers allows the units to function as humidifiers During winter months, the units function only as humidifiers During summer months, they both chill & humidify

36 January September 2011 COOLING SYSTEM CONSTRUCTION

37 Machine Room Before Construction

38 4 Duct Holes

39 Air Handler Unit Lifted to Roof

40 8 am, Wed., March 30 th hours

41 Six Trades at Once: Mechanical, Electrical, Plumbing, Sprinkler, Sheet Metal, Controls

42 Ready for re-start, 1 am Saturday morning

43 Back Online Saturday April 2, 3 pm

44 Reconfigured Data Center Servers in Hot Aisles

45 Installed Cooling Units with Input Air Duct

46 Perseverance for Performance: Investigate CRAC replacement options NSIDC desire for off-the-shelf system A Three-Year Odyssey NSF solicitation for facility upgrades NASA agrees to $200,000 budget NSIDC wins internal competition to submit to NSF $375,000 design proposed Tour $27,000 solution NREL consults with NSIDC and RMH OCT 2008 JAN 2009 MAR 2009 MAY JUN AUG 2009 SEP 2009 JAN 2010 RMH proposes design with 95% reduction in cooling energy NSF awards NSIDC facility upgrade grant $525,000 University issues overcome, construction RFP released, $225,000 bids NSF releases funds! Cooling system installation Hot aisles and systems shakedown Cooling phase completed FEB 2010 JUN 2010 SEP 2010 JAN 2011 APR 2011 MAY 2011 SEP 2011

47 February May 2012 SOLAR SYSTEM CONSTRUCTION

48 Solar System Design Considerations Cd/Te cadmium/telluride solar PV panels 50 kw output (78,000kWh/year) watt panels ($83/panel) Flat-mount panels for higher density High wind load (rated to 120mph) No penetrations of roof (concrete block ballast) Acts as backup generator (powers SAN and cooling system) Automatic solar failover to battery array 3 hours of cooling at maximum power Data center is carbon-zero in daylight

49 Construction begins with racking system and lift of solar panels

50 Removed the old 20/10-ton CRAC unit condensers

51 50 kw of Solar Panels are Next 720 Abound Solar 70W Panels 5 miles+ of wire

52 RESULTS

53 Perseverance Pays Off! 91% reduction in cooling energy consumption New hot-aisle isolation configuration Solar array as backup generator 70% reduction in total power consumption CRAC used 9 hours as dehumidifier per year! Increased data capacity Only maintenance cost is filters (90% reduction)

54 30000 New Cooling System Energy Use with Projection Construction + Transition Period Energy Use (kwh) ,500 kwh Average CRAC Energy Actual New Cooling System Energy Use Projected New System Energy Use (10 C cutoff) Old CRAC Cooling System Energy Use Jun-10 Jul-10 Aug-10 Sep-10 Oct-10 Nov-10 Dec-10 Jan-11 Feb-11 Mar-11 Apr-11 May-11 Jun-11 Jul-11 Aug-11 Sep-11 Oct-11 Nov-11 Dec-11 Jan-12 Feb-12 Mar-12 Apr-12 May-12 Jun-12 Jul-12 IT Energy 2,560 kwh Average Coolerado Energy

55

56 Solar Output This system should generate about 78 megawatt-hours of energy/year

57 Even on this snowy April 2, we generated enough power to run the cooling system (~1kw) at midday

58 Energy and Dollar Savings Original Conditions NSIDC Data Center, through : Current Conditions NSIDC Data Center, August 2011 to present: 60-Tons Liebert DX cooling (30,20,10 ton) Average Cooling Power - 45kW Average UPS conversion 6kW (87% efficient) Average IT Load - 45kW Average annual power consumption: 96kW*24*365 = 841,000 kwh/yr Average power cost at $0.11/kWh = $92,500/year 40 tons Coolerado Cooling (8 5-ton) Average Cooling Power - 4kW Average UPS conversion 4kW (87% efficient) Average IT Load - 32kW (Virtual servers) Average annual power consumption: 40kW*24*365 = 350,000 kwh/yr Average power cost at $0.11/kWh = $38,500/year Average annual savings = 58% or $54,000 50kW Solar Array savings = $8580/year Total Annual Savings = $62,580 Total Energy Reduction = 70%

59 NSIDC NASA NSF RMH Group NREL Coolerado UCB FacMan CU Dept of Engineering CIRES Project Participants University of Colorado Xcel Energy Raytheon Abound Solar SMA Whitestone Construction Lighthouse Solar All the critical components were built in locally

60 SUMMARY

61 Striving for Innovation: Keys to Success Calculated Risk Taking. Strong desire by NSIDC to consider new ideas and approaches. Localized Think Tank. Generating ideas with NREL, NSIDC, RMH and the University Forming the Right Team. Pushing, encouraging and supporting the design effort; true collaboration Getting Support. Soliciting funding/support from the NASA and NSF Accountability. Test and measure performance. Knowledge Sharing. Publicize success.

62 A few thoughts 1. Don't kill a good idea just because there are those who feel (strongly at times) that it is impossible. The fact that it has never been done is generally a virtue. 2. Build/show your vision by way of a tangible and easy-to-comprehend graphic. A picture really is worth a thousand words. 3. Bring as many collaborators in as early as possible (government, universities, associations, manufactures, sponsors) to spread the risk. 4. Failed projects die as orphans, successful projects have multiple fathers. Accept this. 5. Engineering is inherently conservative and moves in small evolutionary steps. The best engineers will break out of the box; don t settle for less. 6. Be persistent and don't settle for lame answers. Be prepared to fire those who do not deliver. 7. Manage risk. Do not attempt to eliminate it. The project that avoids all risks will accomplish little and cost too much.

63 A few thoughts 8. Have the full team understand the importance/significance of the project. This will give them buy-in. 9. Have your contractors sign their work where everyone can see it (we had them sign on the duct work). It improves quality of workmanship. 10. The Project Manager must be the project champion (push through the difficulties). Do whatever it takes; turn the screw, mop the floors if you must. 11. Every project will have at least 3 crises, which could derail the project. Expect them and do not delay in resolving them. 12. Scope creep: "As long as we are here, we might as well..." will kill any project. Stay focused. 13. If successful, seek out awards and be prepared to show others what can be done. You cannot change the world if no one knows what you did. 14. Celebrate a successful completion with all involved parties. Credit your sponsors, subordinates and superiors.

64 And the best reward of all daylight zero-carbon footprint

65 Conclusion This is a prototype project of what can be done on a small-scale operating data center Cooling energy was reduced 91% (about what is takes to run your car s A/C) Carbon and overall energy use were reduced by 70% Payback on the cooling system is under 3 years (difference of CRAC replacement vs. Coolerado solution and energy savings) Project won the Governor s Sustainability Award, CU Sustainability Award, CIRES Service Innovation Award and the Uptime Institute Green Energy IT Retrofit Award

66 Courtesy Ted Scambos, NSIDC Courtesy Ted Scambos and Rob Bauer, NSIDC Courtesy Andy Mahoney QUESTIONS? CONTACT: David Gallaher: