Research Topic Acceptance Request (1675-RTAR), Experimental Benchmark Data for Data Center Numerical Modeling

Transcription

1 [Type text] 1791 Tullie Circle NE Atlanta, Georgia Tel Fax Michael R. Vaughn, P.E. Manager Research & Technical Services TO: FROM: James VanGilder, Chair TC 4.10, Jelena Srebric, Research Subcommittee Chair TC 4.10, Srinivas Garimella, Research Liaison Section 4.0, Michael Vaughn, MORTS, DATE: July 20, 2012 SUBJECT: Research Topic Acceptance Request (1675-RTAR), Experimental Benchmark Data for Data Center Numerical Modeling During their spring teleconference, the Research Administration Committee (RAC) reviewed the subject Research Topic Acceptance Request (RTAR) and voted to return it. The following list summarizes the mandatory comments and questions that need to be fully addressed in the RTAR re-submission: 1. Work with TC 9.9 (Mission Critical Facilities, Technology Spaces and Electronic Equipment) to develop this topic. TC 9.9 may also able to assist in recruiting industry contributions for the project (in-kind services and equipment donations, co-funding, etc.) 2. RAC Research Liaison, Srinivas Garimella, should be invited to review and comment on the revised RTAR and/or draft work statement prior to formal submission to MORTS for RAC review. 3. The benchmark data being sought needs to be better defined/explained in the RTAR. Please address or incorporate the above information into the RTAR with the help of your Research Liaison prior to resubmitting it to the Manager of Research and Technical Services for further consideration by RAC. In addition, a separate document providing a point by point response to each of these mandatory comments and questions must be submitted with the RTAR. The response to each item should explain how the RTAR has been revised to address the comment, or a justification for why the technical committee feels a revision is unnecessary or inappropriate. The RTAR and response to these comments and questions must be approved by the Research Liaison prior to submitting it to RAC. An RTAR evaluation sheet is attached as additional information and it provides a breakdown of comments and questions from individual RAC members based on specific review criteria. This should give you an idea of how your RTAR is being interpreted and understood by others. You also have the option of skipping the RTAR stage at this point and proceeding to the work statement (WS) stage since most of the comments from RAC on the RTAR relate to the need for more information and details. The risk you take with this approach is investing a lot of time in a document that RAC may not accept. The next submission deadline for RTARs and WSs is August 15, 2012 for consideration at the Society s 2012 fall meeting. The submission deadline after that is December 15, 2012.

2 Project ID 1675 Project Title Sponsoring TC Cost / Duration Submission History Classification: Research or Technology Transfer Experimental Benchmark Data for Data Center Numerical Modeling TC 4.10, Indoor Environmental Modeling $125,000/18M 1st submission Basic/Applied Research 2012 Annual Meeting Review RTAR SUMMARY VOTES & COMMENTS - Version 1 Check List Criteria VOTED NO Comments & Suggestions Is there a well-established need? The RTAR should include some level of literature review that documents the importance/magnitude of a problem. If not, then the RTAR should be returned for revision. #2, #7 #2 - Believe that the proposal is too simplified. CFD is being used in the data center market but has been used in more complex arenas for a long time (semiconductor clean rooms, etc.). #10 - Multiple TC's would benefit from the output. #7 - Hasn't it been established that CFD software such as FLUENT has the capability to capture the fundamental airflow physics. If you eliminate real-world data-center details, then what makes this modeling challenge any different from some other CFD problem? Why not use a model from an accepted CFD simulation tool as the benchmark if the primary goal is to use the data to validate simplified numerical models? #8 - Simulations of data centers could benefit from such benchmark data Is this appropriate for ASHRAE funding? If not, then the RTAR should be rejected. Examples of projects that are not appropriate for ASHRAE funding would include: 1) research that is more appropriately performed by industry, 2) topics outside the scope of ASHRAE activities. #2 #2 - project will not add value to the industry or marketplace. #3 - sponsor support? Is there an adequate description of the approach in order for RAC to be able to evaluate the appropriateness of the budget? If not, then the RTAR should be returned for revision. #10, #8 #2 - Wrong approach. #10 - The deliverables need more detail; what will the engineer have to work with? Will this be a design guide? #8 - The description is OK in general, and the authors seem to have put some thought into it, but what benchmark data are being sought should be specified explicitly, because this could mean quite different things to different proposers and evaluators. #3 - not convinced that these data are really usable for the software developers, perhaps this could be a requirement, a statement of these developers will be convincing? Is the budget reasonable for the project scope? If not, then RTAR could be returned for revision or conditionally accepted with a note that the budget should be revised for the WS. Have the proper administrative procedures been followed? This includes recording of the TC vote, coordination with other TCs, proper citing of the Research Strategic Plan, etc. If not, then the RTAR could be returned for revision or possibly conditionally accepted based on adequately resolving these issues. #14- I think this is a decent estimate, but is probably a little low given the laboratory effort. #10 - Co-funding not identified. #10 - RL not listed as having been involved. Decision Options Initial Decision Additional Comments or Approval Conditions #10- This RTAR needs more refinement and clarification. #8- Needs better definition of what the benchmark data being sought are ACCEPT COND. ACCEPT RETURN X REJECT ACCEPT Vote - Topic is ready for development into a work statement (WS). COND. ACCEPT Vote - Minor Revision Required - RL can approve RTAR for development into WS without going back to RAC once TC satisfies RAC's approval condition(s) RETURN Vote - Topic is probably acceptable for ASHRAE research, but RTAR is not quite ready. REJECT Vote - Topic is not acceptable for the ASHRAE Research Program

3 Research Topic Acceptance Request Cover Sheet Date: May 14, 2012 (Please Check to Insure the Following Information is in the RTAR ) Title: Experimental Benchmark Data for Data A. Title X Center Numerical Modeling B. Applicability to ASHRAE Research Strategic Plan X C. Application of the Results X D. State-of-the-Art (background) X E. Advancement to State-of-the-Art X F. Justification and Value to ASHRAE X RTAR# G. Objective X (To be assigned by MORTS) Results of this Project will affect the following Special Handbook Publications, Chapters, etc.: H. Estimated Duration X Fundamentals Chapter 13, Indoor Environmental I. References X Modeling Applications Chapter 19 Data Processing and Telecommunications Future TC4.10 Applications Handbook Thermal Guidelines for Data Processing Environments Design Considerations for Datacom Equipment Centers Publication in ASHRAE Transactions or HVAC&R Research Responsible TC/TG: TC4.10, Indoor Environmental Modeling Date of Vote: ed on 5/7/12. Voting closed 5/11/12 For 11 Co-sponsoring TC/TG/MTG/SSPCs (give vote and date): Against 0 Abstaining 0 Absent or not 2 returning Total 13 Ballot Voting Members Jim VanGilder Jim VanGilder RTAR Lead Author: Expected Work Statement Lead Author: Research Classification: (Basic/Applied Research; Advanced Concepts; or Technology Transfer) Basic/Applied Research Potential Co-funders (organization): Has an electronic copy been furnished to the MORTS? Has the Research Liaison reviewed the RTAR? ( ed on 5/7/12) * Reasons for negative vote(s) and abstentions Yes X No X 1

4 Unique Tracking Number Assigned by MORTS 1675 RESEARCH TOPIC ACCEPTANCE REQUEST (RTAR) FORM (Generally 2 to 6 pages, with 10 pt Times New Roman font) Sponsoring TC/TG/MTG/SSPC/EHC/REF: TC4.10 Title: Experimental Benchmark Data for Data Center Numerical Modeling Applicability to ASHRAE Research Strategic Plan: This project will produce high-quality experimental data against which numerical models of data center applications will be benchmarked. This is a necessary step for the broader utilization of existing CFD-based modeling tools and the further development new, potentially simpler and cheaper, alternatives. Given that data centers consume more than 2% of all power generated in the US [1] a figure that continues to rise, there is great opportunity for such tools to improve the reliability and energy efficiency of such facilities. Accordingly, this project directly supports the following ASHRAE research goals: Goal 1: Maximize the actual operational efficiency of buildings and facilities. Goal 5: Support the development of ASHRAE energy standards and reduce effort required to demonstrate compliance. Goal 7: Support development of tools, procedures and methods suitable for designing low-energy buildings. Research Classification: (Basic/Applied Research; Advanced Concepts; or Technology Transfer) Basic/Applied Research TC/TG/MTG/SSPC Vote: Reasons for Negative Votes and Abstentions: (For Against-Abstentions-Absent-Total) (Negative Votes) None (Abstentions) None Estimated Cost: $125,000 (Estimate total dollars) RTAR Lead Author Jim VanGilder jim.vangilder@schneider-electric.com Estimated Duration: 18 months (Months to complete) Expected Work Statement Lead Author Jim VanGilder jim.vangilder@schneider-electric.com Co-sponsoring TC/TG/MTG/SSPCs and votes: This RTAR has been shared with TC9.9 which may choose to co-sponsor pending review. Possible Co-funding Organizations: None 2

5 Application of Results: Handbook Fundamentals Chapter 13, Indoor Environmental Modeling, TC4.10 Applications Chapter 19 Data Processing and Telecommunications, TC9.9 This study will provide material for the future Applications Handbook material chapter currently being discussed within TC4.10. Datacom Series Publications o Thermal Guidelines for Data Processing Environments o Design Considerations for Datacom Equipment Centers Publication in ASHRAE Transactions or HVAC&R Research The contractor will be required to publish a summary of the findings within ASHRAE. State-of-the-Art (Background): CFD is increasingly being used to optimize cooling performance and minimize energy consumption in data centers. This assertion is evidenced by the number of papers presented on the subject within ASHRAE and other forums and also by the fact that there are no fewer than 3 data-center-specific CFD tools sold commercially today. Despite the implicit acceptance of CFD for data center applications, there are few, if any, high-quality validation studies in the literature which can be used to benchmark CFD and other numerical models. Challenges to obtaining benchmarkquality measurement data include lack of access to working facilities because of potential business disruptions and the fact that real data centers (or real data center equipment in lab settings) have unique and complex characteristics (server airflow rates and flow patterns, geometric detail of racks and servers, perforated tile type, raised-floor depth, etc.) which are either not well- known as inputs or are impractical to capture in sufficient detail in a numerical model. Bash et al [2] studied a dedicated test facility and realized perhaps the best comparison to date between CFD and experiment for a practical-size data center application with most predicted temperatures within 10% of experiment. Others [3-7] have achieved comparisons between CFD and experimental data in the 25% difference range. In many cases, CFD results differ from measurements by as much as 60% or more. Furthermore, these publications generally do not provide sufficient detail such that the studies may be repeated with new or alternative modeling tools. A recent ASHRAE study [8] provided recommendations for modeling racks in a compact manner while providing additional CFD-experimental data comparisons. This was a major and necessary step in achieving practical, accurate simulations. A next logical step is to validate the accuracy of different numerical models which utilize multiple instances of such compact rack models on a larger scale. While CFD has been extensively used and validated for other various indoor-environment applications (see, for example, [9-11] and their references), data centers are made unique by virtue of their relatively high heat densities of 100 W/ft 2 or greater [12] and airflow patterns dominated by the fans in racks and cooling units. Consequently, existing benchmark data for occupied spaces are not be appropriate for validating numerical model of data centers. Several reduced-order models have been recently proposed which trade some potential modeling accuracy for improvements in solution speed and usability [13-18] with the goal of making predictive tools more economical and widely available. These models generally provide better predictions for the low Archimedes Numbers (ratio of buoyancy to momentum) typical of data centers [18] as they typically do not include a buoyancy model. Such tools need to be validated against high-quality measurement data for their intended data-center applications. 3

6 Advancement to the State-of-the-Art: The proposed research project will provide benchmark airflow and temperature data for a simple laboratory data center configuration. The configuration may not be an actual data center or even utilize actual data center equipment but should be representative of geometric sizes, airflow rates, heat loads, etc. found in (at least a portion of) an actual facility. For example, the configuration may include a few (perhaps plywood models of) racks and a single CRAC (Computer Room A/C) unit. The laboratory configuration will necessarily avoid real-world datacenter details such as extreme geometric detail, unintended leakage paths through the raised floor, transient variations in IT load, etc. which distract from the primary goal of benchmarking the ability of numerical models to capture the fundamental data center airflow physics. While such details may affect actual facility conditions, highquality benchmark data can only come from a sanitized laboratory interpretation of a data center. Additionally, the simplicity of such a configuration will afford a succinct description of input data, e.g., by way of a couple of 2D drawings and a table of values such as airflow rates, rack power dissipation, cooler supply temperature, etc. If numerical models can successfully predict performance for this benchmark configuration, it is reasonable to assume that the same basic models will provide better predictions as additional real-world detail is added. Some progress has been made to date on the subject of determining how much rack detail is required in numerical models [8]. The project will report input and output data in sufficient detail so that numerical-experimental comparisons can be made by developers and users of predictive software. Reported results will include temperatures and airflow patterns at multiple locations throughout the room. Such results should be presented both quantitatively and also somewhat qualitatively so that it will be possible to assess if relevant numerical models, at least, predict the correct general flow features and temperature distributions. Finally, in order to benchmark typical or best practical accuracy, the project will also provide simulation results using a typical RANS CFD model of the laboratory configuration. Justification and Value to ASHRAE: Objectives: ASHRAE members employing numerical models to analyze cooling performance and energy consumption in data centers will use the benchmark data to validate the use of their numerical models for data center applications. ASHRAE members new to CFD modeling will use the benchmark data for training purposes. Developers of numerical algorithms will be able to benchmark their tools. ASHRAE will further improve its leadership position in terms of modeling the indoor environment and data center cooling by being responsible for the benchmark experimental data center model which will be frequently referenced. Project Goals: 1) Provide benchmark airflow and temperature measurement data for a simple, generic, data center configuration with which data center numerical models can be validated. 2) Provide a typical or best practical comparison of a RANS CFD prediction to the benchmark data. The benchmark data will be used by researchers, software developers, and software users to produce reliable numerical models of data centers. Further, the CFD comparison will put into perspective the value of analyses commonly performed today. Estimated Duration: 18 months 4

7 References: [1] EPA, ENERGY STAR Program, 2007, Report to Congress on Server and Data Center Energy Efficiency Public Law [2] Patel, C.D., C.E. Bash, and C. Belady Computational Fluid Dynamics Modeling of High Compute Density Data Centers to Assure System Inlet Air Specifications. Pacific Rim ASME International Electronic Packaging Technical Conference and Exhibition (IPACK 2001), Kauai, Hawaii, July [3] VanGilder, J.W., and R.R. Schmidt Airflow Uniformity Through Perforated Tiles in a Raised-Floor Data Center. Pacific Rim ASME International Electronic Packaging Technical Conference and Exhibition (IPACK 2005), San Francisco, California, July [4] Shrivastava, S.K., M. Iyengar, B.G. Sammakia, R.R. Schmidt, and J.W. VanGilder Experimental- Numerical Comparison for a High-Density Data Center: Hot Spot Heat Fluxes in Excess of 500 w/ft 2. Intersociety Conference on Thermal and Thermomechanical Phenomena in Electronic Systems (ITHERM 2006), San Diego, California, May 30 June 2. [5] Iyengar, M., Schmidt, R.R., Hamann, H. and VanGilder, J., 2007, Comparison Between Numerical and Experimental Temperature Distributions in a Small Data Center Test Cell. Proceedings of InterPACK 07, International Electronic Packaging Technical Conference and Exhibition, July, Vancouver, Canada. [6] Y. Amemiya, M. Iyengar, H.F. Herman, M. O Boyle, M. Schappert, J. Shen, T. van Kessel, Comparison of Experimental Temperature Results with Numerical Modeling Predictions of a Real-World Compact Data Center Facility. Proceedings of InterPACK 07, International Electronic Packaging Technical Conference and Exhibition, July, Vancouver, Canada. [7] Kailash C. Karki, Amir Radmehr, and Suhas V. Patankar, Use of Computational Fluid Dynamics for Calculating Flow Rates Through Perforated Tiles in Raised-Floor Data Centers. International Journal of Heating, Ventilation, Air-Conditioning, and Refrigeration Research, Volume 9, Number 2, April 2003, pp [8] ASHRAE RP The Development of Simplified Rack Boundary Conditions for Numerical Data Center Models. Atlanta: American Society of Heating, Refrigerating and Air-Conditioning Engineers, Inc. [9] ASHRAE RP-1009 Simplified Diffuser Boundary Conditions for Numerical Room Airflow Models. Atlanta: American Society of Heating, Refrigerating and Air-Conditioning Engineers, Inc. [10] ASHRAE RP-1418 Optimizing the Trade Off Between Grid Resolution and Simulation Accuracy: Coarse Grid CFD Modeling. Atlanta: American Society of Heating, Refrigerating and Air-Conditioning Engineers, Inc. [11] ASHRAE RP-1133 How to Verify, Validate, and Report Indoors Environmental Modeling CFD. Atlanta: American Society of Heating, Refrigerating and Air-Conditioning Engineers, Inc. [12] ASHRAE Thermal Guidelines for Data Processing Environments, 2nd Ed.. Atlanta: American Society of Heating, Refrigerating and Air-Conditioning Engineers, Inc. [13] Toulouse, M., Doljac, G., Carey, V., and Bash, C., 2009, Exploration of A Potential-Flow-Based Compact Model of Air-Flow Transport in Data Centers, Proceedings of IMECE, November 13-19, Lake Buena Vista, Florida. 5

8 [14] Lopez, V. and Hamann, H., 2010, Measurement-Based Modeling for Data Centers, Proceedings of ITHERM, June 2-5, Las Vegas, Nevada. [15] Hamann, H., Lopez, V., and Stepanchuk, A., 2010, Thermal Zones for More Efficient Data Center Energy Management, Proceedings of ITHERM, June 2-5, Las Vegas, Nevada. [16] Healey, C., VanGilder, J., Sheffer, Z. and Zhang, X. 2011, Potential-Flow Modeling for Data Center Applications, Proceedings of InterPACK 11, International Electronic Packaging Technical Conference and Exhibition, July, Portland, OR. [17] VanGilder, J., Sheffer, Z., Zhang, X., and Healey, C., 2011, Potential Flow Model for Predicting Perforated Tile Airflow in Data Centers, ASHRAE Transactions, Vol. 117, Part 2. [18] Michael M. Toulouse, David J. Lettieri, Van P. Carey, Cullen E. Bash, and Amip J. Shah, Computational and Experimental Validation of a Vortex superposition-based Buoyancy Approximation For The Compact Code In Data Centers, Proceedings of the ASME 2011 International Mechanical Engineering Congress & Exposition IMECE2011November 11-17, Denver, Colorado [19] Demetriou, D. And Khalifa, H. E Evaluation of a Data Center Recirculation Non-Uniformity Metric Using Computational Fluid Dynamics. Proceedings of InterPACK 11, International Electronic Packaging Technical Conference and Exhibition, July, Portland, OR. 6