INVITATION TO SUBMIT A RESEARCH PROPOSAL ON AN ASHRAE RESEARCH PROJECT

Transcription

1 INVITATION TO SUBMIT A RESEARCH PROPOSAL ON AN ASHRAE RESEARCH PROJECT Attached is a Request-for-Proposal (RFP) for a project dealing with a subject in which you, or your institution have expressed interest. Should you decide not to submit a proposal, please circulate it to any colleague who might have interest in this subject TRP, Guidance for CFD Modeling of Data Centers Sponsoring Committee: TC 4.10, (Indoor Environmental Modeling) Co-Sponsors: TC 9.9, (Mission Critical Facilities, Data Centers, Technology Spaces and Electronic Equipment) Budget Range: $150,000 proposals. may be more or less as determined by value of proposal and competing Scheduled Project Start Date: April 1, 2016 or later. All proposals must be received at ASHRAE Headquarters by 8:00 AM, EST, December 15, NO EXCEPTIONS, NO EXTENSIONS. Electronic copies must be sent to rpbids@ashrae.org. Electronic signatures must be scanned and added to the file before submitting. The submission title line should read: and Bidding Institutions Name (electronic pdf format, ASHRAE s server will accept up to 10MB) If you have questions concerning the Project, we suggest you contact one of the individuals listed below: For Technical Matters Technical Contact James VanGilder Schneider Electric 146 Heald St Pepperell, MA Phone: Jim.VanGIlder@schneider-electric.com For Administrative or Procedural Matters: Manager of Research & Technical Services (MORTS) Michael R. Vaughn ASHRAE, Inc Tullie Circle, NE Atlanta, GA Phone: Fax: MORTS@ashrae.net Contractors intending to submit a proposal should so notify, by mail or , the Manager of Research and Technical Services, (MORTS) by November 30, 2015 in order that any late or additional information on the RFP may be furnished to them prior to the bid due date. All proposals must be submitted electronically. Electronic submissions require a PDF file containing the complete proposal preceded by signed copies of the two forms listed below in the order listed below. ALL electronic proposals are to be sent to rpbids@ashrae.org. All other correspondence must be sent to ddaniel@ashrae.org and mvaughn@ashrae.org. In all cases, the proposal must be submitted to ASHRAE by 8:00 AM, EST, December 15, NO EXCEPTIONS, NO EXTENSIONS. The following forms (Application for Grant of Funds and the Additional Information form have been combined) must accompany the proposal: (1) ASHRAE Application for Grant of Funds (electronic signature required) and (2) Additional Information for Contractors (electronic signature required) ASHRAE Application for Grant of Funds (signed) and ASHRAE reserves the right to reject any or all bids.

2 State of the Art (Background) The use of CFD to optimize cooling performance and minimize energy consumption in data centers has grown dramatically in recent years. Many papers have been published on the subject and no fewer than 5 data-center-specific CFD tools are sold commercially today. Data centers now represent a large class of indoor CFD applications. Data center CFD modeling tools in frequent use range from simple [5-13] to complex [14-18]; many are aimed at inexperienced users and are now being applied to large operational data centers [19]. Complicating matters, the most appropriate CFD modeling assumptions and solver choice typically vary depending on the specific application and simulation goals. As with RP-1133, which provides guidelines for modeling other indoor environmental applications, experimentally-verified airflow and temperature data is a key starting point for developing modeling guidance. Additionally, the raw data will be presented as transparent and open support for the conclusions reached. It could, potentially, also allow users to directly compare their modeling with that presented in the study. Experimental data in the literature is quite limited. Patel et al [20] studied a dedicated test facility and realized some of the better comparisons to date between CFD and experiment for a practical-size data center application with most predicted temperatures within 10% of experiment. Others [21-22] have achieved comparisons between CFD and experimental data in the 25 % difference range. In many cases, CFD results differ from measurements by as much as 60% or more. These publications generally do not provide sufficient detail or achieve sufficient accuracy for guidance to be developed. As far as guidance is concerned, several studies in the literature address certain aspects of modeling data center objects and features including under-floor plenums [23-24], perforated tiles [25-27], racks [28-31], and computational grid [32]. These existing studies provide supporting evidence but are not sufficiently comprehensive or consistent with one another for general guidance to be developed. While this work should be considered and leveraged where possible in the present research, the scope of applications and quantity/quality of experimental data are insufficient for the development of reasonably general modeling guidance. In summary, CFD is an enormously powerful tool and its application to data centers offers the potential to meaningfully impact global energy consumption and drive efficiency and reliability. However, the breadth and usefulness of its implementation today - consistent with ASHRAE s Research Strategic Plan - is partly held back by the lack of guidance and recognition from a trusted independent entity such as ASHRAE. Justification and Value to ASHRAE This research project will improve the reliability and energy efficiency of data centers through the use of modeling tools and is, therefore, aligned with ASHRAE s strategic research plan. Data centers are immensely important to the world economy and consume huge quantities of energy; even small improvements to data center design and operational characteristics will have a significant global impact. ASHRAE is the logical independent organization to sponsor the needed research with its recognized expertise in both indoor environmental modeling (i.e., TC4.10) and data centers (i.e., TC9.9). Objective The primary objective of this research project is to provide CFD modeling guidance for data center applications. The guidance will be based on experimental and CFD analyses of several data center configurations to be conducted as part of this study as well as other work available in the literature. Scope: The contractor will need to work closely with the PMS throughout this project especially during the phase when data center test configurations are being selected. During this phase, CFD modeling should be used before substantial laboratory effort is undertaken to ensure that interesting (i.e., includes meaningful rack-inlet-temperature variations, complex airflow patterns, etc.) configurations are selected. Once configurations are selected, modeling and simulation should proceed in parallel until reasonable

3 agreement can be achieved. All task deliverables will be reviewed by the PMS and may result in modifications to subsequent tasks. The baseline data center (laboratory) configuration should be geometrically simple and, while it can be fairly small, must include the fundamental building blocks common to most data centers including a few IT equipment racks (a minimum of 10 racks in 2 rows) and at least one down-flow Computer Room Air Handler (CRAH) which supplies airflow into a raised-floor plenum configured with perforated floor tiles. The plenum depth should be in the inch ( mm, chosen to be both typical and provide greater variations in airflow) range and perforated tiles should have an open area in the 40% or greater range. If the room contains more than one down-flow CRAH but only one is to be used in a test, it must be possible to seal off the unused unit(s) to prevent any unintended leakage. Note that (chilled-water cooled) CRAHs are preferred over CRACs (Computer Room A/C) as the latter tend to cycle making steady-state conditions difficult to achieve. All boundary conditions should be made simple and easy to define by sealing up unintended leakage paths, ensuring that rack airflow is fairly uniform across front and rear faces, and that there is no unnecessary equipment present, etc. As rack power and airflow and CRAH airflow and supply temperature are to be assumed known input, these parameters must be either measured during the experimental trials or calibrated beforehand and operated under known conditions during experimental trials. A well-recognized general purpose CFD tool should be used which is capable of capturing all aspects of data center airflow physics. Task 1 Literature Review Review existing data center literature related to comparisons of CFD to measured values, modeling perforated tiles, racks, cooling units, etc. and key questions listed in Advancement to the State-of-the- Art section. Previously referenced work [23-32] (and any other data center literature related to comparisons of CFD to measured values, modeling perforated tiles, racks, cooling units, etc.) should be used as a starting point for selecting appropriate modeling strategies. Intermediate deliverable: 1-2 page summary of findings to PMS Task 2 Calibrated CFD Model for Baseline Configuration Select a baseline configuration. The air ratio (of total cooling airflow to total rack airflow) should be approximately 0.8 in order to intentionally generate some recirculation of rack exhaust air back to rack inlets providing a more widely-varying temperature distribution in the room. Calibrate the CFD model and experimental measurements until reasonable agreement can be achieved. Compare data for all applicable items under Experimental Measurements below. Intermediate deliverable: 1-2 page illustration (plots, tables, etc.) showing layout selected and calibration of model to experiment Task 3 Calibrated Model for Blocked-Off-Plenum Configuration Repeat Task 2 with a portion of the plenum blocked-off in order to create indirect and more complex flow patterns. Intermediate deliverable: 1-2 page illustration (plots, tables, etc.) showing layout selected and calibration of model to experiment Task 4 Calibrated Model for Configuration with Non-Uniform Rack Loading Repeat Task 1 with an uneven distribution of rack loading with minimum peak-to-average-power and average-to-minimum power ratios of two. Ideally, a distribution of different rack temperature rises, TITs, will also be employed varying from about 20 F (11 C) 40 F (22 C) which is equivalent to a rack-airflow-

4 per-power range of cfm/kw ( m 3 /hr/kw). The variation in rack TITs will make the effect of buoyancy forces more or less significant in different locations of the test data center configuration The goal is to create a complex airflow and temperature distribution which provides sufficiently challenging variations. Intermediate deliverable: 1-2 page illustration (plots, tables, etc.) showing layout selected and calibration of model to experiment Task 5 Additional CFD modeling with Calibrated Model Starting from the calibrated models developed in Tasks 2-4, run alternative CFD models to consider additional and more extreme variations than could be investigated experimentally. Features to be investigated include perforated tile types, floor plenum depth, room-blockage location and size, rack loading distribution, segregation leakage (e.g. cable cut-outs, poor cabinet segregation), etc. Intermediate deliverable: 1-2 page illustration (plots, tables, etc.) showing layout selected and calibration of model to experiment Task 6 Document and Develop Guidance Document the layouts and all data of Tasks 2-4 for both CFD prediction and experimental measurement. The layout description must include a dimensioned drawing and/or dimensions must be readily inferred from a standard (e.g., 2 ft [600 mm] x 2ft [600 mm]) floor-tile grid. Tables should be provided as necessary documenting rack airflow and power dissipation, CRAC airflow rate and supply temperature, and any other details not obvious from the layout drawing. A discussion of the likely sources of disagreement between measured and predicted values should be provided for each configuration so that a reader can understand whether simulation differences are reasonable. Finally, provide specific CFD modeling recommendations based on the literature review plus what was learned in the course of this study. The recommendations should answer the questions posed in Advancement to the State-of-the-Art and additional findings as appropriate. Notes on Experimental Measurements: a) Cooling unit airflow. It is normally easier to measure the return airflow. Ensure an adequate array of measurements are made as close as possible to the inlet and use techniques to avoid measuring conflicting cross velocities. b) Cooing unit cooling power/load. Measure the return air temperatures and supply air temperatures. If the return is in the open, measure temperatures around the perimeter as well as over the over the face of the return to capture non-uniformities. If there are multiple supply air streams, measure the temperature in each stream. c) IT equipment power draw. Measure as close to IT equipment as possible. If the measured power is not at the IT equipment level, the connectivity for power must be documented. d) Perforated tile airflow. Measure the airflow at every perforated tile. Be careful to check that the measurement device is not significantly affecting the measured airflow or take steps to correct measurements. Note that back pressure compensation may be inadequate. The sum shall be compared with the cooling unit total supply. If significant discrepancies exist, significant efforts shall be made to identify the cause of the discrepancy and if possible correct it. e) Inlet temperatures of selected racks. Measure inlet temperatures (at a minimum) of end-row and center-of-row racks adjacent to the front face of the inlet vent at: i) Bottom-of-rack height (approximately 6 in [0.15 m] above the raised floor); ii) Half-rack height (e.g., 39 in [1 m] above the raised floor for a typical rack but elevation shall be appropriate for typical rack height); and iii) Top-of-rack height (e.g., 72 in [1.85 m] above the raised floor for a typical rack but elevation shall be appropriate for typical rack height).

5 f) Airflow and velocity distributions. Finally, the airflow velocity and temperature distribution shall also be measured in a vertical plane through the centerline of the perforated tile/rack, at least to top-of-rack height across the entire cold aisle, for the following locations: i) The lowest-flow perforated tile, ii) A typical-flow perforated tile, and iii) The highest-flow perforated tile. Laser anemometry or other more sophisticated measurement systems will be an advantage. Note that during measurement and in reporting the contractor must be aware of the accuracy of the measurement tools they are using and take steps to ensure that reported data is consistent; measured cooling temperatures/power and airflow must be balanced by the measured use/demand in the facility. Intermediate deliverable: Draft final report Deliverables: Progress, Financial and Final Reports, Technical Paper(s), and Data shall constitute the deliverables ( Deliverables ) under this Agreement and shall be provided as follows: a. Progress and Financial Reports Progress and Financial Reports, in a form approved by the Society, shall be made to the Society through its Manager of Research and Technical Services at quarterly intervals; specifically on or before each January 1, April 1, June 10, and October 1 of the contract period. Furthermore, the Institution s Principal Investigator, subject to the Society s approval, shall, during the period of performance and after the Final Report has been submitted, report in person to the sponsoring Technical Committee/Task Group (TC/TG) at the annual and winter meetings, and be available to answer such questions regarding the research as may arise. b. Final Report A written report, design guide, or manual, (collectively, Final Report ), in a form approved by the Society, shall be prepared by the Institution and submitted to the Society s Manager of Research and Technical Services by the end of the Agreement term, containing complete details of all research carried out under this Agreement, including a summary of the control strategy and savings guidelines. Unless otherwise specified, the final draft report shall be furnished, either electronically or hardcopy format (6 copies) for review by the Society s Project Monitoring Subcommittee (PMS). Tabulated values for all measurements shall be provided as an appendix to the final report (for measurements which are adjusted by correction factors, also tabulate the corrected results and clearly show the method used for correction). Following approval by the PMS and the TC/TG, in their sole discretion, final copies of the Final Report will be furnished by the Institution as follows: -An executive summary in a form suitable for wide distribution to the industry and to the public. - One bound copy -Two copies on CD-ROM disks; one in PDF format and one in Microsoft Word. c. Science & Technology for the Built Environment or ASHRAE Transactions Technical Papers One or more papers shall be submitted first to the ASHRAE Manager of Research and Technical Services (MORTS) and then to the ASHRAE Manuscript Central website-based manuscript review system in a form and containing such information as designated by the Society suitable for publication. Papers specified as deliverables should be submitted as either Research Papers for HVAC&R Research or Technical Paper(s) for ASHRAE Transactions. Research papers contain

6 d. Data generalized results of long-term archival value, whereas technical papers are appropriate for applied research of shorter-term value, ASHRAE Conference papers are not acceptable as deliverables from ASHRAE research projects. The paper(s) shall conform to the instructions posted in Manuscript Central for an ASHRAE Transactions Technical or HVAC&R Research papers. The paper title shall contain the research project number (1675-RP) at the end of the title in parentheses, e.g., (1675-RP). All papers or articles prepared in connection with an ASHRAE research project, which are being submitted for inclusion in any ASHRAE publication, shall be submitted through the Manager of Research and Technical Services first and not to the publication's editor or Program Committee. Data is defined in General Condition VI, DATA e. Project Synopsis A written synopsis totaling approximately 100 words in length and written for a broad technical audience, which documents 1. Main findings of research project, 2. Why findings are significant, and 3. How the findings benefit ASHRAE membership and/or society in general shall be submitted to the Manager of Research and Technical Services by the end of the Agreement term for publication in ASHRAE Insights The Society may request the Institution submit a technical article suitable for publication in the Society s ASHRAE JOURNAL. This is considered a voluntary submission and not a Deliverable. Technical articles shall be prepared using dual units; e.g., rational inch-pound with equivalent SI units shown parenthetically. SI usage shall be in accordance with IEEE/ASTM Standard SI-10. Level of Effort The project is estimated to cost $150,000 and cover a period of about 24 months. The estimation is based on a total of 3 months effort of the Principal Investigator and 24 months of a research assistant. Proposal Evaluation Criteria: Proposals should describe and justify methods to be used in meeting the objectives of this work. Bidders must submit their current test procedures, practices, qualifications and an estimated time schedule of the proposed research activities. 1. Contractor's understanding of Work Statement as revealed in proposal. 15% a) Logistical problems associated b) Technical problems associated 2. Quality of methodology proposed for conducting research. 20% a) Organization of project b) Management plan 3. Contractor's capability in terms of facilities. 25% a) Managerial support b) Data collection c) Technical expertise 4. Qualifications of personnel for this project. 15% a) Project team 'well rounded' in terms of qualifications and experience in related work b) Project manager person directly responsible; experience and corporate position c) Team members' qualifications and experience d) Time commitment of Principal Investigator 5. Student involvement 5% a) Extent of student participation on contractor's team

7 b) Likelihood that involvement in project will encourage entry into HVAC&R industry 6. Probability of contractor's research plan meeting the objectives of the Work Statement.15% a) Detailed and logical work plan with major tasks and key milestones b) All technical and logistic factors considered c) Reasonableness of project schedule 7. Performance of contractor on prior ASHRAE or other projects. 5% (No penalty for new contractors.) References [1] ASHRAE RP How to Verify, Validate, and Report Indoor Environmental Modeling CFD. Atlanta: American Society of Heating, Refrigerating and Air-Conditioning Engineers, Inc. [2] EPA, ENERGY STAR Program, 2007, Report to Congress on Server and Data Center Energy Efficiency Public Law [3] ASHRAE Thermal Guidelines for Data Processing Environments, 2nd Ed.. Atlanta: American Society of Heating, Refrigerating and Air-Conditioning Engineers, Inc. [4] ASHRAE Design Considerations for Datacom Equipment Centers, 2nd Ed.. Atlanta: American Society of Heating, Refrigerating and Air-Conditioning Engineers, Inc. [5] Toulouse, M., Doljac, G., Carey, V., and Bash, C., 2009, Exploration of A Potential-Flow-Based Compact Model of Air-Flow Transport in Data Centers, Proceedings of IMECE, November 13-19, Lake Buena Vista, Florida. [6] Lopez, V. and Hamann, H., 2010, Measurement-Based Modeling for Data Centers, Proceedings of ITHERM, June 2-5, Las Vegas, Nevada. [7] Hamann, H., Lopez, V., and Stepanchuk, A., 2010, Thermal Zones for More Efficient Data Center Energy Management, Proceedings of ITHERM, June 2-5, Las Vegas, Nevada. [8] Healey, C., VanGilder, J., Sheffer, Z. and Zhang, X. 2011, Potential-Flow Modeling for Data Center Applications, Proceedings of InterPACK 11, International Electronic Packaging Technical Conference and Exhibition, July, Portland, OR. [9] VanGilder, J., Sheffer, Z., Zhang, X., and Healey, C., 2011, Potential Flow Model for Predicting Perforated Tile Airflow in Data Centers, ASHRAE Transactions, Vol. 117, Part 2. [10] Healey, C., VanGilder, J., and Zhang, X., 2013, Efficient Implementation of Potential-Flow Airflow Prediction for Data Centers. Proceedings of InterPACK 2013, International Electronic Packaging Technical Conference and Exhibition, July 16-18, Burlingame, CA. [11] Demetriou, D. And Khalifa, H. E Evaluation of a Data Center Recirculation Non-Uniformity Metric Using Computational Fluid Dynamics. Proceedings of InterPACK 11, International Electronic Packaging Technical Conference and Exhibition, July, Portland, OR. [12] Michael M. Toulouse, David J. Lettieri, Van P. Carey, Cullen E. Bash, and Amip J. Shah Computational and Experimental Validation of a Vortex Superposition-Based Buoyancy Approximation for the Compact Code in Data Centers, Proceedings of the ASME 2011 International Mechanical Engineering Congress & Exposition IMECE2011November 11-17, Denver, Colorado [13] VanGilder, J., Zhang, X., and Healey, C., 2013, Data Center Airflow Prediction with an Enhanced Potential Flow Model. Proceedings of InterPACK 2013, International Electronic Packaging Technical Conference and Exhibition, July 16-18, Burlingame, CA.

8 [14] Seymour, M. and Ikemoto, S Design and Management of Data Center Effectiveness, Risks and Costs /12 Proceedings of 28 th IEEE SEMI-THERM Symposium, San Jose, California, March 18-22, [15] Abdelmaksoud, W., Thong Q., Dang, H., Khalifa, E., and Schmidt, R., Improved Computational Fluid Dynamics Model for Open-Aisle Air-Cooled Data Center Simulations. Journal of Electronic Packaging, 135(3), July , [16] Alkharabsheh, S., Sammakia, B., Shrivastava, S., and Schmidt, R., "A Numerical Study for Contained Cold Aisle Data Center Using CRAC and Server Calibrated Fan Curves", ASME 2013 International Mechanical Engineering Congress and Exposition, Volume 10: Micro- and Nano-Systems Engineering and Packaging, San Diego, California, USA, November 15 21, [17] Khalil, E., and Aziz, M., On the Computations of Flow Regimes and Thermal Patterns in Large Scale High Compute Density Data Centers, ASME 2013 International Mechanical Engineering Congress and Exposition, Volume 11: Emerging Technologies, San Diego, California, USA, November 15 21, [18] King, D., Ross,M., Seymour, M., Gregory, T., Comparative analysis of data center design showing the benefits of server level simulation models Proceedings of 30th IEEE SEMI-THERM Symposium, San Jose, California, March 9-13, 2014, pp [19] Cane, G., At the End of the Day It s Lost Capacity. A Road Map to a Fully Deployed Data Centre. CBRE Ltd White Paper [20] Patel, C.D., C.E. Bash, and C. Belady Computational Fluid Dynamics Modeling of High Compute Density Data Centers to Assure System Inlet Air Specifications. Pacific Rim ASME International Electronic Packaging Technical Conference and Exhibition (IPACK 2001), Kauai, Hawaii, July [21] Shrivastava, S.K., M. Iyengar, B.G. Sammakia, R.R. Schmidt, and J.W. VanGilder Experimental-Numerical Comparison for a High-Density Data Center: Hot Spot Heat Fluxes in Excess of 500 w/ft 2. Intersociety Conference on Thermal and Thermomechanical Phenomena in Electronic Systems (ITHERM 2006), San Diego, California, May 30 June 2. [22] Iyengar, M., Schmidt, R.R., Hamann, H. and VanGilder, J., 2007, Comparison Between Numerical and Experimental Temperature Distributions in a Small Data Center Test Cell. Proceedings of InterPACK 07, International Electronic Packaging Technical Conference and Exhibition, July, Vancouver, Canada. [23] VanGilder, J.W., and R.R. Schmidt Airflow Uniformity through Perforated Tiles in a Raised-Floor Data Center. Pacific Rim ASME International Electronic Packaging Technical Conference and Exhibition (IPACK 2005), San Francisco, California, July [24] Kailash C. Karki, Amir Radmehr, and Suhas V. Patankar, Use of Computational Fluid Dynamics for Calculating Flow Rates through Perforated Tiles in Raised-Floor Data Centers. International Journal of Heating, Ventilation, Air-Conditioning, and Refrigeration Research, Volume 9, Number 2, April 2003, pp [25] Abdelmaksoud, W. A., T. Q. Dang, H. E. Khalifa, B. Elhadidi, R. R. Schmidt, and M. Iyengar, "Experimental and Computational Study of Perforated Floor Tile in Data Centers", Proc. ITherm Conference, Las Vegas, NV, June, 2010.

9 [26] Abdelmaksoud, W. A., T. Q. Dang H. E. Khalifa, R. R. Schmidt and M. Iyengar, "Perforated tile models for improving data center CFD simulations", Proceedings of ITherm 2012, San Diego, CA, May 30-June [27] Arghode, Vaibhav K. and Joshi, Yogendra, "Rapid Modeling of Air Flow through Perforated Tiles in a Raised Floor Data Center", Proceedings of ITherm 2014, Orlando, FL, May 27 30, [28] Y. Amemiya, M. Iyengar, H.F. Herman, M. O Boyle, M. Schappert, J. Shen, T. van Kessel, Comparison of Experimental Temperature Results with Numerical Modeling Predictions of a Real-World Compact Data Center Facility. Proceedings of InterPACK 07, International Electronic Packaging Technical Conference and Exhibition, July, Vancouver, Canada. [29] ASHRAE RP The Development of Simplified Rack Boundary Conditions for Numerical Data Center Models. Atlanta: American Society of Heating, Refrigerating and Air- Conditioning Engineers, Inc. [30] Abdelmaksoud, W. A., H. E. Khalifa, T. Q. Dang, R. R. Schmidt, and M. Iyengar, "Improved CFD Modeling of a Small Data Center Test Cell", Proc. ITherm Conference, Las Vegas, NV, June, [31] Abdelmaksoud, W. A., T. Q. Dang, H. E. Khalifa and R. R. Schmidt, "Improved CFD model for open-aisle, air-cooled data center simulations", ASME Trans. J. of Electronic Packaging, Vol. 6(3), pp , [32] ASHRAE RP Optimizing the Trade Off Between Grid Resolution and Simulation Accuracy: Coarse Grid CFD Modeling. Atlanta: American Society of Heating, Refrigerating and Air-Conditioning Engineers, Inc.