Ultra-Efficient HPC Data Centre - Gary Bernstein, McGill University
2 likes953 views
The document outlines the design and strategic planning for an ultra-efficient high-performance computing (HPC) data center located at McGill University's McDonald Campus in Quebec, leveraging natural low energy cooling methods. Key features include utilizing medium temperature chilled water and seasonal ice storage to optimize cooling costs and efficiency, targeting a Power Usage Effectiveness (PUE) of 1.06. The project aims for environmentally sustainable operations with significant cost savings and minimal reliance on compressor-based cooling.
Ultra-Efficient HPC Data Centre - Gary Bernstein, McGill University
- 1. Ultra Efficient HPC Data Center Natural Low Energy Cooling Conceptual Design 10/8/10 1
- 3. Project Funding Canada's Advanced Research and Innovation Network (CANARIE) Canada California Strategic Innovation Partnership (CCSIP) • ISTP Canada • University of California • McGill University
- 4. Site Selection Three candidate locations in Quebec • McDonald Campus of McGill University in St. Anne de Bellevue • Campus of the Institut de recherche d’Hydro-Quebec (IREQ) in Varennes • IREQ campus in Shawinigan McDonald Campus of McGill University selected as site for project All enjoy Cold climate Renewable energy resource (hydroelectric) Inexpensive electricity
- 5. The System Approach: An Overview Goals: Most Efficient Class One Data Center Climate Evaluation Define Loads and How to Best Serve Them • Water cooled equipment • Medium temperature chilled water (65F, 75F) Optimize Heat Rejection for Climate and Loads • Evaporative free cooling – Primary cooling • Seasonal ice storage – Top up cooling Backup Approach Results
- 6. Goals Provide ASHRAE TC9.9 Class 1 Datacenter No compressor based cooling Lower construction cost Lower operating cost Best efficiency Environmentally friendly Construction materials Recycle heat, water
- 7. Proposed Annual Electrical Costs Comparison $10,000,000 $9,000,000 $8,000,000 $7,000,000 $5 Million/yr Annual $6,000,000 Savings Target $5,000,000 $4,000,000 $3,000,000 $2,000,000 $1,000,000 $0 San Diego (1.35 PUE) Montreal (1.06 PUE) at $0.09/kWh at $0.05/kWh
- 8. Aerial Perspective – “Farm” at McDonald Campus Cooling Towers and Mechanical Infrastructure 20,000 SF Phase 1 8 MW IT Load VA Hospital Fuel Tanks Cooling Ice Pond Stormwater Detension Office, Shipping/ Receiving Electrical Infrastructure
- 9. Climate: Free Cooling Analysis with 65F CHWS 0.030 Data Source: Government of Canada - National Climate Data & Information Archive 0.028 Data Set: WMO #71627, Montreal/Pierre Elliott Trudeau Airport, Typical Year 0.026 Elevation: 118 feet Humidity Ratio (lbs H2O per lbs dry air) 0.024 Air Pressure: 14.633224 psia 0.022 0.020 Auxillary Cooling 80 hrs/yr 0.018 0.016 Partial Free Cooling 1234 hrs/yr 0.014 0.012 0.010 0.008 Full Free Cooling 0.006 7446 hrs/yr 0.004 0.002 0.000 -30 -25 -20 -15 -10 -5 0 5 10 15 20 25 30 35 40 45 50 55 60 65 70 75 80 85 90 95 100 Dry Bulb Temperature (F)
- 10. Climate: Free Cooling Analysis With 75F CHWS 0.030 Data Source: Government of Canada - National Climate Data & Information Archive 0.028 Data Set: Montreal/Pierre Elliott Trudeau Intl Airport, Typical Year 0.026 Elevation: 118 feet Auxillary Cooling Air Pressure: 14.633224 psia 0 hrs/yr Humidity Ratio (lbs H2O per lbs dry air) 0.024 0.022 0.020 Partial Free Cooling 0.018 114 hrs/yr 0.016 0.014 0.012 0.010 0.008 Full Free Cooling 0.006 8646 hrs/yr 0.004 0.002 0.000 -30 -25 -20 -15 -10 -5 0 5 10 15 20 25 30 35 40 45 50 55 60 65 70 75 80 85 90 95 100 Dry Bulb Temperature (F)
- 11. Load Strategies Climate analysis shows higher temperature chilled water offers many more hours of free cooling Highly concentrated heat loads • A single high density rack can put off as much waste heat as a VW Beetle (40kW) • Air exiting racks typically exceeds 90F
- 12. Load Heat Collection Strategies Higher temperature Chilled Water Supply (CHWS) offers many more hours of free cooling: Design to use 75F and 65F CHWS • Direct water based cooling most efficient • Hot aisle / cold aisle for minority of load
- 13. Primary Cooling Strategies: Medium Temp. Cooling Water and Free Cooling
- 14. Primary Cooling Strategies: Medium Temp. Cooling Water and Free Cooling Design to cool with 65F/18C and 75F/24C water • 90% of 65F load served with cooling tower provided free cooling; 99.3% of 75F load • 590,000 ton-hrs (2,100 MWh) Top up Cooling Required
- 15. Supplemental Cooling: Seasonal Ice Storage Slush Pond System Fill in winter with plowed snow collection Melt water cools data center
- 17. Slush Pond Paved collection basin, 75,000 ft3 (2,100 m3) Drive-in slope on one side for plow loading Lightweight, waterproof insulating cover or roof to protect from warm rains Extensive drain system to collect meltwater Berms for sides, or dig into ground
- 18. Sundsvall, Sweden, Snow Storage - Empty
- 19. Sundsvall, Sweden, Snow Storage - Full
- 20. Snow dump overruns in Montreal, 2008-09
- 21. Slush Pond System – Pumping and Filtration Mature waste water handling technology Remove gravel, wood, grit from melt water Remove oils and road chemicals prior to release as required Filter Select heat exchangers for highly corrosive fluid Maintain complete separation between pond water and building loop water Integrate settling tank to also serve as emergency storage
- 22. Key approaches Keep storage pond simple Leverage local snow removal program if possible Collect snow dumpage fees? Provide appropriate maintenance Provide for pile grooming, drain clearing, filter cleaning, etc Use in lieu of chillers to save cost Consider emergency chiller rental for backup Design properly Simple concept but careful design required
- 23. Office Approaches Much lower load Design for comfort and optimal use of medium temperature water
- 24. Backup – Do Not Invest in Chillers 'Just in Case'! Pay for it only when (if) you ever need it Design for portable air-cooled chillers to connect in an emergency
- 25. Results McGill-USCD HPC Data Center PUE Itemization Fans; 1.5% CRAH Fans; 0.0% Humidifier; 0.0% CHW Plant; 2.1% Transformer Loss; 0.5% UPS Loss; 0.6% Racks; 94.0% PDU Loss; 1.0% Data Center Lights; 0.2% Power Usage Effectiveness (PUE) = Total Energy / Rack Energy = 1.06
- 26. Results Supply Temperatures Annual Energy Use Mechanical Cooling Needed Water Usage Hours of Free Cooling / year PUE Additional Load at Air Water Cost Evaporation + Cooled Cooled Energy ( $0.058/kWh) Hours Extreme Weather Carry Over per Year (wetbulb = 68.7°F) °C °F °C °F hrs/yr % of yr MWh/yr $ tons gallons 23.9 75.0 23.9 75.0 8,532 97% 1.06 74,567 $4,325,000 228 0 30,100,000