10 dos and donts of data centre design
4 likes2,279 views
The document discusses several key factors for designing an efficient data center. It recommends (1) virtualizing servers and storage to increase utilization, (2) optimizing space, power, and cooling usage to reduce wasted capacity, and (3) rearranging equipment layouts to address fragmentation and maximize available capacity. It also stresses the importance of selecting an appropriate location and tier level of resilience based on the data center's needs. Overall, the document provides guidance on foundational strategies to design an efficient and cost-effective data center.
10 dos and donts of data centre design
- 1. The 10 and of Data Centre Design Barry Shambrook RCDD MD Tuckers Consultancy Ltd, and DC Professional Trainer
- 2. build a new data centre! • Consider cloud computing • Virtualise • Optimise facilities
- 3. Cloud efficiency Source: Cloud Computing and Sustainability, The Environmental Benefits of Moving to the Cloud Accenture and WSP
- 4. make the best of what you have • Virtualisation • Optimise – Space – Power – Cooling
- 5. Virtualise, increase utilisation Dell PowerEdge R815 power against load Source: SpecPower_ssj2008
- 6. Data Centre Fragmentation Capacity is lost when there is no location in the data centre where all the components are available together in sufficient capacity for the proposed install. Available U: 13 Available U: 15 Available U: 26 Available U: 17 Available U: 17 Available kW:1.2 Available kW:1.7 Available kW: 0 Available kW: 2 Available kW:1.7 Next Change: Cluster Total Space: 11U Total Power: 1.4kW Total Space Total Power All cabinets fail on at least one of the parameters DCCP Module 6 v2.2 CFD Fundamentals 6
- 7. Lost Capacity – it all adds up Fragmented Space Total Space 53% 47% Fragmented Power Total Power 39% 61% DCCP Module 6 v2.2 CFD Fundamentals 7
- 8. Lost cooling capacity The Future Assumption 40% 80kW 60% 15kW 15kW Reality 15kW 30% 40% 15kW Total Load 140kW 80kW 120kW 140kW 30% DCCP Module 6 v2.2 CFD Fundamentals 8
- 9. Re-arrangement increases capacity Cooling Capacity 10% 25kW 25kW 80kW 100 % 10% Lost Capacity 90 % 25kW 10kW 15kW 10kW 25kW Total Load 140kW 80kW 120kW 180kW DCCP Module 6 v2.2 CFD Fundamentals 9
- 10. locate the data centre:
- 11. Risk of wrong location
- 12. Other risks relevant to SA
- 13. consider locations • Risks – see TIA 942/BICSi 002 • Climate • Power • Access, resources, communications
- 14. Temperature
- 16. Power map
- 18. start with how many racks • Servers • Storage • Network
- 19. More space efficient options
- 20. Allow for support areas Goods Entrance Main Entrance
- 21. forget resilience has a cost Poor efficiency Poor efficiency Poor efficiency Item N N+1 N+N IT equipment 5285421 kWhr 5285421 kWhr 5285421 kWhr PUE 1.91 1.98 2.34 Cost/kWhr R 0.52 R 0.52 R 0.52 TOTAL COST R 5,250,762 R 5,439,394 R 6,436,616 Emissions 5554 T CO2 5753 T CO2 6808 T CO2 Calculated annual costs and emissions for typical poor efficiency data centre
- 22. Even at high efficiency High efficiency High efficiency High efficiency Item N N+1 N+N IT equipment 5285421 kWhr 5285421 kWhr 5285421 kWhr PUE 1.42 1.46 1.53 Cost/kWhr R 0.52 R 0.52 R 0.52 TOTAL COST R 3,904,462 R 4,006,124 R 4,196,825 Emissions 4130 T CO2 4237 T CO2 4439 T CO2 Calculated annual costs and emissions for typical high efficiency data centre
- 23. Tier 1 power and cooling Power Cooling Mains Generator Change Over UPS (N) Heat Rejection (N) Equipment Room Units
- 24. Tier 2 power and cooling Power Cooling Mains Generator (N+1) Change Over UPS (N+1) Heat Rejection (N+1) Equipment Room Units (N+1)
- 25. Tier 3 power and cooling Power Cooling Mains Generator Mains Generator Change Over Change Over UPS (N+N) UPS Heat Rejection (N+N) Heat Rejection Equipment Room Units (N+N) Room Units
- 26. Tier 4 power and cooling Power Cooling Mains Generator Mains Generator Change Over Change Over UPS (N+N) UPS Heat Rejection (N+N) Heat Rejection (N+1) (N+1) (N+1) (N+1) Equipment Room Units (N+N) Room Units (N+1) (N+1)
- 27. consider how much resilience is really needed Operational 500 – 5000 min 50 – 500 min 5.0 – 50 min 0.5 – 5.0 min Level unplanned unplanned unplanned unplanned (maintenance) downtime downtime downtime downtime Level 1 1 1 2 3 (any time) Level 2 1 2 2 3 Level 3 2 2 3 4 Level 4 3 3 4 4 (never)
- 28. TIA 942/BICSi 002 Classes Impact of Availability Availability Availability Availability Availability Ranking 0 Ranking 1 Ranking 2 Ranking 3 Ranking 4 Downtime Enterprise wide Class F1 Class F2 Class F3 Class F4 Class F4 Multi Regional Class F1 Class F2 Class F3 Class F3 Class F4 Regional Class F1 Class F2 Class F2 Class F3 Class F3 Local Class F0 Class F1 Class F2 Class F3 Class F3 Sub Local Class F0 Class F0 Class F1 Class F2 Class F2
- 29. ignore the fundamentals of data centre efficiency • IT utilisation and efficiency • Resilience • Power conversions IT Load • Heat rejection (cooling) Cooling Electrical Lighting Misc.
- 30. Low efficiency IT Idle time Power Virtual Average Average Unit Max power Active idle Standby hours management/ Item quantity virtualisation utilisation quantity /unit power /unit power /unit /year EEE enabled Annual energy use Servers N+N PSU and NIC 4000 1 10% 4000 303 W 221 W 60 W 1000 no 7998368 kWhr Storage N+N PSU and NIC 800 1 10% 800 200 W 100 W 20 W 1000 no 762880 kWhr Network ports - edge N+N 100% 9600 12 W 6W 1W 1000 no 951552 kWhr Network ports Core N+N 100% 400 12 W 6W 1W 1000 no 39648 kWhr Other equipment 40 50 W 17520 kWhr 9769968 TOTAL per annum kWhr Average consumption 1115 kW ProLiant DL380 G5 (2.66GHz, Intel Xeon L5430 processor), 2009, 734 ops/watt
- 31. High efficiency IT Max Active idle Standby Idle time Power Virtual Average Average Unit power power power hours management/EE Item quantity virtualisation utilisation quantity /unit /unit /unit /year E enabled Annual energy use Servers N+N PSU and NIC 7200 10 75% 720 303 W 221 W 60 W 500 yes 1701684 kWhr Servers N PSU and NIC 800 10 75% 80 253 W 170 W 30 W 500 yes 154671 kWhr Storage N+N PSU and NIC 1440 10 75% 144 200 W 100 W 20 W 500 yes 209592 kWhr Storage N PSU and NIC 160 10 75% 16 180 W 80 W 15 W 500 yes 20605 kWhr Network ports - edge N+N 100% 1728 12 W 6W 1W 500 yes 172143 kWhr Network ports Core N+N 100% 72 12 W 6W 1W 500 yes 7173 kWhr Network ports - edge N 100% 96 12 W 6W 1W 500 yes 9564 kWhr Network ports Core N 100% 4 12 W 6W 1W 500 yes 398 kWhr Other equipment 40 50 W 17520 kWhr 2293350 TOTAL per annum kWhr Average consumption 262 kW ProLiant DL380 G7 (3.07 GHz, Intel Xeon X5675 processor), 2011, 3197 ops/watt
- 33. SSD efficiency Source: “Environmental Leadership in Storage Solutions” STEC Inc
- 34. Number of switch ports per link?
- 36. UPS partial load efficiency Efficiency vs Load % Load 36
- 37. select the most efficient heat rejection • Temperature ONTO equipment >25˚C • No humidity control • Fan speed control • Preferably air side economisation with adiabatic (outside air up to 35˚C, water supply, lots of space for louvres) • If not water side economisation
- 38. Example - IBM Blade Server NEBS Environment Air temperature: • Blade server on: 5°C to 40°C (41°F to 104°F). Altitude: -60 m to 1800 m (-197 ft to 6000 ft) • Blade server on: 5°C to 30°C (41°F to 86°F). Altitude: 1800 m to 4000 m (6000 ft to 13000 ft) Humidity: 8% to 85%
- 39. Fan speed control Other heat gains (lighting, UPS losses) 100 kW Resultant temperature rise 3.0 °C * IT equipment peak load 293 kW Specific heat of air 1.2 W/kg°C Mixed air temperature Assume 1kg air = 1m³ 33.0 °C Average load 262 kW Δ temperature 12 °C Air flow 18.2 m³/s Sensible cooling unit capacity 130 kW Number of units required 26 Cooling unit(s) total capacity 3380 kW Cooling unit(s) 100% air flow 234.7 m³/s Total cooling unit fan power 234.0 kW Fan energy/annum @ 100% 2050 MWhr Bypass air flow Average cooling load 362 kW 9.2 m³/s Reduced fan speed 12% Return temperature set point 36 °C Adjusted air flow 27.4 m³/s Δ temperature 11 °C Fan power @ reduced flow 0.4 kW Fan energy/annum @ reduced flow 3 MWhr Reduction in total energy/annum 2388 MWhr (assumes annualised cooling COP of 6) Supply temperature (fixed) 25 °C
- 40. Direct air side economisation - Facebook
- 41. Indirect air side economisation
- 43. Other 5 • EPO • Fire suppression (unless really warranted) • Raised floor? • Too many boxes • Assume that BREEAM/LEED will give an efficient data centre
- 46. Is a raised floor needed?
- 47. Boxes in boxes
- 48. BREEAM
- 49. Other 5 • Plan and prioritise • Metering • Fire prevention • Grounding and bonding • IST
- 51. Metering
- 52. Fire prevention
- 53. Grounding and bonding Source: TIA942 /BICSi 002
- 55. Thank you!
Editor's Notes
- #7: For this intended deployment the space and power is lost.