07. datacenters
Download as PPTX, PDF5 likes3,791 views
Datacenters house most IT infrastructure hardware in racks and provide power, cooling, fire protection and equipment space. Today's large datacenters contain thousands of servers in shipping container sized blocks. Datacenters are categorized based on size and tenant usage. Proper location, physical structure, layout, power supply, cooling, fire protection and availability tiers are important considerations for datacenter design and operation. Physical security restricts access to protect equipment.
07. datacenters
- 1. IT Infrastructure Architecture Datacenters (chapter 7) Infrastructure Building Blocks and Concepts
- 2. Introduction • Most IT infrastructure hardware, except for end user devices, are hosted in datacenters • A datacenter provides: – Power supply – Cooling – Fire prevention and detection – Equipment racks
- 3. History • Early datacenters (computer rooms) were designed and built for large mainframe systems • A mainframe, together with its peripheral systems easily filled up a large computer room
- 4. Today’s datacenters • Today’s datacenters are equipped with standardized 19" racks – They house servers, storage devices, and network equipment – They are installed in rows forming corridors between them • Very large datacenters today contain shipping containers packed with thousands of servers each – When repairs or upgrades are needed, entire containers are replaced
- 6. Datacenter categories • Typical datacenter categories: – Sub Equipment Room (SER) – a SER is also known as a patch closet – Main Equipment Room (MER) – a MER is a small datacenter in the organization’s subsidiaries or buildings – Organization owned datacenter – a datacenter that contains all central IT equipment for the organization – Multi-tenant datacenter – used by service providers that provide services for multiple other organizations. These datacenters are typically the largest
- 7. Datacenter location • Many variables should be considered to determine where a datacenter could be installed • Examples: – Environment of the datacenter – Visibility of the datacenter – Utilities available to the datacenter – Datacenters located in foreign countries
- 8. Physical structure • The physical structure of a datacenter includes components that need special attention: – Floors – Walls – Windows – Doors – Water and gas pipes
- 9. Floors • In a typical datacenter, the floor must be able to carry 1500 to 2000 kg/m2 – One fully filled 19” computer rack weighs up to 700 kg – The footprint of a rack is about 60x100 cm, leading to a floor load of 1166 kg/m2 – In office buildings typically the floor can carry approximately 500 kg/m2
- 10. Floors • Raised floors consist of a metal framework carrying removable floor tiles • Tiles are usually 60×60 cm • Tiles can be lifted individually to reach cables installed under the raised floor • Vents provide cool air flow to the racks placed on the floor • Under the raised floor, data and power cables are installed – As alternative, overhead cable trays can be used
- 11. Floors • Raised floors have the following disadvantages: – They are expensive – The total available height in the datacenter is decreased – The maximum floor load is limited – Doors and equipment loading slopes are hard to install due to the difference in floor height – Under the raised floor, a fire could easily spread through the entire datacenter
- 12. Walls, windows, and doors • Walls should reach from the floor to the building’s ceiling – Because of fire safety and physical intrusion prevention – Adequate fire rating is needed to serve as a physical firewall • Windows are not desirable in a datacenter • Windows must be: – Translucent – Shatterproof – Impossible to open • Doors should be large enough to have equipment brought in • Doors must resist forced entry
- 13. Water and gas pipes • Water or gas pipes may have been installed: – Under the floor – In the walls – Above the ceiling of the datacenter • Leakage from water pipes in the ceiling of a datacenter could lead to damage of equipment • Datacenter operators should know where the shutoff valves are
- 15. Power supply • Energy usage is a key issue for datacenters • Power drawn by datacenters: – A few kilowatts (kW) for one rack of servers – Dozens of megawatts (MW) for large facilities • The amount of power available in a datacenter is expressed as the number of kilowatts per m2 – Known as power density • Normal-density datacentre: between 2 to 6 kW/m2 • High-density datacentre: between 10 and 20 kW/m2
- 16. Power supply • Example: – If a HP DL380 server uses 250W power – A rack could physically hold 40 servers – Available energy density is 2 to 6 kW/m2 – Only between 8 and 24 servers can be placed in the rack • In practice most server racks cannot be fully equipped • A high-density datacentre allows racks to be filled with approximately 40 to 80 servers
- 17. Uninterruptable Power Supply (UPS) • Power issues can occur in the utility power supply • Possibly leading to: – Downtime – Damage to equipment • Types of power issues: – Blackout – Surge – Spike – Brownout – Waveform issues • An Uninterruptable Power Supply (UPS): – Is independent of the utility power supply – Provides high quality electrical power
- 18. Uninterruptable Power Supply (UPS) • A UPS installation consists of: – Filters – A diesel power generator – A set of batteries or a flywheel system
- 19. Uninterruptable Power Supply (UPS) • Battery powered UPSs: – Standby UPS systems (also known as off-line systems) • Used in small setups (a few workstations or servers) – Line interactive UPS systems • Use a transformer between the utility power and the IT equipment • Works as a filter for many of the power issues – Double conversion UPS systems • Convert the AC utility power to DC power and then back to high quality AC power
- 20. Power distribution • A power distribution unit (PDU) is a device with multiple power outlets – Distributes power to equipment located in the datacenter • Two types of PDUs: – Large floor mounted PDUs take main feeds (usually 3 phase power) and distribute it into multiple smaller feeds to computer racks – Power Strips that feed equipment in racks • Most Infrastructure components can be equipped with two power supplies for redundancy • For availability reasons at least two power strips are needed to power equipment in a rack – Each feeds one of the two power supplies in the equipment
- 21. Uninterruptable Power Supply (UPS) • Two independent power feeds to racks • UPS should not be a Single Point of Failure
- 22. Cooling • > 90% of all power used by IT infrastructure components is converted into heat • All heat has to be dissipated by a cooling system • Two types of cooling systems: – Computer Room Air Conditioners (CRAC) • Refrigerant-based units connected to outside condensing units – Computer Room Air Handlers (CRAH) • Chilled water based and connected to outside chillers • A chiller produces chilled water via a refrigeration process
- 23. Cooling • The efficiency of a cooling system is specified in: – EER - Energy Efficiency Ratio • The measure of efficiency at maximum air conditioning load • The ratio between output cooling in BTU per hour and the electric energy input in Watts at a given operating point – SEER - Seasonal Energy Efficiency Ratio • Same as EER, but seasonal data is used for the measurement • The time of year the cooling system is used most (typically in the summer) – COP - Coefficient Of Performance • The ratio between cooling load in kW and the electric energy input in kW • Normal values are between 3 and 10
- 24. Operating temperatures • Infrastructure components have maximum operating temperatures – Servers shut themselves down at an air inlet temperature of 40 degrees Celsius • The air temperature in the datacenter usually ranges from 18 degrees to 27 degrees Celsius • Using higher temperatures saves cooling capacity and power – Raising the temperature in a datacenter with one degree Celsius lowers the cost for cooling by approximately 5%
- 25. Airflow
- 26. Humidity and dust • The humidity of the air in a datacenter is critical for the IT infrastructure components – Air humidity should between 40% and 60% • The number of dust particles in a datacenter should be minimized – Don’t allow visitors in the datacenter – People should wear dust-free clothing (like white coats) and protective sleeves around their shoes
- 27. Fire prevention, detection, and suppression • Fire is one of the main enemies of a datacenter – A short circuit in a cable – Defect equipment • Fires can spread around very quickly – Because of the air flow in the datacenter and the frequent use of raised floors • Smoke could damage equipment in the datacenter – Even if a fire starts outside of the datacenter’s computer room
- 28. Fire prevention, detection, and suppression • Suppressing fire in a datacenter consists of four levels: – Fire prevention – Avoid a fire – Passive fire protection – Limit the exposure of the fire once it has started – Fire detection systems – Detect smoke and fire – Fire suppression systems – Extinguish the fire once it is detected
- 29. Equipment racks • A 19” rack is a standardized metal enclosure to house IT infrastructure components • The height of a rack is measured in rack unit or 'U‘ – One U is 44.5 mm • A typical rack is 42U high
- 30. Datacenter energy efficiency • IT accounts for approximately 2% of all the world’s CO2 emissions • During the lifetime of a server the amount of money spent on electricity can be much higher than the cost of the server itself • The datacenter itself uses power as well – Cooling system – Lighting – Heating of the operator rooms – Etc.
- 31. Datacenter energy efficiency • The Power Usage Effectiveness (PUE) metric measures the power used by the datacenter • The PUE is calculated by dividing the amount of power used by the datacenter, by the power used to run the IT equipment in it • Typical PUE value of a datacenter is between 1.1 and 2.0 – Running a datacenter with a PUE of 1.5 means that for each watt of power used by the IT equipment an extra half watt is used by the rest of the datacenter
- 33. Availability tiers Tier Measures Expected downtime Tier 1 Availability 99.671% Type Basic Single path for power and cooling distribution No redundant components Downtime very likely for planned and unplanned maintenance Tier 2 Availability 99.741% Type Redundant components Fulfills all Tier 1 requirements Single path for power and cooling distribution Redundant components Downtime likely for planned and unplanned maintenance
- 34. Availability tiers Tier 3 Availability 99.982% Type Concurrently maintainable Fulfills all Tier 1 and Tier 2 requirements Multiple active power and cooling distribution paths Only one path active Redundant components All IT equipment must be dual-powered No downtime due to planned maintenance Downtime unlikely for unplanned maintenance Tier 4 Availability 99.995% Type Fault tolerant Fulfills all Tier 1, Tier 2, and Tier 3 requirements Multiple active power and cooling distribution paths Redundant components All cooling equipment is independently dual-powered, including chillers and Heating, Ventilating and Air Conditioning (HVAC) systems No downtime due to planned or unplanned maintenance
- 35. Availability tiers • The tier classification only describes the availability of the datacenter facilities – Not the availability of the IT infrastructure components • A tier 3 datacenter running an IT infrastructure with an availability of 99.990% will have a total availability of 0.99982 × 0.9990 = 0.99972 = 99.972%
- 36. Redundant datacenters • Multiple redundant datacenters can be used to increase availability • Multiple datacenters are a must when higher availability than 99.995% is needed – If a datacenter with all its equipment has an availability lower than tier 1, two datacenters can reach an availability of the same level as one tier 4 datacenter • Redundant datacenters should be at least 5 km apart – Based on the effect of incidents like the 9/11 terrorist attacks in the USA and reports of explosions in factory plants and fireworks storage
- 38. Datacenter performance • The datacenter itself does not provide performance to IT Infrastructures, except for the bandwidth of the internet connectivity and the scalability of the location
- 40. Physical security • Ensure that equipment is physically safe behind the datacenter doors • Physical access to the datacenter must be restricted to selected and qualified staff • An entry registration system should be used • A log should be maintained containing all staff entering and leaving the datacenter • Doors must be secured using conventional locks (for instance for dock loading doors) or electronic locks – Electronic locks should open only after proper authentication
- 41. Physical security • Entry points can be implemented as: – Regular doors – Mantraps • Staff is routed through a set of double doors that may be monitored by a guard – Revolving doors • Only one person at a time can enter the datacenter’s restricted area • Entries can be equipped with weighing scales to ensure only one person enters the restricted area