Going Modular? Lessons learned from 500 modular data center implementations
1 like645 views
The document discusses lessons learned from over 500 modular data center implementations around the world. Key lessons include: (1) Good modular design allows deferring up to 50% of electrical and mechanical capacity costs until needed, saving millions compared to retrofitting; (2) Plans should account for unpredictability over 10-30 years and support 3-5x power density growth; (3) No single cooling approach is optimal for all sizes and densities, requiring tailored solutions.
Going Modular? Lessons learned from 500 modular data center implementations
- 1. Going Modular? Lessons from 500 modular data center implementations around the world John M. Kundtz Data Center Services Manager IBM Global Services jkundtz@us.ibm.com © Copyright IBM Corp. 2011 All Rights Reserved Boston Seminar: Financial Services Infrastructure Management
- 2. Today’s Agenda • Background – what are we seeing • Definition of Modular Data Center • Lessons Learned • Summary • More Information
- 3. Change is accelerating … IT demand 82M Servers installed by 2013 20% Server workloads have been virtualized 650% Storage growth by 2012 <30% Disk storage is used effectively Cost pressures 75% Of CIOs anticipate a strongly centralized infrastructure in 5 years 2-3% Increase forecast for IT spending Need for flexibility 70¢ Of every $1 is spent on maintaining existing environment 71% Of data centers are > 7 years old 5-60% Of IT workloads may be cloud-enabled
- 4. Our research suggests that Clients can respond to these challenges in 4 ways. Future proof existing data center infrastructure Double IT capacity or reduce operational expenses by 50% Rationalize the data center infrastructure across the company Improve operational efficiencies while reducing operational expenses by 50% Flexible design to be responsive to change Pay as you grow by deferring 40-50% of capital and operational costs Integrated management of IT and data center operations Lower operational costs up to 20%
- 5. Capital cost gets 100% of the focus & are 15% of the total cost. Data center capital costs 60% costs from mechanical / electrical systems Shell 7% Mechanical 20% Fees 24% Fit-Up Costs 9% Instrumentation & Controls 4% Power 36% Source: IBM Estimates Operating costs 3- 5X capital costs CumulativeCostofOperations($Millions) Year 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 Energy Cost Staffing Bldg. Maint. & Mgmt. R E Tax $100 $150 $200 $250 $50 0 CumulativeCostofOperations($Millions) Year 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 Energy Cost Staffing Bldg. Maint. & Mgmt. R E Tax Energy Cost Staffing Bldg. Maint. & Mgmt. R E Tax $100 $150 $200 $250 $50 0 Major cost savings is the deferral of 40-50% of the electrical and mechanical capacity until it is required.
- 6. We have analyzed over 500 global modular data center design and builds in the last 3 years ... Custom data centers Scalable modular data center SCALABLE MODULAR DATA CENTER Enterprise modular data center data center ENTERPRISE MODULAR DATA CENTER PORTABLE MODULAR DATA CENTER HIGH DENSITY ZONE
- 7. Past industry definition of “Modular Data Center” • Modular data center = Shipping container • Modular = subsystems such as UPS systems, rack/row level cooling components • Prefabricated “modules” that provide the ability to easily add space, power and cooling More flexible definition of modular data center • Plan for unpredictability in requirements • Provide capacity when you need it • Upgrade with no disruption to operations • Maximize flexibility for future technology and computing models • Tailored to each client’s requirements, not a fixed architectural answer • Suitable for mid-size and enterprise clients Definitions of modular data centers …
- 8. 1. Data center projects often go off track … • No governance process to make decisions based on facts • Operating vs. capital costs are not a standard part of the design and evaluation process • The real estate, technology and operational management teams are not all equal players Decisions made on wants versus needs • Plans need to reflect unpredictability of business and technology over 10-30 year period • No clear or detailed statement of requirements • Clients take on the responsibility by themselves with minimal current data center skills or experience Unclear control over how decisions are made Not finding a trusted partner from concept to completion
- 9. Business challenge Server power density trends1: 20x Data Center challenge Support expected IT kW/rack power density 2. Plan to maximize capacity & flexibility for technology & computing models, investment is an upfront statement of requirements … 1. ASHRAE Publication: Datacomm Equipment Power and Cooling Applications, 2005, page 24 Source: Page 4, Figure 1.3, Best Practices for Datacom Facility Energy Efficiency Second Ed. (c) 2009. Kilowatts/rack 0 10 20 30 40 2000 2002 2004 2006 2008 2010 2012 2014 0% 5% 10% 15% 20% 25% 30% 35% 0 5 10 15 20 25 30 35 h Kilowatts/rack • Support 5 kw/rack and 32 kW/Rack for high density servers • Design for air cooling as primary; provide liquid cooling without disrupting operations • Able to support 2.5x power density growth by pre-plumbing infrastructure • Leverage IBM Rear Door Heat Exchanger as most cost-effective high density support
- 10. 3. Plan for unpredictability to manage the lifecycle of a data center. Dodge the next 2-3 retrofits by smart planning for unknowns … 0.0% 5.0% 10.0% 15.0% 20.0% 25.0% 30.0% 35.0% 40.0% 45.0% 0 5 10 15 20 25 30 35 h Based on IBM estimates from client engagements Today 50% of server density is < 5 kw/Rack or less 2020 60% of server density is > 10-20 kw/rack Kilowatts/rack 0.0% 10.0% 20.0% 30.0% 40.0% 50.0% 60.0% 0 5 10 15 20 25 30 35 h Racks loaded at 40% Racks at 90% Kilowatts/rack • Unknown loads. Create a density requirement model which plans for changes over time • Unknown capacity. Support 3-5x power density over the 10 year timeframe • Predicted load density is a pre-requisite to a better design • Design included efficient dual zone capacity ; saving $1M USD per year for client
- 11. 4. A Modular design allows for expansion of power & cooling density with minimal disruption to operations … 0 5 10 15 20 25 30 35 40 1x 3x Pay now Pay later • Size electrical and mechanical systems to meet future capacity needs • Provision for potential water cooling • Modular UPS design and chiller plant design • Electrical switchgear sized to grow. Electrical distribution scalable per module $(Millions) Power Density Growth (# times) Piping for water cooling Capital Costs 5-10% up front to support 3x growth versus 60% higher cost & disruptive retrofit later Good design is an insurance policy with a 5-10% premium versus paying 60% later.
- 12. 5. No single electrical and mechanical approach is cost effective for all sizes and densities … Example: 20,000 square foot data center, computer room air conditioners Capital Costs Capital costs for computer room air conditioners can vary by 5x Under-floor Cooling Combined Systems Close Coupled Cooling $2 1.5 1 .5 0 $Millions Operational Costs Operational cost for computer room air conditioners can vary by 140% 2 1.5 1 .5 0 Year 1 2 3 4 5 6 7 8 9 10 2.5 $3 $Millions Under-floor Cooling Combined Systems Close Coupled Close Coupled, RDHX • Although highly efficient, close coupled cooling doesn't always provide the lowest TCO. • Rear door heat exchanger is the lowest cost solution to support high density equipment. • Raised floor cooling is cost effective for low density environment. Source: IBM Estimates
- 13. In Summary: Lessons learned from 500 modular data center implementations include … • Good modular design is an insurance policy with a 5% premium versus 60% later. • Major cost savings is the deferral of electrical and mechanical capacity until it is required. • No single electrical and mechanical approach is cost effective for all sizes and power densities. Decisions made on wants versus needs • Insure your data center strategy can meet business and IT requirements over the long term. • Best investment is an upfront statement of requirements. • Plan for unpredictability. Dodge the next 2-3 retrofits by smart planning for unknowns. • There is not a one- size-fits-all strategy. • Find a partner who doesn’t use a fixed architectural approach. Not finding a trusted partner from concept to completion Unclear control over how decisions are made
- 14. Scalable modular data center. Enterprise modular data center. Portable modular data center. High density zone. • Turnkey center for 500-2,500 sq ft. • Implement in 8- 12 weeks. • Standardized design starting with modules as small as 5,000 sq ft. • Save up to 50% operational costs. • Rapidly deploy in 12-14 weeks. • Without impacting operations. Watch a video at :http://www-03.ibm.com/systems/data/flash/dynamicinfrastructure/datacenterdesignsolutions/ 15-25% lower TCO than traditional data centers Defer 40-50% of capex and opex cost Fully functional data center 35% lower cost than site retrofit IBM has designed over 500 modular data centers in the last 3 years to be responsive to change … Design new infrastructure for flexibility with modularity IBM’s data center familyTM solutions align to your business and cost objectives.
- 15. Portable modular data center (PMDC) Enterprise modular data center (EMDC) Scalable modular data center (SMDC) Modular data centers Data center consolidation and relocation Data center strategy Extend the life of an existing data center IBM site and facilities services For more information on “going modular”