Open-IX Presentation: Datacenter Selection by Adam Rothschild
Download as PPTX, PDF1 like1,809 views
The document discusses the complexities of datacenter selection, emphasizing the importance of operational expenditure and careful planning to avoid downtime. It outlines characteristics of retail and wholesale datacenters and provides guidance on creating effective Request for Proposals (RFPs) for colocation services, including requirements for space, power, commercial terms, and communication. Additionally, it highlights the significance of certifications like OIX-2 in promoting transparency and a diverse interconnection ecosystem.
![CHARACTERISTICS OF
WHOLESALE DATACENTER
WHOLESALE
○ Power is metered and billed based upon consumption (KW/hr or mean KW/month).
○ Common increments of space are private cages or partitioned suites.
○ Limited providers on-site, but should connect to one or more dark fiber provider(s) with entrance
diversity. Tethering is common. May also offer extended cross-connect service to nearby MMR(s).
○ Cross-connection is not a profit center for the operator; commonly NRC to cover time & materials.
○ Access to remote hands may be limited. Clients typically provide their own operational staffing.
○ Greater options for power diversity (e.g. 2N[+n] designs). Some customization on power, cooling, and
rack designs to meet customer’s needs.](https://image.slidesharecdn.com/cluxgkktiiwyascc9yoz-signature-3804cae31282b42117cbee3f91dad89937106fb9863d3c7c06127df18a053177-poli-150417122655-conversion-gate01/85/Open-IX-Presentation-Datacenter-Selection-by-Adam-Rothschild-9-320.jpg)
![RFP: COMMUNICATIONS
WHOLESALE
○ “Provider ABC is on-net” is, itself, not a complete answer:
○ Develop list of on-site IP and telco providers you’re looking to purchase from.
○ Drill down to determine which building entrance[s] are utilized; underlying dark fiber providers and
routes are into the street. Also look for convergence points further away.
○ Vendor KMZ files overlayed into Google Earth are your friends. Some vendors have commissioned
reports to help with this planning, also.
○ Discuss relative strengths and weaknesses of aerial vs. buried fiber.](https://image.slidesharecdn.com/cluxgkktiiwyascc9yoz-signature-3804cae31282b42117cbee3f91dad89937106fb9863d3c7c06127df18a053177-poli-150417122655-conversion-gate01/85/Open-IX-Presentation-Datacenter-Selection-by-Adam-Rothschild-15-320.jpg)
![RFP: OIX-2 CERTIFICATION
WHOLESALE
○ My employer (Packet) asks this of all RFP respondents.
○ We view OIX-2 as a good indicator of transparency in operations, willingness to promote a diverse
carrier/interconnection ecosystem.
○ We understand that certifying every single site in a provider’s portfolio won’t happen overnight,
however [un]willingness to certify a particular site is a telling conversation.](https://image.slidesharecdn.com/cluxgkktiiwyascc9yoz-signature-3804cae31282b42117cbee3f91dad89937106fb9863d3c7c06127df18a053177-poli-150417122655-conversion-gate01/85/Open-IX-Presentation-Datacenter-Selection-by-Adam-Rothschild-16-320.jpg)
Open-IX Presentation: Datacenter Selection by Adam Rothschild
- 1. Datacenter Selection Adam Rothschild <asr@packet.net> Open-IX - Americas Interconnection Summit San Diego, CA - April 2015
- 2. DISCLAIMERS ○ Personal experiences and opinions only, not those of the Open-IX Association, working groups, or any employer. ○ Not intended as legal advice. Datacenter contracts are extremely complex and require review by counsel. ○ Datacenters presented are completely fictitious. Any resemblances, living or dead, are coincidental.
- 3. WHY DATACENTERS MATTER○ Significant operational expenditure, often exceeding cost of IP transit or interconnection. ○ Often selected by the “network person” or “vendor negotiator” (hence, this audience) ○ Can be a major source of downtime if planned poorly. ○ Power, Plumbing, and Ping is not an “attractive” topic like “cloud” or “SDN”, though it underpins it; therefore not often discussed around these parts. ○ Lack of best-practice sharing among operators. Not a germane topic for a NANOG, etc.
- 5. WHEN YOU’RE NOT ALIGNED: SANDY EDITION Force majeure, or poor site selection by the customer?
- 8. CHARACTERISTICS OF RETAIL DATACENTER OR MMR WHOLESALE ○ Power is billed at a fixed MRC, based on breakered (or capped) capacity. ○ Common increments of space are RUs, racks, or private cages. ○ Diverse meet-me room, with access to dozens of IP, lit services, dark fiber providers. ○ Cross-connection is a large profit center for the operator, and may carry significant MRC. ○ Access to 24 x7 remote hands for basic troubleshooting, rack & stack, managed services. ○ Limited options for power redundancy beyond N or N+1; so called “primary” and “redundant” power feeds may terminate on common infrastructure.
- 9. CHARACTERISTICS OF WHOLESALE DATACENTER WHOLESALE ○ Power is metered and billed based upon consumption (KW/hr or mean KW/month). ○ Common increments of space are private cages or partitioned suites. ○ Limited providers on-site, but should connect to one or more dark fiber provider(s) with entrance diversity. Tethering is common. May also offer extended cross-connect service to nearby MMR(s). ○ Cross-connection is not a profit center for the operator; commonly NRC to cover time & materials. ○ Access to remote hands may be limited. Clients typically provide their own operational staffing. ○ Greater options for power diversity (e.g. 2N[+n] designs). Some customization on power, cooling, and rack designs to meet customer’s needs.
- 10. SCOPE OF THIS TALK WHOLESALE ○ Most relevant to wholesale colocation, in the hundreds of KWs and larger. ○ Some elements are applicable to retail colocation, e.g. a cage, though you’re probably wasting your and your account rep’s time if you’re using this method to quote out a single cabinet. ○ Some elements are applicable to larger wholesale suites/pods in the MWs range, however further due diligence may be helpful. ○ We’re not discussing greenfield construction, though the TCO comparatives may help.
- 11. RFP: INTRO AND BASICS WHOLESALE ○ RFP should be short and sweet, while explaining your requirements. Should not exceed several pages of core requirements. ○ Should be conducive to vendor providing concise, binary, responses, without the hassle of flowery marketing narratives. ○ Should clearly state nature of customer’s business and use case; establish expectations for a response and go-live date. ○ Should clearly establish search radius, leave no room for interpretation as to whether or not a particular location is in-scope for consideration.
- 12. RFP: BUILD AND POWER WHOLESALE ○ Explain clearly what type of equipment racks or cabinets you need to place, and quantity. Floor plan and rack layout can be helpful as attachments. Establish provider’s “sweet spot” for density (W/ft^2 or KW/cabinet) so you can tweak your layouts appropriately. ○ Establish “day one” and future (12mo, 24mo, …) power requirements, in terms of both base commit and ceiling. Negotiate a ramp beneficial to both parties, if you’re not using all power initially. ○ Establish how power is metered and invoiced; base commitments vs. variable. ○ Establish mean Power Usage Effectiveness (PUE) for facility, and its impact on how power is billed. ○ Establish costs for installing incremental cabinets, NEMA receptacles.
- 13. RFP: COMMERCIAL TERMS WHOLESALE ○ Establish term length. Long terms (>= 5 years) ensure stability and cost predictability for the purchaser, as well as certainty for the vendor. ○ Establish YoY escalations, if applicable. Solidify language for additional pricing increases, e.g. notice period, commensurate to increased costs from power utility. ○ Establish “look back” period for incorrect billings. Ensure it’s bilateral; same period for billing disputes initiated by customer, and billing corrections issued by vendor…
- 14. RFP: CABLING WHOLESALE ○ Establish rates (NRC and MRC) for all common types of cross-connections: Cat5e, SMF; are increases permitted during the contract term? ○ Establish policies for running your own data connections inside your datacenter space, using your own materials and labor. ○ Establish facility’s policy on unionized labor. Is there any type of non-electrical work (e.g. bolting of cabinets, intra-suite data cabling) that you’re not allowed to perform by yourself? ○ Develop with the facility a mutually agreeable plan for wire tray placement, rack placement, cable plant, etc.
- 15. RFP: COMMUNICATIONS WHOLESALE ○ “Provider ABC is on-net” is, itself, not a complete answer: ○ Develop list of on-site IP and telco providers you’re looking to purchase from. ○ Drill down to determine which building entrance[s] are utilized; underlying dark fiber providers and routes are into the street. Also look for convergence points further away. ○ Vendor KMZ files overlayed into Google Earth are your friends. Some vendors have commissioned reports to help with this planning, also. ○ Discuss relative strengths and weaknesses of aerial vs. buried fiber.
- 16. RFP: OIX-2 CERTIFICATION WHOLESALE ○ My employer (Packet) asks this of all RFP respondents. ○ We view OIX-2 as a good indicator of transparency in operations, willingness to promote a diverse carrier/interconnection ecosystem. ○ We understand that certifying every single site in a provider’s portfolio won’t happen overnight, however [un]willingness to certify a particular site is a telling conversation.
- 17. RFP: OTHER IDEAS WHOLESALE ○ Review history of planned maintenance events for past 12+ months, and upcoming 3+ months. ○ Review history of unplanned site outages for past 24+ months. ○ Establish access methods/controls into the datacenter, and private cage or suite; multiple “factors”. ○ Review policy on CCTV placement, retention, customer access. Discuss placement of your own cameras, to augment the provider’s systems. ○ Confirm that temperature and RH will be maintained in accordance with (e.g. ASHRAE) norms. ○ Review SLA threshold and remedies for loss of power or cooling.
- 18. FLOOD MAPS Source: FEMA MSC
- 21. Documents are here: RFP: http://baremet.al/packet-dc- rfp TCO Model: http://baremet.al/packet-dc-tco Questions or comments: asr@packet.net