"Morphology of Modern Data Center Networks: Overview". Dinesh Dutt, Cumulus Networks
1 like2,450 views
The document discusses the evolution and challenges of modern data center networks, focusing on the shift from traditional L2-centric to IP-centric applications, which handle dynamic workloads and high capacities. It highlights the use of folded Clos networks for scalability and improved failure management, and emphasizes the need for automation and Linux-based solutions for network configuration and management. Key topics include network topology, routing protocols, and the importance of validation tools for connectivity in complex environments.
"Morphology of Modern Data Center Networks: Overview". Dinesh Dutt, Cumulus Networks
- 1. Data Center Topologies Morphology of Modern Data Center Networks Dinesh G Dutt | Chief Scientist
- 2. Dawn of the Modern Data Center Network 2.0 Routing Taming the Configuration Beast Agenda 10/1/13 2YAC 2013 (Pictures courtesy of Wikimedia, where not stated)
- 3. Dawn of the Modern Data Center Network 2.0 Routing Taming the Configuration Beast Agenda 10/1/13 3YAC 2013
- 4. Evolution of the Data Center Application 10/1/13 4YAC 2013 Traditional Enterprise Applications § L2-centric § Sensitive to network failures § Mostly static § VLANs § No Server Virtualization § Mostly North-South Lower Capacity 100s-few thousand endpoints Modern Data Center Applications § IP-centric § Workaround network failures § Dynamic § Clouds § Server Virtualization § Mostly East-West High Capacity Thousands to millions of endpoints
- 5. Challenges: § Large failure domain § Agg box failure § Unscalability of agg boxes § MAC/ARP § VLANs § Choke point for E-W § Complex § HA § Too many protocols § Many proprietary enhancements § Each vendor has their version of the same feature 10/1/13 5YAC 2013 Traditional Enterprise DC Network Design L3 L2 Access Aggregation Core VRRP VRRP STP/VTP/GVRP/UDLD ECMP STP/VTP/GVRP/UDLD
- 6. 10/1/13 6YAC 2013 Network's Function is to Serve the Application Needs
- 7. Dawn of the Modern Data Center Network 2.0 Routing Taming the Configuration Beast Agenda 10/1/13 7YAC 2013
- 8. Folded CLOS Network 10/1/13 8YAC 2013
- 9. § ECMP § IP fabric ubiquitous § Better Failure Handling § Predictable Latency § Simple Feature Set § Scalable § L2/L3 Boundary § ToR vs. EoR design Characteristics Of CLOS Network 10/1/13 9YAC 2013 LEAF SPINE
- 10. Calculating Network Size 10/1/13 10YAC 2013 TIER-1 TIER-2 TIER-3 2 Tier Fabric For smaller environments 3 Tier Fabric For large-sale environments Pods can be of dissimilar size LEAF SPINE
- 11. Calculating Network Size 10/1/13 11YAC 2013 2 Tier Fabric • #ports @ToR = (m*n)/2 • Max #ports @ToR= 2K with 64px10GE at Tor/Spine • Max #ports @ToR= 4608 with 96px10GE at Tor/Spine 3 Tier Fabric • #ports @ToR = (m*n*o)/4 • Max #ports @ToR= 65K with 64px10GE at Tor/Spine/Spine • Max #ports @ToR= 884K with 96x10GE at Tor/Spine/Spine m m n on
- 12. Oversubscription & Such 10/1/13 12YAC 2013 Number of servers: Number of uplinks Non-blocking after this first layer Using Trident and 40 servers per rack: Oversubscription is 2.5 Using Trident2 in same config: Oversubscription can be 1
- 13. Paganini Variations 10/1/13 13YAC 2013
- 14. Size Does Matter Fine grained failure domain Large boxes vs small boxes Interconnect link Scheduling Downtime Trying on new clothes Multi-vendor 10/1/13 14YAC 2013 Failure Analysis
- 15. Dawn of the Modern Data Center Network 2.0 Routing Taming the Configuration Beast Agenda 10/1/13 15YAC 2013 Picture courtesy Nanoer.com @flickr
- 16. What Protocol Link state (OSPF/ISIS) or BGP Managing IPv4/v6 Separate session/protocol or unified Multi-Vendor Support Deployment Experience 10/1/13 16YAC 2013 Questions That Affect Routing Protocol
- 17. Commonly deployed protocol within enterprises Simplify config: Only 2 area IDs, backbone and non-BB Unnumbered interfaces Run OSPFv3 also if you have IPv6 Route summarization possible, not desired due to non-optimal routing 10/1/13 17YAC 2013 OSPF Backbone area Area 0.0.0.1 Area 0.0.0.1
- 18. Simple up-down routing Use Private AS numbers Route summarization not possible Interface addresses only Single BGP session for v4/v6 or separate sessions 10/1/13 18YAC 2013 eBGP ASx ASx1 ASx2 ASxn ASy1 ASy1 ASy1 ASy1ASy ASy ASy ASy ASz ASz ASz ASz ASx3 ASx ASx1 ASx2 ASxnASx3
- 19. Simple up-down routing No IGP Eliminates AS number distraction Use of NH Self with RR Single-hop BGP peer, use interface address Single BGP session for v4/v6 or separate sessions 10/1/13 19YAC 2013 iBGP RR RR RR RR RR RR RR RR RR RR RR RR
- 20. VM VM VM Logical switch Great fit for modern data center apps Layer complex applications such as clouds as an overlay L2 as a service 10/1/13 20 Network Virtualization YAC 2013
- 21. Dawn of the Modern Data Center Network 2.0 Routing Taming the Configuration Beast Agenda 10/1/13 21YAC 2013
- 22. To err is human, to automate divine But traditional networking gear is a black box OS functions more like an embedded OS No programmable way to configure the box Primitive network management tool chain Vendor-specific 10/1/13 22YAC 2013 Automate Configuration
- 23. Turn Black box into White & use Linux as the network OS Why Linux ? Well established and open API Vibrant community fueling innovation Sophisticated management tool chain Excellent networking support Linux As The Network OS 10/1/13 23YAC 2013
- 24. Server management tools to manage networks Puppet, Chef, Ansible or in house Common Toolset 10/1/13 24YAC 2013
- 25. Verify connectivity is as per operator specified cabling plan User defined actions on topology check result § For example, routing adjacency is brought up only if physical connectivity check passes Example: § T1, port1 is connected to M1, port1 § T1, port2 is connected to M2, port1 § … § M1, port 3 is connected to S1, port1 § M1, port 4 is connected to S2, port1 … 10/1/13 25YAC 2013 Validating Physical Topology S2 M2 M1 T2 T1 M4 M3 T4 T3 S1
- 26. Graphviz: Network topology specified via DOT language Well understood graph modeling language Wide range of supported tools Open source Central management tool: Network topology is pushed out to all nodes Each node determines its relevant information LLDP: Use the discovery protocol to verify connectivity Graph G { S1:p1 – M1:p3; S1:p2 – M2:p3; S1:p3 – M3:p3; S1:p4 – M4:p3; S2:p1 – M1:p4; S2:p2 – M2:p4; S2:p3 – M3:p4; S2:p4 – M4:p4; M1:p1 – T1:p1; M1:p2 – T2:p2; … M4:p2 – T4:p2; } 10/1/13 26YAC 2013 ptmd: Prescriptive Topology Manager h*ps://github.com/CumulusNetworks/ptm
- 27. CLOS Fabric as the foundation for modern data center networks Layer Complex applications such as Clouds on top with overlays Automate Configuration & Simplify Networking Linux as the network OS to use sophisticated management tools Simplify networking further with tools such as ptmd 10/1/13 27YAC 2013 Conclusion