Simplifying OpenStack Networks with Routing on the Host: Gerard Chami + Scott Laffer, Cumulus Networks
1 like3,384 views
The document outlines strategies for simplifying OpenStack networks using Cumulus Networks' solutions. It discusses various ML2 type driver options, including VLAN and VXLAN configurations, and their implications for network architecture. The presentation emphasizes operational efficiencies and scalability through the integration of open networking and DevOps practices.
Simplifying OpenStack Networks with Routing on the Host: Gerard Chami + Scott Laffer, Cumulus Networks
- 1. v Simplifying OpenStack Networks with Routing on the Host Scott Laffer and Gerard Chami 5th of May, 2016
- 2. cumulusnetworks.com 2 • Overview • Who are we? • Why care about the plumbing? • What options are there? • Demo Agenda
- 3. Who are we? cumulusnetworks.com 3 Scott Laffer Technical Support Engineer slaffer@cumulusnetworks.com @slaffah Gerard Chami Technical Support Engineer gchami@cumulusnetworks.com @gerardchami
- 4. Transformation: First Servers, Now Networking cumulusnetworks.com 4 First: Compute Transformed LOCKED Now: Networking Transforms OPEN Open Networking Enables Platform Choice and Affordable Capacity cumulusnetworks.com Applications, OS and Hardware Open Ecosystem Agile, open, scalable with unprecedented cost savings 4
- 5. “NetDevOps” – using existing DevOps tools for networking Operational efficiencies, increased deployment speed OpenStack + Cumulus – Own the Rack with Linux cumulusnetworks.com 5 . Why OpenStack? . Why Cumulus Linux? Open source and associated ability to innovate No vendor lock-in Affordable Commoditized hardware Bridge the gap between your sysadmins and network engineers Treat your switch like a server OpEx and CapEx savings Disaggregated HW and SW Linux throughout your entire rack!
- 6. Why care about the plumbing? cumulusnetworks.com 6
- 7. ML2 Type Driver Choices – “The What” Flat Type Driver All subnets assigned are placed in the same Layer-2 broadcast domain. Commonly used for defining a single provider network (single pool of external IP addresses). cumulusnetworks.com 7 VLAN Type Driver Each OpenStack subnet is assigned to a different VLAN. Discussed in detail in the Cumulus OpenStack Validated Design Guide. VxLAN Type Driver Each OpenStack subnet is assigned to a different VxLAN. Looks similar to a typical Cumulus VxLAN design except VTEP can be in the host. Other Linux Bridge OvS VendorOther GRE VLAN VxLAN Core Plugin (ML2) Type Manager Type Driver Mechanism Manager Mechanism Driver
- 8. Other Linux Bridge OvS VendorOther GRE VLAN VxLAN ML2 Mechanism Driver Choices – “The How” Linux Bridge Provides Layer-2 and Layer-3 connectivity on a compute node using traditional bridging constructs. cumulusnetworks.com 8 OpenVswitch (OVS) Provides Layer-2 and Layer-3 connectivity on a compute node using networking stack that sits on top of the Linux Kernel. It does not use the Linux Kernel API. Cumulus Linux Instantiates/Destroys VLANs on a Cumulus Switch after a tenant network is created/deleted on the OpenStack Compute Nodes Core Plugin (ML2) Type Manager Type Driver Mechanism Manager Mechanism Driver ML2 Framework providing a way to configure L2/L3 connectivity on any networking platform such as the linux kernel (linuxbridge) or OpenVSwitch
- 9. Design 1: ML2 + VLAN: MLAG Between Host/Leaf and Leaf/Spine § Overall: A well known and common design using MLAG at the spine layer, MLAG at leaf layer, but least scalable and least flexible. An “old school” but proven network design. cumulusnetworks.com 9 § Considerations: § VLANs statically assigned but doesn’t scale well § STP heavy between Leaf/Spine and Leaf/Host § MLAG difficult to manage at scale. § Using Cumulus ML2 Mechanism driver to dynamically add/remove VLANs doesn’t make sense. How do you add/remove VLANs from spines consistently? § Cumulus “Stickiness”: § Better automation story § Better operational story • Common tools for operation switch and server § Validated Design Guide certified L2 ML2 Pair
- 10. Design 2: MLAG at Top-of-rack, IP Fabric Between Leaf/Spine Overall: Uses less MLAG, more Layer-3, VxLAN, and is therefore more scalable. Caveat: Utilizes third-party SDN overlays, which could add to overall complexity. cumulusnetworks.com 10 § Considerations: § Scales better than L2 + MLAG § SDN Overlays dynamically provision VxLAN on the switch • SDN overlay – Midokura, Nuage, PLUMgrid • Hierarchical Port Binding with Cumulus Mechanism Driver (alpha) § Future “Upsides”: § Scales better than L2 + MLAG § SDN Overlays dynamically provision VxLAN on the switch • SDN overlay – Midokura, Nuage, PLUMgrid • Hierarchical Port Binding with Cumulus Mechanism Driver (alpha) § Cumulus “Stickiness”: § Simple Layer-3 config for IP fabric § BGP/OSPF unnumbered § HPB + Cumulus ML2 in production ML2 Pair L2 L3 ECMP
- 11. Design 3– Layer-3 to the Host: Single Attach Overall: “Good Enough” for single links from hosts to switches, and recommended by Openstack.org Caveat: Not unique/novel - other networking vendors can accomplish this cumulusnetworks.com 11 § Considerations: § Application need to be distributed § Not recommended for those who believe in dual attaching host § VTEP on the host § VXLAN offload NICs recommended § Cumulus “Stickiness”: § Eliminated STP (Spanning Tree) L3 ECMP
- 12. L3 Design 4: Layer-3 to the Host: Multiple Attach (Quagga on the Host) § Overall: The best overall networking solution with OpenStack and Cumulus Networks in large configurations. 100% simple and flexible architecture with Layer-3 networking using Linux quagga package extendable to other software solutions. cumulusnetworks.com 12 § Considerations: § VXLAN offload network interfaces recommended § Succeeds in docker container environments § Supports more than two links from hosts to switches for load balancing § Cumulus “Stickiness”: § Simplified infrastructure config § Server/switch/rack mobility § Major reduction in IPv4 addressing § Requires Cumulus Quagga package ECMP ECMP
- 13. OpenStack Network Design Decision Tree cumulusnetworks.com 13 Tenant Separation method? VLAN Type Driver on host VxLAN Type Driver on host IP Fabric Between Leaf/Spine Number of Host to Switch Links? 1 host to switch link 2 or more host to switch links All L2/MLAG - Leaf/Spine MLAG, Host/Leaf MLAG, applies Cumulus Validated Design Guide All L3 - Assign L3 address on host interfaces. Unnumbered for IP fabric switch interfaces. All L3 - L3 BGP/OSPF unnumbered config all the way to the host. Install Linux Quagga package from Cumulus on each host. “past” “present” “future”
- 15. Not just a party trick… cumulusnetworks.com 15
- 16. © 2016 Cumulus Networks. Cumulus Networks, the Cumulus Networks Logo, and Cumulus Linux are trademarks or registered trademarks of Cumulus Networks, Inc. or its affiliates in the U.S. and other countries. Other names may be trademarks of their respective owners. The registered trademark Linux® is used pursuant to a sublicense from LMI, the exclusive licensee of Linus Torvalds, owner of the mark on a world-wide basis. § Thank You! cumulusnetworks.com 16 Unleashing the Power of Open Networking
- 18. Getting Started with the Validated Design Guide cumulusnetworks.com 18 § Detailed 54-page HOWTO deployment guide - two spine and four leaf switches § Install and configure switches and compute nodes § Closely mimics production architecture VMware® vSphere and Cumulus Networks® Validated Solution Guide Deploying VMware ® vSphere with Cumulus ® Linux ® Switches Big Data Hadoop and Cumulus Networks ® Validated Solution Guide Deploying Apache Hadoop with Cumulus ® Linux ® Switches 0 0 00 0 0 0 00 0 0 01 1 0 1 11 1 1 1 11 1 1 11 1
- 19. Demo: Cumulus VX "Rack-on-a-Laptop" Part I (L2 + MLAG) cumulusnetworks.com 19 VirtualBox Appliance Contains: § Two Cumulus VX leaf nodes + Two RDO compute nodes § Custom tenant creation and tear-down script § Command line input via any local Web browser § Cumulus ML2 mechanism driver enabled – create 1 or 2 tenants http://tinyurl.com/RackOnALaptop OpenStack Controller Compute Node (Nova) Network Node (Neutron) Dashboatd Node (Horizon) Compute Node 192.168.100.4/24192.168.100.3/24 192.168.100.2/24192.168.100.1/24 Mgmt Bridge Leaf 1 Leaf 2802.1q bond Virtual Experience Cumulus VX Virtual Experience Cumulus VX swp18 swp17 swp18 swp17 swp32s0 ens0p9 swp32s0 ens0p9 host1 host2
- 20. Demo: Cumulus VX "Rack-on-a-Laptop" Part II (L3 to the Host) cumulusnetworks.com 20 VirtualBox Appliance Contains: § One Cumulus VX spine node + Two Cumulus VX leaf nodes + Two RDO compute nodes + One Debian external router § Custom tenant creation and tear-down script § Command line input via any local Web browser § Quagga packages on each compute node for Layer-3 to the host with BGP unnumbered http://tinyurl.com/RackOnALaptop-2
- 21. OpenStack Network Design: Layer 2 vs. Layer 3 cumulusnetworks.com 21 VMVM bridge - <>bridge - <> subinterface taptap subinterface 802.1q trunk 802.1q trunk 802.1q bond VMVM bridge - <>bridge - <> subinterface taptap tap tap taptap subinterface vRouter L3 Agent DHCP AgentDHCP Agent 3 4 VXLAN –> Tunnel IP Server1 Network Node 172.16.1.1 172.16.1.2 192.168.40.2192.168.40.3/24 VM br-<random> br-<random> br-external TAP VXLAN-2061 eth0 eth0 eth0 eth0 swp1 swp8 swp47 vRouter VXLAN-2061 Mgmt Network 1 2 5 6 203.0.113.1/24 203.0.113.2/24 Layer 2 + VLAN Layer 3 + VXLAN