![Background on SDN [5]
3
[5] "Software-Defined Networking: A Comprehensive Survey."](https://image.slidesharecdn.com/routeflowixps1-141103140633-conversion-gate02/85/RouteFlow-IXPs-3-320.jpg)
![Separating Routing From Routers
• Routing Control Platform (2004) [1]
• Path computation based on a consistent view of network state
• Controlled interactions between routing protocol layers
• Expressive specification of routing policies
6
Figure 1 – Routing Control Platforms (RCPs) [1]](https://image.slidesharecdn.com/routeflowixps1-141103140633-conversion-gate02/85/RouteFlow-IXPs-6-320.jpg)
![RouteFlow [2]
• Background
• Glue of IP routing stacks with OpenFlow
• Controller-centric hybrid IP networks
• Migration path to SDN (e.g., Cardigan)
• Benefits:
• Cheap network gear with minimal embedded software
• Provide interoperability with legacy network elements
• Avoid re-writing legacy protocols in a centralized fashion
• Innovation power to stakeholders
• network operators, service providers
11
[2] “Revisiting routing control platforms with the eyes and muscles of software-defined networking”](https://image.slidesharecdn.com/routeflowixps1-141103140633-conversion-gate02/85/RouteFlow-IXPs-11-320.jpg)
![RouteFlow: High Level Architecture
• Modular
• 3 components
• Hierarchical
• Distributed
• Multi-controller
• POX, Ryu, ODL
• Routing stacks
• Quagga, BIRD, XORP
12
Figure 2 – RouteFlow Architecture [2]](https://image.slidesharecdn.com/routeflowixps1-141103140633-conversion-gate02/85/RouteFlow-IXPs-12-320.jpg)
![IXPs
17
Figure 4 – Typical IXP Architecture [3]](https://image.slidesharecdn.com/routeflowixps1-141103140633-conversion-gate02/85/RouteFlow-IXPs-17-320.jpg)
![Anatomy of a large European IXP [3]
• A particular Internet ecosystem (2012)
• Agnostic of any tier structure (flatenning 400 members)
• Peer-Peer links (more than 50,000)
• High-densely connected (visible / invisible / cannot tell P-P links)
• Prefix exchange ratio
• Traffic matrix (10 PB of IP traffic daily)
• Misconceptions reassessment
• Tier-1 ISPs do not peer at IXP (they do)
• IXPs are not used for transit (they are)
• The number of peer-peer links on the internet is small (larger than
assumed)
• IXP links are mostly used for back-up (they are not)
18](https://image.slidesharecdn.com/routeflowixps1-141103140633-conversion-gate02/85/RouteFlow-IXPs-18-320.jpg)
![Cardigan [4]
• SDN-based distributed router in a live IXP
• Reduces operational complexity
• Hybrid SDN-IP network side-by-side
• New approach to the router abstraction model
• RouteFlow Vandervecken branch
19
Figure 5 – Cardigan initial 2-switch deployment [4]](https://image.slidesharecdn.com/routeflowixps1-141103140633-conversion-gate02/85/RouteFlow-IXPs-19-320.jpg)
![Cardigan 2.0
22
Figure 6 – Cardigan with MPLS design [4]](https://image.slidesharecdn.com/routeflowixps1-141103140633-conversion-gate02/85/RouteFlow-IXPs-22-320.jpg)
![Research & Open Innovation
• Open Innovation
• Google Summer of Code [ 2013, 2014, … new proposals]
• Github Community [ forks, pull requests ]
• Code Contributions [5K – 10K LOC, testing hours, bug reports ]
• Mailing Lists [ +200 Members, 160 Topics ]
• Community contributions & collaborations
• Web-based UI & Internet 2 HW pilot [C. Small, Indiana]
• Aggregated BGP Routing Service [C. Corrêa, Unirio]
• SNMP plugin [J. Stringer, Google]
• IPv6, MPLS, IXP [REANZ, Google]
• Optimal BGP best path reflection [R. Raszuk, NTT-MCL]
• Intra data center BGP routing [R. V. Rosa, Unicamp]
• MPLS LDP support [Vikram Dham]
• ….
28](https://image.slidesharecdn.com/routeflowixps1-141103140633-conversion-gate02/85/RouteFlow-IXPs-28-320.jpg)
![References
• [1] Nick Feamster, Hari Balakrishnan, Jennifer Rexford, Aman Shaikh, and Jacobus van der
Merwe. 2004. The case for separating routing from routers. In Proceedings of the ACM
SIGCOMM workshop on Future directions in network architecture (FDNA '04). ACM, New
York, NY, USA, 5-12.
• [2] Christian Esteve Rothenberg, Marcelo Ribeiro Nascimento, Marcos Rogerio Salvador,
Carlos Nilton Araujo Corrêa, Sidney Cunha de Lucena, and Robert Raszuk. 2012. Revisiting
routing control platforms with the eyes and muscles of software-defined networking. In
Proceedings of the first workshop on Hot topics in software defined networks (HotSDN '12).
ACM, New York, NY, USA, 13-18.
• [3] Bernhard Ager, Nikolaos Chatzis, Anja Feldmann, Nadi Sarrar, Steve Uhlig, and Walter
Willinger. 2012. Anatomy of a large european IXP. SIGCOMM Comput. Commun. Rev. 42, 4
(August 2012).
• [4] Jonathan Philip Stringer, Qiang Fu, Christopher Lorier, Richard Nelson, and Christian
Esteve Rothenberg. 2013. Cardigan: deploying a distributed routing fabric. In Proceedings of
the second ACM SIGCOMM workshop on Hot topics in software defined networking
(HotSDN '13). ACM, New York, NY, USA, 169-170
• [5] Diego Kreutz, Fernando M. V. Ramos, Paulo Verissimo, Christian Esteve Rothenberg,
Siamak Azodolmolky, Steve Uhlig. "Software-Defined Networking: A Comprehensive Survey."
To appear in Proceedings of the IEEE, 2015.
• https://github.com/SDN-Survey/latex/wiki
• http://arxiv.org/abs/1406.0440
31](https://image.slidesharecdn.com/routeflowixps1-141103140633-conversion-gate02/85/RouteFlow-IXPs-31-320.jpg)
![32
Thank you!
Questions?
• Raphael Vicente Rosa
• raphaelvrosa [at] gmail [dot] com
Ask and contribute!
routeflow-discuss@googlegroups.com
Get the code!
https://github.com/routeflow/RouteFlow/](https://image.slidesharecdn.com/routeflowixps1-141103140633-conversion-gate02/85/RouteFlow-IXPs-32-320.jpg)
RouteFlow & IXPs
- 1. ROUTEFLOW & SOFTWARE-DEFINED EXCHANGES Raphael Vicente Rosa Christian Esteve Rothenberg University of Campinas (UNICAMP), Brazil
- 2. Agenda • Background: • Software-Defined Networking (SDN) • Separating Routing from Routers (BGP-based pre-SDN) • RouteFlow: Software-Defined IP Routing • Basics • Cardigan deployment at live IXP (>1 year) • Software-Defined Exchanges • Research perspectives and open innovation 2
- 3. Background on SDN [5] 3 [5] "Software-Defined Networking: A Comprehensive Survey."
- 4. SDNAND OPEN SOURCEARE EVOLVINGAND ACCELERATING THE PATH OF STANDARDIZATION Components of the New Frontier
- 5. SDN & NFV: The Frontier of Networking Existing • CLIs • Closed Source • Vendor Lead • Classic Network Appliances New • APIs • Open Source • Customer Lead • Network Function Virtualization (NFV) Adapted from: Kyle Mestery, Next Generation Network Developer Skills
- 6. Separating Routing From Routers • Routing Control Platform (2004) [1] • Path computation based on a consistent view of network state • Controlled interactions between routing protocol layers • Expressive specification of routing policies 6 Figure 1 – Routing Control Platforms (RCPs) [1]
- 7. Redefinition of Inter-AS Routing • Inter-AS Routing Issues • Routing tables scalability (route entries) • BGP policy definitions (communities, filter tags) • Dynamic negotiation of network resources (e.g., bandwidth prices) • … • Complexity can be better managed • Simpler routing configuration • Network-wide traffic engineering • Better network diagnostics and troubleshooting • New interdomain routing protocols 7
- 8. RCP Challenges • Robustness • Consistent view: loop-free paths • Scalability • Session establishment, path computation, management • Convergence speed • Intra and Inter AS route computation • Configuration languages • Correctness and security • Troubleshooting and diagnostics • Detach control and data planes 8
- 9. RouteFlow: Basics Data Plane Control Plane
- 11. RouteFlow [2] • Background • Glue of IP routing stacks with OpenFlow • Controller-centric hybrid IP networks • Migration path to SDN (e.g., Cardigan) • Benefits: • Cheap network gear with minimal embedded software • Provide interoperability with legacy network elements • Avoid re-writing legacy protocols in a centralized fashion • Innovation power to stakeholders • network operators, service providers 11 [2] “Revisiting routing control platforms with the eyes and muscles of software-defined networking”
- 12. RouteFlow: High Level Architecture • Modular • 3 components • Hierarchical • Distributed • Multi-controller • POX, Ryu, ODL • Routing stacks • Quagga, BIRD, XORP 12 Figure 2 – RouteFlow Architecture [2]
- 13. RouteFlow 13 Figure 3 – RouteFlow General Architecture
- 14. RouteFlow: SDX for Layer 3 • RouteFlow – an approach to build BGP in the controller • Create a virtual model of the router in the controller • Forward BGP update packets to the virtual interface • Compute BGP route tables (RIB and FIB) based on virtual model • Translate FIB into Flow Tables based on real switch model • Install Flow Tables in the data plane • Build a SDN-based Router • vandevecken code fork on RouteFlow (higher performance, simpler, 1.3 capable) • Build peering fabric based on dynamic SDN policies • rather than static network constructs • Use SDN rules to scrub bad behavior* • Deal with Router hygiene issues (default deny instead of default allow). • Deal with “Capacity Theft” issues (as well as “noise” traffic) (Adapted from – Inder Monga NVS'14) 14
- 15. Abstracting SDX: what are the common functions? • A place where peering domains come together • Peering domains need not be physically connected • Supports establishment of inter-domain connectivity/routing • A broker of inter-domain and inter-SRP trust • Each SDX defines its rules of engagement • Enforcer of individual domain policies • Marketplace for Software Defined Services (SDS) can be built as a SDX overlay • For example, an application-specific exchange (a video marketplace) • Resources may be virtual or physical (Adapted from – Inder Monga NVS'14) 15
- 16. Internet eXchange Points (IXPs) • It's a network fabric that participants connect to • They advertise their networks • Any packet you place into the exchange finds the appropriate destination • It is not a: router, switch, layer 3 switch… • It's just a place you poke two things... • Advertisements for things you'll accept • and packets you want to get rid of 16
- 17. IXPs 17 Figure 4 – Typical IXP Architecture [3]
- 18. Anatomy of a large European IXP [3] • A particular Internet ecosystem (2012) • Agnostic of any tier structure (flatenning 400 members) • Peer-Peer links (more than 50,000) • High-densely connected (visible / invisible / cannot tell P-P links) • Prefix exchange ratio • Traffic matrix (10 PB of IP traffic daily) • Misconceptions reassessment • Tier-1 ISPs do not peer at IXP (they do) • IXPs are not used for transit (they are) • The number of peer-peer links on the internet is small (larger than assumed) • IXP links are mostly used for back-up (they are not) 18
- 19. Cardigan [4] • SDN-based distributed router in a live IXP • Reduces operational complexity • Hybrid SDN-IP network side-by-side • New approach to the router abstraction model • RouteFlow Vandervecken branch 19 Figure 5 – Cardigan initial 2-switch deployment [4]
- 20. Initial Limitations and Easy Fixes • Extensible message formats • RFProtocol flexibility • Set of matches, actions, options... • IPv6, MPLS on Ethernet • Inneficient gateway resolution • RFClient: Reception of its Netlink announcement and discovery of the associated gateway MAC address • Cache IPv4 and IPv6 routes until ARP resolutions • Scalable router abstraction • Router abstraction dependent on the physical topology • No arbitrary paths inside the ISP network • Traffic classification using VLAN tags per static inter-switch links (ISLs) configuration • Fine granular traffic control introducing MPLS paths 20
- 21. Cardigan 2.0 • MPLS Label Switched Path (LSP) • A prefix to a path (set of paths) • Ingress node through a set of transit nodes • Operator freely defines packet circuits over the network (arbitrary level of detail) • Forwarding Path Manager (FPM) component On RouteFlow architecture • Feeds all calculated routes (even if initially discarded) 21
- 22. Cardigan 2.0 22 Figure 6 – Cardigan with MPLS design [4]
- 23. Deployment • Pronto Switches (PicOs) - 1G SFPs • Out-of-band VM controller by layer 2 VLANs • Traffic forwarded directly by OpenFlow switches • In production for 9 months • 90 organizations - forwarding customer traffic and sharing routes • 1134 flows on each swicth (1028 layer 3 routes) 23
- 24. Discussion Itens (1/2) • Protocol compliance • OF 1.0 TTL decrement • MAC addressing • Scalability of flow tables • OF agent implementation • Vendor switches memory leaks and flow counters • Encapsulation Hazards • MTU size for Ethernet, VLAN, MPLS, etc 24
- 25. Discussion Itens (2/2) • Gateway Address Resolution - Increased performance • Separation of gateway resolution and route processing • Scalability • Distribution of the FIB accross multiple devices, different data plane technologies (NPU/FPGA) • Resilience • High-available non-stop forwarding solution and systematical SDN troubleshooting • Policy enforcements at IXPs – tedious tasks • Manual time-of-the-day routing, dynamic traffic engineering, route preferences, etc)’ 25
- 26. Related Work • IXP: an interesting networking landscape • SDX: A Software Defined Internet Exchange • Arpit Gupta (Georgia Institute of Technology), Laurent Vanbever (Princeton University), Muhammad Shahbaz (Georgia Institute of Technology), Sean P. Donovan (Georgia Institute of Technology), Brandon Schlinker (University of Southern California), Nick Feamster (Georgia Institute of Technology), Jennifer Rexford (Princeton University), Scott Shenker (UC Berkeley), Russ Clark (Georgia Institute of Technology), Ethan Katz-Bassett (University of Southern California) • ACM SIGCOMM, Chicago, IL. August 2014. • Enforcing RPKI-Based Routing Policy on the Data Plane at an Internet Exchange • HotSDN ’14 26
- 27. Future Work • Rethink peering between SDN domains • Strictly enforce routing and security policy • Prevent crafted BGP packets from exploiting policy • Prevent misdirected traffic • Simplify operations • Increase reliability 27
- 28. Research & Open Innovation • Open Innovation • Google Summer of Code [ 2013, 2014, … new proposals] • Github Community [ forks, pull requests ] • Code Contributions [5K – 10K LOC, testing hours, bug reports ] • Mailing Lists [ +200 Members, 160 Topics ] • Community contributions & collaborations • Web-based UI & Internet 2 HW pilot [C. Small, Indiana] • Aggregated BGP Routing Service [C. Corrêa, Unirio] • SNMP plugin [J. Stringer, Google] • IPv6, MPLS, IXP [REANZ, Google] • Optimal BGP best path reflection [R. Raszuk, NTT-MCL] • Intra data center BGP routing [R. V. Rosa, Unicamp] • MPLS LDP support [Vikram Dham] • …. 28
- 29. RouteFlow • Ongoing Work • OpenFlow hardware upfront: Pica8, NoviFlow, Corsa (OF 1.3) • Open Issues • Replicate data plane events in the virtual environment • Extensive evaluation on realistic networking conditions • OpenFlow table abstractions • Towards a routing control PaaS • IXP Perspective • Scale to large networks • Future implementation of policies • Load balancing, closest exit usage, complex setups https://github.com/routeflow/RouteFlow/wiki/Projects-and-ideas-for-RouteFlow 29
- 30. Summary • Routing Control Platforms possible through SDN • IXP is a particular microcosm of the Internet ecosystem • SDN may solve IXP challenges • Software Defined Exchanges • RouteFlow as a SDX layer 3 • Rethink peering between IP and SDN domains (and SDN-SDN) • More related work at below L3 and at application-layer • Cardigan RouteFlow pilot deployment is a real SDN- based legacy IXP implementation 30
- 31. References • [1] Nick Feamster, Hari Balakrishnan, Jennifer Rexford, Aman Shaikh, and Jacobus van der Merwe. 2004. The case for separating routing from routers. In Proceedings of the ACM SIGCOMM workshop on Future directions in network architecture (FDNA '04). ACM, New York, NY, USA, 5-12. • [2] Christian Esteve Rothenberg, Marcelo Ribeiro Nascimento, Marcos Rogerio Salvador, Carlos Nilton Araujo Corrêa, Sidney Cunha de Lucena, and Robert Raszuk. 2012. Revisiting routing control platforms with the eyes and muscles of software-defined networking. In Proceedings of the first workshop on Hot topics in software defined networks (HotSDN '12). ACM, New York, NY, USA, 13-18. • [3] Bernhard Ager, Nikolaos Chatzis, Anja Feldmann, Nadi Sarrar, Steve Uhlig, and Walter Willinger. 2012. Anatomy of a large european IXP. SIGCOMM Comput. Commun. Rev. 42, 4 (August 2012). • [4] Jonathan Philip Stringer, Qiang Fu, Christopher Lorier, Richard Nelson, and Christian Esteve Rothenberg. 2013. Cardigan: deploying a distributed routing fabric. In Proceedings of the second ACM SIGCOMM workshop on Hot topics in software defined networking (HotSDN '13). ACM, New York, NY, USA, 169-170 • [5] Diego Kreutz, Fernando M. V. Ramos, Paulo Verissimo, Christian Esteve Rothenberg, Siamak Azodolmolky, Steve Uhlig. "Software-Defined Networking: A Comprehensive Survey." To appear in Proceedings of the IEEE, 2015. • https://github.com/SDN-Survey/latex/wiki • http://arxiv.org/abs/1406.0440 31
- 32. 32 Thank you! Questions? • Raphael Vicente Rosa • raphaelvrosa [at] gmail [dot] com Ask and contribute! routeflow-discuss@googlegroups.com Get the code! https://github.com/routeflow/RouteFlow/
- 33. BACKUP 33
- 34. RouteFlow Project History 34 • Start Msc. Thesis work by Marcelo N. • First Prototype • QuagFlow Poster @ SIGCOMM • Open-Source Release • Demos @ ONS11 • Demo @ SuperComputing 11 • Tutorial & Demo @ OFELIA/CHANGE SS• First Short-Paper @ WPEIF • Evaluation on NetFPGA testbed • Indiana University - Pronto OF switches + BGP peering with Juniper MX • Demos @ ONS12 • Running on FIBRE / OFELIA testbed • HotSDN Paper • Collaboraion with NTT
- 35. RouteFlow 2013 / 2014 • Significant contributions from CARDIGAN and the VANDERWECKEN fork • Live in production network in New Zealand • Inter-VLAN routing at UNESP, Brazil. • Google Summer of Code (GSOC 2013 / 2014) • … 35