Introducing XRd Lightweight, Programmable and Containerized
- 1. Raja Kolagatla, Product Manager, Cisco Networking BRKSPG-1552 Lightweight, Programmable and Containerized Introducing XRd
- 2. Enter your personal notes here Questions? Use Cisco Webex App to chat with the speaker after the session Find this session in the Cisco Live Mobile App Click “Join the Discussion” Install the Webex App or go directly to the Webex space Enter messages/questions in the Webex space How Webex spaces will be moderated until February 24, 2023. 1 2 3 4 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public Cisco Webex App 2 BRKSPG-1552
- 3. © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public A Few Questions to Start With … • Are Service Providers interested in network virtualization, which functions and for what? • What has changed to make Virtual Routing more relevant and going mainstream? • SDN and NFV have been around for 10+ years. • To what degree does the Virtual Routing complement the Physical Routing? • The value of software routing functions being moved around at will. • Can Virtual Routing run on modern servers crank up enough throughput for SP use cases? 3 BRKSPG-1552
- 4. © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public Agenda • Introduction and Market Trends • Product Overview • XRd Use Cases and Deployment • Automation • Conclusion BRKSPG-1552 4
- 6. © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public How does NetFlix Open-Connect work? Open Connect is Netflix’s in-house content distribution network specifically built to deliver its TV shows and movies. 6 BRKSPG-1552
- 7. © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public Market and Architectural Evolution • Key Inflection Points in Networking • Emergence of edge computing • New edge applications and services • Localized content • New revenue creation Key Inflection Points and Telco Edge 7 • Telco Edge is Evolving • On-Premise • Far Edge • Near and Metro Edge • Public Cloud BRKSPG-1552 Calls for Dynamic Workload Placement Floating Edge
- 8. © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public Market and Architectural Evolution Dynamic work-load placement: low latency, high bandwidth, massive scale Near Edge and Metro Edge Far Edge High Perf. LAN or Wireless CRAN SDAN MEC Device to the Edge Edge to the Device Business Edge Subscriber Edge Content UPF 5GC End-User Apps IT Apps Web-scale or Public Clouds Typical Latency: 1ms 2-5ms 10-20ms >20ms Telco Cloud – SP Play Hyperscale Cloud Connectivity Infrastructure Tooling BRKSPG-1552 8
- 9. © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public Hybrid-Cloud Adoption Combination of on-premise and off-premise infrastructure and applications CSP Telco hybrid cloud environment: Any combination of CSP managed and HCP (Hyperscale cloud provider) managed infrastructure. Trend Drivers: • TCO Optimization • Open and Multi-vendor environments • Enterprise Focused Use cases *decides edge, private or public cloud deployment CSP Managed HCP Private APPs Telco + Enterprise HCP Public CSP Managed Infrastructure HCP or AAV Managed Transport Regional / Core Public Cloud CSP Managed APPs Telco + Enterprise HCP Public CSP Managed Infrastructure HCP or AAV Managed Transport Regional / Core Public Cloud On-Premise DC, Edge or Private Cloud BRKSPG-1552 9 Azure’s partnership with operators: “We meet operators where they are––on-premises, at the edge, and in the cloud” https://azure.microsoft.com/en-us/resources/future-of-mobile- networks-and-cloud-computing/
- 10. 10 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public BRKSPG-1552 Mechanical Popular pre 1900 Electromechanical Solid- state Relay Vacuum Tube Transistor Integrated Circuit 1800 1600 1700 Boolean Logic Binary Code 10 10 106 10 4 10 2 1 10-2 10 -4 10 -6 Moore’s Law 1965 10 8 Calculations per Second per $1,000 2010 2020 Era of Modern µProcessors Remember 1965 ”Moore’s Law” – Well, It Surely Does Ramble on … 2025 Source: Ray Kurzweil, "The Singularity Is Near: When Humans Transcend Biology", Page 67, The Duckworth Publishers 2009. Data points between 1600 and 1900, and after 2000 represent presenter’s perspective and benchmarked data. Popular Since 2017 Server µProcessor Xeon® Skylake 14 nm https://youtu.be/aLJ0XLeV3V4 erabit on sockets in PCIe Gen 3 Server µProcessor Popular Since 2021 Xeon® Icelake 10 nm https://youtu.be/ipQQmjzE_g0 sec erabiton sockets in PCIe Gen 4 … Just Released 2023 Xeon® Sapphire Rapids More to Come In 2024+ 7 nm Server µProcessor sec erabitson sockets in https://shorturl.at/fglL1 PCIe Gen 5 animated Technology Leadership Demonstrators (not products)
- 11. © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public https://youtu.be/aLJ0XLeV3V4 erabit on sockets in https://youtu.be/ipQQmjzE_g0 sec erabiton sockets in sec erabitson sockets in https://shorturl.at/fglL1 Factors contributing to the increased throughput on servers: • Increasing packet processing power (and less Watts) with better ISA* and lower nm*** • More transistors, more network optimized PCIe buffering • PCIe speeds doubling every 3-4 years New Moore’s law for NFV: Performance doubles every 3 years! Intel® Xeon® Skylake PCIe Gen 3.0 x16 lanes for 100 GbE* Intel® Xeon® Icelake PCIe Gen 4.0 x16 lanes for 200 GbE* Intel® Xeon® Sapphire Rapids PCIe Gen 5.0 x16 lanes for 400 GbE* *Terabit demo: 48x lanes per socket ** Instruction Set Architecture *** nano meter fab technologies BRKSPG-1552 11
- 12. © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public Advancements in Server hroughput … • x86 CPU evolution does not follow the Moore’s Law anymore, but network performance does ☺ • And there is a growing number of hardware accelerators to further boost performance …are accompanied by cost-performance improvements 12 BRKSPG-1552 CPU DPU FPGA/ASIC Flexible Efficient … 12.6 5.9 4.1 300 800 1362 0 200 400 600 800 1000 1200 1400 1600 0 2 4 6 8 10 12 14 Intel® Xeon® Cascade Lake 6252N Q2CY19 Intel® Xeon® Ice Lake 8351N Q2CY21 Intel® Xeon® Sapphire Rapids 8462Y+ Q1CY23 Gbps $/Gbps Cost per Gbps Max Throughput (Gbps) 12.6 8.1 6 300 588 934 0 100 200 300 400 500 600 700 800 900 1000 0 2 4 6 8 10 12 14 Intel® Xeon® Cascade Lake 6252N Q2CY19 Intel® Xeon® Ice Lake 8351N Q2CY21 Intel® Xeon® Sapphire Rapids 8462Y+ Q1CY23 Gbps $/Gbps Cost per Gbps Max Throughput (Gbps) Server Data Plane Cost-Performance Guesstimates Based on VPP performance on Intel® Xeon® Processor (single socket)
- 13. Product Overview
- 14. © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public BRKSPG-1552 14 XRd architecture evolution from XRv9K- working on VPP stack since a decade! XR Route processor Protocols Interface Mgr Route Information Base Data plane GD Plane VPP + DPDK Calvados Admin Plane XRv Linux Kernel WRL7 Hypervisor Virtual & PCIe Interfaces x86 processor x86 processor Host OS Host OS PIRD Linux Interface s XR Route processor SPP Protocols Interface Mgr Route Information Base XR Route processor Protocols Interface Mgr Route Information Base Data plane GD Plane VPP + DPDK SPP Linux Interfa ces PCI Interfac es XRv9K: LXCs within VM XRd: Control Plane & vRouter Containers 2015 ………......... 2022
- 15. © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public Glossary • Control plane: Cisco developed packages for core network functions (BGP, MPLS, etc.); Yocto packages for standard Linux tools and libraries(bash, python, tcpdump, etc.). • PIRD: Platform Independent Reference Data plane - provide a functional platform layer to enable the XR software packet path, designed to be used on virtual platforms. • GD Plane: General Data Plane • VPP: Vector Packet Processing • DPDK: Data Plane Development Kit that consists of libraries to accelerate packet processing workloads running on a wide variety of CPU architectures. 15 BRKSPG-1552
- 16. © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public • Two variants: Control Plane only and complete vRouter (Control Plane + Data Plane) • Lyndt a.k.a. XR7 based architecture (No Admin Plane) • Optimizations for low footprint core (vCPU) use cases • Throughput scales up with additional resources (vCPUs) • Boot time: About 90 seconds • Resource requirements: Typical 8G RAM, 2 CPUs (Min) and 7GB disk XRd Architecture XRd CP XRv9K GDPlane Low footprint optimizati ons XRd vRouter BRKSPG-1552 16
- 17. © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public Life of a Packet for XRd 17 BRKSPG-1552 DPDK Driver DPDK Driver VPP Interface Output GD Plane Ingress Forwarding And Features GD Plane Traffic Manager VPP Ethernet Interface Classification VPP RX Processing GD Plane Egress Forwarding And Features NIC NIC vRouter container
- 18. © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public Performance: ’s Gbps/core, erabit-level/2-socket system • ‘Realistic’ configuration uses L V N over SR-MPLS over ECMP VLANs with Egress HQoS • Latency: ~50 usec average through XRd • Ice Lake CPU @ 3.5 GHz turbo 11 25 47 14 31 59 2 0 10 20 30 40 50 60 70 353 800 1514 Throughput / Gbps Packet Size / Bytes 3.9 Mpps - Realistic 4.9 Mpps - Vanilla IPv4 BRKSPG-1552 18 With multi-core close to linear speedup, terabit level performance is becoming possible.
- 19. © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public Model Driven Manageability APIs at every layer for the stack 19 BRKSPG-1552 Network Infrastructure / Service Adaptation RIB, Label Manager, BFD, Interface and more Applications / Protocol Stack BGP, ISIS, OSPF, LDP, SR, L2 Protocols Management CLI, Netconf, SNMP, Syslog, SSH SAL APL NBI 3rd Party Agent + Telemetry OSS Network Infrastructure Layer / Service Adaptation Layer • Acts as the bridge between the Application Layer and the HW • Presents abstractions to the Application/Protocol Layer Application/Protocol Layer Provides APIs into the Routing Protocols (BGP, IGP, SR, etc.) Management/Presentation Layer Provides access to configure and manage the stack through Network config/oper DB: Yang Models, CLI. SL API Complexity & Performance
- 20. © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public Solution for HA on Public Cloud with Service Layer API Use-case for AWS 20 • AWS underlay doesn’t support broadcast packets, to run HSRP/VRRP to be run between active & standby XRd’s. • Service-Layer API– “Service-Layer” acts as client for BFD allowing us to add/modify BFD sessions directly WITHOUT any IOS-XR config. BFD session state is transient/ephemeral – tied to the client. • Real-time BFD events are then received over gRPC, allowing the SLAPI client to react to Peer-router unreachable/down events BRKSPG-1552
- 21. © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public Streaming telemetry Supported by virtue of XR7 21 BRKSPG-1552 XRd Control Plane Host OS Telemetry Agent YANG Model Telemetry Receiver Telemetry Configuration
- 23. © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public Cisco XRd Use Cases VIRTUAL ROUTE REFLECTOR / PCE Industry Leading Scale Up to 70M Paths 20M Routes 100 RR Groups CLOUD ROUTER Public Cloud Gateway Routes: Up to 100K# GRE Tunnels: ~400# BGP Sessions: 1000# SRv6$, SR-MPLS, GRE Overlay VIRTUAL PROVIDER EDGE Consistent architecture w/ ASR 9000 ’put: ~100Gbps Business VPNs, M’cast, Peering VIRTUAL CELL SITE ROUTER Lowest XR footprint ’put: ~30 Gbps# Routes: 20K VPNv4/V6 SRv6/SR-MPLS/SR-TE HQoS #Linearly scalable with the vCPU allocation $Dependent on underlay BRKSPG-1552 23 Light weight lab simulation CI/CD deployment Thorough XR Coverage Fast boot time: ~2 Min
- 24. © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public virtual Cell Site Router (vCSR) Deployment with XRd 24 • Key features for vCSR role • Transport: SR-MPLS, SRv6 • SRv6-TE, SR-TE • ISIS, BGP L3VPN (v4/v6) • TI-LFA • ACL • VRRP • HQoS • GRE, ECMP – Load-balancing BRKSPG-1552 Intel CPU + HW Accel NIC VFs (SR-IOV) Mid Haul GPS Multi-device connectivity Precision Clocking Port Fan-out, multiplexing Management & Orchestration Cloud Infra Automation NSO vCSR GNSS O-CU IP Network COTS Server Front Haul/ ePRI GMC O-DU
- 25. © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 25 Cloud Networking for Public Cloud BRKSPG-1552 • Key features for Cloud Router role • BFD, GRE Tunnelling • BGP VPNs (v4/v6) & ISIS • QoS for GRE Interfaces • ACLs • Load balancing – GTP, ECMP • PBR, ABF • SR-MPLS, SRv6 (if supported by underlay) VPC VPC Inter-subnet communication within VPC or inter-VPC communication happens via vRouter Distributed Routing simplify the routing architecture Traditional vRouter XRd vRouter EC2 Instance XRd vRouter ENI interface VPC
- 26. © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public IoT Cloud-ready Converged SDN Transport Cisco’s Cloud-ready Converged SDN Transport is extended to support public cloud infra and cloud service availability. Residential End User Use Cases * Access Direct Peering Direct Connect Enterprise & Private 5G SDWAN Cloud RAN MH, BH Access CSP Network Edge DC Edge DC Regional DC Regional DC Internet BRKSPG-2315 BRKSPG-1552 26
- 27. © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public XRd vRouter Role in Cloud-ready Converged SDN Transport Transport Automation & Service Assurance Unified XR experience across Physical and Virtual environments Cloud RAN xHaul D-RAN Small Cell / Low BW sites O-DU + vRouter On x86 CSR CSR O-CU O-DU vRouter vRouter Converged SDN Transport 27 • L3VPN, Anycast & BGP extensions to VPCs • vCSR integrated with vDU server for low bandwidth sites • End to End SR with Slicing • GRE tunnels are option in case the public cloud doesn’t support v6 • Automation and Service Assurance • vRouter functions: • XRd as CNF* • XRv9k as VNF** (if needed) BRKSPG-1552 27 * Containerized Network Function ** Virtualized Network Function
- 28. © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public BRKSPG-1552 28
- 29. © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public Telco Workload & Network Requirements • Telco workloads have strict requirements • Latency budgets, jitter • Seamless failover to ensure service continuity, and regulatory requirements are mandated by federal and/or state agencies. • Data persistency & Storage management • Multiple limits in cloud provider networks are imposed on customers based on traffic flows and virtual machine templates that aren’t fully aligned with Telco specific requirements. • Public cloud providers does not provide the level of visibility and control in their underlay network that Telco requires BRKSPG-1552 29
- 31. © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public XRd Host Requirements • Any linux distribution, with a recent ( .x) kernel and built from ‘official packages’ • cgroups version v1 • Drivers: vfio-pci or igb_uio • Host privileges • Container tooling: Docker (v18+) and Kubernetes (v1.22) • Supports any OCI-compliant tools withing K8S ecosystem • Some tweaks to host OS capabilities • Beyond vanilla Kubernetes: • AWS EKS (supported from 7.8.1onwards) BRKSPG-1552 31
- 32. © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public Host Check Script Walkthrough for setting up Host Environment: https://xrdocs.io/virtual-routing/tutorials/2022-08-22- setting-up-host-environment-to-run-xrd/ Publicly Available Host Check Script https://github.com/ios-xr/xrd- tools/blob/main/scripts/host-check BRKSPG-1552 32
- 33. © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public Helm chart snapshot for XRd • Helm: a package manager for Kubernetes • Specify details of your deployment in a YAML file • Pass the YAML file to helm for deployment: • helm install my-xrd xrd/xrd- vrouter -f myvalues.yaml • https://github.com/ios-xr/xrd- helm Specify PCI Interfaces Use a subset of cpus myvalues.yaml BRKSPG-1552 33
- 34. © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public XRd deployment workflows BRKSPG-1552 34 On Vanilla K8S On AWS EKS Detailed steps can be found at : https://xrdocs.io/virtual-routing/tutorials
- 35. Automation
- 36. © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public Day “- ” Day “ ” Day “ ” Day “N” Day “- ” Integration to IaaC and IaaS layers • Instance sizing • Helm charts • Templating via Infra Build tools (Terraform, Cloud Formation etc) Day “- ” Deployment and Activation • Infra Deployment by the Infra automation layer • CNF Deployment via Infra build tools • ZTP and Golden Config Day “- ” Fulfillment & Service provisioning • Provisioning • Inventory Management • Config Change Management • Run-rate Operations Day “- ” Service Assurance and CLA • Fault Management • Performance Management • Event Management & Closed Loop Actioning XRd Automation & Operationalization BRKSPG-1552 39
- 37. © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public VPC Availability Zone: AZ(a) Instance Provisioning driven by Terraform/CloudFormation • Manage XRd/XRv9k using CLI/YANG over SSH/Netconf/gRPC • Monitor using XR Telemetry • Provide a Unified dashboard for CloudOps (Terraform Plugin events, cloud-specific instance details and stats) XRd Automation & Operationalization with Terraform – AWS example CNC BRKSPG-1552 40
- 38. Wrap up!
- 39. © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public Industry Leading Control Plane 15+ years of advanced networking CP 250K+ Devices Deployed Across XR Common XR, Multiple form-factors - Runs Across VM/Containers; Multiple CP/DP Deployment Use Cases Lightweight and Fast Cloud Focus Docker, Kubernetes, Multus CNI Orchestration Versatile Optional RPMs, Low foot-print, speedy core and multi- core DP XRd Value Proposition BRKSPG-1552 42
- 40. © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public Continue your education!! 43 BRKSPG-1552 Related sessions Cloud-Ready Converged SDN Transport - BRKSPG-2315 Introduction to SRv6 uSID Technology - BRKSPG-2203 SRv6 and Cloud-Native: a Platform for Network Service Innovation - LTRSPG-2212 Meet the engineer in 1:1 meetings Try out the XRd in the Docker/K8S environment or AWS https://xrdocs.io/virtual-routing/ https://github.com/ios-xr/xrd- tools
- 41. © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public Complete your Session Survey • Please complete your session survey after each session. Your feedback is important. • Complete a minimum of 4 session surveys and the Overall Conference survey (open from Thursday) to receive your Cisco Live t-shirt. • All surveys can be taken in the Cisco Events Mobile App or by logging in to the Session Catalog and clicking the "Attendee Dashboard” at https://www.ciscolive.com/emea/learn/sessions/session-catalog.html 44 BRKSPG-1552
- 42. Thank you