Vxlan control plane and routing

VxLAN Routing and Control Plane on Nexus
9000 Series Switches
• Lilian Quan – Technical Marketing Engineering, INSBU
• Chad Hintz – TSA, US Commercial

© 2014 Cisco and/or its affiliates. All rights reserved. Cisco Public
• Lukas Krattiger
• Victor Moreno
• Yves Louis
• Brenden Buresh
• Jason Gmitter
• Chad Hintz
• Errol Roberts
• Cesar Obediente
Contributors and Acknowledgements
 Leo Boulton
 Vaughn Suazo
 Dave Malik
 Lilian Quan
 Mike Herbert
 Juan Lage
 Jason Pfiefer
 Lilian Quan
 David Jansen
 Kevin Corbin
 Babi Seal
 James Christopher
 Jim Pisano
 Matt Smorto
 Priyam Reddy

Agenda
 VxLAN Overview
 MP-BGP EVPN Basics
 MP-BGP EVPN Control Plane
 VXLAN Design Options
 MP-BGP EVPN VXLAN Configuration
 VxLAN Capability on Nexus 9000
Series Switches

STP
VPC
Data Center “Fabric” Journey
MAN/WA
N
FabricPath
MAN/WA
N
FabricPath
/BGP
MAN/WA
N
VXLAN
/EVPN
VXLAN
ACI Fabric
Application Policy
Infrastructure
Controller
APIC
Application Centric Infrastructure

Trend: Flexible Data Center Fabrics
Hosts
V
M
O
S
V
M
O
S
Virtual
Physical
Create Virtual Networks on top
of an efficient IP network
Workload Mobility
Workload Placement
Segmentation
Scale
Automation & Programmability
L2 + L3 Connectivity
Physical + Virtual
Open

Why Do We Need Overlays?
Location and Identity Separation
IP core
Device IPv4 or IPv6
Address Represents
Identity and Location
Traditional Behaviour
Loc/ID “Overloaded” Semantic
10.1.0.1 When the Device Moves, It Gets
a New IPv4 or IPv6 Address for
Its New Identity and Location
20.2.0.9
Device IPv4 or IPv6
Address Represents
Identity Only.
When the Device Moves, Keeps
Its IPv4 or IPv6 Address.
It Has the Same Identity
Overlay Behaviour
Loc/ID “Split”IP core
1.1.1.1
2.2.2.2
Only the Location Changes
10.1.0.1
10.1.0.1
Its Location Is Here!

VTE
P
Local LAN Local LAN Local LAN Local LAN
IP Transport
Network
VTE
P
VTEP VTEP
VXLAN VNI
LAN Segment
Underlay Network:
• IP routing – proven, stable, scalable
• ECMP – utilize all available network paths
Overlay Network:
• Standards-based overlay
• Layer-2 extensibility and mobility
• Expanded Layer-2 name space
• Scalable network domain
• Multi-Tenancy

 No VXLAN control plane
 Data driven flood-&-learn
 Multicast transport for VXLAN BUM (Broadcast, Unknown Unicast and Multicast) traffic.
VTEP-1
End System
A
MAC-A
IP-A
VTEP
-3
End
System
EndSystem
VTEP-
2
End System B
MAC-B
IP-B
Multicast
Group
IP Network
VTEP 1
IP-1
VTEP 2
IP-2
VTEP
3
IP-3

Sound Familiar?

FabricPath
• Single address lookup at the ingress edge identifies the exit port
across the fabric
• Traffic is then switched using the shortest path available
• Reliable L2 and L3 connectivity any to any
(L2 as if it was within the same switch, no STP inside)
Shortest path any to any
FabricPath
A B
s3 s8
MAC IF
A e1/1
… …
B s8, e1/2
e1/1 e1/2

The Secret Sauce is the Control
Plane, not the Encapsulation

Agenda
 VxLAN Overview
 MP-BGP EVPN Basics
 MP-BGP EVPN Control Plane
 VxLAN Design Options
 MP-BGP EVPN VXLAN Configuration
 VxLAN Capability on Nexus 9000
Series Switches

MP-BGP with MPLS VPN Route Distribution
Exchange of VPN Policies Among PE Routers
• Full mesh of BGP sessions
among all PE routers
– BGP Route Reflector
• Multi-Protocol BGP extensions
(MP-iBGP) to carry VPN policies
• PE-CE routing options
– Static routes
– eBGP
– OSPF
– IS-IS
Label Switched Traffic
P
P
PE
PE
CE
PE-CE
Link
PE-CE
Link
P
P
CE
PE
PE
CE
CE
Blue VPN
Policy
BlueVPN
Policy`
Red VPN
Policy
Red VPN
Policy
BGP Route Reflector

VPN Control Plane Processing
VRF Parameters
Make customer routes unique:
• Route Distinguisher (RD):
8-byte field, VRF parameters; unique value to make VPN IP routes unique
• VPNv4 address: RD + VPN IP prefix
Selective distribute VPN routes:
• Route Target (RT): 8-byte field, VRF parameter, unique value to define the
import/export rules for VPNv4 routes
• MP-iBGP: advertises VPNv4 prefixes + labels

Blue VPN
VPN Control Plane Processing
Interactions Between VRF and BGP VPN Signaling
1. CE1 redistribute IPv4 route to PE1
via eBGP
2. PE1 allocates VPN label for prefix
learnt from CE1 to create unique
VPNv4 route
3. PE1 redistributes VPNv4 route into
MP-iBGP, it sets itself as a next
hop and relays VPN site routes to
PE2
4. PE2 receives VPNv4 route and, via
processing in local VRF (green), it
redistributes original IPv4 route to
CE2
BGP advertisement:
VPN-IPv4 Addr =
RD:16.1/16
BGP Next-Hop = PE1
Route Target = 100:1
Label=42
PPE1
eBGP:
16.1/16
IP
Subnet
P
CE1
PE2
CE2
eBGP:
16.1/16
IP Subnet
ip vrf blue-vpn
RD 1:100
route-target export 1:100
route-target import 1:100
VRF parameters:
Name = blue-vpn
RD = 1:100
Import Route-Target = 100:1
Export Route-Target = 100:1

EVPN – Ethernet VPN
VXLAN Evolution
18
Control-
Plane
EVPN MP-BGP
draft-ietf-l2vpn-evpn
Data-
Plane
Multi-Protocol Label Switching
(MPLS)
draft-ietf-l2vpn-evpn
Provider Backbone Bridges
(PBB)
draft-ietf-l2vpn-pbb-evpn
Network Virtualization Overlay
(NVO)
draft-sd-l2vpn-evpn-overlay
 EVPN over NVO Tunnels (VXLAN, NVGRE, MPLSoE) for
Data Center Fabric encapsulations
 Provides Layer-2 and Layer-3 Overlays over simple IP
Networks

Ethernet VPN
Highlights
• Next generation solution for Ethernet
multipoint connectivity services
– Leverage similarities with L3VPN
• PEs run Multi-Protocol BGP to advertise &
learn MAC addresses over Core
• Learning on PE Access Circuits via data-
plane transparent learning
• No pseudowire full-mesh required
– Unicast: use MP2P tunnels
– Multicast: use ingress replication over MP2P tunnels
or use LSM
• Under standardization at IETF – draft-ietf-
l2vpn-evpn
MPLS
PE1
CE1
PE2
PE3
CE3
PE4
VID 100
SMAC: M1
DMAC: F.F.F
BGP MAC adv. Route
E-VPN NLRI
MAC M1 via PE1
Data-plane address
learning from Access
Control-plane address
advertisement / learning
over Core

EVPN
• Multi-Protocol BGP (MP-BGP)
based Control-Plane using EVPN
NLRI (Network Layer Reachability
Information)
• Make Forwarding decisions at
VTEPs for Layer-2 (MAC) and
Layer-3 (IP)
• Discovery: BGP, using MPLS
VPN mechanisms (RT)
• Signaling: BGP
• Learning: Control plane (BGP)
PE
PE
CE2
CE1
PE
PE
CE4
CE3
Emulated Virtual Switch
BGP RR
BGP advertisement:
L2VPN/EVPN Addr = CE1.MAC
BGP Next-Hop = PE1
Route Target = 100:1
Label=42
20

• Host Route Distribution decoupled
from the Underlay protocol
• Use MultiProtocol-BGP (MP-BGP)
on the Leaf nodes to distribute
internal Host/Subnet Routes and
external reachability information
• Route-Reflectors deployed for
scaling purposes
Host and Subnet Route Distribution
VXLAN/EVPN
21
RR RR
V2
V1
V3
BGP Route-ReflectorRR
iBGP Adjacency

Overlay services
– Layer-2
– Layer-3
– Layer-2 + Layer-3
Tunnel
Encapsulation
Underlay transport
network
• Peer discovery mechanism • Overlay L2/L3 Unicast traffic
• Route learning and distribution
mechanism
– Local learning
– Remote learning
Control Plane
• Overlay Broadcast, Unknown
(Layer-2) traffic, Multicast traffic
(BUM traffic) forwarding
Data Plane

FCS
Outer
Mac Header
Outer
IP Header
UDP Header
VXLAN
Header
Original L2 Frame
FCS
8 Bytes
20 Bytes
IPHeader
MiscData
Protocol
0x11
Header
Checksum
Outer
Src.IP
Outer
Dst.IP
72 8 16 32 32
8 Bytes
UDP
Src.Port
UDP
DstPort
UDPLength
Checksu
m
0x0000
16 16 16 16
VXLAN
RRRR1RRR
Reserved
VNID
Reserved
8 24 24 8
10 or 14 Bytes
Dst.
MACAddr.
Src.
MACAddr.
VLANType
0x8100
VLANID
Tag
EtherType
0x0800
48 48 16 16 16
Source and Destination
VTEP addresses,
allowing transport across
the underlay IP network
Allows for possible
16M segments
The well known VXLAN
port 4789. Indicates a
VXLAN packet
Hash of the internal L2/L3/L4 header
of the original frame. Can be used as
entropy for better ECMP/LACP load
sharing
For next-hop transport in
the underlay network

• IP routed Network
• Support any routing protocols --- OSFP, EIGRP, IS-IS, BGP, etc.
IP Transport Network
• Flexible topologies
• Recommend a network with redundant paths using ECMP for load sharing
• Multicast is needed if using multicast for overlay BUM replication and transport

DC Core
DC Aggregation
DC Access
3-Tier Design
DC Core/ Aggregation
DC Access
Collapsed Core/Aggregation
2-Tier Design
DC Spine
DC Leaf
Fabric Design
DC-1 DC-2
WAN
DC Interconnect

VXLAN terminates its tunnels on VTEPs (Virtual Tunnel End Point).
Each VTEP has two interfaces, one is to provide bridging function for local hosts, the
other has an IP identification in the core network for VXLAN
encapsulation/decapsulation.
Local LAN Segment
IP Interface
End SystemEnd System
VTEP
Transport IP Network
Local LAN Segment
IP Interface
End SystemEnd System
VTEP

Flood-&-Learn EVPN Control Plane
Overlay Services L2+L3 L2+L3
Underlay Network IP network with ECMP IP network with ECMP
Encapsulation MAC in UDP MAC in UDP
Peer Discovery Data-driven flood-&-learn MP-BGP
Peer Authentication Not available MP-BGP
Host Route Learning Local hosts: Data-driven flood-&-learn
Remote hosts: Data-driven flood-&-learn
Local Host: Data-driven
Remote host: MP-BGP
Host Route Distribution No route distribution. MP-BGP
L2/L3 Unicast Forwarding Unicast encap Unicast encap
BUM Traffic forwarding Multicast replication
Unicast/Ingress replication
Multicast replication
Unicast/Ingress replication

MP-BGP for EVPN
 MP-BGP is the routing protocol for EVPN
 Multi-tenancy construct using VRF (Rout
Distinguisher, Route Targets)
 New address-family “l2vpn evpn” for distributing
EVPN routes
 EVPN routes = [MAC] + [IP]
 iBGP or eBGP support
vrf context evpn-tenant-1
vni 39000
rd auto
address-family ipv4 unicast
route-target both auto
route-target both auto evpn
evpn
vni 20000 l2
rd auto
route-target import auto
route-target export auto
router bgp 100
router-id 10.1.1.11
log-neighbor-changes
address-family l2vpn evpn
neighbor 10.1.1.1 remote-as 100
update-source loopback0
send-community extended
vrf evpn-tenant-1
advertise l2vpn evpn

VXLAN EVPN Control Plane Functions in Bronte Release
 Host MAC/IP advertisements through MP-BGP
 VTEP Peer Auto-discovery and Authentication via MP-BGP
 Anycast IP gateway
 ARP Suppression
 Ingress Replication with Head-end Auto-discovery (planned for
Bronte+)

 Use MP-BGP with EVPN Address Family on VTEPs to distribute internal
host MAC/IP addresses, subnet routes and external reachability information
 MP-BGP enhancements to carry up to 100s of thousands of routes with
reduced convergence time
EVPN Control Plane -- Host and Subnet Route Distribution
BGP Update
• Host-MAC
• Host-IP
• Internal IP Subnet
• External Prefixes
MP-BGP for VXLAN EVPN Control PlaneEVPN Control Plane – Reachability Distribution
LeafVTEPVTEPVTEPVTEP
Spine

Host
IP
VNI VTEP
C-1 H-IP-1 VNII-1 VTEP-1
VTEP-2
VTEP-1
VTEP-3
H-MAC-1
H-IP-1
VLAN-1 /VNI-1
BGP Update:
H-MAC-1
H-IP-1
VTEP-1
VNI-1
BGP Update:
H-MAC-1
H-IP-1
VTEP-1
VNI-1
BGP Update:
H-MAC-1
H-IP-1
VTEP-1
VNI-1
Install host info to RIB/FIB:
H-MAC-1  MAC table
H-IP-1  VRF IP host table
Route
Reflector
Install host info to RIB/FIB:
H-MAC-1  MAC table
H-IP-1  VRF IP host table
Local learning of host info:
H-MAC-1 (MAC table)
H-IP-1 (VRF IP host table )
1
2
3
3
4
4
MAC Host
IP
VNI VTEP
H-MAC-1 H-IP-1 VNII-1 VTEP-1
MAC Host
IP
VNI VTEP
H-MAC-1 H-IP-1 VNII-1 VTEP-1

1. VTEP-1 detects Host1 and advertise an EVPN route for Host1 with seq# 0
2. Host1 Moves behind VTEP-3
3. VTEP-3 detects Host1 and advertises an EVPN route for Host1 with seq #1
4. VTEP-1 sees more recent route and withdraws its advertisement
MAC IP VNI Next-Hop Encap Seq
MAC-1 IP-1 5000 VTEP-1 VXLAN 0
MAC IP VNI Next-Hop Encap Seq
MAC-1 IP-1 5000 VTEP-3 VXLAN 1
VXLAN BGP Control Plane
Host 1
MAC1
IP 1
VNI 5000
EVPN Control Plane --- Host Movement
VTEP-4VTEP-3VTEP-2VTEP-1
NLRI:
• Host MAC1, IP1
• NVE IP 1
• VNI 5000
• Next-Hop: VTEP-3
Ext. Community:
• Encapsulation: VXLAN
• Cost/Sequence: 1
NLRI:
• Host MAC1, IP1
• NVE IP 1
• VNI 5000
• Next-Hop: VTEP-1
Ext. Community:
• Encapsulation: VXLAN
• Cost/Sequence: 0

Host 1
MAC1
IP 1
VLAN A
VXLAN A
VTEPVTEPVTEPVTEP
SVI
GW IP
GW MAC
# VLAN to VNI mapping
vlan 200
vn-segment 5200
# Anycast Gateway MAC, identically configured on all VTEPs
fabric forwarding anycast-gateway-mac 0002.0002.0002
# Distributed IP Anycast Gateway (SVI)
# Gateway IP address needs to be identically configured on all
VTEPs
interface vlan 200
no shutdown
vrf member Tenant-A
ip address 20.0.0.1/24
fabric forwarding mode anycast-gateway
SVI
GW IP
GW MAC
SVI
GW IP
GW MAC
SVI
GW IP
GW MAC
Host 2
MAC2
IP 2
VLAN A
VXLAN A
Host 3
MAC3
IP 3
VLAN A
VXLAN A
Host 4
MAC4
IP 4
VLAN A
VXLAN A
The same anycast gateway virtual IP
address and MAC address need to
be configured on all VTEPs in the
VNI

ARP Suppression in MP-BGP EVPN
38
Host-1 in VLAN 10 sends an
ARP request for Host-2’s IP-2
address.
1
Host 1
MAC1
IP 1
VLAN 10
VXLAN 5000
Host 1
MAC1
IP 1
VLAN 10
VXLAN 5000
VTEP
1
VTEP
2
VTEP
3
VTEP
4
VTEP-1 intercepts the ARP request and checks in its
ARP suppression cache. It finds a match for IP-2 in
VLAN 10 in its ARP suppression cache.*
2
IP Address MAC Address VLAN Physical Interface
Index (ifindex)
Flags
IP-1 MAC-1 10 E1/1 Local
IP-2 MAC-2 10 Null Remote
IP-3 MAC-3 10 Null Remote
VTEP-1 sends an ARP response back to Host-1 with
MAC-2.*
3
Host-1 learns the IP-2 and MAC-2 mapping.
4
* If VTEP-1 doesn’t have a match for IP-2 in its ARP suppression cache table, it will flood the ARP request to all other VTEPs in this VNI
ARP suppression reduces network flooding due to host learning

interface nve1
no shutdown
source-interface loopback0
host-reachability protocol bgp
member vni 20000
suppress-arp
mcast-group 239.1.1.1
member vni 21000
suppress-arp
member vni 39000 associate-vrf
ARP Suppression in MP-BGP EVPN (Cont’ed)
• ARP Suppression can be enabled on
a per-VNI basis under the interface
nve1 configuration.
VTEP
1
VTEP
2
VTEP
3
VTEP
4
n9396-vtep-1.sakommu-lab.com# sh ip arp suppression topo-info
ARP L2RIB Topology information
Topo-id ARP-suppression mode
100 L2 ARP Suppression
200 L2/L3 ARP Suppression
201 L2/L3 ARP Suppression

Head-end Replication
40
Leaf
Spine
VTEP
1
VTEP
2
VTEP
3
VTEP
4
Host-1 sends BUM
traffic into the
VXLAN VNI
1
VTEP-1 receives the overlay BUM traffic,
encapsulates the packets into unicast VXLAN
packets, sends one copy to each remote VTEP
peer in the same VXLAN VNI
2
Multicast-Free
Underlay
Head-end Replication (aka. Ingress replication):
Eliminate the need for underlay multicast to transport overlay BUM
traffic

Different integrated Route/Bridge (IRB) Modes
• Overlay Networks do follow two
slightly different integrated
Route/Bridge (IRB) semantics
• Asymmetric
– Uses different “path” from Source to
Destination and back
• Symmetric
– Uses same “path” from Source to
Destination and back
• Cisco follows Symmetric IRB
VXLAN Routing
Host 1
H-MAC-1
H-IP-1
VNI-A
Host 2
H-MAC-2
H-IP-2
VNI-B
SVI
A
SVI
B
IP Transport Network
Routing ?

Host 1
H-MAC-1
H-IP-1
VNI-A
Host 2
H-MAC-2
H-IP-2
VNI-B
VNI
A
VNI
B
VNI
B
VNI
A
S-MAC: H-MAC-1
D-MAC: H-MAC-2
S-IP: H-IP-1
D-IP: H-IP-2
Ingress VTEP
routes packets
from source VNI to
destination VNI. D-
MAC in the inner
header is the
destination host
MAC
1
S-MAC: H-MAC-1
D-MAC: H-MAC-2
S-IP: H-IP-1
D-IP: H-IP-2
S-IP: VTEP-1
D-IP: VTEP-4
VNI: VNI-B
S-MAC: H-MAC-1
D-MAC: H-MAC-2
S-IP: H-IP-1
D-IP: H-IP-2
Egress VTEP
bridges packets in
the destination
VNI
2
Asymmetric
• Routing and Bridging on the ingress VTEP
• Bridging on the egress VTEP
• Both source and destination VNIs need to reside on the ingress VTEP

1. All VTEPs in a VNI can be the virtual IP gateway for the local hosts
2. Optimized south-north bound forwarding for routed traffic without hair-pinning
VXLAN BGP Control PlaneVTEP VNI Membership Asymmetric IRB
Host 1
MAC1
IP 1
VLAN 100
VXLAN 5100
Host 2
MAC2
IP 2
VLAN 100
VXLAN 5100
SVI 100
Host 3
MAC3
IP 3
VLAN 200
VXLAN 5200
VTEP
VTEPVTEPVTEP
SVI 200SVI 200 SVI 100 SVI 200SVI 100
SVI 200SVI 100
Every VTEP needs to be in all
VNIs
Every VTEP needs to maintain
MAC tables for all VNIs, including
those they don’t have local hosts
for.

• Routing on both ingress and egress VTEPs
• Layer-3 VNI
• Tenant VPN indicator
• One per tenant VRF
• VTEP Router MAC
• Ingress VTEP routes packets onto the Layer-3 VNI
• Egress VTEP routes packets to the destination Layer-2 VNI

VTEP VTEP VTEP VTEP
Layer-3 VNI
(VRF VNI)
Layer-2 VNI
(Network VNI)
Layer-2 VNI
(Network VNI)
vlan 200
vn-segment 20000
vlan 201
vn-segment 20100
vlan 3900
name l3-vni-vlan-for-tenant-1
vn-segment 39000
interface Vlan3900
description l3-vni-for-tenant-1-routing
no shutdown
vrf member evpn-tenant-1
vni 39000
rd auto
interface Vlan200
no shutdown
interface Vlan201
no shutdown

Host 1
H-MAC-1
H-IP-1
VNI-A
VTEP-4
Router MAC-4
VTEP-3VTEP-2
VTEP-1
Router MAC-1
Host 2
H-MAC-2
H-IP-2
VNI-B
VNI
A
L3
VNI
VNI
B
L3
VNI
S-MAC: H-MAC-1
D-MAC: H-MAC-2
S-IP: H-IP-1
D-IP: H-IP-2
Ingress VTEP
routes packets
from source VNI to
L3 VNI. D-MAC in
the inner header is
the egress VTEP
router MAC
1
S-MAC: Router-MAC-1
D-MAC: Router-MAC-4
S-IP: H-IP-1
D-IP: H-IP-2
S-IP: VTEP-1
D-IP: VTEP-4
VNI: L3 VNI
S-MAC: H-MAC-1
D-MAC: H-MAC-2
S-IP: H-IP-1
D-IP: H-IP-2
Egress VTEP
routes packets
from L3 VNI to the
destination
VNI/VLAN2

1. Optimal utilization of ARP and MAC tables
2. A VTEP only needs to be in the VNIs which it has local hosts for.
VXLAN BGP Control PlaneVTEP VNI Membership Symmetric IRB
Host 1
MAC1
IP 1
VLAN 100
VXLAN 5100
Host 2
MAC2
IP 2
VLAN 100
VXLAN 5100
SVI
100
Host 3
MAC3
IP 3
VLAN 200
VXLAN 5200
VTEPVTEPVTEPVTEP SVI
100
SVI
200
Every VTEP only needs to be in VNIs
that it has local hosts for.
VTEPs don’t need to maintain MAC
tables for VNIs that they don’t have local
hosts for.

VTEP-1
VTEP VTEP VTEP-2
Host 1
H-MAC-1
H-IP-1
VNI-A
L3-VNI-A
VRF-A
Host 2
H-MAC-2
H-IP-2
VNI-B
L3-VNI-A
VRF-A
IP Transport
Network
S-MAC: H-
MAC-1
D-MAC: H-
MAC-2
S-IP: H-IP-1
D-IP: H-IP-2
S-MAC: H-
MAC-1
D-MAC: H-
MAC-2
S-IP: H-IP-1
D-IP: H-IP-2
S-MAC: Router-MAC-1
D-MAC: Router-MAC-2
S-IP: H-IP-1
D-IP: H-IP-2
S-IP: VTEP-1
D-IP: VTEP-2
VNI: L3-VNI-A
Use VTEP addresses in the outer
header to route encapsulated
packets to the egress VTEP
S-MAC: Router-MAC-1
D-MAC: Router-MAC-4
S-IP: H-IP-1
D-IP: H-IP-2
S-IP: VTEP-1
D-IP: VTEP-2
VNI: L3 –VNI-A Use L3-VNI to identify
the tenant VRF
Tenant A
VRF-A
L3-VNI-A
H-IP-2
Tenant B
VRF-B
L3-VNI-B
Tenant C
VRF-C
L3-VNI-C
Tenant A
VRF-A
L3-VNI-A
H-IP-2

• Symmetric IRB has optimal utilization of ARP and MAC tables on a VTEP
• Symmetric IRB scales better for end hosts
• Symmetric IRB scales better in terms of the total number of VNIs a VXLAN
overlay network can support
Multi-vendor interoperability:
• Some vendors implemented Asymmetric IRB
• It’s been agreed upon among multiple vendors that Symmetric IRB is the
ultimate solution
• Cisco implemented Symmetric IRB
• Cisco will introduce backward compatability with asymmetric IRB by adding the
support for it.

Remote host route learning through
MP-BGP
Optimal VXLAN Routing
with Symmetric IRB and Anycast Gateway
LeafVTEP-4VTEP-3VTEP-2VTEP-1 VTEP-5
Spine
SVI
100
SVI
200
SVI
100
SVI
200
Host-based fabric routing and bridging
with optimal and flexible VXLAN VNI placement
Every VTEP is an anycast
gateway for its VXLAN subnets.
Anycast gateway VTEPs share:
• The same virtual Gateway IP
• The same virtual MAC
address
Distributed inter-vxlan
host-based routing on
local VTEP
Host IP Port
IP-A Eth1/1
IP-B VTEP-4
Host IP VTEP
IP-A VTEP-2
IP-B Eth1/1
VLAN 100
SVI VLAN 100
VNI 5100
Host IP-A
VLAN 200
SVI VLAN 200
VNI 5200
Host IP-B
SVI
100
SVI
200
SVI
300

51
Local Scoping of VLANs –ToR Local
16 million possible VNIs global scope
VLANS are Locally
Scoped at Top of
Rack/ Gateway
VNI 5000 maps to VLAN 10
Possible VLAN IDs 1-4K
VNI 5000 maps to VLAN 60
VLANS are Locally
Scoped at Top of Rack/
Gateway

52
Local Scoping of VLANs – Port Local*
16 million possible VNIs global scope
VLANS are Locally Scoped
VLAN to VNI mapping is per-port
significant
VNI 5000 maps to (E1/1, VLAN 10)
VNI 5000 maps to (E1/2, VLAN 60
(Eth1/1, Vlan10) => VNI 10000
(Eth1/2, Vlan10) => VNI 10001
(Eth1/2, Vlan11) => VNI 10000
* Available in
Q2CY2015
VLANS are Locally Scoped
VLAN to VNI mapping is per-port
significant

VXLAN
VLANvPC
vPC VTEP vPC VTEP
• When vPC is enabled an ‘anycast’ VTEP
address is programmed on both vPC
peers
• Symmetrical forwarding behavior on both
peers provides
• Multicast topology prevents BUM traffic
being sent to the same IP address across
the L3 network (prevents duplication of
flooded packets)
• vPC peer-gateway feature must be
enabled on both peers
• VXLAN header is ‘not’ carried on the vPC
Peer link (MCT link)
interface loopback0
ip address 10.1.1.13/32
ip address 10.1.1.134/32 secondary

Underlay IP Network
Virtual
PortChannel
Layer 3 Link
Layer 2 Link
vPC VTEP-1 vPC VTEP-2
vPC VTEP with
Anycast VTEP
Address
BGP Router ID 1 BGP Router ID 2

EVPN Control Plane Advantages
A multi-tenant fabric solution with host-based forwarding
• Industry standard protocol for multi-vendor interoperability
• Build-in multi-tenancy support
• Leverage MP-BGP to deliver VXLAN with L3VPN characteristics
• Truly scalable with protocol-driven learning
• Host MAC/IP address advertisement through EVPN MP-BGP
• Fast convergence upon host movements or network failures
• MP-BGP protocol driven re-learning and convergence
• Upon host movement, the new VTEP will send out a BGP update to advertise
the new location of the host

EVPN Control Plane Advantages (Cont’ed)
• Optimal traffic forwarding supporting host mobility
• Anycast IP gateway for optimal forwarding for host generated traffic
• No need for hair-pinning to to reach the IP gateway
• ARP suppression
• Minimize ARP flooding in overlay
• Head-end Replication with dynamically learned remote-VTEP list
• Head-end replication enables multicast-free underlay network
• Dynamically learned remote-VTEP list minimizes the operational overhead of
head-end replication
• VTEP peer authentication via MP-BGP authentication
• Added security to prevent rogue VTEPs or VTEP spoofing
A multi-tenant fabric solution with host-based forwarding

DC Core
VTEP
L2 Link
L3 Link
DC Aggregation
DC Access
VTEP VTEP VTEP
VXLAN Overlay

LeafVTEPVTEPVTEPVTEP VTEP VTEP
SpineRRRR
VXLAN Overlay
MP-iBGP EVPN
MP-iBGP Sessions
• VTEP Functions are on leaf layer
• Spine nodes are iBGP route reflector
• Spine nodes don’t need to be VTEP

Spine
Leaf
VTEP VTEPVTEP VTEP VTEP
iBGP iBGP
iBGP iBGP iBGP RRRR
VXLAN Overlay
Spine switches are not capable of
running MP-BGP EVPN.
Leaf switches are chosen to provide
iBGP route-reflector functions to the
other iBGP VTEP leaf nodes. All other
leaf nodes peer with them through
iBGP.

Spine
Leaf
VTEP
Cisco Nexus 9300
VTEP
Cisco Nexus 9300
VTEP
Cisco Nexus 9300
VTEP
Cisco Nexus 9300
VTEP
Cisco Nexus 9300
iBGP iBGP iBGP iBGP iBGP
RR
RR
Spine switches don’t need to be
able to run MP-BGP EVPN. They
are purely IP transport devices.
Dedicated MP-BGP EVPN route
reflectors provide better scalability
and control-plane performance.
They can be connected to the
fabric network in the same way as
a leaf node.
All leaf VTEPs run iBGP sessions with the
dedicated route reflectors.

• VTEP Functions are on leaf layer
• Spine nodes are MP-eBGP Peers
• Spine nodes don’t need to be VTEP
Spine
MP-eBGP Sessions
AS 65001 AS 65002 AS 65003 AS 65004 AS 65005 AS 65006
AS 65000
BGP on Spine needs to have the
following in address-family l2vpn
evpn:
• BGP next-hop unchanged
• retain route-target all
Need to manually configure Route-
targets on each VTEP

[BGP configuration on a spine switch as in Figure 16 design]
route-map permit-all permit 10
set ip next-hop unchanged
router bgp 65000
router-id 10.1.1.1
redistribute direct route-map permitall
nexthop route-map permit-all
retain route-target all
route-map permit-all out
route-map permit-all out
Set next-hop policy to not change
the next-hop attributes.
Retain routes with all route targets
when advertising the EVPN BGP
routes to eBGP peers.
Set outbound policy to advertise all
routes to this eBGP neighbor.

Spine
MP-eBGP Sessions
AS 65100 AS 65100 AS 65100 AS 65100 AS 65100 AS 65100
AS 65000
VTEP leafs are in the
same BGP AS
BGP on Spine needs to have the
following in address-family l2vpn
evpn:
• BGP next-hop unchanged
• retain route-target all

EVPN VXLAN Fabric External Routing
SpineRRRR
Global Default VRF
Or User Space VRFs
Border Leaf
VXLAN Overlay
EVPN MP-BGP
IP Routing
Routing
Protocol
of
Choice
VXLAN Overlay
EVPN VRF/VRFs Space

EVPN VXLAN Fabric External Routing (Cont’ed)
Tenant VRF or Default VRF
VRF OSPF Process
Overlay
EVPN
VRF A
Overlay
EVPN
VRF B
Overlay
EVPN
VRF C
VRFA
For Layer 3 interfaces, use
one per overlay VRF
instance. The routing
protocol neighbor is in the
EVPN VRF address family.
Layer 3 interfaces on the
external devices can be in
either tenant VRF
instances or the global
default VRF instance.
External Router
VRFB VRFC
Interface-Type Options:
• Physical Routed Ports
• Subinterfaces
• VLAN SVIs over Trunk Ports
Border Leaf

IP Routing in the Default
VRF Instance
VTE
P
VTE
P
VTE
P
VTE
P
Spine
RRRR
VXLAN Overlay
EVPN VRF Instance Space
VTEP
Border Leaf interface Ethernet2/9.10
mtu 9216
encapsulation dot1q 10
ip address 30.10.1.1/30
interface Ethernet1/50.10
mtu 9216
ip address 30.10.1.2/30
Router bgp 100
vrf evpn-tenant-1
network 20.0.0.0/24
prefix-list outbound-no-hosts out
router bgp 200
network 100.0.0.0/24
network 100.0.1.0/24

router bgp 100
router-id 10.1.1.16
vrf evpn-tenant-1
network 20.0.0.0/24
prefix-list outbound-no-hosts out
ip prefix-list outbound-no-hosts seq 5 deny 0.0.0.0/0 eq 32
ip prefix-list outbound-no-hosts seq 10 permit 0.0.0.0/0 le 32
On the VXLAN Border Leaf
The eBGP neighbor is on the outside.
It’s in address-family ipv4 unicast of the tenant VRF routing
instance.
For better scalability, apply prefix-list to filter out /32 IP host routes.
Advertise prefix routes only to the external eBGP neighbor.

n9396-vtep-1# sh ip bgp vrf evpn-tenant-1 100.0.0.0
BGP routing table information for VRF evpn-tenant-1, address family IPv4 Unicast
BGP routing table entry for 100.0.0.0/24, version 70
Paths: (1 available, best #1)
Flags: (0x08041a) on xmit-list, is in urib, is best urib route
vpn: version 75, (0x100002) on xmit-list
Advertised path-id 1, VPN AF advertised path-id 1
Path type: internal, path is valid, is best path, no labeled nexthop
Imported from unknown dest
AS-Path: NONE, path sourced internal to AS
10.1.1.16 (metric 3) from 10.1.1.1 (10.1.1.1)
Origin IGP, MED not set, localpref 100, weight 0
Received label 39000
Extcommunity: RT:100:39000 ENCAP:8 Router MAC:6412.2574.6ae7
Originator: 10.1.1.16 Cluster list: 10.1.1.1
VRF advertise information:
Path-id 1 not advertised to any peer
VPN AF advertise information:
Path-id 1 not advertised to any peer
n9396-vtep-1#
n9396-vtep-1# sh ip route vrf evpn-tenant-1 100.0.0.0/24
IP Route Table for VRF "evpn-tenant-1"
'*' denotes best ucast next-hop
'**' denotes best mcast next-hop
'[x/y]' denotes [preference/metric]
'%<string>' in via output denotes VRF <string>
This is the external route.
The tenant is VRF L3 VNI.
The next hop is the VTEP address of the border lea
10.1.1.16 is the BGP router ID of the border
leaf. 10.1.1.1 is the spine route reflector.
This is the iBGP route. The next hop is the VTEP
address of the border leaf.

IP Routing in the Default
VRF Instance
VTE
P
VTE
P
VTE
P
VTE
P
Spine
RRRR
VXLAN Overlay
EVPN VRF and VRF
Instance Space VTEP
Border Leaf
mtu 9216
ip address 30.10.1.1/30
ip router ospf 1 area 0.0.0.0
mtu 9216
ip address 30.10.1.2/30
ip router ospf 1 area 0.0.0.0
route-map permit-bgp-ospf permit 10
route-map permit-ospf-bgp permit 10
router ospf 1
router-id 10.1.1.16
vrf evpn-tenant-1
redistribute bgp 100 route-map permit-bgp-ospf
router bgp 100
router-id 10.1.1.16
vrf evpn-tenant-1
redistribute ospf 1 route-map permit-ospf-bgp

ip prefix-list bgp-ospf-no-hosts seq 5 permit 0.0.0.0/0 eq 32
route-map permit-bgp-ospf deny 5
match ip address prefix-list bgp-ospf-no-hosts
route-map permit-bgp-ospf permit 10
route-map permit-ospf-bgp permit 10
router ospf 1
router-id 10.1.1.16
vrf evpn-tenant-1
redistribute bgp 100 route-map permit-bgp-ospf
router bgp 100
router-id 10.1.1.16
Redistribute BGP routes to OSPF. Filter
out /32 IP host routes.
A BGP router will modify route targets in l2vpn
evpn routes when it is an autonomous system
boundary router. The original route target must
be retained.
Redistribute OSPF to BGP. Advertise
the redistributed routes to L2VPN
EVPN.

n9396-vtep-1# sh vrf evpn-tenant-1 detail
VRF-Name: evpn-tenant-1, VRF-ID: 3, State: Up
VPNID: unknown
RD: 10.1.1.11:3
VNI: 39000
Max Routes: 0 Mid-Threshold: 0
Table-ID: 0x80000003, AF: IPv6, Fwd-ID: 0x80000003, State: Up
Table-ID: 0x00000003, AF: IPv4, Fwd-ID: 0x00000003, State: Up
n9396-vtep-1# sh bgp l2vpn evpn rd 10.1.1.11:3 100.0.0.0
BGP routing table information for VRF default, address family L2VPN EVPN
Route Distinguisher: 10.1.1.11:3 (L3VNI 39000)
BGP routing table entry for [5]:[0]:[0]:[24]:[100.0.0.0]:[0.0.0.0]/224, version 396
Paths: (1 available, best #1)
Flags: (0x00001a) on xmit-list, is in l2rib/evpn
Advertised path-id 1
Path type: internal, path is valid, is best path, no labeled nexthop
Imported from 10.1.1.16:3:[5]:[0]:[0]:[24]:[100.0.0.0]:[0.0.0.0]/120
AS-Path: NONE, path sourced internal to AS
10.1.1.16 (metric 3) from 10.1.1.1 (10.1.1.1)
Origin IGP, MED not set, localpref 100, weight 0
Received label 39000
Extcommunity: RT:100:39000 ENCAP:8 Router MAC:6412.2574.6ae7
Originator: 10.1.1.16 Cluster list: 10.1.1.1
The external route learned through MP-
BGP EVPN is imported into the tenant
VRF.
This is the Layer 3 VNI of the tenant
VRF routing instance.
The next hop is the VTEP address of
the border leaf.

Alternative Design for EVPN VXLAN External Routing
Spine/Aggr.
Global Default VRF
Or User Space VRFs
IP Routing
Routing Protocol of
Choice
VXLAN Overlay
EVPN VRF/VRFs
Space
VXLAN Overlay
EVPN MP-BGP Border Leaf
RR
Overlay
EVPN
VRFs
Default
VRF
Default VRF in
IP Routing
EVPN MP-BGP
IP Routing MP-BGP

Tenant A (VRF A)
• 1 Layer-3 VNI per
Tenant (VRF) for routing
• VNI X’ is used for routed
packets
• One VLAN maps to one Layer-2 VNI Layer-2 VNI per
Layer-2 segment
• A Tenant can have multiple VLANs, therefore multiple
Layer-2 VNIs
• Traffic within one Layer-2 VNI is bridged
• Traffic between Layer-2 VINs is routed
VLAN A
Layer-2 VNI A’
SVI
A
VLAN B
Layer-2 VNI B’
SVI
B
VLAN X
Layer-3 VNI X’
SVI
X
VLAN N
Layer-2 VNI N’
SVI
N

feature nv overlay
feature vn-segment-vlan-based
feature bgp
nv overlay evpn
Initial configuration – Per Switch
Enable VXLAN and MP-BGP EVPN Control Plane
Enable VXLAN
Enable VLAN-based VXLAN (the currently only
mode)
Enable OSPF if it’s chosen to be the underlay
IGP routing protocol
Enable VLAN SVI interfaces if the VTEP
needs to be IP gateway and route for the
VXLAN VLAN IP subnet.
Enable EVPN control plane for VXLAN
feature ospf
feature pim
feature interface-vlan
Other features may need to be anabled
Enable BGP
Enable IP PIM multicast routing in the
underlay network

vni 39000
rd auto
EVPN Tenant VRF
Create VXLAN tenant VRF Create a VXLAN Tenant VRF
Specify the Layer-3 VNI for VXLAN routing
within the tenant VRF
Define VRF Route Target and import/export
policies in address-family ipv4 unicast
Define VRF RD (route distinguisher)
vni 39010
rd auto
Example to create a 2nd tenant VRF following
the above steps

vlan 3900
vn-segment 39000
interface Vlan3900
no shutdown
vni 39000
rd auto
Layer-3 VNI Per Tenant for EVPN Routing
Configure Layer-3 VNI per EVPN Tenant VRF Routing Instant
Create the VLAN for the Layer-3 VNI.
One Layer-3 VNI per tenant VRF routing
instance
Create the SVI interface for the Layer-3 VNI
Put this SVI interface into the tenant VRF context
Associate the Layer-3 VNI with the tenant VRF
routing instance.

vlan 3901
vn-segment 39010
interface Vlan3901
no shutdown
vni 39010
rd auto
EVPN Layer-3 VNI Per Tenant for Routing Instance
Configure Layer-3 VNI per EVPN Tenant VRF Routing Instant
Define Layer-3 VNI for a 2nd tenant following
the same steps in the previous slide

vlan 200
vn-segment 20000
vlan 210
vn-segment 21000
EVPN Layer-2 Network VXLAN VNI
Map VLANs to VXLAN VNIs and Configure their MP-BGP EVPN Parameters
Map VLAN to VXLAN VNI
evpn
vni 20000 l2
rd auto
vni 21000 l2
rd auto
Under EVPN configuration, define RD and
RT import/export policies for each Layer-2
VNIs

interface Vlan200
no shutdown
interface Vlan210
no shutdown
EVPN Layer-2 Network VXLAN VLAN SVI Interface
Create SVI interface for Layer-2 VNIs for VXLAN routing
Create SVI interface for a Layer-2 VNI.
Associate it with the tenant VRF.
Enable distributed anycast gateway for this
VLAN/VNI
All VTEPs for this VLAN/VNI should have the
same SVI interface IP address as the
distributed IP gateway.

fabric forwarding anycast-gateway-mac 0002.0002.0002
interface Vlan210
no shutdown
EVPN Distributed Gateway
Configure virtual IP address
All VTEPs for this VLAN should have the same
virtual IP address
Configure distributed gateway virtual MAC address
One virtual MAC per VTEP
All VTEPs should have the same virtual MAC
address
Enable distributed gateway for this VLAN

interface nve1
no shutdown
source-interface loopback0
host-reachability protocol bgp
member vni 20000
suppress-arp
member vni 21000
suppress-arp
interface loopback 0
ip address 10.1.1.11/32
VXLAN Tunnel Interface Configuration
Configure VXLAN tunnel interface nve1
Specify loopback0 as the source interface
Define BGP as the mechanism for host
reachability advertisement
Add Layer-3 VNIs, one per tenant VRF
Associate tenant VNIs to the tunnel interface
nve1
Define the mcast group on a per-VNI basis
Enable arp suppression on a per-VNI basis

router bgp 100
router-id 10.1.1.11
vrf evpn-tenant-1
vrf evpn-tenant-2
MP-BGP Configuration on VTEP
Address-family ipv4 unicast for prefix-based
routing
Define MP-BGP neighbors.
Under each neighbor define address-family
ipv4 unicast and l2vpn evpn
Under address-family ipv4 unicast of each
tenant VRF instance, enable advertising EVPN
routes
Send extended community in l2vpn evpn
address-family to distribute EVPN route
attributes
Address-family l2vpn evpn for evpn host
routes

router bgp 100
router-id 10.1.1.1
template peer vtep-peer
remote-as 100
send-community both
route-reflector-client
send-community both
route-reflector-client
neighbor 10.1.1.11
inherit peer vtep-peer
neighbor 10.1.1.12
neighbor 10.1.1.13
neighbor 10.1.1.14
MP-BGP Configuration on iBGP Route Reflectore
Address-family ipv4 unicast for prefix-based
routing
iBGP RR client peer template
Send both standard and extended community
in address-family l2vpn evpn
Send both standard and extended community
in address-family ipv4 unicast
Address-family l2vpn evpn for EVPN vxlan
host routes
Retain route-targets attributes

VXLAN is supported across the Nexus 9000 series platforms. The VXLAN
Gateway functionality is supported across all form factors and line cards.
Integrated routing functionality is supported on Nexus 9300 switches and
ACI-enabled Modules for Nexus 9500 switches.
Nexus 9500 SeriesNexus 9300 Series

• VXLAN Encapsulation and De-encapsulation
occur on T2
• Bridging and Gateway are independent of the
port type (1/10/40G ports)
• Encapsulation happens on the egress port
• Decapsulation happens on the ingress port
ALE
(NorthStar)
Encap/Decap
(NFE)
VXLAN /VLAN
VXLAN /VLAN

• VXLAN Routing is not supported currently on Broadcom
• Additional recirculation required for VXLAN routing through NS
Insieme
Encap/De
cap
Recirculate
Route
Packet
VLAN
Subnet 10.20.20.0/24
VXLAN
Subnet 10.10.10.0/24
Insieme
Encap/De
cap
Recirculate
Route
Packet
VLAN
Subnet 10.20.20.0/24
VXLAN
Subnet 10.10.10.0/24

VXLAN Scales on Nexus 9000 Series Switches
Scale Parameter Bronte Target
VxLAN enabled VLANs 1000
VxLAN enabled VRFs 900
VxLAN SVIs 1000
Total VNIs (L2/L3) 1900 (1000/900)
ECMP paths 64
Local VTEPs 1
Remote VTEPs 255

Hardware -
• DC Power Supply for N93xx
Software -
• 10G Dynamic Per-Port Break-out Support:
X9636PQ, X9432PQ, N9332PQ, N3164PQ,
X9536PQ
• IEEE 1588 PTP
• 93xx (except ALE uplink ports)
• X9500, X9600 Linecards
• 1588 Timestamp in ERSPAN packets
• 93xx (ERSPAN destination is ALE port)
• DHCP Snooping
• IPSG
• BFDv6
• BFDv6 client support: BGP, OSPF, IS-IS,
EIGRP
• VRRPv3
• DAI
• Egress Policing on Uplink only T2 (ALE 40 G
links are not supported)
• SNMP Module Stats/Trap
• EIGRP NSF Enhancements
• Multicast ECMP
Bronte1.0
Target: Q1CY15
Software (contd) –
• VxLAN Flood & Learn MIB , VXLAN counters on 9300
• VxLAN Flood & Learn on 9300
• Ingress Replication
• VxLAN BudNode
• Bridging/Gateway
• Static MAC to VTEP binding
• VxLAN eVPN Control Plane on 9300
• Bridging/Gateway
• VxLAN routing (SVI Based)
• VxLAN BudNode
• ARP suppression
• Anycast Gateway
Bronte1.0
Target: Q1CY15
•

Hardware -
 100G LC, 8p CFP2 100G
 100G GEM Module, 4p CFP2 100G
 N93120TX ((96p SFP+ & 6p 40G QSFP+)
 HVDC Power Supply for N9504,N9508, N9516
Software –
• Per-port VLAN significance (VXLAN)
• VXLAN MIB, Counters on N95xx
• VXLAN Bud Node support on 9500
• FEX (Straight-through) on 9500
• Host vPC FEX on 9500
• PFC(802.1Qbb) for 9300
• 802.3x
Bronte1.2
Target: Q2CY15

Software –
Routing(SVI) Support Flood & Learn with multicast in
underlay for ToR (Routing)
• Routing(SVI) Support for VXLAN Flood & Learn with
multicast in underlay for 9300
• Routing(SVI) and Bridging Support for VXLAN Flood &
Learn with multicast in underlay for 9500
• Routing(SVI) Support for VXLAN Flood & Learn with static
ingress replication for 9300
• Routing(SVI) and Bridging Support for VXLAN Flood &
Learn with static ingress replication for 9500
• BGP EVPN Control Plane for VXLAN on 9500 with both
Bridging and Routing support
Bronte1.3
Target: Q3CY15

Hardware -
• New FEX models supported
• Tiburon FEX support (2348UPQ)
• FEX Support – B22-IBM, B22-Fujitsu
Software -
• N9K-N3K converged software release
• NAT(Static & Dynamic)
• PVLAN
• Static MPLS label stripping
• Static MPLS label pop & swap support
• FEX QoS
• Port Security
• sFLOW
• Scale enhancement 4K VLAN’s per port
• PIM BiDir Support – no vPC Support, use /32 for
destination Groups
• PIMv4 SSM
• PIMv6 ASM
• PIMv6 Sparse Mode
• PIMv6 BSR
• BFD HSRPv2
• BFD VRRPv3 Camden
Target: Q3CY15
Software -
• VXLAN Q-in-Q support
• VXLAN on FEX HIF Ports
• GRE – IPv6 over IPv4, Unicast IPv4 over IPv6
• IPIP Tunneling Filter Mask
• Resilient Hashing(v4/v6)
• Config Sync
• Config Session Manager
• Graceful Insertion & Removal
• Configurable Login Blockout
• Config Checkpoint & Rollback
• Openflow 1.3 (9300)
• IP SLA
• Weighted ECMP
• Enhanced RPM Support
• Kstack
• Model Driven/Object Based NX-API
• Native Support of Chef & Puppet
• PXE & ONIE Support
• Q in Q VLAN Tunneling
Camden
Target: Q3CY15

Vxlan control plane and routing

Vxlan control plane and routing

More Related Content

What's hot (20)

Similar to Vxlan control plane and routing (20)

Recently uploaded (20)

Vxlan control plane and routing

Editor's Notes