Software Stacks to enable SDN and NFV

0Copyright©2017 NTT corp. All Rights Reserved.
Software Stacks to enable
Software-Defined Networking and
Network Functions Virtualization
Yoshihiro Nakajima
<nakajima.yoshihiro@lab.ntt.co.jp, ynaka@lagopus.org>
NTT Network Innovation Laboratories

 Yoshihiro Nakajima
 Work for Nippon Telegraph and Telephone Corporation,
R&D Division
Network Innovation Laboratories
 Project lead of Lagopus SDN software switch
 Background
• High performance computing
• High performance networking and data processsing
About me

 Trends
 Software-Defined Networking (SDN)
 Network Functions Virtualization (NFV)
 Dataplane
 Lagopus: SDN/OpenFlow software switch
• Overview and NFV
• Trials
 Controller/C-plane
 O3 project
 Ryu and gobgp
 Zebra2
 Future plan
Agenda

Network is so stuck….
 Many technologies has emerged in system development area
 Language, Debugger, Testing framework, Continuous Integration, Continuous
Deployment….
 Network area …
 CLI with vender format 
 CLI with serial or telnet 
 Expect script 
 Good for Human,
But not for software

Trend shift in networking
 Closed (Vender lock-in)
 Yearly dev cycle
 Waterfall dev
 Standardization
 Protocol
 Special purpose HW / appliance
 Distributed cntrl
 Custom ASIC / FPGA
 Open (lock-in free)
 Monthly dev cycle
 Agile dev
 DE fact standard
 API
 Commodity HW/ Server
 Logically centralized cntrl
 Merchant Chip

What is Software-Defined Networking?
5
Innovate services and applications
in software development speed!
Reference: http://opennetsummit.org/talks/ONS2012/pitt-mon-ons.pdf
Decouple control plane and data plane
→Free control plane out of the box
（APIs: OpenFlow, P4, …）
Logically centralized view
→Hide and abstract complexity of networks,
provide entire view of the network
Programmability via abstraction layer
→Enables flexible and rapid service/application
development
SDN conceptual model

Why SDN?
6
Differentiate services (Innovation)
• Increase user experience
• Provide unique service which is not in the
market
Time-to-Market（Velocity）
•Not to depend on vendors’ feature roadmap
•Develop necessary feature when you need
Cost-efficiency
•Reduce OPEX by automated workflows
•Reduce CAPEX by COTS hardware

Open-Source Cloud Computing
Open Northbound APIs
Open-Source Controller
Open-Source Hardware
Open Southbound Protocol
Open-Source
Switch Software
Infrastructure
Layer
Application Layer
Business Applications
Control Layer
Network
Services
Network Services
API API API
v
v v v
Open source network stacks for SDN/NFV
FBOSS

SDN/NFV history
201020092008 2011 2012 2013
OpenFlow deveopment
Open Networking Foundation
ETSI
NFV
Open
Daylight
OpenFlow Switch Consortium
Stanford University
Clean Slate Program
NFV
SDN
activity
Standarlization
OpenFlow
2014
trema Ryu
OpenDaylight
OF1.1 OF1.2
OpenFlow
protocols OF1.4OF 0.8 OF 0.9 OF 1.0
NOX
Lagopus

History of software vswitch/router and I/O library
1995 2000 2005 2010 2015
DPDK
R1.0 OSS LF join
2012 BT vBRAS demo
Research phase
Internal product phase
OSS phase
netmap
XDP
Click
BESS
OVS
Nicira OVN
Lagopus
VPP

Network Functions Virtualization
 Replace dedicated network nodes to
virtual appliance
 Runs on general servers
 Leverage cloud provisioning and
monitoring system for management
 Virtual network function (VNF) may be run on
virtual machine or baremetal server

11
Evaluate the benefits of SDN
by implementing our control plane and switch

 High performance network I/O for all packet sizes
 Especially in smaller packet size (< 256 bytes)
 Low-latency and less-jitter
 Network I/O & Packet processing
 Isolation
 Performance isolation between NFV VMs
 Security-related VM-to-VM isolation from untrusted apps
 Reliability, availability and serviceability (RAS) function for long-term
operation
NFV requirements from 30,000 feet
Virtua lma chine
Ma na gement a nd
API
Virtua lSwitch
(vSwitch)
NIC
Ha rd wa re resources
Core Core
VM VM
Core
VM VMNf-Vi-H
Instruction,
Policing ma p p ing
a nd emula tion
CoreCore CoreCore
Seq uentia lthrea d
emula tion

 Still poor performance of NFV apps 
 Lower network I/O performance
 Big processing latency and big jitter
 Limited deployment flexibility 
 SR-IOV has limitation in performance and configuration
 Combination of DPDK apps on guest VM and DPDK-enabled vSwitch is
configuration
 Limited operational support 
 DPDK is good for performance, but has limited dynamic reconfiguration
 Maintenance features are not realized
What’s matter in NFV

 Not enough performance 
 Packet processing speed < 1Gbps 
 10K flow entry add > two hours 
 Flow management processing is too heavy 
 Develop & extension difficulties 
 Many abstraction layer cause confuse for me 
• Interface abstraction, switch abstraction, protocol abstraction Packet abstraction… 
 Many packet processing codes exists for the same processing 
• Userspace, Kernelspace,…
 Invisible flow entries cause chaos debugging 
Existing vswitch is  @2013

vSwitch requirement from user side
 Run on the commodity PC server and NIC
 Provide a gateway function to allow connect different
various network domains
 Support of packet frame type in DC, IP-VPN, MPLS and access NW
 Achieve 10Gbps-wire rate with >= 1M flow rules
 low-latency packet processing
 flexible flow lookup using multiple-tables
 High performance flow rule setup/delete
 Run in userland and decrease tight-dependency to OS
kernel
 easy software upgrade and deployment
 Support various management and configuration protocols.

Lagopus:
High-performance
SDN/OpenFlow Software Switch

Goal of Lagopus project
 Provide NFV/SDN-aware switch software stack
 Provide dataplane API with OpenFlow protocol and gRPC
 100Gbps-capable high-performance software dataplane
 DPDK extension for carrier requirements
 Cloud middleware adaptation
 Expand software-based packet processing to carrier
networks

 Lagopus is a small genus of birds in the grouse subfamily,
commonly known as ptarmigans. All living in tundra or
cold upland areas.
 Reference: http://en.wikipedia.org/wiki/Lagopus
What is Lagopus (雷鳥属)?
© Alpsdake 2013© Jan Frode Haugseth 2010

 Provide High performance software switch on Intel CPU
 Over-100Gbps wire-rate packet processing / port
 High-scalable flows handling
 Expands SDN idea to many network domain
 Datacenter, NFV environment, mobile network
 Various management /configuration interfaces
Target of Lagopus switch
TOR
Virtual Switch
Hypervisor
VM VM
Virtual Switch
Hypervisor
NFV NFV
Virtual Switch
Hypervisor
VM VM
Gateway CPE
Data Center Wide-area Network Access Network Intranet

 Open Source High performance SDN software switch
 Multicore-CPU-aware packet processing with DPDK
 Supports NFV environment
 Runs on Linux and FreeBSD
 Best OpenFlow 1.3 compliant software switch by Ryu
certification
 Many protocol frame matches and actions support
• Ethernet, VLAN, MPLS, PBB, IPv4, IPv6, TCP, UDP, VxLAN, GRE, GTP
 Multiple-Flow table, Group table, meter table
 1M flow entries handling (4K flow mod/sec)
 Over-40Gbps-class packet processing (20MPPS)
 Open source under Apache v2 license
 http://lagopus.github.io/
What is Lagopus SDN software switch

0
2,000,000
4,000,000
6,000,000
8,000,000
10,000,000
12,000,000
14,000,000
16,000,000
0 256 512 768 1024 1280
#ofpacketsperseconds
Packet size (Byte)
How many packets to be proceeded for 10Gbps
Short packet 64Byte
14.88 MPPS, 67.2 ns
• 2Ghz: 134 clocks
Computer packet 1KByte
1.2MPPS, 835 ns
L1 cache access: 4 clocks
Main memory: 100 clocks
Max PPS between cores: 20MPPS

PC architecture and limitation
NIC
CPU CPUMemory
Memory
NIC
NICNIC
QPI
PCI-Exp PCI-Exp
Reference: supermicro X9DAi

0 1000 2000 3000 4000 5000 6000
L2 forwarding
IP routing
L2-L4 classification
TCP termination
Simple OF processing
DPI
# of required CPU cycles
# of CPU cycle for typical packet processing
10Gbps 1Gbps

 Simple is better for everything
 Packet processing
 Protocol handling
 Straight forward approach
 Full scratch (No use of existing vSwitch code)
 User land packet processing as much as possible, keep
kernel code small
 Kernel module update is hard for operation
 Every component & algorithm can be replaced
Approach for vSwitch development

What is Lagopus vSwitch
switch configuration datastore
(config/stats API, SW DSL)
None-DPDK NIC DPDK NIC/vNIC
DPDK libs/PMD driver
Lagopus soft dataplane
flow lookup flow cache
OpenFlow pipeline
queue/
policer
Flow table
Flow table
flow table
Flow table
Flow tableGroup
table
Flow table
Flow tablemeter
table
switch HAL
OpenFlow1.3
agent
JSON IF
SNMP
CLI
CLI
JSON
OSNWstack
Agent
SDN switch Agent
• Full OpenFlow 1.3 support
• Controller-less basic L2 and
L3 support with
action_normal
SDN-aware
management API
• JSON-based control
• Ansible support
DPDK-enabled
OpenFlow-aware
software dataplane
• Over-10-Gbps performance
• Low latency packet processing
• high performance multi-layer flow
lookup
• Cuckoo hash for flow cache
Switch configuration
datastore
• Pub/sub mechanism
• Switch config DSL
• JSON-based control
OS NIFVarious I/O support
• DPDK-enabled NIC
• Standard NIC with raw socket
• tap
Virtualization support
• QEMU/KVM
• Vhost-user
• DPDK-enabled VNF

General packet processing on UNIX
NIC
skb_buf
Ethernet Driver API
Socket API
vswitch
packet
buffer
Data plane
User-space implementation
(Event-triggered)
1. Interrupt
& DMA
2. system call (read)
User
space
Kernel
space
Driver
4. DMA
3. system call (write)
Kernel-space implementation
(Event-triggered)
NIC
skb_buf
Ethernet Driver API
Socket API
vswitch
packet
buffer
1. Interrupt
& DMA
vswitch
Data plane
agentagent
2. DMA
Contexts switch
Massive
Interrupt
Many memory
copy / read

 x86 architecture-optimized data-
plane library and NIC drivers
 Memory structure-aware queue, buffer
management
 packet flow classification
 polling mode-based NIC driver
 Low-overhead & high-speed
runtime optimized with data-
plane processing
 Abstraction layer for hetero
server environments
 BSD-license 
Data Plane Development Kit (DPDK)
NIC
Ethernet Driver API
Socket API
DPDK apps
packet
buffer
1. DMA
Write
2. DMA
READ
DPDK
dataplane
DPDK apps

What DPDK helps
Network Platforms Group
Kernel
Space
Driver
95
Packet Processing Kernel Space vs. User Space
User
Space
NIC
Applications
Stack
System Calls
CSRs
Interrupts
Memory (RAM)
Packet Data
Copy
Socket Buffers
(mbuf’s)
Configuration
Descriptors
Kernel Space Driver
Configuration
Descriptors
DMA
Benefit #1
Removed Data copy
from Kernel to User
Space
Benefit #2
No Interrupts
Descriptors
Mapped from Kernel
Configuration
Mapped from Kernel
Descriptor
Rings
Memory (RAM)
User Space Driver with Zero Copy
Kernel
Space
User
Space
NIC
DPDK PMD
Stack
UIO Driver
System Calls
CSRs
DPDK Enabled App
DMA
Descriptor
Rings
Socket
Buffers
(skb’s)
1
2
3
1
2
Benefit #3
Network stack can
be streamlined and
optimized
DATA

Processing bypass for speed
NIC
skb_buf
Ethernet Driver API
Socket API
vswitch
packet
buffer
packet buffer
memory
Standard linux application
1. Interrupt & DMA
2. system call (read)
User space
Kernel
space
Driver
4. DMA
3. system call (write)
NIC
Ethernet Driver API
User-mode I/O & HAL
vswitch
packet
buffer
Application with intel DPDK
1. DMA Write 2. DMA READ
DPDK Library
Polling-base
packet handling
Event-base
packet handling

Implementation strategy for vSwitch
 Massive RX interrupts handling for NIC device
=> Polling-based packet receiving
 Heavy overhead of task switch
=> Thread assignment
(one thread/one physical CPU)
 Lower performance of PCI-Express I/O and memory
bandwidth compared with CPU
=> Reduction of # of access in I/O and memory
 Shared data access is bottleneck between threads
=> Lockless-queue, RCU, batch processing

Basic packet processing
Network I/O RX
packet
Frame
processing
Flow lookup &
Action
QoS・Queue
Network I/O
TX
Packet classification &
packet distribution to buffers
Packet parsing
lookup, Header rewrite
Encap/decap
Policer, Shaper
Marking
packet

What we did for performance 
Network I/O RX
packet
Frame
processing
Flow lookup &
Action
QoS・Queue
Network I/O
TX
packet
• Delayed packet frame
evaluation
• Delayed action (processing)
evaluation
• Packet batching to improve
CPU $ efficiency
• Delayed flow stats
evaluation
• Smart flow classification
• Thread assignment optimization
• Parallel flow lookup
• Lookup tree compaction
• High-performance lookup
algorithm for OpenFlow
(multi-layer, mask, priority-aware
flow lookup)
• Flow $ mechanism
• Batch size tuning

 Exploit many core CPUs
 Reduce data copy & move (reference access)
 Simple packet classifier for parallel processing in I/O RX
 Decouple I/O processing and flow processing
 Improve D-cache efficiency
 Explicit thread assign to CPU core
Packet processing using multi core CPUs
NIC 1
RX
NIC 2
RX
I/O RX
CPU0
I/O RX
CPU1
NIC 1
TX
NIC 2
TX
I/O TX
CPU6
I/O TX
CPU7
Flow lookup
packet processing
CPU2
Flow lookup
packet processing
CPU4
Flow lookup
packet processing
CPU3
Flow lookup
packet processing
CPU5
NIC 3
RX
NIC 4
RX
NIC 3
TX
NIC 4
TX
NIC RX buffer
Ring buffer
Ring buffer NIC TX buffer

 OpenFlow semantics includes
 Match
• Protocol headers
– Port #, Ethernet, VLAN, BPP, MAC-in-MAC, MPLS, IPv4, IPv6, ARP, ICMP, TCP,
UDP…
– Mask-enabled
• Priority
 Action
• Output port
• Packet-in/Packet-out
• Header rewrite, Header push, Header push
How OpenFlow match is so hard

Many header match
eth_dst
eth_src
eth_type
eth_dst
eth_src
0x0800
(ip_v, ip_hl)
ip_tos
(ip_len)
(ip_id, ip_off)
ip_ttl
ip_p
(ip_sum)
ip_src
ip_dst
eth_dst
eth_src
0x0800
(ip_v, ip_hl)
ip_tos
(ip_len)
(ip_id, ip_off)
ip_ttl
17
(ip_sum)
ip_src
ip_dst
udp_src
udp_dst
eth_dst
eth_src
0x0800
(ip_v, ip_hl)
ip_tos
(ip_len)
(ip_id, ip_off)
ip_ttl
1
(ip_sum)
ip_src
ip_dst
icmp_type
icmp_code
eth_dst
eth_src
0x0806
(ar_hrd, ar_pro)
(ar_hln, ar_pln)
ar_op
ar_sha
ar_spa
ar_tha
ar_tpa
eth_dst
eth_src
0x8847
eth_dst
eth_src
0x8100
Ethernet IPv4 UDP(v4) ICMP ARP MPLS VLAN
mpls_label
(ip_v, ip_hl)
ip_tos
(ip_len)
(ip_id, ip_off)
ip_ttl
ip_p
(ip_sum)
ip_src
ip_dst
vlan_pcp
(vlan_cfi)
vlan_vid
0x0800
mpls_exp
mpls_bos
mpls_ttl
And GRE, L2TP,
OSPF…
Same as TCP, SCTP L3 header continues
Includes IP and other
protocols

Linear search (first implementation)
eth_dst
eth_src
eth_type
(ip_v, ip_hl)
ip_tos
(ip_len)
(ip_id, ip_off)
ip_ttl
ip_p
(ip_sum)
ip_src
ip_dst
tcp_src
tcp_dst
Is eth_type
0x0800?
Is ip_p
17?
Is udp_dst
53?
0x0800?
6?
53?
packet entry 0 entry 1
Is eth_type
Is ip_p
Is tcp_dst
0x0800?
6?
80?
entry 2
Is eth_type
Is ip_p
Is tcp_dst

0: Linear search (first implementation)
eth_dst
eth_src
eth_type
(ip_v, ip_hl)
ip_tos
(ip_len)
(ip_id, ip_off)
ip_ttl
ip_p
(ip_sum)
ip_src
ip_dst
tcp_src
tcp_dst
Is eth_type
0x0800?
Is ip_p
17?
Is udp_dst
53?
0x0800?
6?
53?
packet entry 0 entry 1
Is eth_type
Is ip_p
Is tcp_dst
0x0800?
6?
80?
entry 2
Is eth_type
Is ip_p
Is tcp_dst

 Simplify comparison routine
 Flow entry is composed by mask and value
 Still linear search
1: bitmap comparison with bitmask
eth_dst
eth_src
eth_type
(ip_v, ip_hl)
ip_tos
(ip_len)
(ip_id, ip_off)
ip_ttl
ip_p
(ip_sum)
ip_src
ip_dst
tcp_src
tcp_dst
eth_dst
eth_src
0xffff
(ip_v, ip_hl)
ip_tos
(ip_len)
(ip_id, ip_off)
ip_ttl
0xff
(ip_sum)
ip_src
ip_dst
tcp_src
0xffff
eth_dst
eth_src
0x0800
(ip_v, ip_hl)
ip_tos
(ip_len)
(ip_id, ip_off)
ip_ttl
6
(ip_sum)
ip_src
ip_dst
tcp_src
80
& ==
packet mask flow entry

 Compose search tree for dedicate fields to narrow lookup space
 Then compare each fileds
2: Fixed-type search tree
eth_dst
eth_src
eth_type
(ip_v, ip_hl)
ip_tos
(ip_len)
(ip_id, ip_off)
ip_ttl
ip_p
(ip_sum)
ip_src
ip_dst
tcp_src
tcp_dst
packet
eth_dst
eth_src
eth_type
(ip_v, ip_hl)
ip_tos
(ip_len)
(ip_id, ip_off)
ip_ttl
ip_p
(ip_sum)
ip_src
ip_dst
tcp_src
tcp_dst
array
eth_dst
eth_src
eth_type
(ip_v, ip_hl)
ip_tos
(ip_len)
(ip_id, ip_off)
ip_ttl
ip_p
(ip_sum)
ip_src
ip_dst
tcp_src
tcp_dst
hash table
eth_dst
eth_src
eth_type
(ip_v, ip_hl)
ip_tos
(ip_len)
(ip_id, ip_off)
ip_ttl
ip_p
(ip_sum)
ip_src
ip_dst
tcp_src
tcp_dst
array
Linear sarch

 Scan all flow entries and compose search tree with arrangement
in order from the most frequently searched field
3: Search tree with most searched field first
eth_dst
eth_src
eth_type
(ip_v, ip_hl)
ip_tos
(ip_len)
(ip_id, ip_off)
ip_ttl
ip_p
(ip_sum)
ip_src
ip_dst
tcp_src
tcp_dst
packet
(ip_v, ip_hl)
ip_tos
(ip_len)
(ip_id, ip_off)
ip_ttl
ip_p
(ip_sum)
ip_src
tcp_src
tcp_dst
hash table
eth_dst
eth_src
eth_type
(ip_v, ip_hl)
ip_tos
(ip_len)
(ip_id, ip_off)
ip_ttl
ip_p
(ip_sum)
ip_src
ip_dst
tcp_src
tcp_dst
hash table
eth_dst
eth_src
eth_type
(ip_v, ip_hl)
ip_tos
(ip_len)
(ip_id, ip_off)
ip_ttl
ip_p
(ip_sum)
ip_src
ip_dst
tcp_src
tcp_dst
hash table
eth_dst
eth_src
eth_type
ip_dst
Linear
search

 Reduce # of lock in flow lookup table
 Frequent locks are required
• Switch OpenFlow agent and counter retrieve for SNMP
• Packet processing
Packet batching
Input packet
Lookup table
Input packet
Lookup table
Lock Unlock
ロック Unlock
●Naïve implementation
●Packet batching implementation
Lock is required
for each packet
Lock can be reduced
due to packet batching

 Reduce # of flow lookup in multiple table
 Generate composed flow hash function that contains flow tables
 Introduce flow cache for each CPU core
Bypass pipeline with flow $
42
●Naïve implementation table1 table2 table3
Input packet
table1 table2 table3
Input packet
Flow cache
1. New flow
2. Success flow
Output
packet
Multiple flow $
generation & mngmtWrite flow $
●Lagopus implementation
Output
packet
Packet

Best OpenFlow 1.3 compliant switch
Type Action Set field Match Group Meter Total
# of test scenario
(mandatory, optional)
56
(3 , 53)
161
(0 , 161)
714
(108 , 606)
15
(3 , 12)
36
(0 , 36)
991
(114 , 877)
Lagopus
2014.11.09
59
(3, 56)
161
(0, 161)
714
(108, 606)
15
(3, 12)
34
(0, 34)
980
(114, 866)
OVS (kernel)
2014.08.08
34
(3, 31)
96
(0, 96)
534
(108, 426)
6
(3, 3)
0
(0, 0)
670
(114, 556)
OVS (netdev)
2014.11.05
34
(3, 31)
102
(0, 102)
467
(93, 374)
8
(3, 5)
0
(0, 0)
611
(99, 556)
IVS
2015.02.11
17
(3, 14)
46
(0, 46)
323
(108, 229)
3
(0, 2)
0
(0, 0)
402
(111, 291)
ofswitch
2015.01.08
50
(3, 47)
100
(0, 100)
708
(108, 600)
15
(3, 12)
30
(0, 30)
962
(114, 848)
LINC
2015.01.29
24
(3, 21)
68
(0, 68)
428
(108, 320)
3
(3, 0)
4
(0, 4)
523
(114, 409)
Trema
2014.11.28
50
(3, 47)
159
(0 , 159)
708
(108, 600)
15
(3, 12)
34
(0, 34)
966
(114, 854)

 Summary
 Throughput: 10Gbps wire-rate
 Flow rules: 1M flow rules
 Evaluation models
 WAN-DC gateway
• MPLS-VLAN mapping
 L2 switch
• Mac address switching
Performance Evaluation
5Copyright©2015 NTTcorp. All Rights Reserved.
Typical carrier usecase
Usecase： Coud-VPN gatew ay
Tomorrow：
• Automatic connection setup via North Bound APIs
• SDN controller maintain mapping between tenant logical
network and VPN
• Routes are advertised via eBGP, no need to configure ASBRs
on provider side
Data center
Network
MPLS-VPN
MPLS
Inter-AS option B
SDN
controller
eBGPOF
API
From ONS2014 NTTCOM
Ito-san presentation
VRF-enabled
router with MPLS

 Evaluation setup
 Server spec.
 CPU: Dual Intel Xeon E5-2660
• 8 core(16 thread), 20M Cache, 2.2 GHz, 8.00GT/s QPI, Sandy bridge
 Memory: DDR3-1600 ECC 64GB
• Quad-channel 8x8GB
 Chipset: Intel C602
 NIC: Intel Ethernet Converged Network Adapter X520-DA2
• Intel 82599ES, PCIe v2.0
Performance Evaluation
Server
Lagopus
Flow
table
tester Flows
Throughput (bps/pps/%)
Flow
rules
Packet size
Flow
cache
(on/off)

WAN-DC Gateway
0
1
2
3
4
5
6
7
8
9
10
0 200 400 600 800 1000 1200 1400 1600
Throughput(Gbps)
Packet size (byte)
10 flow rules
100 flow rules
1k flow rules
10k flow rules
100k flow rules
1M flow rules
Throughput vs packet size, 1 flow, flow-cache
0
1
2
3
4
5
6
7
8
9
10
1 10 100 1000 10000 100000 1000000
Throughput(Gbps)
flows
10k flow rules
100k flow rules
1M flow rules
Throughput vs flows, 1518 bytes packet

 10000 IP subnet entries test
L3 forwarding performance with 40G NIC and
CPU E5-2667v3 3.20GHz x2
Memory DDR4 64GB
NIC Intel X710 x2
OS Ubuntu 14.04LTS
DPDK 2.2
Lagopus 0.2.4 with default options

 Full OpenFlow 1.3 support
 Limited OpenFlow 1.5 support (Flowmod-related instructions)
 General tunnel encap/decap extension (EXT-382 and EXT-566) support
• GRE, VxLAN, GTP, Ethernet, IPv4, IPv6
• Updated draft will be implemented soon
 Flexible & high-performance dataplane
 Hybrid-mode support
• ACTION_NORMAL (L2, L3)
 Various network I/O support
• DPDK NIC, vNIC (vhost-user with virtio-net), none-DPDK NIC (raw-socket)
 Leverage network stack in OS kernel for OpenFlow switch
• ARP, ICMP, Routing control packet are escalated to network stack (with tap IF)
 Queue, Meter table
 Linux, FreeBSD, NetBSD support
 Virtualization support for NFV
 DPDK-enabled VNF on DPDK-enabled vSwitch
 QEMU/KVM support through virsh
Lagopus version 0.2.10

Hands-on and seminar

Collaboration with Lagopus
Bussiness Research institutes and networks
Software switch collaboration White box switch collaboration
Yokoyama Laboratory

High-performance vNIC framework for
hypervisor-based NFV with userspace
vSwitch
• To provide novel components to enable high-
performance NFV with general propose hardware
• To provide high-performance vNIC with operation-
friendly features

Issues on NFV middleware

Performance bottleneck in NFV with HV domain
User space on host
Guest VM
Kernel space
Kernel space on host
bridge
NIC driver
TAP driver
QEMU
KVM driver
TAP client
NIC HW
emulator
User space
e1000 NIC driver
NW stack
Legacy NW apps
NIC
Packet buffer
interrupt
copy
copy
register access
interrupt register access
copy
Pkt recv / send
cause
VM transition
HW emulation
needs CPU cycle
& VM transition
Privileged register accesses
for vNIC
cause VM transition
System call cause
context switch on
guest VM
VM transition: 800 CPU cycles

 Use para-virtualization NIC framework
 No full-virtualization (emulation-based)
 Global-shared memory-based packet exchange
 Reduce memory copy
 User-space-based packet data exchange
 No kernel-userspace packet data exchange
vNIC strategy for performance & RAS

 DPDK apps or legacy apps on guest VM
+ userspace DPDK vSwitch
 Connected by shared memory-based vNIC
 Reduce OS kernel implementation
Target NFV architecture with hypervisor
Virtua lma chine
Ma na gement a nd
API
Virtua lSwitch
(vSwitch)
NIC
Ha rd wa re resources
Core Core
VM VM
Core
VM VMNf-Vi-H
Instruction,
Policing ma p p ing
a nd emula tion
CoreCore CoreCore
Seq uentia lthrea d
emula tion
Run in userspace
to avoid VM
transition and
context switch
Memory based
packet transfer

Existing vNIC for u-vSW and guest VM (1/2)
DPDK e1000 PMD with QEMU's e1000 FV and
vSwitch connected by tap
DPDK virtio-net PV PMD with QEMU virtio-net framework
and
DPDK virtio-net PV PMD with vhost-net framework and
User space on host
DPDK
ETHDEV / PMD
UIO driver TAP driver
QEMU
KVM driver
TAP client
NIC HW
emulator
DPDK-enabled
NIC
interruptcopy
copy
DPDK-enabled vSwitch
DPDK ETHDEV
/ pcap PMD
copy
Software dataplane
DMA
Guest VM
Kernel space
User space
DPDK NW apps
UIO driver
DPDK
ETHDEV/
e1000 PMD
register access
Packet buffer
register access
User space on host
DPDK
ETHDEV / PMD
DPDK-enabled
NIC
DPDK ETHDEV /
pcap PMD
copy
Software dataplane
DMA
Guest VM
Kernel space
KVM driver
User space
vhost-net
DPDK NW apps
TAP client
ioeventfd
register access
copy
copy
DPDK
ETHDEV/
virtio-net PMD
UIO
Packet buffer
virtio
ring
User space on host
DPDK
ETHDEV / PMD
DPDK-enabled
NIC
copy
DPDK ETHDEV /
pcap PMD
copy
Software dataplane
DMA
Guest VM
Kernel space
QEMU
KVM driver
TAP client
Virtio-net device
User space
UIO
copy
DPDK NW apps
register access
register access
DPDK ETHDEV/
virtio-net PMD
Packet buffer
virtio
ring
Pros: legacy and DPDK support, opposite status detection
Cons: bad performance, many VM transitions,
context switch
Cons: bad performance, many VM transitions,
context switch
Cons: Cons: bad performance, many VM transitions,
context switch

User space on host
DPDK
ETHDEV / PMD
UIO driver
DPDK-enabled
NIC
Software dataplane
Guest VM
Kernel space
QEMU
KVM driver
IVSHMEM
device
User space
Network apps
DPDK
RING API
Hugepage
DPDK
RING API
IVSHMEM driver
Packet
buffer
Packet
buffer
DMA
mma p
mma p
Packet
buffer
mma p
Ring
Ring
Ring
User space on host
DPDK
ETHDEV / PMD
UIO driver
DPDK-enabled
NIC
DPDK
vhost-user API
Software dataplane
DMA
Guest VM
Kernel space
User space
UIO driver
copy
DPDK NW apps
DPDK
ETHDEV/
virtio-net PMD
KVM driver
Packet buffer
virtio
ring
vhost-user
backend
QEMU
virtio-net device
Existing vNIC for u-vSW and guest VM (2/2)
DPDK ring by QEMU IVSHMEM extension and
vSwitch connected by shared memory
DPDK virtio-net PV PMD with QEMU virtio-net framework
and vSwitch with DPDK vhost-user API to connect to virtio-
net PMD.
Pros: Best performance
Cons: only DPDK support, static configuration, no
RAS
Pros: good performance,
both support of legacy and DPDK
Cons: no status tracking of opposite device

High performance
vNIC framework for NFV
This patch has been already merged to DPDK

 High-Performance
 10-Gbps network I/O throughput
 No virtualization transition between a guest VM and u-vSW
 Simultaneous support DPDK apps and DPDK u-vSW
 Functionality for operation
 Isolation between NFV VM and u-vSW
 Flexible service maintenance support
 Link status notification on the both sides
 Virtualization middleware support
 Support open source hypervisor (KVM)
 DPDK app and legacy app support
 No OS (kernel) modification on a guest VM
vNIC requirements for NFV with u-vSW

 vNIC as an extension of virtio-net framework
 Para-virtualization network interface
 Packet communication by global shared memory
 One packet copy to ensure VM-to-VM isolation
 Control msg by inter-process-communication between pseudo devices
vNIC deisgn
User space on host
Software dataplane
Guest VM
Kernel space
User space
virtio-net-compatible
device
DPDK NW apps
DPDK ETHDEV/
virtio-net PMD
Global shared memory
DPDK ETHDEV /
PMD
pseudo PMD-
enabled device
IPC-based
control
communication

 Virtq-PMD driver: 4K LOC modification
 Virtio-net device with DPDK extension
 DPDK API and PV-based NIC (virtio-net) API
 Global shared memory-based packet transmission on hugeTLB
 UNIX domain socket based control message
• Event notification (link-status, finalization)
• Pooling-based the opposite device check mechanism
 QEMU: 1K LOC modification
 virtio-net-ipc device on shared memory space
 Shared memory-based device mapping
vNIC implementation
User space on host
DPDK-enabled
vSwitch
Software
dataplane
Guest VM
Kernel space
User space
virtio-net-ipc
device
DPDK NW apps
DPDK ETHDEV/
virtio-net PMD
Global shared memory/HugeTLB
DPDK ETHDEV /
virtq-PMD
Unix domain socket
kick/queue addr
virtio-net device
Event notification

Performance

vhost application on host
Measurement
point
virtio-net PMD
virtq PMD
testpmd on Guest VM
Null PMD
Null PMD
Vhost app
testpmd on host
Null PMD
Bare-metal configuration
virtq PMD
Measurement
point
Testpmd on host
Measurement
point
pcap PMD
testpmd on Guest VM
Null PMD
Null PMD
virtq PMD
virtqueue
TAP driver
Virtio-net driver
Kernel-driver
testpmd on host
virtq PMD
Measurement
point
virtio-net PMD
virtqueue
testpmd on Guest VM
Null PMD
Null PMD
virtq-pmd
 micro benchmarking tool: Testpmd apps
 Polling-based DPDK bridge app that reads data from a NIC and writes data to
another NIC in both directions.
 null-PMD: a DPDK-enabled dummy PMD to allow packet generation from
memory buffer and packet discard to memory buffer
Performance benchmark

Performance evaluation
MPPSGBPS
 Virtq PMD achieved great performance
 62.45 Gbps (7.36 MPPS) unidirectional throughput
 122.90 Gbps (14.72 MPPS) bidirectional throughput
 5.7 times faster than Linux driver in 64B, 2.8 times faster than Linux drvier in 1500B
 Virtq PMD achieved better performance in large packet to vhost app
0.00
2.00
4.00
6.00
8.00
10.00
12.00
64 256 512 1024 1500 1536
MPPS
Packet size
virtq PMD (off) virtq PMD (on) vhost app (off) vhost app (on) Linux drv (off) Linux drv (on)
0.00
10.00
20.00
30.00
40.00
50.00
60.00
70.00
64 256 512 1024 1500 1536
GBPS
Packet size
virtq PMD (off) virtq PMD (on) vhost app (off) vhost app (on) Linux drv (off) Kernel drv (on)

Container adaptation

Vhost-user for container
 Vhost-user compatible
PMD for container
 Virtio-net-based backend
 Shared-memory-based
packet data exchange
 Event-trigger by shared file
 27.27 Gbps throughtput

 Packet flow
 pktgen -> physical-> vswitch -> Container(L2Fwd) -> vswitch -> physical -> pktgen
Performance
Lagopus or docker0
Server
Container
L2Fwd or Linux Bridge
Container
pktgen-dpdk
OS: Ubuntu 16.04.1
CPU: Xeon E5-2697 v2 @ 2.70GHz
Mem: 64GB

Performance

SDN IX
@ Interop Tokyo 2015 ShowNet
Interop Tokyo is
the biggest Internet-related technology show in Japan.
This trial was collaboration with NECOMA project
(NAIST & University of Tokyo)

 IX (Internet eXchange)
 Packet exchange point between ISP and
DC-SP
 Boarder router of ISP exchanges route
information
 Issue
 Enhance automation in provisioning and
configuration
 DDoS attack is one of the most critical
issues
• ISP wants to reduce DDoS-related traffic in
origin
• DDoS traffic occupies link bandwidth
Motivation of SDN-IX
IX
ISP-CISP A ISP-DISP B
SW
SWSW
SW
ISP-EISP F
IX
SW
SWSW
SW
ISP-EISP F

What is SDN IX?
 Next generation IX with SDN technology
 Web portal-based path provisioning between ISPs
• Inter-AS L2 connectivity
– VLAN-based path provisioning
– Private peer provisioning
 Protect network from DDoS attack
• On-demand 5-tuple-baesd packet filtering
 SDN IX controller and distributed SDN/OpenFlow IX core switch
Developed by NECOMA project
(NAIST and University of Tokyo)
ISP-EISP F
SW
SWSW
SW
ISP-EISP F

 Two Lagopus (soft switch) are deployed for
SDN-IX core switch
 Multiple 10Gbps links
 Dual Xeon E5 8core CPUs
Lagopus @ ShowNet 2015

Lagopus @ ShowNet rack

Connectivity between AS
qfx10k ne5kx8
AS290AS131154
DIX-IEJPIXKDDI
CRS-4
10G-LR
lagopus-1
(DPID:2)
pf5240-1
(DPID:1)
ax.noteJGNX
lagopus-2
(DPID:4)
pf5240-2
(DPID:3)
xg-89:0.1
(port 4)
xg-83:0.0
(port 1)
xg-89:00.0
(port 3)
xg-83:00.1
(port 2)
xg-83:0.0
(port 1)
xg-83:0.1
(port 2)
xg-89:0.0
(port 3)
xg-1-0-49
(port 49)
xg-1-0-51
(port 51)
xg-1-0-52
(port 52)
xg-1-0-50
(port 50)
xg-1-0-49
(port 49)
xg-1-0-50
(port 50)
xg-1-0-51
(port 51)
799, 1600, 1060, 810, 910, 920 (tmporally)2, 3000
???
100
???
Otemachi
Makuhari (Veneue)

 Average 2Gbps throughput
 No packet drop
 No reboot & no trouble for 1 week during Interop Tokyo
 Sometimes 10Gbps burst traffic
Traffic on Lagopus @Makuhari

Big change happened
Before After
 
vSwitch has
lots of issues on
performance,
scalability,
stability, …..
vSwitch works
well without
any trouble!
Good
performance,
Good stability.

 The SDI special prize of Show Award
in Interop Tokyo 2015
 http://www.interop.jp/2015/english/exhibition/bsa.html
 Finalist
 The SDI
 ShowNet demonstration
Award

DPDK-enabled SDN/NFV middleware
with Lagopus & VNF with Vhost
@Interop Tokyo 2016
This trial was collaboration with
University of Tokyo and IPIfusion

NFV middleware for scale-out VNFs
 Flexible load balance for VNFs with
smart hash calculation and flow direction
 Hash calc: NetFPGA-SUME
• Hash calculation using IP address pairs
• Hash value are injected to MAC src for flow direction for VNF
 Classification and flow direction: Lagopus
• Flow direction with MAC src lookup
HV VNF VNF VNF
lagopus
lagopus
uplink
downlink
hash calc &
mac rewrite
MAC-based
classification for VMs
hash dl_src
type1 52:54:00:00:00:01
type2 52:54:00:00:00:02
… …
Type 256 52:54:00:00:00:FF

Bird in ShowNet
 Two Lagopus deployments
 NFV domain, SDN-IX
https://www.facebook.com/interop.shownet

Challenges in Lagopus
 vNIC between DPDK-enabled Lagopus and DPDK-enabled
VNF (Virnos)
 Many vNICs and flow director (loadbalancing)
 8 VNFs and total 18 vNICs
HV VirNOS VirNOS VirNOS VirNOS
lagopus
lagopus
port2
port4 port6 port8 port10
port9port7port5port3
port1
Eth0
Eth1
Eth0
Eth1
Eth0
Eth1
Eth0
Eth1

Explicit resource assignment for performance
 Packet processing workload aware assignment is required
for Lagopus and VNF
Memory
Memory
NIC
core
core
core
core
core
core
core
core
core
core
core
core
core
core
core
core
CPU0 CPU1
Traffic

Resource assign impacts in packet processing
performance
Memory
Memory
NIC
core
core
core
core
core
core
core
core
core
core
core
core
core
core
core
core
CPU0 CPU1
Traffic
Lagopus 8 VNFs
Memory
Memory
NIC
core
core
core
core
core
core
core
core
core
core
core
core
core
core
core
core
CPU0 CPU1
Traffic
Lagopus8 VNFs
10Gbps
4.4Gbps

lagopus
lagopus
port2
port1
Eth0
Eth1
Eth0
Eth1
Eth0
Eth1
Eth0
Eth1
 DPDK-based system needs CPUs for I/O because polling-
based network I/O in DPDK
 Physical I/O are intensive compared to vNICs
CPU resource assignment for I/O (1/2)
84
10/4 Gbps 10Gbps

lagopus
lagopus
port2
port1
Eth0
Eth1
Eth0
Eth1
Eth0
Eth1
Eth0
Eth1
 Traffic-path-aware CPU assign
 4 CPU core were assigned to I/O thread of Lagopus
CPU resource assignment for I/O (2/2)
85
10Gbps
10Gbps
5Gbps 5Gbps

Performance evaluation
 Good performance and scalability
 But long packet journey
 Packet-in -> Physical NIC -> Lagopus -> vNIC -> VNF -> vNIC ->
Lagopus -> Physical NIC -> Packet-out
[byte]
[Mbps]
0
1000
2000
3000
4000
5000
6000
7000
8000
9000
10000
0 500 1000 1500
wire rate
lagopus
Packet size
Traffic

Other trials

 Location-aware packet forwarding
+ Service Chain (NFV integration)
 Location-aware transparent security
check by NFV
 Virtual Network
 Intra network
• Web service and clients
• Malware site blocking
 Lab network
• Ixia tester for demo
• Policy management (Explicit routing for TE)
#1: Segment routing with Lagopus for campus network
lago0-0
lago0-1 lago1-1
lago1-0
lago2-1
lago2-0
nfv0
win00
Serv01
Serv00
win01
p1
p1
vFW vFW
web
Untrusted server
block service
lago0
-0
lago0
-1
lago1
-1
lago1
-0
lago2
-1
lago2
-0
nfv0
Ixia
p
3
vFW vFW
Ixia

 Flexible video stream transmission
for multiple-sites and devices
 Lagopus switch as stream duplicator
 Simultaneous 4K video – 50 sites streaming
#2: transparent video stream duplication
111
Encoder
Decoder
Live
IP NW
Cinema or public space

#2: Blackboard streaming without Teacher’s shadow
Human shadow transparent module
Pattern A
Pattern B
 Realtime image processing with flow direction switch
 User can select modes with teacher or without teacher
 No configuration change without video options
Students can not see due to shadow Transparent processing help
students 
Realtime image
processing

TM
O3 project: providing flexible wide
area networks with SDN
This research is executed under a part of a Research and Development of Network
Virtualization Technology” program commissioned by the Ministry of Internal Affairs and
Communications. (Y2013-2016)

TM
Open Innovation over Network Platform
• 3 kinds of Contributions for User-oriented SDN
(1) Open development with OSS
(2) Standardization of architecture and interface
(3) Commercialization of new technologies
Toward open User-oriented SDN
©O3 Project 92
(1) Open (2) Standardization (3) Commercialization

TM
• Open, Organic, Optima
– Anyone, Anything, Anywhere
– Neutrality & Efficiency for Resource, Performance, Reliability, ….
– Multi-Layer, Multi-Provider, Multi-Service
• User-oriented SDN for WAN
– Softwarization: Unified Tools and Libraries
– On-demand, Dynamic, Scalable, High-performance
• Features
– Object-defined Network Framework
– SDN WAN Open Source Software
– SDN Design & Operations Guideline
• Accelerates
– Service Innovation, Re-engineering, Business Eco-System
The O3 Project Concept, Approach, & Goal
©O3 Project 93

TM
The O3 User-oriented SDN Architecture
©O3 Project 94
Path Nodes
（Opt・Pkt Transport）
Switch Nodes
（Lagopus, OF ）
D-plane
C-plane
D-plane consists of Switch and Path Nodes; Switching Nodes provide programmability,
and Path Nodes provide various type of network resources.
Orchestrator & Controllers can create and configure virtual
networks according to SDN Users, and enable to customized
control on individual D-Plane.
Virtual NW Virtual NW
OTTs
Carriers
OTT-A
Cnt. Appl.
OTT-B
Cnt. Appl.
Controls on Virtual
NW
View from
Virtual NW
Network Orchestrator
Switch Nodes
（Lagopus, OF）
Controller
(スイッチ部)
Controller
(スイッチ部)
Controller
（Switch Nodes）
Controller
(パス部)
Controller
(パス部)
Controller
（Path Nodes）
Controller
(スイッチ部)
Controller
(スイッチ部)
Controller
（Switch Nodes）
Common
Control
Framework
SDN Nodes
Multi-Layer,
Multi-Domain
Control

Ryu SDN Framework
http://osrg.github.io/ryu/

 OSS SDN Framework founded by NTT
 Software for building SDN control plane agilely
 Fully implemented in Python
 Apache v2 license
 More than 350 mailing list subscribers
 Supporting the latest southbound protocols
 OpenFlow 1.0, 1.2, 1.3, 1.4 (and Nicira extensions)
 BGP
 Ofconfig 1.2
 OVSDB JSON
What’s RYU?

Many users
and more…

 Developed mainly for network operators
 Not for one who sells the specific hardware switch
 Integration with the existing networks
 Gradual SDN’ing’ the existing networks
Ryu development principles

 Your application are free from OF wire format (and some details
like handshaking)
What ‘supporting OpenFlow’ means?
Python
Object
OF wire
protocol
Data
Plane
Ryu converts it
Python
Object
OF wire
Protocol
Ryu generates
Your application
does something here

Ryu development is automated
github
Push the new code
Unit tests are executed
Docker hub image
is updated
Ryu certification is
executed on test
lab
Ryu certification
site is updated
You can update
your Ryu
environment
with one command

Lessons leaned

 What’s OpenStack?
 OSS for building IaaS
 You can run lots of VMs
 Many SDN solutions are supported
 What SDN means for OpenStack?
 The network for your VMs are separated from others
 Virtual L2 network on the top of L3 network
SDN in OpenStack

 Virtual L2 on tunnels (VXLAN, GRE, etc)
Typical virtual L2 implementation
OVS
Agent
Compute node
VM VM
OVS
Agent
Compute node
VM VM
OVS
Agent
Compute node
VM VM
OVS
Agent
Compute node
VM VM
Tunnel

People advocated something like this
Data
Plane
Data
Plane
Data
Plane
OpenFlow Controller
OpenFlow Protocol
Application Logic

 Same as other OpenFlow controllers
 The controller are connected with all the OVSes
Our first version OpenStack integration
Plugin
Neutron
Server
Ryu
OVS
RYU
Agent
Compute node
VM VM
Custom REST API
OpenFlow
OVS
Agent
Compute node
VM VM
OVS
Agent
Compute node
VM VM
OpenStack REST API
SDN
Operational
Intelligence

 Scalability
 Availability
What’s the problems?

 How many a single controller can handle?
 Can handle hundreds or thousands?
 Controller does more than setting up flows
 Replying to ARP packet requests rather than sending ARP packets to all
compute nodes
 Making OVS work as L3 router rather than sending packets to a central router
 You could add more here
Scalability

 The death of a controller leads to the dead of the whole cloud
 No more network configuration
Availability

 OFC on every compute node
 One controller handles only one OVS
Our second verion (OFAgent driver)
Neutron
Server
OVS
RYU
Agent
Compute node
VM VM
OVS
RYU
Agent
Compute node
VM VM
OVS
RYU
Agent
(OFC)
Compute node
VM VM
OpenStack standard RPC
Over queue system
Released in Icehouse
OpenStack REST API
Openflow is used only inside a compute node
• Scalable with the number of compute nodes
• No single point of failure in OFAgent
SDN
Operational
Intelligence

 Push more features to edges
 Distribute features
 Place only a feature (e.g. TE) on central node you can’t distribute
 Couple loosely a central node and edges
 Tight coupling doesn’t scale (e.g. OpenFlow connections between a controller
and switches)
 The existing technology like queue works
SDN deployment for scale

 NSA (National Security Agency)
More users: Tracking network activities
“The NSA is using NTT’s Ryu SDN controller. Larish says it’s a few thousand
lines of Python code that’s easy to learn, understand, deploy and troubleshoot”
http://www.networkworld.com/article/2937787/sdn/nsa-uses-openflow-for-tracking-its-network.html

 TouIX (IX in France) : Replacing expensive legacy switch with
whitebox switch and Ryu
More users: IX (Internet Exchange)
“The deployment is leveraging Ryu, the NTT Labs open-source controller”
http://finance.yahoo.com/news/pica8-powers-sdn-driven-internet-120000932.html

Zebra 2.0
Open Source Routing Software

Open Source Revisited
• Apache License
• Written From Scratch in Go
• Go routine & Go channel is used for
multiplexing
• Task Completion Model + Thread Model
• Single SPF Engine for OSPFv2/OSPFv3/IS-IS
• Forwarding Engine Abstraction for DPDK/OF-
DPA
• Configuration with Commit/Rollback
• gRPC for Zebra control

Architecture
• Single Process/Multithread Architecture
BGP OPSF RSVP-TE LDP
FEA

OpenConfigd
• Commit & Rollback support configuration system
• Configuration is defined by YANG
• CLI, NetConf, REST API is automatically
generated
• `confsh` - bash based CLI command
• OpenConfig is fully supported

OpenConfigd Architecture
• gRPC is used for transport
• completion/show/config APIs between shell
OpenConfigd
Zebra 2.0 Lagopus
confsh
DB
completion show config
API API

Forwarding Engine Abstraction
• Various Forwarding Engine Exists Today
• OS Forwarder Layer
• DPDK
• OF-DPA
• FEA provides Common Layer for Forwarding Engine
• FEA provides
• Interface/Port Management
• Bridge Management
• Routing Table
• ARP Table

Current limitation of software switch
 # of OF apps is limited
 Leverage Network OS for whtitebox switch
• Opensnaproute, Linux kernel, OpenSwitch
 Hard to integrate with other management system
 OpenStack, libvirt,
 nFlow, sFlow, BGP-extension
 OF pipeline does not cover all our requirements
 Tunnel termination
• IPsec, VxLAN,
 Control packet escalation/injection
 User-defined OAM functionality
 Heavy packet processing for OpenFlow flow entry
 Lookup, action, long pipeline
 Balance of programmability and existing network protocol support
 L2, L3 (IPv4, IPv6), GRE, VxLAN, MPLS
 Hybrid traffic control

 Provide router-aware programmable
dataplane for network OS
 Protocol-aware pipeline and APIs, OpenFlow
 Integration with network OS
 Existing forwarding &routing protocols
support (BGP, OSPF)
 VPN framework over IP networks
 IP as a transport protocol
 Vxlan, GRE, IPsec tunnel support
 Decouple OpenFlow semantics and
Wireprotocol from OpenFlow
protocol
 Provide gRPC switch control API
Next major upgrade: Lagopus SDN switch router

Forwarding Engine Integration
dataplane
Lagopus
BGP OPSF RSVP-
TE
LDP
FEA
Zebra 2.0
OpenConfigd DB
Dataplane
Manager
Configuration
datastore
RIB/FIB
control Interface/Port
bridge mngmt
Interface/Port
bridge mngmt
C-plane related packet
User-traffic User-traffic
C-plane related packet
C-plane packet
escalation via tap IF
FIB
ARP
Stats
DB
C-plane packet
escalation via tap IF

 New version will available this summer
Availability?

Hirokazu Takahashi, Tomoya Hibi, Ichikawa, Masaru Oki,
Motonori Hirano, Kiyoshi Imai, Takaya Hasegawa,
Tomohiro Nakagawa, Koichi Shigihara, Keisuke Kosuga,
Takanari Hayama, Tetsuya Mukawa, Saori Usami,
Kunihiro Ishiguro
Thanks our development team

 Comments and collaboration are very welcome!
 Web
 https://lagopus.github.io
 Github
 Lagopus vswitch
• https://github.com/lagopus/lagopus
 Lagopus book
• http://www.lagopus.org/lagopus-book/en/html/
 Ryu with general tunnel ext
• https://github.com/lagopus/ryu-lagopus-ext
Conclusion

Questions?

Software Stacks to enable SDN and NFV

More Related Content

What's hot (20)

Viewers also liked (20)

Similar to Software Stacks to enable SDN and NFV (20)

Recently uploaded (20)

Software Stacks to enable SDN and NFV