The n00bs guide to ovs dpdk

The n00bs' guide to
DPDK and OVS-DPDK
Mark Gray (mark.d.gray@intel.com)
Intel

Disclaimers
This presentation is targeted
towards beginners who have never
used DPDK or OVS-DPDK.
Text may be small if you are at the
back, please feel free to move to
the front! Or go to
http://goo.gl/21ixdy
Methodology is correct at this
time but things change quickly!
If you want to try this then
check the versions!

Host OS
Traffic Generator
testpmd
Software
Versions
• Ubuntu 14.0.4
LTS
• DPDK v2.0.0

/fosdem # git clone http://dpdk.org/git/dpdk
Cloning into 'dpdk'...
remote: Counting objects: 37650, done.
remote: Compressing objects: 100% (7810/7810), done.
remote: Total 37650 (delta 29652), reused 37579 (delta
29605)
Receiving objects: 100% (37650/37650), 23.27 MiB | 3.60
MiB/s, done.
Resolving deltas: 100% (29652/29652), done.
Checking connectivity... done.
/fosdem #
/fosdem #
/fosdem #

/fosdem # cd dpdk/
/fosdem/dpdk # git checkout v2.0.0
Note: checking out 'v2.0.0'.
You are in 'detached HEAD' state. You can look around,
make experimental
changes and commit them, and you can discard any commits
you make in this
state without impacting any branches by performing another
checkout.
If you want to create a new branch to retain commits you
create, you may
do so (now or later) by using -b with the checkout command
again. Example:
git checkout -b new_branch_name
HEAD is now at 493db80... version: 2.0.0
/fosdem/dpdk #

/fosdem/dpdk # make install T=x86_64-native-linuxapp-gcc
================== Installing x86_64-native-linuxapp-gcc
Configuration done
== Build lib
== Build lib/librte_compat
SYMLINK-FILE include/rte_compat.h
== Build lib/librte_eal
== Build lib/librte_eal/common
SYMLINK-FILE include/generic/rte_atomic.h
SYMLINK-FILE include/generic/rte_byteorder.h
SYMLINK-FILE include/generic/rte_cycles.h
SYMLINK-FILE include/generic/rte_prefetch.h
SYMLINK-FILE include/generic/rte_spinlock.h
SYMLINK-FILE include/generic/rte_memcpy.h
SYMLINK-FILE include/generic/rte_cpuflags.h
SYMLINK-FILE include/rte_branch_prediction.h
<SNIP>

<SNIP>
INSTALL-APP testpmd
INSTALL-MAP testpmd.map
== Build app/cmdline_test
CC cmdline_test.o
CC commands.o
LD cmdline_test
INSTALL-APP cmdline_test
INSTALL-MAP cmdline_test.map
== Build app/dump_cfg
CC main.o
LD dump_cfg
INSTALL-APP dump_cfg
INSTALL-MAP dump_cfg.map
Build complete
/fosdem/dpdk #
/fosdem/dpdk #
/fosdem/dpdk # vim ./x86_64-native-linuxapp-gcc/.config

#CONFIG_RTE_TOOLCHAIN="gcc"
#CONFIG_RTE_TOOLCHAIN_GCC=y
# Use intrinsics or assembly code for key routines
CONFIG_RTE_FORCE_INTRINSICS=n
# Compile to share library
CONFIG_RTE_BUILD_SHARED_LIB=n
# Combine to one single library
CONFIG_RTE_BUILD_COMBINE_LIBS=n
CONFIG_RTE_LIBNAME="intel_dpdk"
# Compile Environment Abstraction Layer
CONFIG_RTE_LIBRTE_EAL=y
CONFIG_RTE_MAX_LCORE=128
CONFIG_RTE_MAX_NUMA_NODES=8
CONFIG_RTE_MAX_MEMSEG=256
CONFIG_RTE_MAX_MEMZONE=2560

#CONFIG_RTE_TOOLCHAIN="gcc"
#CONFIG_RTE_TOOLCHAIN_GCC=y
# Use intrinsics or assembly code for key routines
CONFIG_RTE_FORCE_INTRINSICS=n
# Compile to share library
CONFIG_RTE_BUILD_SHARED_LIB=n
# Combine to one single library
CONFIG_RTE_BUILD_COMBINE_LIBS=y
CONFIG_RTE_LIBNAME="intel_dpdk"
# Compile Environment Abstraction Layer
CONFIG_RTE_LIBRTE_EAL=y
CONFIG_RTE_MAX_LCORE=128
CONFIG_RTE_MAX_NUMA_NODES=8
CONFIG_RTE_MAX_MEMSEG=256
CONFIG_RTE_MAX_MEMZONE=2560

/fosdem/dpdk # cd x86_64-native-linuxapp-gcc/
/fosdem/dpdk/x86_64-native-linuxapp-gcc # make
== Build lib
== Build lib/librte_compat
SYMLINK-FILE include/rte_compat.h
== Build lib/librte_eal
== Build lib/librte_eal/common
SYMLINK-FILE include/generic/rte_atomic.h
SYMLINK-FILE include/generic/rte_byteorder.h
SYMLINK-FILE include/generic/rte_cycles.h
SYMLINK-FILE include/generic/rte_prefetch.h
SYMLINK-FILE include/generic/rte_spinlock.h
SYMLINK-FILE include/generic/rte_memcpy.h
SYMLINK-FILE include/generic/rte_cpuflags.h
SYMLINK-FILE include/rte_branch_prediction.h
SYMLINK-FILE include/rte_common.h
<SNIP>

/fosdem/dpdk # export RTE_SDK=`pwd`
/fosdem/dpdk # export RTE_TARGET=x86_64-native-linuxapp-g
cc
/fosdem/dpdk # cd app/test-pmd/
/fosdem/dpdk/app/test-pmd # make
CC testpmd.o
CC parameters.o
CC cmdline.o
<SNIP>

/fosdem/dpdk/app/test-pmd # modprobe uio

/fosdem/dpdk/app/test-pmd # insmod /fosdem/dpdk/x86_64-
native-linuxapp-gcc/kmod/igb_uio.ko

/fosdem/dpdk/app/test-pmd # insmod /fosdem/dpdk/x86_64-
native-linuxapp-gcc/kmod/igb_uio.ko
/fosdem/dpdk/app/test-pmd #
$RTE_SDK/tools/dpdk_nic_bind.py --status
Network devices using DPDK-compatible driver
============================================
<none>
Network devices using kernel driver
===================================
0000:05:00.0 '82599ES 10-Gigabit SFI/SFP+ Network
Connection' if=eth1 drv=ixgbe unused=igb_uio,vfio-pci
Connection' if=eth4 drv=ixgbe unused=igb_uio,vfio-pci
<SNIP>

$RTE_SDK/tools/dpdk_nic_bind.py -b igb_uio 0000:81:00.0
0000:81:00.1

$RTE_SDK/tools/dpdk_nic_bind.py -b igb_uio 0000:81:00.0
0000:81:00.1
$RTE_SDK/tools/dpdk_nic_bind.py --status
============================================
0000:81:00.0 'Ethernet 10G 2P X710 Adapter' drv=igb_uio
unused=vfio-pci
0000:81:00.1 'Ethernet 10G 2P X710 Adapter' drv=igb_uio
unused=vfio-pci
<SNIP>

/fosdem/dpdk/app/test-pmd # mkdir –p /mnt/huge

/fosdem/dpdk/app/test-pmd # echo 1024 >
/sys/kernel/mm/hugepages/hugepages-2048kB/nr_hugepages

/fosdem/dpdk/app/test-pmd # echo 1024 >
/sys/kernel/mm/hugepages/hugepages-2048kB/nr_hugepages
/fosdem/dpdk/app/test-pmd # mount -t hugetlbfs nodev
/mnt/huge

/fosdem/dpdk/app/test-pmd # ./testpmd -c 0xF -n 4 -- -i --
forward-mode=io --portmask=0x3 --nb-cores=2

/fosdem/dpdk/app/test-pmd # ./testpmd -c 0xF -n 4 -- -i --
forward-mode=io --portmask=0x3 --nb-cores=2
EAL: Detected lcore 0 as core 0 on socket 0
<SNIP>
EAL: Support maximum 128 logical core(s) by configuration.
EAL: Detected 20 lcore(s)

<SNIP>
EAL: Setting up memory...
EAL: Ask a virtual area of 0x200000 bytes
EAL: Virtual area found at 0x7f62e8000000 (size =
0x200000)
EAL: Virtual area found at 0x7f62e2a00000 (size =
0x5400000)
EAL: Virtual area found at 0x7f62e2000000 (size =
<SNIP>
EAL: PCI device 0000:05:00.0 on NUMA socket 0
EAL: probe driver: 8086:10fb rte_ixgbe_pmd
EAL: Not managed by a supported kernel driver, skipped
<SNIP>
EAL: probe driver: 8086:1572 rte_i40e_pmd
EAL: PCI memory mapped at 0x7f6261df0000

<SNIP>
Port 1: 00:00:00:00:01:01
Checking link statuses...
Port 0 Link Up - speed 10000 Mbps - full-duplex
Done
testpmd>

<SNIP>
Port 1: 00:00:00:00:01:01
Checking link statuses...
Done
testpmd> help
Help is available for the following sections:
help control : Start and stop forwarding.
help display : Displaying port, stats and config
information.
help config : Configuration information.
help ports : Configuring ports.
help registers : Reading and setting port registers.
help filters : Filters configuration help.
help all : All of the above sections.

testpmd> help control
Control forwarding:
-------------------
start
Start packet forwarding with current configuration.
start tx_first
Start packet forwarding with current config after
sending one burst of packets.
stop
Stop packet forwarding, and display accumulated
statistics.
quit
Quit to prompt.

testpmd> start
io packet forwarding - CRC stripping disabled -
packets/burst=32
nb forwarding cores=2 - nb forwarding ports=2
RX queues=1 - RX desc=128 - RX free threshold=32
RX threshold registers: pthresh=8 hthresh=8 wthresh=0
TX queues=1 - TX desc=512 - TX free threshold=32
TX threshold registers: pthresh=32 hthresh=0 wthresh=0
TX RS bit threshold=32 - TXQ flags=0xf01

testpmd> stop
Telling cores to stop...
Waiting for lcores to finish...
---------------------- Forward statistics for port 0 ---------
RX-packets: 75883815 RX-dropped: 0 RX-total:
TX-packets: 0 TX-dropped: 0 TX-total:
---------------------------------------------------------------
---------------------- Forward statistics for port 1 ---------
---------------------------------------------------------------
+++++++++++++++ Accumulated forward statistics for all ports+++
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Done.
testpmd>

testpmd> quit
Stopping port 0...done
Stopping port 1...done
bye...

Host OS
Guest OS
Traffic Generator
testpmd
Software
Versions
• Ubuntu 14.0.4
LTS
• DPDK v2.0.0
• OVS v2.4
• Qemu v2.3.1
• Fedora 21 in VM

/fosdem # git clone https://github.com/openvswitch/ovs.git
Cloning into 'ovs'...
remote: Total 92972 (delta 6), reused 0 (delta 0), pack-reused 92
Receiving objects: 100% (92972/92972), 40.80 MiB | 1.76 MiB/s, do
/fosdem #

/fosdem # cd ovs

/fosdem # cd ovs
/fosdem/ovs # git checkout branch-2.4
Branch branch-2.4 set up to track remote branch branch-2.4 from o
Switched to a new branch 'branch-2.4'
/fosdem/ovs #

/fosdem/ovs # DPDK_BUILD=/fosdem/dpdk/x86_64-native-li
nuxapp-gcc

/fosdem/ovs # DPDK_BUILD=/fosdem/dpdk/x86_64-native-li
nuxapp-gcc
/fosdem/ovs # ./boot.sh
libtoolize: putting auxiliary files in AC_CONFIG_AUX_DIR, `build-
libtoolize: copying file `build-aux/ltmain.sh'
libtoolize: putting macros in AC_CONFIG_MACRO_DIR, `m4'.
libtoolize: copying file `m4/libtool.m4'
libtoolize: copying file `m4/ltoptions.m4'
libtoolize: copying file `m4/ltsugar.m4'
libtoolize: copying file `m4/ltversion.m4'
libtoolize: copying file `m4/lt~obsolete.m4'
configure.ac:24: installing 'build-aux/compile'
configure.ac:43: installing 'build-aux/config.guess'
configure.ac:43: installing 'build-aux/config.sub'
configure.ac:22: installing 'build-aux/install-sh'
configure.ac:22: installing 'build-aux/missing'
Makefile.am: installing 'build-aux/depcomp'
/fosdem/ovs #

/fosdem/ovs # ./configure --with-dpdk=$DPDK_BUILD
checking for a BSD-compatible install... /usr/bin/install -c
checking whether build environment is sane... yes
checking for a thread-safe mkdir -p... /bin/mkdir -p
checking for gawk... gawk
checking whether make sets $(MAKE)... yes
checking whether make supports nested variables... yes
checking how to create a pax tar archive... gnutar
checking for style of include used by make... GNU
checking for gcc... gcc
checking whether the C compiler works... yes
checking for C compiler default output file name... a.out
checking for suffix of executables...
checking whether we are cross compiling... no
checking for suffix of object files... o
checking whether we are using the GNU C compiler... yes
checking whether gcc accepts -g... yes
checking for gcc option to accept ISO C89... none needed
checking whether gcc understands -c and -o together.
<SNIP>

/fosdem/ovs # make CFLAGS='-O3 -march=native’
make all-recursive
make[1]: Entering directory `/fosdem/ovs'
Making all in datapath
make[2]: Entering directory `/fosdem/ovs/datapath'
make[3]: Leaving directory `/fosdem/ovs/datapath'
depbase=`echo lib/bfd.lo | sed 's|[^/]*$|.deps/&|;s|.lo$||'`;
/bin/bash ./libtool --tag=CC --mode=compile gcc -std=g
-I ./include -I ./include -I ./lib -I ./lib -Wstrict-prototype
compare -Wpointer-arith -Wformat-security -Wswitch-enum -Wunused-
cast -Wcast-align -Wmissing-prototypes -Wmissing-field-initialize
I/fosdem/dpdk/x86_64-native-linuxapp-gcc/include -mssse3 -D_FILE_
march=native -MT lib/bfd.lo -MD -MP -MF $depbase.Tpo -c -o lib/bf
mv -f $depbase.Tpo $depbase.Plo
<SNIP>

PYTHONPATH=./python":"$PYTHONPATH PYTHONDONTWRITEBYTECODE=yes /us
aux/check-structs -I./include include/openflow/openflow.h &&
nclude/openflow/openflow.hstamp
make[2]: Leaving directory `/fosdem/ovs'
/fosdem/ovs #
/fosdem/ovs #
/fosdem/ovs #

PYTHONPATH=./python":"$PYTHONPATH PYTHONDONTWRITEBYTECODE=yes /us
aux/check-structs -I./include include/openflow/openflow.h &&
nclude/openflow/openflow.hstamp
/fosdem/ovs #
/fosdem/ovs #
/fosdem/ovs # make install
make install-recursive
Making install in datapath
make[4]: Nothing to be done for `install-exec-am'.
make[4]: Nothing to be done for `install-data-am'.
<SNIP>

/fosdem/ovs # modprobe uio
/fosdem/ovs # insmod $DPDK_BUILD/kmod/igb_uio.ko
/fosdem/ovs # RTE_SDK=/fosdem/dpdk
/fosdem/ovs # $RTE_SDK/tools/dpdk_nic_bind.py -b igb_uio 00
00:05:00.0 0000:05:00.1
/fosdem/ovs # $RTE_SDK/tools/dpdk_nic_bind.py –status
============================================
0000:05:00.0 '82599ES 10-Gigabit SFI/SFP+ Network Connection' drv
0000:05:00.1 '82599ES 10-Gigabit SFI/SFP+ Network Connection' drv
===================================
0000:09:00.0 '82599ES 10-Gigabit SFI/SFP+ Network Connection' if=
unused=igb_uio
<SNIP>

/fosdem/ovs # echo 8196 > /sys/kernel/mm/hugepages/hugepages-
2048kB/nr_hugepages
/fosdem/ovs #
/fosdem/ovs # mount -t hugetlbfs nodev /mnt/huge
/fosdem/ovs #
/fosdem/ovs # cat /proc/meminfo | grep -i huge
AnonHugePages: 4096 kB
HugePages_Total: 8196
HugePages_Free: 8196
HugePages_Rsvd: 0
HugePages_Surp: 0
Hugepagesize: 2048 kB

/fosdem/ovs # mkdir -p /usr/local/etc/openvswitch
/fosdem/ovs # mkdir -p /usr/local/var/run/openvswitch
/fosdem/ovs #
/fosdem/ovs #
/fosdem/ovs # rm -f /usr/local/etc/openvswitch/conf.db
/usr/local/etc/openvswitch/.conf.db~lock~
/fosdem/ovs #
/fosdem/ovs #
/fosdem/ovs # /usr/local/bin/ovsdb-tool create /usr/local/e
tc/openvswitch/conf.db /usr/local/share/openvswitch/vswitch
.ovsschema
/fosdem/ovs #

/fosdem/ovs # /usr/local/sbin/ovsdb-server --
remote=punix:/usr/local/var/run/openvswitch/db.sock --
remote=db:Open_vSwitch,Open_vSwitch,manager_options –-pidfile
--detach
2016-01-25T10:23:27Z|00001|ovsdb_server|INFO|ovsdb-server (Open v
/fosdem/ovs #
/fosdem/ovs # /usr/local/bin/ovs-vsctl --no-wait init
/fosdem/ovs #

/fosdem/ovs # /usr/local/sbin/ovs-vswitchd --dpdk -c 0x1 -n 4
--socket-mem 1024,0 -- unix:/usr/local/var/run/openvswitch/db
.sock --log-file=/tmp/vswitchd.log --pidfile --detach
2016-01-25T10:24:14Z|00001|dpdk|INFO|No -vhost_sock_dir provided
/usr/local/var/run/openvswitch
<SNIP>
EAL: Support maximum 128 logical core(s) by configuration.
EAL: Detected 20 lcore(s)
<SNIP>

EAL: Setting up memory...
EAL: Virtual area found at 0x7f89cba00000 (size = 0x200000)
EAL: Ask a virtual area of 0x2e000000 bytes
EAL: Virtual area found at 0x7f899d800000 (size = 0x2e000000)
EAL: Virtual area found at 0x7f899d400000 (size = 0x200000)
EAL: Virtual area found at 0x7f899d000000 (size = 0x200000)
EAL: Ask a virtual area of 0x1cfc00000 bytes
EAL: Virtual area found at 0x7f87cd200000 (size = 0x1cfc00000)
EAL: Virtual area found at 0x7f87cca00000 (size = 0x600000)
EAL: Virtual area found at 0x7f87cb800000 (size = 0x1000000)
EAL: Virtual area found at 0x7f87cb400000 (size = 0x200000)
EAL: Virtual area found at 0x7f87cae00000 (size = 0x400000)
EAL: Virtual area found at 0x7f87ca800000 (size = 0x400000)
EAL: Virtual area found at 0x7f87ca400000 (size = 0x200000)

<SNIP>
EAL: PCI memory mapped at 0x7f89cbc00000
PMD: eth_ixgbe_dev_init(): MAC: 2, PHY: 17, SFP+: 5
PMD: eth_ixgbe_dev_init(): port 0 vendorID=0x8086 deviceID=0x10fb
EAL: PCI memory mapped at 0x7f89cbd04000
PMD: eth_ixgbe_dev_init(): MAC: 2, PHY: 17, SFP+: 6
PMD: eth_ixgbe_dev_init(): port 1 vendorID=0x8086 deviceID=0x10fb
EAL: probe driver: 8086:1572 rte_i40e_pmd
EAL: Not managed by a supported kernel driver, skipped
<SNIP>
Zone 3: name:<rte_eth_dev_data>, phys:0x87802080, len:0x1f400, vi
socket_id:0, flags:0
2016-01-25T10:24:22Z|00002|vlog|INFO|opened log file /tmp/vswitch
2016-01-25T10:24:22Z|00003|ovs_numa|INFO|Discovered 10 CPU cores

/fosdem/ovs # /usr/local/bin/ovs-vsctl add-br br0 -- set
bridge br0 datapath_type=netdev
/fosdem/ovs #
/fosdem/ovs #

/fosdem/ovs # /usr/local/bin/ovs-vsctl add-br br0 –- set
/fosdem/ovs #
/fosdem/ovs # /usr/local/bin/ovs-vsctl add-port br0 dpdk0
-- set Interface dpdk0 type=dpdk
-- set Interface dpdk1 type=dpdk
/fosdem/ovs #

/fosdem/ovs #
-- set interface dpdk0 type=dpdk
/fosdem/ovs # /usr/local/bin/ovs-vsctl add-port br0 vhost
-user-0 -- set Interface vhost-user-0 type=dpdkvhostuser
-user-1 -- set interface vhost-user-1 type=dpdkvhostuser
/fosdem/ovs #
/fosdem/ovs #

/fosdem/ovs #
/fosdem/ovs #
/fosdem/ovs #
/fosdem/ovs #
/fosdem/ovs # ls -al /usr/local/var/run/openvswitch/vhost
-user*
srwxr-xr-x 1 root root 0 Jan 25 10:26 /usr/local/var/run/
openvswitch/vhost-user-0
srwxr-xr-x 1 root root 0 Jan 25 10:26 /usr/local/var/run/
openvswitch/vhost-user-1

/fosdem/ovs # cd ..
/fosdem # wget http://wiki.qemu-project.org/download/qemu-2.3.1.t
--2016-01-25 10:27:06-- http://wiki.qemu-project.org/download/qe
Length: 24692020 (24M) [application/x-bzip2]
Saving to: ‘qemu-2.3.1.tar.bz2’
100%[============================================================
===============================>] 24,692,020 244KB/s in 45s
2016-01-25 10:27:52 (535 KB/s) - ‘qemu-2.3.1.tar.bz2’ saved [2469
/fosdem #

/fosdem/ovs # cd ..
100%[============================================================
===============================>] 24,692,020 244KB/s in 45s
/fosdem # tar -xf qemu-2.3.1.tar.bz2
/fosdem #

/fosdem/ovs # cd ..
100%[============================================================
===============================>] 24,692,020 244KB/s in 45s
/fosdem # cd qemu-2.3.1
/fosdem/qemu-2.3.1 #

/fosdem/ovs # cd ..
100%[============================================================
===============================>] 24,692,020 244KB/s in 45s
/fosdem # cd qemu-2.3.1
/fosdem/qemu-2.3.1 # ./configure --enable-kvm --target-list=x86_6
Install prefix /usr/local
BIOS directory /usr/local/share/qemu
binary directory /usr/local/bin
library directory /usr/local/lib
<SNIP>

lzo support no
snappy support no
bzip2 support no
NUMA host support no
/fosdem/qemu-2.3.1 # make
GEN x86_64-softmmu/config-devices.mak.tmp
GEN x86_64-softmmu/config-devices.mak
GEN config-all-devices.mak
GEN config-host.h
(cd /fosdem/qemu-2.3.1/pixman; autoreconf -v --install)
autoreconf: Entering directory `.'
autoreconf: configure.ac: not using Gettext
autoreconf: running: aclocal
autoreconf: configure.ac: tracing
autoreconf: running: libtoolize --copy
libtoolize: putting auxiliary files in `.'.
libtoolize: copying file `./ltmain.sh‘
<SNIP>

CPP optionrom/linuxboot.asm
AS optionrom/linuxboot.o
Building optionrom/linuxboot.img
Building optionrom/linuxboot.raw
Signing optionrom/linuxboot.bin
CPP optionrom/kvmvapic.asm
AS optionrom/kvmvapic.o
Building optionrom/kvmvapic.img
Building optionrom/kvmvapic.raw
Signing optionrom/kvmvapic.bin
/fosdem/qemu-2.3.1 # make install
make all-recursive
Making all in pixman
make[3]: Nothing to be done for `all'.
Making all in demos
Making all in test
LEX convert-dtsv0-lexer.lex.c
make[1]: flex: Command not found
BISON dtc-parser.tab.c
make[1]: bison: Command not found

/fosdem/qemu-2.3.1 # cd ..
/fosdem # /usr/local/bin/qemu-system-x86_64 -cpu host -boot c
-hda /fosdem/tmp/fedora-21.img -m 4096M -smp 2 --enable-kvm -
vnc :1 -net user,hostfwd=tcp::10022-:22 -net nic -chardev
socket,id=char0,path=/usr/local/var/run/openvswitch/vhost-
user-0 -netdev type=vhost-
user,id=mynet1,chardev=char0,vhostforce -device virtio-net-
pci,mac=00:00:00:00:00:01,netdev=mynet1 -chardev
pci,mac=00:00:00:00:00:02,netdev=mynet2 -object memory-
backend-file,id=mem,size=4096M,mem-path=/mnt/huge,share=on -
numa node,memdev=mem -mem-prealloc &

/fosdem/qemu-2.3.1 # cd ..
/fosdem # /usr/local/bin/qemu-system-x86_64 -cpu host -boot c
-hda /fosdem/tmp/fedora-21.img -m 4096M -smp 2 --enable-kvm -
vnc :1 -net user,hostfwd=tcp::10022-:22 -net nic -chardev
pci,mac=00:00:00:00:00:01,netdev=mynet1 -chardev
pci,mac=00:00:00:00:00:02,netdev=mynet2 -object memory-
backend-file,id=mem,size=4096M,mem-path=/mnt/huge,share=on -
numa node,memdev=mem -mem-prealloc &
[1] 50459
/fosdem/tmp #

/fosdem # scp -r -P 10022 $RTE_SDK root@localhost:/root/dpdk
root@localhost's password:
ixgbe_logs.h
100% 2985 2.9KB/s 00:00 ixgbe_bypass.c
100% 11KB 10.6KB/s 00:00 ixgbe_82599.c
100% 79KB 79.1KB/s 00:00 ixgbe_common.h
100% 8519 8.3KB/s 00:00 ixgbe_phy.h
100% 7171 7.0KB/s 00:00 ixgbe_api.h
<SNIP>

/fosdem # ssh root@localhost -p 10022
root@localhost's password:
Last login: Mon Jan 25 11:35:30 2016
[root@localhost ~]# cd /root/dpdk/
[root@localhost dpdk]# export RTE_SDK=`pwd`
[root@localhost dpdk]# export RTE_TARGET=x86_64-native-linux-gcc
[root@localhost dpdk]# make install T=$RTE_TARGET
<SNIP>
[root@localhost dpdk]# cd app/test-pmd
[root@localhost test-pmd]# make
<SNIP>
[root@localhost test-pmd]# cd ..
GREEN – Commands executed in the guest

[root@localhost dpdk]# $RTE_SDK/tools/dpdk_nic_bind.py –status
============================================
<none>
===================================
0000:00:03.0 '82540EM Gigabit Ethernet Controller' if=ens3 drv=e1
0000:00:04.0 'Virtio network device' if= drv=virtio-pci unused=vi
0000:00:05.0 'Virtio network device' if= drv=virtio-pci unused=vi
Other network devices
=====================
<none>
[root@localhost dpdk]# modprobe uio
[root@localhost dpdk]# insmod $RTE_SDK/x86_64-native-linuxapp-gcc
[root@localhost dpdk]# $RTE_SDK/tools/dpdk_nic_bind.py -b igb_uio
[root@localhost dpdk]#

[root@localhost dpdk]# echo 1024 > /sys/kernel/mm/hugepages/hugep
[root@localhost dpdk]# mkdir -p /mnt/huge
[root@localhost dpdk]# mount -t hugetlbfs nodev /mnt/huge

[root@localhost dpdk]# ./app/test-pmd/testpmd -c 0x3 -n 4 --socke
i --txqflags=0xf00 --disable-hw-vlan
<SNIP>
Done
testpmd> start
io packet forwarding - CRC stripping disabled - packets/burst=6
nb forwarding cores=1 - nb forwarding ports=2
RX queues=1 - RX desc=128 - RX free threshold=0
RX threshold registers: pthresh=0 hthresh=0 wthresh=0
TX queues=1 - TX desc=512 - TX free threshold=0
TX threshold registers: pthresh=0 hthresh=0 wthresh=0
TX RS bit threshold=0 - TXQ flags=0xf00

/fosdem # /usr/local/bin/ovs-ofctl dump-flows br0
NXST_FLOW reply (xid=0x4):
cookie=0x0, duration=24340.628s, table=0, n_packets=1125, n_byte
priority=0 actions=NORMAL
/fosdem #

/fosdem # /usr/local/bin/ovs-ofctl dump-flows br0
NXST_FLOW reply (xid=0x4):
cookie=0x0, duration=24340.628s, table=0, n_packets=1125, n_byte
priority=0 actions=NORMAL
/fosdem #
/fosdem # /usr/local/bin/ovs-ofctl del-flows br0
/fosdem #

/fosdem # /usr/local/bin/ovs-ofctl show br0 | grep addr
1(dpdk0): addr:00:1b:21:74:5b:64
2(dpdk1): addr:00:1b:21:74:5b:65
3(vhost-user-0): addr:00:00:00:00:00:00
4(vhost-user-1): addr:00:00:00:00:00:00
LOCAL(br0): addr:00:1b:21:74:5b:64
/fosdem #

1(dpdk0): addr:00:1b:21:74:5b:64
2(dpdk1): addr:00:1b:21:74:5b:65
3(vhost-user-0): addr:00:00:00:00:00:00
4(vhost-user-1): addr:00:00:00:00:00:00
LOCAL(br0): addr:00:1b:21:74:5b:64
/fosdem #
1 2
3 4

1(dpdk0): addr:00:1b:21:74:5b:64
2(dpdk1): addr:00:1b:21:74:5b:65
3(vhost-user-0): addr:00:00:00:00:00:00
4(vhost-user-1): addr:00:00:00:00:00:00
LOCAL(br0): addr:00:1b:21:74:5b:64
/fosdem #
/fosdem #
/fosdem #
/fosdem #
/fosdem #
/fosdem #
/fosdem #
/fosdem #
/fosdem #
/fosdem # /usr/local/bin/ovs-ofctl add-flow br0 in_port=1,
action=output:3
1 2
3 4

1(dpdk0): addr:00:1b:21:74:5b:64
2(dpdk1): addr:00:1b:21:74:5b:65
3(vhost-user-0): addr:00:00:00:00:00:00
4(vhost-user-1): addr:00:00:00:00:00:00
LOCAL(br0): addr:00:1b:21:74:5b:64
/fosdem #
/fosdem #
/fosdem #
/fosdem #
/fosdem #
/fosdem #
/fosdem #
/fosdem #
/fosdem #
action=output:3
action=output:2
/fosdem #
1 2
3 4

/fosdem # top -H
top - 17:17:04 up 7:51, 2 users, load average: 2.00, 1.78, 1.42
Threads: 833 total, 3 running, 830 sleeping, 0 stopped, 0 zombie
%Cpu(s): 4.7 us, 0.1 sy, 0.0 ni, 95.2 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st
KiB Mem: 65871880 total, 33673988 used, 32197892 free, 119144 buffers
KiB Swap: 1998844 total, 0 used, 1998844 free. 15128148 cached Mem
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
23749 root 20 0 16.745g 13008 2284 R 97.8 0.0 411:40.15 pmd44
50463 root 20 0 4693852 237856 3848 R 97.8 0.4 7:51.55 qemu-system-x86
50651 root 20 0 21516 1900 1000 R 11.5 0.0 0:00.03 top
1 root 20 0 34556 3864 1460 S 0.0 0.0 0:05.39 init
2 root 20 0 0 0 0 S 0.0 0.0 0:00.00 kthreadd

/fosdem # taskset -pc 50463
pid 50463's current affinity list: 0-19
/fosdem # taskset -pc 3 50463
pid 50463's current affinity list: 0-19
pid 50463's new affinity list: 3

/fosdem # top –H
P
23749 root 20 0 16.745g 13008 2284 R 99.9 0.0 412:01.41 pmd44
0
3
50652 root 20 0 21520 2240 1096 R 0.4 0.0 0:00.11 top
6
1571 avahi 20 0 32480 1888 1404 S 0.1 0.0 0:01.91 avahi-daemon
5

/fosdem # /usr/local/bin/ovs-vsctl set Open_vSwitch .
other_config:pmd-cpu-mask=30
/fosdem #
/fosdem # top –H
50656 root 20 0 16.818g 15804 2284 R 99.9 0.0 5:48.67 pmd46
50657 root 20 0 16.818g 15804 2284 R 99.9 0.0 5:48.67 pmd45

/fosdem # /usr/local/bin/ovs-appctl list-commands
The available commands are:
autoattach/show-isid [bridge]
autoattach/statistics [bridge]
autoattach/status [bridge]
bfd/set-forwarding [interface] normal|false|true
bfd/show [interface]
bond/disable-slave port slave
bond/enable-slave port slave
bond/hash mac [vlan] [basis]
bond/list
bond/migrate port hash slave
bond/set-active-slave port slave
bond/show [port]
bridge/dump-flows bridge
bridge/reconnect [bridge]
cfm/set-fault [interface] normal|false|true
cfm/show [interface]
coverage/show
dpctl/add-dp
dpctl/add-flow
dpctl/add-if
dpctl/del-dp

/fosdem # /usr/local/bin/ovs-appctl dpif-netdev/pmd-stats-show
main thread:
emc hits:0
megaflow hits:0
miss:0
lost:0
polling cycles:11104860 (100.00%)
processing cycles:0 (0.00%)
pmd thread numa_id 0 core_id 4:
emc hits:529722612
megaflow hits:11
miss:1
lost:0
polling cycles:547445295208 (64.91%)
processing cycles:295908348428 (35.09%)
avg cycles per packet: 1592.07 (843353643636/529722624)
avg processing cycles per packet: 558.61 (295908348428/52
pmd thread numa_id 0 core_id 5:
emc hits:1441418911
megaflow hits:19
miss:2
lost:0
polling cycles:579189882464 (73.80%)

/fosdem # /usr/local/bin/ovs-appctl dpif/dump-flows br0
recirc_id(0),in_port(2),eth_type(0x0800),ipv4(frag=no), packets:5
bytes:35229152580, used:0.000s, actions:4
bytes:35226757020, used:0.000s, actions:3
bytes:61034843460, used:0.000s, actions:drop

The n00bs guide to ovs dpdk

More Related Content

What's hot (20)

Viewers also liked (9)

Similar to The n00bs guide to ovs dpdk (20)

Recently uploaded (20)

The n00bs guide to ovs dpdk

Editor's Notes