Snabb, a toolkit for building user-space network functions (ES.NOG 20)
0 likes221 views
Snabb is a toolkit for developing high-performance user-space network functions, primarily written in Lua. It allows for the manipulation of traffic data through various network functions, such as firewalls and NAT, utilizing commodity hardware while bypassing the limitations of general-purpose operating systems like Linux. The toolkit includes a catalog of applications and programs, supporting advanced networking solutions like lightweight 4over6 (lw4o6) for IPv4 address exhaustion.
![FFI: EXAMPLE
ffi.cdef[[
void syslog(int priority, const char*format, ...);
]]
ffi.C.syslog(2, "error:...");
local ether_header_t = ffi.typeof [[
/* All values in network byte order. */
struct {
uint8_t dhost[6];
uint8_t shost[6];
uint16_t type;
} __attribute__((packed))
]]](https://image.slidesharecdn.com/diegopino-snabb-171106152641/85/Snabb-a-toolkit-for-building-user-space-network-functions-ES-NOG-20-21-320.jpg)
![PACKETS
struct packet {
uint16_t length;
unsigned char data[10*1024];
};](https://image.slidesharecdn.com/diegopino-snabb-171106152641/85/Snabb-a-toolkit-for-building-user-space-network-functions-ES-NOG-20-29-320.jpg)
![LINKS
struct link {
struct packet *packets[1024];
// the next element to be read
int read;
// the next element to be written
int write;
};](https://image.slidesharecdn.com/diegopino-snabb-171106152641/85/Snabb-a-toolkit-for-building-user-space-network-functions-ES-NOG-20-30-320.jpg)
Snabb, a toolkit for building user-space network functions (ES.NOG 20)
- 1. SNABB A TOOLKIT FOR BUILDING USER-SPACE NETWORK FUNCTIONS
- 3. ABOUT IGALIA Consultancy specialized in open-source Base in Coruña but distributed all over the world (>60 people working from 15 different countries) Contributors to projects such as WebKit, Chromium V8, etc Other areas: Graphics, Multimedia, Networking
- 5. AGENDA What is Snabb? How it works? Catalog and programs Use case: lwAFTR
- 6. WHAT IS SNABB? Snabb is a toolkit for developing high-performance network functions in user- space
- 7. WHAT IS A NETWORK FUNCTION? A program that manipulates traffic data Basic operations: read, forward, drop, modify, create... Combining these primitives we can build any network function
- 8. EXAMPLES Firewall: read incoming packets, compare to table of rules and execute an action(forward or drop) NAT: read incoming packets, modify headers and forward packet Tunelling: read incoming packets, create a new packet, embed packet into new one and send it
- 9. WHY SNABB? Increasing improvement of commodity hardware: 10Gbps NICs at very affordable prices High-performance equipment is still very expensive Idea: build an analog high-performance router using commodity hardware
- 10. WHY SNABB? What software to put into this hardware? Common intuition: Linux Drawback: Linux is not suitable for high-performance networking
- 11. WHY NOT LINUX? General-purpose operating system An OS abstracts hw resources to offer high-level interfaces: filesystems, processes, sockets... Our network function will be divided into two lands: user-space and kernel- space Colorary: processing a packet has an inheritent cost => the cost of the OS
- 12. HIGH-PERFORMANCE NETWORKING NIC: 10Gbps Avg Packet-size: 550-byte PPS: 2272727,27 1 packet every 440ns ((1/2272727,27)*10^9) CPU: 2,5 Ghz 1100 cycles to process one packet (2,5 cycles/sec * 440 ns)
- 13. HIGH-PERFORMANCE NETWORKING Packet-size: 64-byte: 51 ns per packet; 128 cycles per packet Lock/Unlock: 16ns; Cache-miss: 32 ns Source: Jonathan Corbet's Small packet size => More packets per second => worse Faster CPU => better "Improving Linux networking performance"
- 14. USER-SPACE DRIVER Do a kernel by-pass and manage the hardware directly from user-space: Tell Linux not to manage the PCI device (unbind) Do a mmap of the registers of the PCI device into addressable memory Whenever we read/write the addressable memory, we're actually poking the registers of the NIC Follow the NIC's datasheet to implement operations such as initialize, receive, transmit, etc
- 15. USER-SPACE NETWORKING Snabb is not an isolated case of user-space networking: Snabb (2012) DPDK (2012) VPP/fd.io (2016) DPDK (Data-plane Development Kit, Intel) VPP (Vector Packet Processing, Cisco)
- 16. RING-BUFFER Very important to avoid packet drops
- 17. INSIDE SNABB
- 18. SNABB Project started by Luke Gorrie User-space networking benefit: freedom of programming language Snabb is mostly written in Lua Network functions are also written in Lua Fast to run, fast to develop Snabb means fast in Swedish :)
- 19. ABOUT LUA Started in 1993 at University of Rio de Janeiro (PUC Rio) Very similar to JavaScript, easy to learn Very small and compact, it's generally embeded in other systems Use cases: microcontrollers (NodeMCU), videogames (Grim Fandango), IA (Torch7)
- 20. ABOUT LUAJIT Just-in-time compiler for Lua Extremely fast virtual machine!! Very good integration with C thanks to FFI (Foreign Function Interface)
- 21. FFI: EXAMPLE ffi.cdef[[ void syslog(int priority, const char*format, ...); ]] ffi.C.syslog(2, "error:..."); local ether_header_t = ffi.typeof [[ /* All values in network byte order. */ struct { uint8_t dhost[6]; uint8_t shost[6]; uint16_t type; } __attribute__((packed)) ]]
- 22. SNABB IN A NUTSHELL A snabb program is an app graph Apps are conected together via links Snabb processes the program in units called breadths
- 23. NF: APP GRAPH
- 24. BREADTHS A breadth has two steps: inhale a batch of packets into the graph process those packets To inhale, the method pull of the apps is executed (if defined) To process, the method push of the apps is executed (if defined)
- 25. # Pull function of included Intel 82599 driver function Intel82599:pull () for i = 1, engine.pull_npackets do if not self.dev:can_receive() then break end local pkt = self.dev:receive() link.transmit(self.output.tx, pkt) end end
- 26. # Push function of included PcapFilter function PcapFilter:push () while not link.empty(self.input.rx) do local p = link.receive(self.input.rx) if self.accept_fn(p.data, p.length) then link.transmit(self.output.tx, p) else packet.free(p) end end end
- 27. PACKET PROCESSING Normally only one app of the app graph introduces packets into the graph The method push gives an opportunity to every app to do something with a packet
- 28. APP GRAPH DEFINITION local c = config.new() -- App definition. config.add(c, "nic", Intel82599, { pci = "0000:04:00.0" }) config.add(c, "filter", PcapFilter, "src port 80") config.add(c, "writer", Pcap.PcapWriter, "output.pcap") -- Link definition. config.link(c, "nic.tx -> filter.input") config.link(c, "filter.output -> writer.input") engine.configure(c) engine.main({duration=1})
- 29. PACKETS struct packet { uint16_t length; unsigned char data[10*1024]; };
- 30. LINKS struct link { struct packet *packets[1024]; // the next element to be read int read; // the next element to be written int write; };
- 31. SNABB: APP CATALOG AND PROGRAMS
- 32. INVENTARY apps: software components that developers combine together to build network functions programs: complete network functions
- 33. APPS I/O Intel i210/i350/82599/XL710 Mellanox Connectx-4/5 Virtio host y guest UNIX socket Linux: tap and "raw" (e.g: eth0) Pcap files
- 34. APPS L2 Flooding and learning bridge VLAN insert/remove ARP/NDP
- 35. APPS L3 IPv4/v6 fragmentation and reassembly IPv4/v6 splitter ICMPv4/v6 echo responder Control-plane delegation (nh_fwd)
- 36. APPS L4 IPsec ESP Lightweight 4-over-6 AFTR Keyed IPv6 Tunnel
- 37. APPS MONITORING IPFix capturer and exporter L7 monitor/filtering (libndpi) Pcap expressions filter (with own backend for code generation)
- 38. APPS TESTING Lots of load generators: loadgen, packetblaster, loadbench...
- 39. USE CASE: LWAFTR
- 40. CONTEXT 2012-2014: Several RIRs run out of IPv4 public addresses 2008: IPv6 adoption starts to peak up Still big dependency on IPv4: services, websites, programs, etc
- 41. SOLUTIONS Carrier-Grade NAT: temporal solution for IPv4 address exhaustion problem Deployment of Dual-Stack networks (IPv4 e IPv6) Dual-Stack implies increasing complexity and costs (maintenance of two separated networks) Dual-Stack Lite (IPv6-only network which also offers IPv4 connectivity relying on CGN) Lightweight 4over6: iteration over Dual-Stack
- 43. LW4O6 - GOALS RFC7596 fully complaint (lwAFTR part) Performance: 2MPPS; 550-byte (packet-size); Binding-table: 1M subscribers. No packet drops
- 44. LW4O6 - DEVELOPMENT Version 1: Prototype Basic functionality (encapsulating/decapsulating) Small binding-table (own format) Development of tools to measure performance
- 45. LW4O6 - DEVELOPMENT Version 2 Production quality Fully standard compliant Big binding-table: 1M subscribers (still customized format but much closer to standard) Add support for other necessary protocols: ARP, NDP, fragmentation, reassembly, ping Tons of optimizations (use of AVX instructions to speed up lookups)
- 46. LW4O6 - DEVELOPMENT Version 3: Added YANG support to Snabb Support binding-table format according to standard Support of execution as leader/worker (leader: control-plane/worker: data- plane)
- 47. LW4O6 - DEVELOPMENT Version 4: Multiprocess (one leader, multiple workers) Improvement of the Intel 10Gbps driver (added support for RSS, Received Side Scaling) Added alarms support according to latest draft
- 48. LIGHTWEIGHT 4OVER6 - TALKS Juniper's vMX Lightweight 4over6 VNF Charla: Kostas Zordabelos's A real-world scale network VF using Snabb for lw4o6 Charla: Juniper Tech Club, Marzo 2017 SDN Meetup, Abril 2017
- 49. OTHER PROGRAMS
- 50. PROGRAM: PACKET BLASTER Generally useful tool: fill TX buffer of NIC with packets and transmit them over and over again Measures received traffic too Easily saturates 10Gbps links snabb packetblaster replay packets.pcap 82:00.1
- 51. PROGRAM: SNABBWALL L7 firewall that optionally uses nDPI Collaboration betwen Igalia and NLnet Foundation Landed upstream in 2017 Website: http://snabbwall.org
- 52. PROGRAM: IPFIX NETFLOW collector and exporter (v9 and IPFIX) Line-rate speed on a single core. Further improvement: parallel processing via RSS Landed upstream very recently
- 53. PROGRAM: L2VPN L2VPN over IPv6 (developed by Alexander Gall from SWITCH) Pending to land upstream; used in production Ideal Snabb use case: programmer/operator builds bespoke tool
- 54. PROGRAM: YOUR VNF Snabb upstream open to include new network functions Repository will grow as people will build new things Igalia can build one for you
- 55. LAST NOTES ABOUT PERFORMANCE
- 56. CONSIDERATIONS Isolcpus: Prevents the kernel to take a CPU to schedule processes Dishable HyperThreading Use HugePages (2MB) (Linux default is 4Kb) Do not neglect NUMA when launching programs Make use of SIMD instructions (AVX, AVX2) to speed up computations (checksum) Keep an eye on regressions: profile often
- 57. SUMMARY Toolkit for developing high-performance network functions in user-space Snabb provides apps which can be combined together forming a graph (network function) Snabb provides programs, complete network functions ready to use Snabb provides libraries, to easy the development of new network functions Completely written in Lua: easy to extend Fast: kernel-by pass + high-level language + fast VM (LuaJIT)
- 58. THANKS! Email: dpino@igalia.com Twitter: @diepg $ git clone https://github.com/snabbco/snabb.git $ cd snabb $ make