Networking (OvS) Data Path Offloads: Hardware Acceleration Architectures

Introduction

 Offloading Open vSwitch (OvS) and Remote Direct Memory Access (RDMA) to hardware is essential because software-based implementations incur prohibitive CPU overhead and latency penalties that fundamentally bottleneck modern high-speed networks and distributed systems. OvS, when processed in software, consumes 30-50% of server CPU cycles just to handle basic virtual switching at 10G+ speeds, starving application workloads and limiting tenant density in cloud environments. Similarly, RDMA's promise of ultra-low latency and zero-copy networking is negated when its transport logic runs in software, adding microseconds of delay and saturating memory bandwidth with data copies. Hardware offloading solves this by moving OvS flow tables (VXLAN encapsulation, ACLs) and RDMA transport layers (reliable connection management, memory registration) into smart NICs/DPUs, achieving three critical benefits: Sub-5µs latency by bypassing kernel stacks, Line-rate throughput at 100G+ speeds via dedicated packet processors, and 80%+ host CPU savings by eliminating protocol processing cycles – making scalable hyperscale infrastructure, real-time AI training, and high-frequency trading feasible where software implementations collapse under load. Imagine a network interface so advanced it can classify, process, and route millions of flows per second — all without a routine consumption of CPU cycles.

That’s modern NICs and DPUs, where advanced Match-Action flow engines, nanosecond-precision congestion control, and zero-copy RDMA pipelines converge into fully programmable silicon. These aren’t just offload devices — they are complete networking stacks, such as Open vSwitch and RDMA data paths running entirely on-chip.

Note that data-path acceleration has so many ways and means – building poll mode (PMD) drivers instead of interrupt mode ones, Kernel bypass implementations such as DPDK/SPDK data-paths in the user space and then of course offloading everything to the hardware. I am deliberately focusing on the chip-offload and omitting other acceleration mechanisms.

In previous articles, we went over a conceptual data-path and got an idea about what it takes to move the packet. In this article, we’ll be dissecting the match-action pipelines, protocol accelerators, and memory translation units that make it possible. We’ll explore integration across OvS and briefly touch upon on the next steps towards the AI networking.

Hardware offload Targets

When it comes to offload target, there are various options such as: Custom Silicon such as Intel Mount family of IPUs., SoC such as Intel Lake family of SoCs or FPGA such as Altera Oak Springs IPU and Xilinx/AMD Alveo series Smart-NICs.

SoC offload: A NIC with a multi-core SoC such as Intel Icelike-D or even a NIC with its own multi-core SoC (ARM/x86 class) and built-in hardware accelerators. It can run a full Linux stack and offload/terminate protocols (RDMA, TLS/IPsec, NVMe-oF, storage etc.) on the device, isolating infrastructure control/IO from the host.

Custom ASIC offload: A high-speed NIC ASIC that implements wire-speed parsing, matching, and actions (L2–L4, tunneling, RDMA, congestion control, PTP, crypto, etc.). The host still runs the application; the NIC offloads data-plane primitives via kernel drivers/DPDK/TC.

FPGA-based offload: A NIC that exposes a regeneratable pipeline to implement or customize protocol handling and in-NIC compute.

It is worth clarifying that both FPGA and Custom ASICs can implement a programmable such as P4 programmable pipeline. However, it makes more case for Custom-ASIC than FPGAs for many reasons that can be covered separately.

A Conceptual Hardware Pipeline

Every hardware adds some custom features, but the following diagram is from Chelsio communications. It appears that it offloads only L2 networking along0with TOE, but it captures the data-path and puts in the context very well. The second diagram A generic description of various functions is provided by me to help understand the diagram.

Packet Parsing & Flow Matching

1. Flow Key Parsing / Packet Parser

When a packet arrives on a datapath, OVS parses it to extract a structured flow key. This consists of packet header fields—like Ethernet, VLAN, MPLS, IP, TCP/UDP, etc.—which form a canonical representation used throughout OVS. The datapath extracts this key and uses it for matching; the same key is shared with user space to preserve compatibility—even if one side recognizes more protocol fields than the other.

2. Datapath Classifier / Matching

OVS employs a tiered lookup architecture to match parsed flow keys to flow actions. In datapath, this may involve various levels, but it contains a mix and match of EMC/LPM etc.

• Exact Match Cache (EMC) – Fast, direct lookup with no wildcard capabilities. But consumes more space. Needs a lot of entries in the cache. Need a good hit rate on this.

• Small TCAM Cache – Small SRAM based lookup cache to route traffic in case of miss.

Once a match is found, its associated actions (e.g., forwarding, dropping, modifying) are executed.

3. Action/Header Modification

The OVS uses the term action to denote commands that dictate what to do with a packet after it matches a flow. These are based on the OpenFlow model and include both standard and extended operations

• Action List: An ordered set of actions executed in sequence as specified. If any action fails to apply, the entire list is typically rejected.

• Action Set: Introduced in OpenFlow 1.1, this is an unordered set of actions where duplicates are merged, and the overall execution order is defined by the switch’s semantics. OVS processes actions in the set following a predetermined order.

• Common Header Modification Actions :

• Field modifications: For instance, rewriting using operations like to control return routing.

• VLAN manipulations: Actions such as , and . These executions are automatically reordered within an action set per OVS semantics.

4. Extended Actions:

• Packet truncation: A newer extension allows limiting frame size in mirroring or output actions. Syntax example: —this sends a maximum of 100 bytes to port 1.

• Recirculation for multi-pass processing: When an outer header is stripped (e.g., ), the same packet can enter processing again to apply further actions (e.g., parsing an inner IP header)—known as recirculation.

• Statistics and Telemetry: Hardware counters for per-flow packet/byte counters (64b), latency histograms (8-bin), and drop counters with cause codes are some of the common telemetry data collected. But there are a lot more to this in OvS.

Networking and RDMA (ROCeV2) Offloads to AI Fabric

The journey of AI-scale-out networking began with RoCEv2 (RDMA over Converged Ethernet), which brought ultra-low latency, zero-copy transfers, and high throughput—crucial for GPU clustering—by enabling routable, lossless Ethernet fabrics complemented by ECN and PFC for congestion control. From there, hyperscalers such as Meta deployed dedicated backend RoCEv2 fabric networks—separate from their front-end networks—to reliably interconnect tens of thousands of GPUs across Clos and Agg-switch topologies.

 As AI workloads ballooned, networking vendors responded with next-gen Ethernet and fabric technologies: companies like Arista have optimized Ethernet for AI "back-end" networks to rival InfiniBand’s performance and scalability while Broadcom launched its Jericho 4 chip to support ultra-wide area AI connectivity (up to ~4,500 interconnected chips with HBM memory, encryption, and massive scale-out support).

 Simultaneously, startups such as Cornelis Networks introduced new OmniPath-based hardware tailored specifically for scaling connections between up to 500,000 AI chips—offering flexibility alongside emerging Ethernet compatibility. Together, these innovations reflect a shift from low-level transport acceleration (RoCEv2) to purpose-built, scalable network fabrics and silicon designed for the explosive growth of distributed AI workloads.

 In next article, I will cover RDMA data-path and the foundations of scale out AI Fabric 

References

1. docs.openvswitch.org

2. patchwork.ozlabs.org