DPDK FPGA with Atomic Rules Arkville
1 like542 views
Arkville is an FPGA-based DPDK packet conduit developed by Atomic Rules LLC that provides high-speed data transfer between DPDK applications running on a GPP and FPGA-based accelerators. It consists of an open-source DPDK Poll Mode Driver software and a proprietary FPGA RTL IP core. Arkville delivers over 150Gbps throughput and 1.25us latency while eliminating CPU overhead through zero-copy of packets between the NIC and application memory. It is supported on major FPGA boards and can be used to accelerate DPDK workloads in smart NICs, network appliances, and other solutions requiring efficient GPP-FPGA communication.
![Roadmap: 17.08
19©2017 Atomic Rules LLC
• Shipped: August 2017
• Support for DPDK 17.08
• Added Support for Vivado 2017.2
• Examples include Atomic Rules TimeServo[1]
– Timestamps from MAC to DPDK metadata
[1] TimeServo is a separate IP product that may be purchased
from Atomic Rules.](https://image.slidesharecdn.com/atomicrules-arkvilledpdkacceleration-171008152519/85/DPDK-FPGA-with-Atomic-Rules-Arkville-19-320.jpg)
DPDK FPGA with Atomic Rules Arkville
- 1. Arkville DPDK Acceleration Shep Siegel, CTO Atomic Rules LLC 1©2017 Atomic Rules LLC Shepard.Siegel@atomicrules.com Arkville DPDK Acceleration 2017-10-08
- 2. AR Background • Providing FPGA Design Services since 2008 – Contributions to numerous Open-Source Projects – Corporate Member of The Linux Foundation • Began augmenting services with IP Core products – UDP Offload Engine (2014) – Arkville DPDK-Aware Data Mover (2016) – TimeServo FPGA System Timer (2017) • Differentiation with Tools and Methodology • Broad and Agnostic set of Partners • Growing list of Recurring Customers and Clients 2©2017 Atomic Rules LLC
- 3. Arkville: A DPDK Packet Conduit 3©2017 Atomic Rules LLC
- 4. Arkville: Why? • DPDK First – Not an Afterthought or Adaptation – Both Higher Throughput and Lower Latency • DPDK Aware – Push GPP cycles to specialized FPGA RTL gates – Zero-Copy GPP Driver = Lowest Possible Overhead • DPDK Acceleration – Certain workloads are better in RTL gateware – Arkville enables a Software-First approach 4©2017 Atomic Rules LLC
- 5. Arkville: Where? • Arkville is used as a building-block component in products and solutions such as – Smart-NIC Devices – Network Appliances – DPDK Accelerators • Anywhere there is the need to efficiently exchange data between DPDK-mbufs and AXI FPGA gates 5©2017 Atomic Rules LLC
- 6. Smart-NIC 6©2017 Atomic Rules LLC 6 Any Rate ENET MAC NIC H/W Offload RTL IP Arkville AXI HW Arkville DPDK PMD SW DPDK Applications GPP SW PCIe FPGA GPP AR Arkville HW/SW DPDK SW API AXI HW API
- 7. Network Appliance 7©2017 Atomic Rules LLC 7 Any Rate ENET MAC Appliance RTL IP Arkville AXI HW Arkville DPDK PMD SW Appliance GPP SW PCIe FPGA GPP AR Arkville HW/SW DPDK SW API AXI HW API
- 8. DPDK Accelerator 8©2017 Atomic Rules LLC 8 AXI Accelerator RTL IP from HLS, OpenCL, BSV Arkville AXI HW Arkville DPDK PMD SW DPDK Applications or fd.io/VPP Nodes GPP SW PCIe FPGA GPP AR Arkville HW/SW DPDK SW API AXI HW API
- 9. Arkville: How? • A Software and Gateware Combination: – Arkville 17.08 DPDK Poll Mode Driver (DPDK PMD) • Open-Source BSD from The Linux Foundation • http://dpdk.org/doc/guides/nics/ark.html – Arkville 17.08 FPGA RTL IP Core (AXI RTL IP) • Atomic Rules sells named-project and site-based licenses • http://www.atomicrules.com/arkville • Arkville performs the data-mover work for products and solutions that require FPGA/GPP communication 9©2017 Atomic Rules LLC
- 10. High-Level Block Diagram 10©2017 Atomic Rules LLC
- 11. Over 150 Gbps Throughput 11©2017 Atomic Rules LLC
- 12. 1.25 us RTT/2 Latency 12©2017 Atomic Rules LLC
- 13. About Arkville Latency • By Itself, Arkville contributes just 100ns to the ingress/egress latency budget • Software: Arkville DPDK PMD: – Constant Time 20ns per packet (e5-2630v4) – Zero-Copy – No GPP Cycles moving packet data! • Hardware: Arkville AXI RTL IP: – 20 Cycles at 250 MHz = 80ns + 4ns/64B – Deterministic, store-and-forward 13©2017 Atomic Rules LLC
- 14. No memcpy() • Arkville is unique among DPDK Poll Mode Drivers in that packet data is landed zero-copy exactly in user-land memory – Zero CPU cycles are used by net/ark PMD for packet data movement – Result is a constant O(1) time for all packets that fit in a single mbuf (See next slide) • More CPU cycles for your application • Fewer CPU cores needed for same amount of work 14©2017 Atomic Rules LLC
- 15. Zero-Copy DPDK Driver 15©2017 Atomic Rules LLC
- 16. Zero Packets Dropped 16©2017 Atomic Rules LLC
- 17. Questions / Feedback • Is FPGA acceleration of packet-based DPDK workloads of interest? • Are there specific challenges you are facing that perhaps Arkville will help solve? • How do you measure success? • Do you have a target platform in mind? • How would you like to get started? 17©2017 Atomic Rules LLC
- 18. Summary and Thank You! 18©2017 Atomic Rules LLC • Arkville is a GPP/FPGA DPDK-Aware Conduit – Software is an Open-Source DPDK Poll Mode Driver – RTL IP Core Gateware is sold by Atomic Rules • Named-Project and Site-Based Licenses are Available • Arkville is Agnostic – Support for Contemporary FPGA Devices – Support for COTS and Custom FPGA Boards • Arkville is Supported – Ongoing developments to the DPDK/LF Community – AR offers both HW and SW services and support
- 19. Roadmap: 17.08 19©2017 Atomic Rules LLC • Shipped: August 2017 • Support for DPDK 17.08 • Added Support for Vivado 2017.2 • Examples include Atomic Rules TimeServo[1] – Timestamps from MAC to DPDK metadata [1] TimeServo is a separate IP product that may be purchased from Atomic Rules.
- 20. Roadmap: 17.11 20©2017 Atomic Rules LLC • Scheduled: November 2017 • Support for DPDK 17.11 • Support for Vivado 2017.3 • Selectable 4/8/16 TX-RX Queue-Pairs – Next-Generation MBUF Prefetcher – Reduced FPGA Logic Area – Still Zero-Copy and Low-microsecond Twire-to-user • Expanded DPDK examples
- 21. Backup Material 21©2017 Atomic Rules LLC
- 22. Core Beliefs and Axioms 22©2017 Atomic Rules LLC • Our Customer’s Success is Key • Separation of Concerns • Divide and Conquer • Automate or Die • Write Things Once • Interface Before Implementation • Functional Correctness First • Performance Correctness Improved Iteratively • Components Must Compose • Components Must Work as Expected • IP Should be Portable, Vendor-Agnostic if possible
- 23. Partner Roster 23©2017 Atomic Rules LLC • 25G / 50G Ethernet Consortium • 25-50-100 Ethernet Alliance • Accellera/OCP-IP Community Member • Amazon F1 Instance Partner • ARM Connected Community Member • BittWare Solution Partner • Bluespec Technology Partner • DPDK Project Corporate Member • FPGA, FCCM and FPL ‘F’ Conference Sponsors • Intel Network Builders • Linux Foundation Corporate Member • MathWorks Connections Partner • NetFPGA Infrastructure Developer • OpenCPI Infrastructure Developer • P4 Language Consortium Member • PCI-SIG Corporate Member • VITA Trade Association Member • Xilinx Alliance Member Partner
- 24. 100 GbE DPDK Offload 24©2017 Atomic Rules LLC
- 25. Arkville IPI Component 25©2017 Atomic Rules LLC
- 26. Arkville Evaluation 26©2017 Atomic Rules LLC • New Performance Evaluation from Atomic Rules… • Evaluate Arkville on-site with your own system components – Measure your own actual system-level performance • Atomic Rules provides pre-compiled bitstreams for your FPGA platform – 100 GbE (x1) and 10 GbE (x4) example designs • Download trusted, unmodified DPDK from dpdk.org • Run DPDK “testpmd” unmodified to observe – Packet Throughput, Latency, Packet Loss – Compare results to DPDK Labs reports • Run supplied AR DPDK examples for deeper insights • Run your own DPDK programs on the example designs