イマドキなNetwork/IO
30 likes3,495 views
The document discusses techniques for improving network and I/O performance between the network interface card (NIC) and CPU. It describes technologies like TCP offloading, receive side scaling, and Intel I/O acceleration features that distribute processing load away from the CPU to improve throughput and reduce latency. Optimization goals include more efficiently handling interrupts and direct memory access to reduce CPU utilization for network tasks.
イマドキなNetwork/IO
- 2. • IO • Linux • Vyatta PC
- 3. Process(User) Process(Kernel) socket queue input SW Intr Handler queue HW Intr Handler
- 4. • NIC • NIC:1GbE→10GbE • CPU:1GHz→3.2GHz :CPU 1/10 • CPU • 1CPU → •
- 5. • NIC NIC •
- 6. Process(User) Process(Kernel) socket queue input SW Intr Handler queue HW Intr Handler
- 8. • • NIC NIC • • →NAPI Linux http://tinyurl.com/LinuxNAPI
- 9. NAPI Process(User) Process(Kernel) socket queue SW Intr Handler HW Intr Handler
- 10. • NIC CPU →CPU • 1Gbps Pentium4 2.4GHz CPU 80% • CPU
- 11. Process(User) Process(Kernel) socket queue SW Intr Handler HW Intr Handler
- 12. TOE (TCP Offload Engine) • NIC TCP/IP • • TOE OS • OS TOE TOE
- 13. TOE (TCP Offload Engine) • Linux • Windows OS http://bit.ly/offload • RDMA, iSCSI HBA
- 14. • TCP Checksum Offload TCP • Large Segment Offload 64KB NIC MTU • Large Receive Offload LSO NIC
- 15. Linux • TCP Checksum Offload TCP • Large Segment Offload 64KB NIC MTU • Large Receive Offload LSO NIC
- 16. • NIC CPU •
- 17. CPU cpu0 cpu1 Process(User) Process(User) Process(Kernel) Process(Kernel) socket socket queue queue SW Intr Handler SW Intr Handler HW Intr Handler HW Intr Handler
- 18. Receive Side Scaling • • CPU CPU • CPU • CPU →
- 19. Receive Side Scaling cpu0 cpu1 cpu2 cpu3 RX RX RX RX Queue Queue Queue Queue #0 #1 #2 #3 hash queue ■ 0 ■ 1 NIC
- 20. Receive Side Scaling • Microsoft Scalable Network Initiative http://bit.ly/ReceiveSideScaling • Windows Linux • • PCI MSI-X • NIC RSS
- 21. RPS(Linux) • RSS NIC • RSS • CPU • CPU CPU • RSS
- 22. cpu0 cpu1 cpu2 cpu3 socket socket queue queue hash queue backlog ■ 0 #1 ■ 1 backlog #2 backlog #3
- 23. RFS(Linux) • CPU RPS •
- 24. RPS
- 25. CPU
- 26. • Intel http://bit.ly/IOATJ • NIC CPU OS • CPU I/O • CPU
- 27. TOE • TOE • CPU CPU →TCP/IP • TCP/IP CPU TOE →
- 28. Intel I/O Acceleration Technology • Intel QuickData Technology • Direct Cache Access • Receive Side Scaling • Large Receive Offload • Low Latency Interrupts
- 29. Intel QuickData Technology • NIC → DMA • CPU • OS
- 30. Intel QuickData Technology Process(User) Process(Kernel) socket queue SW Intr Handler HW Intr Handler
- 31. Direct Cache Access • NIC DMA CPU • NIC • prefetch
- 32. DCA CPU Cache Fetch Snoop invalidate Writeback Memory Memory Controller Memory Wirte DMA Write I/O Device
- 33. DCA CPU Cache HW Prefetch Snoop invalidate Writeback +hint Memory Memory Controller Memory Wirte DMA Write I/O Device
- 34. • Intel VT-c • SR-IOV • OS NIC • VMDq • VM IO
- 35. VM1 VM2 Hypervisor
- 36. Intel VT-d PCI Passthrough VM1 VM2 Hypervisor
- 37. SR-IOV VM1 VM2 Hypervisor
- 38. VMDq VM1 VM2 Hypervisor RX1 RX2 RX1 RX2 RX1 RX1 RX2 RX1 RX1 RX2
- 39. • • • • Intel