Virtual net performance
7 likes2,428 views
This document summarizes virtual networking performance measurements and analysis. It finds that virtual networking introduces overhead compared to physical networking, with legacy emulated NICs performing worse than newer virtual NIC designs. Current work focuses on areas like multiqueue support, zero copy networking, and batching to improve performance. The document outlines future work areas and ways people can help, such as through testing, profiling, and developing optimizations.
Virtual net performance
- 1. Virtual Networking Performance Stephen Hemminger stephen.hemminger@vyatta.com Linuxcon Japan Virtualization Summit
- 3. Bi-directional Virtualization Loss Test 100% 75% Throughput 50% 25% 0% 0 500 1000 1500 Packet Size
- 4. Houston, we have a problem!
- 5. Topics What is virtual router? Performance measurements Analysis Current work How to help
- 6. Performance History 2005 2006 2007 2008 2009 2010 2011 1G line rate Multiqueue NIC 10G line rate 1M pps Xen 3.0 KVM Hyper-V drivers
- 8. Performance measurements Forwarding RFC 2544 minimum packet Throughput TCP bulk transfer Latency Request / Response
- 9. Router Benchmark RFC2544 Router Under Test Spirent
- 10. Virtualized Router Benchmark Router Under Guest Test Hypervisor Bridge Bridge Spirent
- 11. 1 Gigabit bi-directional forwarding 100% 75% Throughput 50% 25% 0% 0 200 400 600 800 1000 1200 1400 1600 Packet Size
- 12. Physical Networks Rate limited by slowest component Adding more layers does not slow down throughput Separate resources
- 13. Virtual Network overhead Hyper- Bridge out visor In Bridge In Hyper- visor Router Under Test
- 14. Network Baseline Intel i7 920 Intel 10G LOM 1G network Loopback – 12G bits/sec Forwarding – 1M packets/sec per core Your mileage may vary...
- 15. Emulated Network Interface Fake PCI region Guest Hypervisor To Network Packet Buffer
- 16. Legacy (emulated) NIC Relative Performance vs Native % -25% -50% -75% Transmit -100% Receive Hyper-V VmWare KVM
- 17. Virtual NIC Guest Hypervisor To Network Packet Buffer
- 18. Virtual NIC performance 0% -25% -50% -75% -100% Transmit Hyper-V VmWare Receive KVM
- 19. VNIC characteristics Hyper-V Vmware Xen KVM vmxnet3 netfront virtio-net MTU 1500 9000 65521 65535 Checksum Y Y Y offload Segmentation Y Y Y offload NAPI Y Y Y LRO Y VLAN Y Multiqueue Y ? ?
- 20. Guest to Guest Throughput TSO Tx csum No offload 0 1000 2000 3000 4000 5000 Megabits/sec
- 21. VM to VM performance 2000 Thorouhput (10^6 bits/sec) 0 100 1000 10000 100000 MTU (bytes)
- 22. TCP performance 2 Tier NAT Route Bridge Bare 0 2 4 6 8 10 Gigabytes / sec
- 23. Observations Slow Legacy emulated NIC NAT Firewall connection tracking Fast Virtual NIC Hardware passthrough Bare Bridge
- 24. Multithread benchmark 20,000 UP Guest SMP Guest Total Transactions/sec 15,000 10,000 5,000 0 Packet Steering (RPS) Baseline Multi Queue NIC
- 25. Multiple Queue HW or SW Guest VM Thread Device queue Thread Flow Device VNIC clas queue sfier Thread Device queue
- 26. TODO More peformance measurement MACVTAP, VEPA, … Configuration Pinning, IRQ assignment, … Profiling
- 27. Future work Virtio-net optimization Michael S. Tsirkin Multiqueue Krishna Kumar KVM ↔ QEMU Zero copy Shirley Ma Skb lifetime Security Batching Rusty Russell Bufferbloat
- 28. How to help Administrator →Tuning and Sizing Tester →Measure and Test Developer →Multiqueue and Offload
Editor's Notes
- #8: Performance is a complex topic, and hard to measure. The blind man who feels a leg says the elephant is like a pillar; the one who feels the tail says the elephant is like a rope; the one who feels the trunk says the elephant is like a tree branch; the one who feels the ear says the elephant is like a hand fan; the one who feels the belly says the elephant is like a wall; and the one who feels the tusk says the elephant is like a solid pipe. A wise man explains to them: "All of you are right. The reason every one of you is telling it differently is because each one of you touched the different part of the elephant. So, actually the elephant has all the features you mentioned."
- #15: I use Virt net every day but always felt uncomfortable that performance is being lost. Rules of thumb, today, on my hardware
- #17: In honor of Australia day wanted to graph it upside down!! Not a benchmark * different hardware * not precise * not allowed by License agreements
- #19: P7 Avoid unnecessary generality
- #22: Relax system requirements