.NET Core Summer event 2019 in Brno, CZ - .NET Core Networking stack and performance -- Karel Zikmund
Download as PPTX, PDF0 likes233 views
.NET Core's networking stack has evolved significantly, with improvements in architecture and performance across platforms including Windows, Linux, and macOS. Key focus areas include the transition to .NET Core 3.0 with support for emerging technologies like HTTP/2 and QUIC, while maintaining high performance and robust reliability. Overall, the document details plans for ongoing performance monitoring, feedback-driven improvements, and the importance of balancing performance with other factors like usability and maintainability.
.NET Core Summer event 2019 in Brno, CZ - .NET Core Networking stack and performance -- Karel Zikmund
- 1. .NET Core Networking stack and Performance .NET Core Summer event 2019 – Brno, CZ Karel Zikmund ( @ziki_cz)
- 2. Agenda • Networking stack architecture evolution • .NET Framework, UWP and .NET Core • Networking stack in .NET Core • Direction and plans • Status & perf results • General BCL performance thoughts & observations
- 3. .NET Framework UWP .NET Core Networking – Architecture Evolution HttpWebRequest + ServicePoint Sockets SslStream, Dns, … Windows HttpClient + HttpClientHandler 4.5
- 4. .NET Framework UWP .NET Core Windows Linux / Mac Sockets, SslStream, … Sockets SslStream, Dns, … Windows Networking – Architecture Evolution HttpWebRequest + ServicePoint Sockets SslStream, Dns, … Windows HttpClient + HttpClientHandler 4.5 HttpWebRequest + ServicePoint HttpClient + HttpClientHandler WinRT APIs win9net.dll HttpClient WinHttpHandler WinHttp.dll CurlHandler libcurl OpenSSL HttpWebRequest + ServicePoint 2.0
- 5. Networking – Architecture Evolution .NET Framework HttpWebRequest + ServicePoint Sockets SslStream, Dns, … Windows HttpClient + HttpClientHandler 4.5 .NET Core Windows Linux / Mac Sockets, SslStream, … HttpClient WinHttpHandler WinHttp.dll CurlHandler libcurl OpenSSL HttpWebRequest + ServicePoint 2.0
- 6. Networking – Architecture Evolution .NET Framework HttpWebRequest + ServicePoint Sockets SslStream, Dns, … Windows HttpClient + HttpClientHandler 4.5 .NET Core Windows Linux / Mac Sockets, SslStream, … HttpClient WinHttpHandler WinHttp.dll CurlHandler libcurl OpenSSL HttpWebRequest + ServicePoint 2.0 .NET Core 2.1 HttpWebRequest + ServicePoint HttpClient ManagedHandler Windows Linux / Mac OpenSSL Sockets SslStream, Dns, …
- 7. Networking – Technical Roadmap https://github.com/dotnet/designs/issues/9 1. Foundation – rock solid • Sockets, SSL, DNS 2. Web stack (client) – high perf & consistency • HttpClient, ClientWebSocket 3. Emerging technologies • HTTP/2 … coming in .NET Core 3.0 as opt-in, mainly for gRPC scenarios • RIO, QUIC 4. Maintenance components • (Http/Ftp/File)WebRequest + ServicePoint, Mail, HttpListener
- 8. Networking – Investments https://github.com/dotnet/designs/issues/9 1. Foundation – rock solid • Sockets, SSL, DNS 2. Web stack (client) – high perf & consistency • HttpClient, ClientWebSocket 3. Emerging technologies • HTTP/2 … coming in .NET Core 3.0 as opt-in, mainly for gRPC scenarios • RIO, QUIC 4. Maintenance components • (Http/Ftp/File)WebRequest + ServicePoint, Mail, HttpListener
- 9. Networking – Investments short-term SocketsHttpHandler – introduced in .NET Core 2.1 (1 year) Next steps: • Documentation (API / conceptual) • Production diagnostics • Testing – multi-machine, load / perf / stress automation • High-perf scenarios – new APIs? • Sync APIs on HttpClient/SocketsHttpHandler • Backlog (15% of all CoreFX) • More perf focus … of course
- 10. Networking – Focus on Perf Scenarios / workloads: • Micro-benchmarks vs. benchmarks vs. real-world scenarios (feedback) Metrics: • RPS (Response per second) • Throughput • Latency • Connection density Important properties: • Percentiles (95% / 99%) • Scale up • Resources (CPU) utilization (90%-95% ideal)
- 11. Networking – Perf Test Environment • Repeatability – isolated environment (reduce noise) • 2 machines: • 4-core • 16 GB RAM • 2x NIC: 1x 1 Gbps + 1x 10 Gbps • 8 servers: • 12-core • 64 GB RAM • 2x NIC: 1x 1 Gbps + 1x 40 Gbps A B External network 10 Gbps 1 Gbps 1 Gbps External network S.1 S.8… 40 Gbps 1 Gbps 1 Gbps 1 Gbps
- 12. Networking – Sockets Perf Results • Micro-benchmark only (disclaimer: Netty/Go impl may be inefficient) • Linux 2 CPUs: • Gap for small payloads (driver problem) • SSL: • 22% gap (to be investigated) GB/s 256 B 4 KB 64 KB 1 MB .NET Core 0.09 0.77 1.09 1.10 Netty 0.11 0.48 0.66 0.67 Go 0.12 0.82 1.10 1.11 SSL - GB/s 256 B 4 KB 64 KB 1 MB .NET Core 0.04 0.31 0.71 0.87 Netty 0.03 0.12 0.15 0.15 Go 0.06 0.56 0.98 1.12
- 13. Networking – SocketsHttpHandler • Introduced in .NET Core 2.1 … May 2018 • Windows: Parity with Go • Linux: 15% gap against Windows (hard problem)
- 14. Networking – SSL Perf • Indicators: • Sockets micro-benchmarks (22% overhead) • TechEmpower benchmark • Customer reports • Attempt for rewrite by community (@drawaes) • Some end-to-end wins, but no micro-benchmark wins • Next steps: • Measure & analyze micro-benchmarks & end-to-end scenarios
- 15. Industry Benchmarks • TechEmpower benchmark • More end-to-end, with DB, etc. • Useful for overall platform performance comparison • Round 17
- 16. Industry Benchmarks - History • Round 15 (from end of 2017 … 1.5 years old) • ASP.NET Core at #5 entry (jump from #14 in Round 14)
- 17. Industry Benchmarks Platform performance shows how fast your app could be, one day … but it is not everything: • Productivity • Tooling • Developer availability (in-house/to hire) • Documentation • Community • etc.
- 18. Performance – Lessons Learned • Plan for performance during design • Understand scenario, set goals • Prototype and measure early • Optimize what’s important – measure • Avoid micro-optimizations • Understand the big picture • Don’t guess root cause – measure • Minimize repro – it’s worth it!
- 19. BCL Performance • Fine-tuned over 15 years • Opportunities are often trade-offs (memory vs. speed, etc.) • How to identify scenarios which matter • Customer feedback • OSS helps: • More eyes on code • More reports • Motivated contributors • Perf improvements in .NET Core (.NET blog post by Stephen Toub) • More Span/friends usage, arrays, strings, parsing, RegEx, Collections, Networking, IO
- 20. BCL Performance – What not to take? • Specialized collections • BCL designed for usability and decent perf for 95% customers • Code complexity (maintainability) vs. perf wins • APIs for specialized operations (e.g. to save duplicate lookup) • Creates complexity • May leak implementation into API surface
- 21. Wrap Up • Proactive investments into .NET Networking stack • Consistency across platforms • Great performance for all workloads • Ongoing scenario/feedback-based improvements in BCL perf • Performance in general is: • Important • But not the only important thing • Tricky to get right in the right place
Editor's Notes
- #3: Why Networking? Networking is even more important in many modern workloads – microservices, Cloud I work on team making some of these cool investments in Networking and BCL These are NOT the only investments in BCL and .NET/.NET Core – just those I have close relation to at my daily job
- #4: Client stack HttpWebRequest since .NET 1.0 Exceptions on errors (404) 4.5 added HttpClient, driven by WCF Pipeline architecture (extensibility) No http2 support Original plan to later re-wire HttpClient directly on fundamentals – huge compat!!!
- #5: UWP: WinRT APIs designed by Windows Networking team at he same time as 4.5 – almost 1:1 mapping win9net.dll is client library - http2 support .NET Core: http2 support – server library HttpWebRequest for compatibility in .NET Core 2.0 (.NET Standard 2.0), but “obsoleted” libcurl with NSS, not just OpenSSL Different behaviors on different OS’s or even Linux distros – inconsistencies in behavior
- #7: Key values: Consistency and Perf wins
- #8: Foundation – performance (& reliability) Usable for both server and client scenarios Web stack – consistency & performance (& reliability) Important for middleware scenarios (not just 1 server) Emerging technologies – new protocols, capabilities and performance motivated RIO = Registered I/O (Win8+) QUIC = Quick UDP Internet Protocol (TCP/TSL replacement) … Latency improvements (esp. for streaming) Maintenance components – minimal investments – mostly for reliability and the most necessary standards support (preserve customers’ investments)
- #9: Foundation – performance (& reliability) Usable for both server and client scenarios Web stack – consistency & performance (& reliability) Important for middleware scenarios (not just 1 server) Emerging technologies – new protocols, capabilities and performance motivated RIO = Registered I/O (Win8+) QUIC = Quick UDP Internet Protocol (TCP/TSL replacement) … Latency improvements (esp. for streaming) Maintenance components – minimal investments – mostly for reliability and the most necessary standards support (preserve customers’ investments)
- #11: RPS (Response per second) – e.g. web servers Throughput – e.g. streaming video/music Latency – e.g. real-time trading Connection density – e.g. messaging services, IoT/devices
- #12: Repeatability Non-networking micro-benchmarks – time (wallclock), memory … classic CLR perf lab disables many OS features Cloud (Azure) attempt Full control of environment needed
- #13: Example how we measure perf and compare, not statement about platform comparison Column is payload size (1B -> 1MB, each column is 16x bigger than previous one) Note: RPS as metric in second table better tells the story of scaling based on payload size 22% gap on SSL against Go Note: .NET Core 1.1 was at 0.47 GB/s at 1MB – 2x improvement in 2.0/2.1
- #15: Sockets micro-benchmarks – 22% gap TechEmpower benchmark - http vs. https larger diff than other platforms Historical reports on some .NET Framework scenarios: https vs. http
- #18: Warning: Not everyone builds hyper-scale service Even if you think you are, don’t forget that most scaling apps are rewritten every 2-3 years You don’t have to be perfect on day 1, evolve over time
- #19: Design performance E.g. for micro-services architecture, model the workloads, use fake data / CPU/network utilization Optimize what’s important: 90-10 rule (95-5) … what contributes to app performance Story: Arguing over d.s. List vs. array when the data is <100 items and the service has 3 hops between servers Guess root cause mistakes: Conclusions based on 1 dump (or 2 in better case)
- #20: Thousands of APIs, hard to pick the right ones Problem: Using telemetry from MS/partners, partner teams reports, working closely with customers and community OSS: We often ask about the scenario, to understand the big picture Examples: (quite often 30% and more)