Skyscanner Engineering: Putting sync async servers to test by Jordi Soucheiron
Download as PPTX, PDF1 like462 views
The document discusses the performance testing of synchronous and asynchronous servers using Docker containers with various configurations. It outlines assumptions, benchmarks, and conclusions regarding latency, memory usage, and suitability for different types of workloads. Ultimately, asynchronous servers show advantages for handling many simultaneous requests, while synchronous servers are simpler and might have better latency for straightforward tasks.
Skyscanner Engineering: Putting sync async servers to test by Jordi Soucheiron
- 1. Putting sync/async servers to test Jordi Soucheiron
- 2. Who am I? Putting sync/async servers to test • Software engineer @ Skyscanner • Python lover • Systems administration and backend systems background • @jordixou http://bit.ly/jordisync
- 3. What are we going to talk about? • Sync/async models • Assumptions • Benchmarking • Conclusions • Questions Putting sync/async servers to test http://bit.ly/jordisync
- 4. How do sync servers (usually) work Putting sync/async servers to test http://bit.ly/jordisync
- 5. How do async servers (usually) work Putting sync/async servers to test http://bit.ly/jordisync
- 6. The test environment • 3 docker containers • aiohttp • Flask • nginx • wrk • used to stress test and gather data • 2 threads, 10-100 connections, 30 seconds • docker stats to gather container performance data • Tested in a macbook pro 13” 2.9GHz i5 • Docker configured to use 4 cores and 4GB of RAM Putting sync/async servers to test http://bit.ly/jordisync
- 7. Nginx container • Using openresty • Hello world with delays: • 0ms • 100ms • 200ms • 1s Putting sync/async servers to test http://bit.ly/jordisync
- 8. Flask container • Dockerfile • Python 3.6.3 • Using gunicorn, 100 worker threads Putting sync/async servers to test http://bit.ly/jordisync
- 9. Flask container • 2 endpoints • IO bottleneck • CPU bottleneck Putting sync/async servers to test http://bit.ly/jordisync
- 10. aiohttp container • Dockerfile • Python 3.6.3 • Using gunicorn, 1 worker per CPU Putting sync/async servers to test http://bit.ly/jordisync
- 11. aiohttp container • 2 endpoints • IO bottleneck Putting sync/async servers to test http://bit.ly/jordisync
- 12. aiohttp container • 2 endpoints • CPU bottleneck Putting sync/async servers to test http://bit.ly/jordisync
- 13. Assumptions • Async servers use less memory • Latency is smaller in sync servers • Sync code is simpler • Async code is harder to reason about • Async servers can handle more requests/seq Putting sync/async servers to test http://bit.ly/jordisync
- 14. Benchmarks Putting sync/async servers to test http://bit.ly/jordisync
- 15. Benchmarks • Disclaimer: • This is not gonna be a fair fight Putting sync/async servers to test http://bit.ly/jordisync
- 16. Benchmarks Putting sync/async servers to test http://bit.ly/jordisync 0.00 200.00 400.00 600.00 800.00 1000.00 1200.00 hello world cpu_waster 100ms 200ms 1s Response time (lower is better) nginx aiohttp flask
- 17. Benchmarks Putting sync/async servers to test http://bit.ly/jordisync 0.00 500.00 1000.00 1500.00 2000.00 2500.00 3000.00 3500.00 4000.00 4500.00 hello world cpu_waster 100ms 200ms 1s Requests per second (higher is better) nginx aiohttp flask
- 18. Benchmarks Putting sync/async servers to test http://bit.ly/jordisync 0 5 10 15 20 25 30 35 hello world cpu_waster 100ms 200ms 1s Memory per request per second (lower is better) nginx aiohttp flask
- 19. Assumptions review • Async servers use less memory • Latency is smaller in sync servers • Sync code is simpler • Async code is harder to reason about • Async servers can handle more requests/seq Putting sync/async servers to test http://bit.ly/jordisync
- 20. Conclusions • Async servers are great for*: • Slow dependencies • Many external dependencies • Data aggregation from multiple sources • Microservices • Lots of I/O • Async is bad for*: • Static content • CPU intensive work Putting sync/async servers to test *generally http://bit.ly/jordisync
- 21. Conclusions • Pros: • Lower memory footprint* • Higher throughput (req/s)* • Cons: • Latency for simple sync requests is higher* • Chasing bottlenecks is harder Putting sync/async servers to test *generally http://bit.ly/jordisync
- 22. Questions? • All feedback is welcome • Slides and links to the code available here: • http://bit.ly/jordisync • Ping me at @jordixou Putting sync/async servers to test http://bit.ly/jordisync
- 23. Thank you