Staying Afloat with Buoy: A High-Performance HTTP Client
- 1. Louis-Philippe Gauthier Director, Platform Engineering Staying afloat with Buoy: a high-performance HTTP client
- 3. Story time 3 1. A • 2011: ibrowse • 2012: dlhttpc (fred’s dispcount lhttpc fork) • 2013: dlhttpc 2.0 (my fork) • 2017: buoy!
- 4. Problems with dlhttpc 2.0 4 • no pipelining! • reconnects on request • complex code base with broken test suite • hacks on hacks on hacks
- 6. Buoy’s architecture 6 1. A 2. C process shackle_pool buoy_pool_utils
- 8. 1. A 2. B 3. C Problem 8 application service ???
- 9. Service…? 9 • communicates over socket • ascii or binary protocols • synchronous or asynchronous protocols
- 10. Goals?! 10 • reusability • speed (low overhead) • concurrency • safety
- 12. Features (last year) 12 1. A • multi-protocol support (TCP / UDP) • fast pool implementation (random, round_robin) • request pipelining • backpressure via backlog (OOM protection) • smart reconnect mechanism (exp. backoff)
- 13. Architecture (last year) 13 service process msg sockets ETS clients (proc_lib) pool options + backlog
- 14. Shackle features (today) 14 1. A • Managed timeouts • SSL support • handle_data callback error • optimizations: • granderl • cache pool options with module • remove setelement/3 calls 2. C
- 17. Shackle features (future) 17 1. A • SCTP network protocol support • more optimizations! • better test suite • better documentation 2. C
- 18. Shackle usage 18
- 19. 19 buoy_client
- 26. buoy_protocol response 26 • small HTTP parser (< 150 lines) • all-at-once vs streaming parser • fast! (10x hackney’s parser) • minimize function calls • minimize sub-binary creation • minimize setelement/3 calls • uses binary:split/{2,3} 2. C
- 28. buoy_protocol response 28 1. A • split 25.7 μs • split-hipe 20.7 μs • split2 16.3 μs
- 30. buoy_protocol response 30 1. A • implements a subset of RFC2616 • unsupported features: • arbitrary number of new lines in headers • random capitalization of headers • protection against malicious servers • chunked encoding 2. C
- 31. buoy_compiler 31 • generates module buoy_pool_utils • function cases generated from ETS state • uses erl_syntax to generate AST • hot-load with code:load_binary/3 2. C
- 36. profiling 36 • make profile target • fprofx (fork of fprof with suspend and GC) • qcachegrind / kcachegrind 2. C
- 37. profiling 37
- 38. profiling 38
- 41. extrospect-beam 41 1. A • sampling profiler using perf events (linux only) • by Julian Squires (AdGear) • erlang-write-perf-map PID 2. C
- 42. perf top -p PID 42
- 45. httpc_bench 45 1. A • timing_hdr • HDR histogram by Gil Tene (Azul Systems) • hdr_histogram_erl (NIF) • It’s on Github, send PRs! 2. C
- 46. httpc_bench 46 1. A • dlhttpc 2.0 • hackney 1.6.2 • httpc (erlang 18.3.3) • ibrowse 4.4.0 • katipo 0.4.0 • buoy 0.1.0 2. C
- 47. httpc_bench 47 1. A • bench servers: • 48 Xeon cores @ 2.20GHz • 256 GB ram • SSDs • latency between servers < 0.1 ms • Erlang 18.3.3 2. C
- 48. httpc_bench 48 1. A • iterations / # requests • concurrency / # processes • pool_size/ # connections 2. C
- 55. buoy 55
- 56. httpc_bench results 56 1. A • buoy (max: 605K/s) • katipo (max: 96K/s) - 6.3x • dlhttpc (max: 70K/s) - 8.6x • ibrowse (max: 59K/s) 10.2x • hackney (max: 33K/s) - 18.3x • httpc (max: 14K/s) - 43x 2. C
- 57. 57 Tips & Tricks
- 58. Performance tips 58 • pattern match • use iolist() • keep your server loops lean (specialized) • use ETS for shared state (R/W) • compile modules for read only states
- 59. Performance tips 59 • bin_opt_info compile opt • measure everything
- 60. Links 60 • http://github.com/lpgauth/buoy • http://github.com/tokenrove/extrospect-beam • http://github.com/tokenrove/granderl • http://github.com/lpgauth/httpc_bench • http://github.com/lpgauth/metal • http://github.com/lpgauth/shackle • https://github.com/lpgauth/timing