Seastar Summit 2019 vectorized.io
0 likes5,313 views
Vectorized is presenting on their RPC system and Redpanda product. They are focused on operational simplicity, safety, and 10x the performance of Kafka. Their RPC goals include using PODs instead of IDLs, isolation, avoiding translation costs, and embracing futures. Their measurements show their RPC system is up to 7x faster than flatbuffers for nested data structures and has lower latency than alternative systems.
Seastar Summit 2019 vectorized.io
- 2. I’m alex, we are vectorized ● CEO ● Have been working w/ seastar for 3 years. ○ Prev: Principal Eng @Akamai; CTO for concord.io, acquired by Akamai 2016 ● First non-scylla fully seastar-based company/product (vectorized.io/redpanda) VECTORIZED
- 3. redpanda by vectorized ● Redpanda is a Kafka replacement for mission critical systems, focused on: ○ operational simplicity ○ safety ○ 10x the performance VECTORIZED
- 4. agenda ● results ● motivation ● reflection ● ergonomics ● measurements ● limitations VECTORIZED
- 5. VECTORIZED talk: 6x faster than flatbuffers for nested tables ~7X lower latency! 24ms 161 ms
- 6. rpc goals, (1) PODs / No IDL
- 7. rpc goals, (2) isolation VECTORIZED Shares quota x-core comms quota
- 8. VECTORIZED rpc goals, (3) No translation cost connected_socket::input() 🠖 input_stream<char>
- 9. rpc goals, (4) native seastar::* types VECTORIZED
- 10. rpc goals, (5) embrace the future<> ● seastar::future<>’s are epidemic, like actors VECTORIZED
- 11. rpc goals, part 2 ● Measurements ○ Large structs of native fields - 52 fields. Nested 6 levels. ○ Large payloads in fragments ● Reflection ● Ergonomics ● Limitations ○ non pointers, they require custom serdes - maybe useful VECTORIZED
- 12. VECTORIZED concept 1 - structure binding
- 13. VECTORIZED concept 2 - constexpr
- 14. VECTORIZED concept 3 - is_standard_layout && is_trivially_copyable
- 15. VECTORIZED concept 4 - reflect & recurse
- 16. VECTORIZED concept 4 - reflect & recurse
- 17. VECTORIZED concept 4 - reflect & recurse
- 18. VECTORIZED concept 4 - reflect & recurse
- 19. VECTORIZED concept 4 - reflect & recurse
- 21. test 1 - large payload test 2 - nested payload
- 22. 5% worse End-to-end latency. 10GB at once, in 100K requests, 1MB per request, in 32K chunks test 1 - large payload
- 23. server-side latency. 230MB at once in 1MM requests. ~7X faster! 24ms 161 ms test 2 - nested payload
- 24. 2X better end-to-end latency. 230MB at once in 1MM requests. test 2 - nested payload