Battle-tested event-driven patterns for your microservices architecture - Scalac conference
- 1. natansil.com twitter@NSilnitsky linkedin/natansilnitsky github.com/natansil Battle-tested event-driven patterns for your microservices architecture Natan Silnitsky Backend Infra Developer, Wix.com
- 2. >200M registered users (website builders) from 190 countries 5% of all Internet websites run on Wix 2,000 Microservices 1.5B Kafka messages produced / Day @NSilnitsky At Wix * Event driven uses cases
- 3. Agenda Event-driven microservices Event-driven patterns: • Consume and Project • Event-driven from end to end • 0-latency KV Store • Events in Transactions * Monolithic
- 5. @NSilnitsky * Separately Monolith to Distributed Microservices System Microservices
- 10. @NSilnitsky Request-reply communication Microservices Communication Request-Reply Model HTTP RPC HTTP RPC HTTP RPC Checkout Service Payments Service Inventory Service User Service
- 11. @NSilnitsky Request-reply communication * 1 by 1- slow Microservices Communication Request-Reply Model Checkout Service
- 13. Broker @NSilnitsky * Reliability Event-driven communications Introduce a message broker in between services Message Consumer Message Producer
- 14. @NSilnitsky * Not ineteract directly Order created Event-driven communications Introduce a message broker in between services Event-driven model Message Consumer Flow control is moved from the sender Broker Message Producer
- 15. @NSilnitsky Order created Event-driven communications Introduce a message broker in between services Event-driven model Message Consumer Flow control is moved from the sender to the receiver. Broker Message Producer * more robust
- 16. Kafka Broker Topic Partition Partition Partition @NSilnitsky * Linear scaling Introduce a message broker in between services Distributed log broker Event-driven communications Message Consumer Message Producer
- 17. @NSilnitsky Append-only log Introduce a message broker in between services Distributed log broker Event-driven communications Kafka Broker Topic Partition Message Consumer Message Producer 0 1 2 3 4 5 0 1 2 3 4 5 0 1 2 3 4 5 0 1 2 3 4 5 0 1 2 3 4 0 1 2 3 4 5 0 1 2 3 4 5 0 1 2 3 4 5 0 1 2 3 4 5 0 1 2 3 4 5
- 18. Order Created Topic 0 1 2 3 4 5 0 1 2 3 4 5 0 1 2 3 4 5 0 1 2 3 4 5 0 1 2 3 4 0 1 2 3 4 5 0 1 2 3 4 5 0 1 2 3 4 5 0 1 2 3 4 5 0 1 2 3 4 5 @NSilnitsky * Data retained, Immutable Introduce a message broker in between services Distributed log broker Event-driven communications Message Consumer Message Producer Kafka Broker
- 19. @NSilnitsky * eventually consumed Distributed log broker eventually consume Event-driven communications Order Created Topic 0 1 2 3 4 5 0 1 2 3 4 5 0 1 2 3 4 5 0 1 2 3 4 5 0 1 2 3 4 0 1 2 3 4 5 0 1 2 3 4 5 0 1 2 3 4 5 0 1 2 3 4 5 0 1 2 3 4 5 Message Consumer Message Producer Kafka Broker
- 20. Checkout Service Event-driven communications Microservices communication & data Event driven model User Service Inventory Service Cart Service Payments Service @NSilnitsky
- 21. Checkout Service Cart Service Inventory Service Payments Service User Service Event-driven communications Microservices communication & data Event driven model Kafka Broker Orders Partition Partition Partition Users Partition Partition Partition Cart Partition Partition Partition Inventory Partition Partition Partition
- 22. Agenda Event-driven microservices Event-driven patterns: • Consume and Project • Event-driven from end to end • 0-latency KV Store • Events in Transactions * used daily. blog
- 24. MetaSite RPC Wix Stores Wix Bookings Wix Restaurants RPC RPC @NSilnitsky Consume and Project For Popular Services
- 25. MetaSite RPC Wix Stores Wix Bookings Wix Restaurants RPC RPC @NSilnitsky Consume and Project Site installed Apps? Site version? Site owner?
- 26. MetaSite RPC Wix Stores Wix Bookings Wix Restaurants RPC RPC @NSilnitsky Consume and Project Request overload Site installed Apps? Site version? Site owner? (~1M RPM requests) Read + Writes Large MetaSite Object
- 27. Request overload MetaSite RPC Wix Stores Wix Bookings Wix Restaurants RPC RPC @NSilnitsky Consume and Project Site installed Apps? Site version? Site owner? Read + Writes Large MetaSite Object (~1M RPM requests)
- 28. Kafka Broker Produce to Kafka Producer MetaSite Site Updated! Consume and Project Entire MetaSite Context
- 29. Kafka Broker Filter Site installed Apps Updated! Installed Apps Context Produce to Kafka Consume and Project MetaSite Consumer Reverse lookup writer Consume and Project Producer
- 30. Kafka Broker Installed Apps Context Produce to Kafka Consume and Project MetaSite Reverse lookup writer Reverse lookup reader RPC RPC RPC Split Read from Write Consume and Project Producer Consumer
- 31. Consume and Project — Produce instead of serve - massive load reduction — Project - client-query-optimized “Materialized View” — Read/write split - only scale read-only DB. more resilient
- 32. Event-driven from end to end 02
- 33. Kafka Broker Consumer Browser Producer Contacts Importer Service Contacts Jobs Service Web Sockets Service @NSilnitsky Event-driven from end to end Use case: Long-running async business process
- 34. Subscribe for notifications Consumer Producer Browser @NSilnitsky Event-driven from end to end Use case: Long-running async business process Kafka Broker Web Sockets Service
- 35. Import CSVs Consumer Producer Browser @NSilnitsky Event-driven from end to end Use case: Long-running async business process Kafka Broker Web Sockets Service
- 36. Web Sockets Service Consumer * Distributed Producer Browser done! Event-driven from end to end Use case: Long-running async business process Kafka Broker
- 37. Web Sockets Service Consumer * Distributed Producer Browser Cross DC! Event-driven from end to end Use case: Long-running async business process Kafka Broker done!
- 38. Event-Driven from End to End — No polling - notify the user on any status change — Scalable - spin up many import job workers — Replication - cross-DC requests are easy to set up
- 40. Kafka Broker Configuration Topic 0 1 2 3 4 5 0 1 2 3 4 5 0 1 2 3 4 5 0 1 2 3 4 5 0 1 2 3 4 0 1 2 3 4 5 0 1 2 3 4 5 0 1 2 3 4 5 0 1 2 3 4 5 K1 V1 K2 V1 K1 V2 K3 V1 K4 V1 K5 V1 @NSilnitsky Producer Key - value Consumer 0-latency KV Store Compacted Kafka Topics
- 41. Kafka Broker Consumer KVStoreReader In-Memory Map Compacted Topic 0 1 2 3 4 5 0 1 2 3 4 5 0 1 2 3 4 5 0 1 2 3 4 5 K1 V1 K2 V1 K1 V2 K3 V1 K4 V1 K5 V1 Key Value K1 V1 K2 V1 * Startup 0-latency KV Store @NSilnitsky Compacted Kafka Topics
- 42. 0 1 2 3 4 5 0 1 2 3 4 5 0 1 2 3 4 5 0 1 2 3 4 5 K1 V1 K2 V1 K1 V2 K3 V1 K4 V1 K5 V1 @NSilnitsky 0-latency KV Store Kafka Broker Consumer KVStoreReader In-Memory Map Key Value K1 V1 -> V2 K2 V1 K3 V1 Compacted Kafka Topics * Finished Compacted Topic
- 43. Consumer Consumer Bookings Business Manager KVStoreReader Time Zones Key Value Albania UTC+1 Algeria UTC+1 Andorra UTC+1 Time Zones KVStoreReader Countries Key Value Albania .al Algeria .dz Andorra .ad Countries @NSilnitsky 0-latency KV Store Kafka Broker
- 44. Producer Kafka Broker Consumer Bookings Business Manager KVStoreReader Countries Countries *New* South Sudan KVStoreWriter Countries @NSilnitsky 0-latency KV Store * Writer producer
- 45. Kafka Broker Producer Bookings KVStoreWriter Time Zones Time Zones Consumer KVStoreReader Time Zones Key Value Albania UTC+1 ... South Sudan UTC+3 *New* Time Zone for South Sudan *New* Country South Sudan @NSilnitsky 0-latency KV Store
- 46. Kafka Broker Producer Bookings KVStoreWriter Time Zones Time Zones Consumer KVStoreReader Time Zones Key Value Albania UTC+2 Algeria UTC+1 Andorra UTC+1 *New* Time Zone for Albania Consumer Events KVStoreReader Time Zones DST in-effect for Albania @NSilnitsky 0-latency KV Store
- 47. 0-latency KV Store — Loading the compacted topic to memory – a 0-latency kv store for dynamic configuration — Compacted topic is still a topic – other services can consume its update events — Small Datasets - for large datasets, the topic should be streamed to a disk-based DB + In-memory Cache
- 48. Events in Transactions 04 * idempotent
- 50. Payments Checkout Kafka Broker Producer Consumer Producer Order (Items) Completed Events in Transactions Classic Ecommerce Flow Payment Completed
- 51. Payments Checkout Inventory Kafka Broker Producer Consumer Producer Consumer Delivery Consumer Consumer Invoices Events in Transactions Classic Ecommerce Flow
- 52. Producer Consumer Producer Consumer Consumer Consumer Completed Orders Events in Transactions Kafka Broker Classic Ecommerce Flow
- 53. Producer Events in Transactions Kafka Broker Enable.idempotence = true Attaches offset to message Classic Ecommerce Flow
- 54. Idempotent Producer Events in Transactions Kafka Broker Classic Ecommerce Flow
- 55. Events in Transactions Kafka Broker Producer 0 1 2 3 4 5 0 1 duplicate! 1 Payments Idempotent Producer
- 56. Transaction Scope Consumer Producer Consumer Consumer Consumer Producer Events in Transactions Kafka Broker
- 57. Events in Transactions — Events transactions are quite complex – More boilerplate, performance tuning via transaction size. — DB updates/3rd party calls are not covered – You can dedup by using Kafka record offset as version — Don’t turn on transactions by default - Use in critical flows like payments processing
- 59. Wix developers have employed these event-driven patterns to make their microservices more decoupled, resilient and scalable.
- 60. A Scala/Java high-level SDK for Apache Kafka. 0.2 is out! github.com/wix/greyhound
- 61. Thank You natansil.com twitter@NSilnitsky linkedin/natansilnitsky github.com/natansil
- 62. Resources 6 Event Driven Architecture Patterns - Part 1 & Part 2 - by Natan Silnitsky The Data Dichotomy: Rethinking the Way We Treat Data and Services - by Ben Stopford Shared Database - by Chris Richardson Exactly once delivery - DevOpsDay TLV 2019 - By Natan Silnitsky @NSilnitsky