From C to Q one event at a time: Event Sourcing illustrated
23 likes4,896 views
The document discusses event sourcing and CQRS patterns. It explains that event sourcing stores a history of events to rebuild the system state, while CQRS separates read and write operations by having different data models optimized for each. Materialized views are then used to generate read models from the event stream to enable efficient querying and address the potential delay in reads compared to a traditional system. The document also provides lessons learned, noting that both the write and read sides need attention, and that event sourcing works best for high volume, low latency writes like in microservices architectures.
From C to Q one event at a time: Event Sourcing illustrated
- 1. Voxxed Days Bristol 2017 Lorenzo Nicora - @nicusX From C to Q, one Event at a time 1 Lorenzo Nicora OpenCredo Event Sourcing illustrated
- 2. Voxxed Days Bristol 2017 Lorenzo Nicora - @nicusX Lorenzo Nicora @nicusX lorenzo.nicora@opencredo.com https://opencredo.com/author/lorenzo/ Senior Consultant @ OpenCredo • Microservices • Cloud • Event-driven, Event Sourcing, Reactive • Java, Spring, Akka… 2
- 3. 3 Concepts from DDD Voxxed Days Bristol 2017 Lorenzo Nicora - @nicusX Aggregate Event Sourcing,CQRS => DDD/ State: f(t) Domain Event
- 4. Voxxed Days Bristol 2017 Lorenzo Nicora - @nicusX 4 Once upon a time… Everything was Synchronous Request <-> Response
- 5. Voxxed Days Bristol 2017 Lorenzo Nicora - @nicusX 5 Scaling out… Updates —> Locks —> Contention! <— Block! <— -> Distributed still Synchronous
- 6. Voxxed Days Bristol 2017 Lorenzo Nicora - @nicusX 6 Let’s go Asynchronous -> Distributed & Message-driven Request/Response ACID Transactions • Distributed: Propagation takes time • Async: No global order strictly guaranteed Updates applied in the wrong order => Inconsistent state!
- 7. Voxxed Days Bristol 2017 Lorenzo Nicora - @nicusX 7 Command Sourcing 💡 • Append Only —> No Contention, No Update • Build State from Commands history K/V Store Distributed
- 8. Command “Submit this Order! (Please)” -> A request (imperative sentence) -> May fail, be rejected -> May affect multiple Aggregates 8 Commands Voxxed Days Bristol 2017 Lorenzo Nicora - @nicusX ✉ Rebuild Aggregate State from Commands
- 9. 9 Events Voxxed Days Bristol 2017 Lorenzo Nicora - @nicusX Event “Order submitted” -> Statement of facts (past tense) -> Never fails -> Can’t be changed -> May be designed to affect a single Aggregate ✉ Rebuild Aggregate State from Events
- 10. Voxxed Days Bristol 2017 Lorenzo Nicora - @nicusX 10 Command > Event Sourcing 💡
- 11. Voxxed Days Bristol 2017 Lorenzo Nicora - @nicusX 11 Commands to Events X Y Z ?
- 12. Voxxed Days Bristol 2017 Lorenzo Nicora - @nicusX 12 (Business) Consistency (ACID) Transactions Eventual (Business) Consistency Guess —> Compensate—> Apologies
- 13. Voxxed Days Bristol 2017 Lorenzo Nicora - @nicusX 13 e.g. Saga Stateful Out of band Corrective Action (Command / Event) Saga
- 14. Voxxed Days Bristol 2017 Lorenzo Nicora - @nicusX –Greg Young “Accountants don’t use pencils. They use pens” Corrective Actions 14
- 15. Voxxed Days Bristol 2017 Lorenzo Nicora - @nicusX 15 Benefits of Event Sourcing History (for free) Rebuild State at a point in Time
- 16. Voxxed Days Bristol 2017 Lorenzo Nicora - @nicusX 16 Benefits of Event Sourcing Easier Eventual Business Consistency —> Corrective Events Robust to data corruption (bugs, malicious, fat fingers…)
- 17. Voxxed Days Bristol 2017 Lorenzo Nicora - @nicusX 17 Benefits of Event Sourcing Horizontal Scalability & Low Latency writes -> Distributed systems & data store —> Append-only Log —> Asynchronous processing
- 18. Voxxed Days Bristol 2017 Lorenzo Nicora - @nicusX 18 Commands to Events Stateful: Events depends 0n current State Stateless: Validate, split by Aggregate… Depends on Business Domain x Point of synchronisation ? Out-of-order commands (IoT, Mobile)
- 19. Voxxed Days Bristol 2017 Lorenzo Nicora - @nicusX 19 Thinking fast or slow Stateless: Think fast, Write fast, More thinking later Stateful: Think slow…(rebuild state), Write fast, Less thinking later
- 20. Voxxed Days Bristol 2017 Lorenzo Nicora - @nicusX What about reads? 20
- 21. Voxxed Days Bristol 2017 Lorenzo Nicora - @nicusX 21 Retrieving the State How do I retrieve the State? “Get details of Order ‘AB123’” ❔ not very efficient, but… …may work
- 22. Voxxed Days Bristol 2017 Lorenzo Nicora - @nicusX 22 Querying (Searching) the State ❓ How do query the State? “Get all Orders delivered to ‘SE1 0NZ’”
- 23. Voxxed Days Bristol 2017 Lorenzo Nicora - @nicusX 23 CQRS Command Query Responsibility Segregation 💡 Separate • Code • muService • Datastore -> Update -—> Retrieve
- 24. Voxxed Days Bristol 2017 Lorenzo Nicora - @nicusX 24 Not a new idea Specialised Downstream
- 25. Voxxed Days Bristol 2017 Lorenzo Nicora - @nicusX 25 CQRS and Event Sourcing for low-latency writes Event Sourcing => CQRS
- 26. Voxxed Days Bristol 2017 Lorenzo Nicora - @nicusX 26 Materialised Views • In Memory K/V Store Graph DB RDBMS • Rebuildable from Events 💡 a.k.a. Read-optimised Views Read Views Projections…
- 27. Voxxed Days Bristol 2017 Lorenzo Nicora - @nicusX 27 Materialised Views * Views are optimised for specific query use cases —> multiple Views from same Events * May be updated asynchronously + low latency writes, scalability - delayed + may reorder Events
- 28. Voxxed Days Bristol 2017 Lorenzo Nicora - @nicusX 28 Materialised Views The Event Log is your Source of Truth * Easy to evolve or fix —> change or fix logic; rebuild view from events
- 29. Voxxed Days Bristol 2017 Lorenzo Nicora - @nicusX 29 Indexes • Search Engines K/V Stores Graph DB + Optimised for querying (less for retrieving) + Reduced delay of State 💡
- 30. Voxxed Days Bristol 2017 Lorenzo Nicora - @nicusX 30 Hybrid solutions: e.g. Snapshots + Speed up rebuilding the State + Use recent Events to rebuild up-to-date 💡 Long delayed
- 31. Distributed + Asynchronous system Distributed ACID Transactions (2PC) 31 No Global Current State Voxxed Days Bristol 2017 Lorenzo Nicora - @nicusX AS S A B Information propagates at a finite speed
- 32. Voxxed Days Bristol 2017 Lorenzo Nicora - @nicusX 32 Lesson from the Trenches
- 33. Voxxed Days Bristol 2017 Lorenzo Nicora - @nicusX 33 Lesson Learned #1 If you put data in… …you will eventually have to get them out! The “Query” side is not secondary
- 34. Voxxed Days Bristol 2017 Lorenzo Nicora - @nicusX 34 Lessons Learned #2 In old days: normalising one DB to support as many queries as possible With CQRS (also No SQL in general) multiple denormalised “data stores” optimised for different queries
- 35. Voxxed Days Bristol 2017 Lorenzo Nicora - @nicusX 35 Lessons Learned #3 Centralised Event Store = Event-sourced monolith Each microservice owns its state (Event Store) Microservices:
- 36. Voxxed Days Bristol 2017 Lorenzo Nicora - @nicusX +++ Summing up +++ 36
- 37. Voxxed Days Bristol 2017 Lorenzo Nicora - @nicusX 37 ES/CQRS Optimal Use Cases • High Volume, Low Latency writes • Out-of-order Commands (IoT, Mobile) Event Sourcing + CQRS 😋 Domain: stateless Command-> Event
- 38. Voxxed Days Bristol 2017 Lorenzo Nicora - @nicusX 38 ES/CQRS Drawbacks x No “One-Size-Fits-All” —> Multiple “Q” implementations x Delayed reads x No ACID Transactions x Additional complexity (!!!) 🙁
- 39. Voxxed Days Bristol 2017 Lorenzo Nicora - @nicusX 39 ES/CQRS Benefits + No “One-Size-Fits-All” —> “Q” optimised per use cases + Distributed systems (Microservices) + Eventual (Business) Consistency + History, Temporal queries + Robust to data corruption 😀
- 40. Voxxed Days Bristol 2017 Lorenzo Nicora - @nicusX That’s all, Folks! 40
- 41. Voxxed Days Bristol 2017 Lorenzo Nicora - @nicusX ??? Questions ??? 41 Thanks. ⏳