A Visual Introduction to Event Sourcing and CQRS by Lorenzo Nicora
36 likes6,220 views
This document provides an introduction to event sourcing and Command Query Responsibility Segregation (CQRS), outlining key concepts such as command and event differentiation, scalable design approaches, and the benefits of eventual consistency. It discusses the significance of having specialized data stores for querying and highlights challenges such as delayed reads and the complexity of implementation. Overall, it emphasizes the importance of optimizing data retrieval and management in modern distributed systems.
A Visual Introduction to Event Sourcing and CQRS by Lorenzo Nicora
- 1. A visual introduction to Event Sourcing and CQRS 1 @nicusX https://opencredo.com/author/lorenzo/ Lorenzo Nicora Senior Consultant @ OpenCredo
- 2. 2 A couple of concepts from DDD Aggregate (Current) State of the Aggregate Lorenzo Nicora Intro to Event Sourcing and CQRS
- 3. 3 Once upon a time… Everything is synchronous Request - Response Lorenzo Nicora Intro to Event Sourcing and CQRS
- 4. 4 Scaling up… Updates —> Locks —> Contention! <— Block <— Lorenzo Nicora Intro to Event Sourcing and CQRS
- 5. 5 Let’s go Asynchronous Pwd —> “secret” Pwd —> “12345” Pwd —> “54321” Pwd —> “secret” Pwd —> “54321” Pwd —> “12345” ===> Out of Order Asynchronous, Message-driven Request/Response ACID Transaction Distributed, Message-based —> No order guaranteed Lorenzo Nicora Intro to Event Sourcing and CQRS
- 6. 6 Command Sourcing 💡 • Append Only —> No Contention • Build State from Command history Write fast, Think later K/V Store Distributed Lorenzo Nicora Intro to Event Sourcing and CQRS
- 7. Command “Submit Order!” —> A request (imperative sentence) —> May fail —> May affect multiple Aggregates 7 Commands vs Events ✉ Rebuild Aggregate State from Commands Lorenzo Nicora Intro to Event Sourcing and CQRS
- 8. 8 Event “Order submitted” —> Statement of facts (past tense) —> Never fails —> May affect a single Aggregate ✉ Rebuild Aggregate State from Events Lorenzo Nicora Intro to Event Sourcing and CQRS
- 9. 9 Commands to Events (DDD patterns: Aggregate / Process Manager) X Y Z Lorenzo Nicora Intro to Event Sourcing and CQRS
- 10. 10 Command > Event Sourcing 💡 Think a little, Write, Think later Lorenzo Nicora Intro to Event Sourcing and CQRS
- 11. 11 Event Sourcing In many domains Commands Events Lorenzo Nicora Intro to Event Sourcing and CQRS
- 12. 12 Additional Benefits Easy Eventual Business Consistency —> Corrective Events Robust to data corruption (bugs, fat fingers…) —> Rebuild state ignoring wrong events Lorenzo Nicora Intro to Event Sourcing and CQRS
- 13. 13 Additional Benefits History (for free) Rebuild State at a point in Time Lorenzo Nicora Intro to Event Sourcing and CQRS
- 14. 14 Main Benefit Scalable —> Append only -> Fits distributed k/v stores —> Low-latency writes —> Allows asynchronous processing Lorenzo Nicora Intro to Event Sourcing and CQRS
- 15. What about reads? 15 Lorenzo Nicora Intro to Event Sourcing and CQRS
- 16. 16 Retrieving the State How do I retrieve the State? “Get details of Order ‘AB123’” ❔ not very efficient, but… …may work Lorenzo Nicora Intro to Event Sourcing and CQRS
- 17. 17 Querying (Searching) the State ❓ ❓ How do query the State? “Get all Orders delivered to ‘SE1 0NZ’” ❓ ❓ Lorenzo Nicora Intro to Event Sourcing and CQRS
- 18. 18 CQRS Command Query Responsibility Segregation 💡 Separate • Code • muService • Datastore -> Update -—> Retrieve “Query” datastore is downstream Lorenzo Nicora Intro to Event Sourcing and CQRS
- 19. 19 Not a new idea Specialised Downstream Lorenzo Nicora Intro to Event Sourcing and CQRS
- 20. 20 CQRS and Event Sourcing Lorenzo Nicora Intro to Event Sourcing and CQRS
- 21. 21 Materialised Views (of current State) Lorenzo Nicora Intro to Event Sourcing and CQRS
- 22. 22 Materialised Views (of State) Latest (known) State (Persistent) Rebuildable from Events In Memory K/V Store Graph … RDBMS Delayed 💡 Lorenzo Nicora Intro to Event Sourcing and CQRS
- 23. 23 Materialised View of State Query a RDBMS?!? Wasn’t it the old way? ❓ RDBMS is just one of our options: easy to use, easily become a bottleneck Lorenzo Nicora Intro to Event Sourcing and CQRS
- 24. 24 Materialised Views of State * Views are optimised for specific query use cases —> multiple Views from same Events * Updated asynchronously, delayed —> to Scale -> may reorder Events Lorenzo Nicora Intro to Event Sourcing and CQRS
- 25. 25 Materialised Views of State * Views can be rebuilt from Events Event Log is our Source of Truth * Easy to evolve or fix —> change or fix logic; rebuild view from events (not the View) Lorenzo Nicora Intro to Event Sourcing and CQRS
- 26. 26 Indexes Lorenzo Nicora Intro to Event Sourcing and CQRS
- 27. 27 Indexes Search Engines K/V Stores • Optimised for querying (less for retrieving) • Latest State; rebuild on the fly 💡 Lorenzo Nicora Intro to Event Sourcing and CQRS
- 28. 28 Hybrid solutions Lorenzo Nicora Intro to Event Sourcing and CQRS
- 29. 29 Hybrid solutions: e.g. Snapshots • Speed up rebuilding the current State • Use recent Events to rebuild up-to-date 💡 Long delayed Lorenzo Nicora Intro to Event Sourcing and CQRS
- 30. 30 Eventual (Business) Consistency Guess —> Compensate—> Apologies Lorenzo Nicora Intro to Event Sourcing and CQRS
- 31. 31 Eventual Consistency: Sagas StatefulOut of band Corrective Command or Event Saga Lorenzo Nicora Intro to Event Sourcing and CQRS
- 32. 32 Lesson from the Trenches Lorenzo Nicora Intro to Event Sourcing and CQRS
- 33. 33 Lesson Learned #1 If you put data in… …you will eventually have to get them out! The “Query” side is not secondary Lorenzo Nicora Intro to Event Sourcing and CQRS
- 34. 34 Lessons Learned #2 In old days: normalising one DB to support as many queries as possible With CQRS: multiple denormalised “data stores” optimised for different queries No single “Q” implementation for all your queries Lorenzo Nicora Intro to Event Sourcing and CQRS
- 35. 35 Lessons Learned #3 A central, shared Event Store may not be the best option No Event-sourced Monolith Prefer persistence per Bounded-Context Lorenzo Nicora Intro to Event Sourcing and CQRS
- 36. +++ Summing up +++ 36 Lorenzo Nicora Intro to Event Sourcing and CQRS
- 37. 37 ES/CQRS Optimal Use Cases High Volume Low Latency writes (big data) Event Sourcing + CQRS 😋 Out-of-order Commands/Events (IoT) Lorenzo Nicora Intro to Event Sourcing and CQRS
- 38. 38 ES/CQRS Drawbacks x No “One-Size-Fits-All” —> Multiple “Q” implementations x Delayed reads x No ACID Transactions x Additional complexity (!) 🙁 Lorenzo Nicora Intro to Event Sourcing and CQRS
- 39. 39 ES/CQRS Benefits + No “One-Size-Fits-All” —> “Q” are optimised for use cases + Eventual (Business) Consistency + History, Temporal queries + Robust to data corruption 😀 Lorenzo Nicora Intro to Event Sourcing and CQRS
- 40. That’s all, Folks! 40 Lorenzo Nicora Intro to Event Sourcing and CQRS
- 41. ??? Questions ??? 41 Thanks. ⏳ Lorenzo Nicora Intro to Event Sourcing and CQRS