Building Microservices with Event Sourcing and CQRS
16 likes3,556 views
This document summarizes a presentation about building microservices with event sourcing and CQRS. It begins by reviewing the characteristics of a traditional n-tier architecture, then introduces event sourcing as an architectural pattern where application state is determined by a sequence of immutable events. Key aspects of event sourcing include storing events in an event store, processing events with handlers, and replaying events to rebuild state. CQRS is also introduced, which separates commands from queries by using different interfaces and models. Consistency challenges with event sourcing architectures are discussed, such as eventual consistency, validation, and handling parallel updates.
Building Microservices with Event Sourcing and CQRS
- 1. Unless otherwise indicated, these slides are © 2013-2015 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ SPRINGONE2GX WASHINGTON, DC Building Microservices with Event Sourcing and CQRS Michael Ploed - innoQ @bitboss
- 2. IncidentSOAPEndpoint IncidentBusinessService IncidentDAO Incident Business Model Client Incident DTO Incident View Model RDBMS Incident ER-Model Network Network Let’s review the classical old school N-Tier architecture
- 4. 1 We read and write data through the same layers IncidentSOAPEndpoint IncidentBusinessService IncidentDAO Incident Business Model Client Incident DTO Incident View Model RDBMS Incident ER-Model Network Network READ WRITE
- 5. 2 We use the same model for read and write access IncidentSOAPEndpoint IncidentBusinessService IncidentDAO Incident Business Model Client Incident DTO Incident View Model RDBMS Incident ER-Model Network Network
- 6. 3 We use coarse grained deployment units that combine read and write code IncidentSOAPEndpoint IncidentBusinessService IncidentDAO Client RDBMS
- 7. 4 We change datasets directly IncidentRestController IncidentBusinessService IncidentDAO Incident ID USER_ID DATE TEXT 1 23423 11.03.2014 Mouse is broken 2 67454 12.03.2014 EMail not working 3 93729 12.03.2014 Office license … … … …
- 8. We’ve done that for ages already and it’s proven
- 9. Many applications will run smooth and fine with this kind of approach
- 10. However there are drawbacks to this kind of architecture
- 11. 1 The data model is a compromise 2 You can’t scale read and write independently 3 No data history, no snapshots, no replay 4 Tendency to a monolithic approach
- 12. Event Sourcing is an architectural pattern in which the state of the application is being determined by a sequence of events
- 13. Building Blocks Applications Event Queue Applications issues events to a queue Event Handler Event Store Events are stored in the event store The Event handler is processing the events
- 14. The sequence of events in the queue is called event stream t now EventEventEventEventEvent
- 15. IncidentCreatedEvent incidentNumber: 1 userNumber: 23423 timestamp: 11.03.2014 12:23:23 text: „Mouse broken“ status: „open“ Event Stream Example IncidentTextChangedEvent incidentNumber: 1 text: „Left button of mouse broken“ IncidentClosedEvent incidentNumber: 1 solution: „Mouse replaced“ status: „closed“
- 16. An event is something that happened in the past t now EventEventEventEventEvent
- 17. The names of the events are part of the Ubiquitous Language D D D
- 19. Code Example public class CustomerVerifiedEvent { private String eventId; private Date eventDate; private CustomerNumber customerNumber; private String comment; public CustomerVerifiedEvent(CustomerNumber custNum, String comment) { this.customerNumber = cusNum; this.comment = comment; this.eventDate = new Date(); } }
- 20. Scope your events based on Aggregates D D D
- 21. An Event is always immutable !
- 22. There is no deletion of events !
- 23. A delete is just another event IncidentCreatedEvent incidentNumber: 1 userNumber: 23423 timestamp: 11.03.2014 12:23:23 text: „Mouse is broken defekt“ status: „open“ IncidentChangedEvent incidentNumber: 1 text: „Maus ist Kaputt“ IncidentRemovedEvent incidentNumber: 1
- 24. The event bus is usually implemented by a message broker t now EventEventEventEventEvent
- 25. Let’s reuse the ESB from the failed SOA project
- 26. NO
- 27. NO
- 28. NO
- 29. ! Prefer dumb pipes with smart endpoints as a suitable message broker architecture
- 30. 1 Complete rebuild is possible 2 Temporal Queries 3 Event Replay
- 31. Well known examples = Version Control Systems or Database Transaction Logs
- 32. The Event Store has a very high business value
- 33. Aren’t there performance issues attached to this kind of data storage?
- 34. YES!
- 35. Application State Think about application state Application Event Queue Event Handler Event Store Application State The application queries the pre-processed application state
- 36. CQRS
- 39. IncidentQueryEndpoint IncidentQueryService IncidentQueryDAO RDBMS Network IncidentCommandEndpoint IncidentCommandService IncidentCommandDAO Basically the idea behind CQRS is simple
- 40. Code Example
- 41. Classic Interface public interface IncidentManagementService { Incident saveIncident(Incident i); void updateIncident(Incident i); List<Incident> retrieveBySeverity(Severity s); Incident retriveById(Long id); }
- 42. CQRS-ified Interfaces public interface IncidentManagementQueryService { List<Incident> retrieveBySeverity(Severity s); Incident retriveById(Long id); } public interface IncidentManagementCommandService { Incident saveIncident(Incident i); void updateIncident(Incident i); }
- 43. Event Store EventHandler EventsEvents Event Sourcing & CQRS IncidentCommandEndpoint IncidentCommandService IncidentCommandDAO IncidentQueryEndpoint IncidentQueryService IncidentQueryDAO Read Storage Events READ
- 44. 1 Individual scalability and deployment options 2 Technological freedom of choice for command, query and event handler code 3 Excellent Fit for Bounded Context (Domain Driven Design)
- 45. Event Sourcing and CQRS are interesting architectural options. However there are various challanges, that have to be taken care of
- 46. 1 Consistency 2 Validation 3 Parallel Updates
- 47. YES
- 48. ! Systems based on CQRS and Event Sourcing are mostly eventually consistent
- 49. Event Store EventHandler EventsEvents Eventual Consistency IncidentCommandEndpoint IncidentCommandService IncidentCommandDAO IncidentQueryEndpoint IncidentQueryService IncidentQueryDAO Read Storage Events READ
- 50. BUT
- 51. ! You can build a fully consistent system which follows Event Sourcing principles
- 52. EventHandler EventsEvents Full Consistency IncidentCommandEndpoint IncidentCommandService IncidentCommandDAO IncidentQueryEndpoint IncidentQueryService IncidentQueryDAO Read Storage Events Select * from Event Store
- 53. Your business domain drives the level of consistency not technology Deeper Insight D D D
- 54. Increased (but still eventual) consistency EventHandler EventsEvents IncidentCommandEndpoint IncidentCommandService IncidentCommandDAO IncidentQueryEndpoint IncidentQueryService IncidentQueryDAO Read Storage Events Select * from Event Store async
- 55. ! There is no standard solution
- 56. 1 Consistency 2 Validation 3 Parallel Updates
- 57. Example Domain User Guid id String email String password RegisterUserCommand ChangeEmailCommand UserRegisteredEvent Guid id Date timestamp String email String password EmailChangedEvent Guid userId Date timestamp String email
- 58. We process 2 million+ registrations per day. A user can change her email address. However the emails address must be unique
- 59. ? How high is the probability that a validation fails Which data is required for the validation Where is the required data stored
- 60. $ What is the business impact of a failed validation that is not recognized due to eventual consistency and how high is the probability of failure
- 61. Your business domain drives the level of consistency Deeper Insight D D D
- 62. 1 Validate from Event Store 2 Validate from Read Store 3 Perform Validation in Event Handler
- 63. Never validate from the event store
- 64. 1 Consistency 2 Validation 3 Parallel Updates
- 65. Example Domain User Guid id String email String password RegisterUserCommand ChangeEmailCommand UserRegisteredEvent Guid id Date timestamp String email String password EmailChangedEvent Guid userId Date timestamp String email
- 66. What happens when Alice and Bob share an account and both update the email address at the same time
- 67. ? What would we do in a „classic old school architecture“
- 68. UserRestController UserBusinessService UserDAO User ID EMAIL PASSWORD … … … 2341 alice_bob@xyz.com lksjdaslkdjas … … … Update Pessimistic or optimistic locking
- 69. Your business domain drives the locking quality Deeper Insight D D D
- 70. ! Pessimistic locking on a data-level will hardly work in event sourcing architectures
- 71. EventHandler EventsEvents Where to „pessimistically“ lock? Read Storage Events Event Store UserRestController UserBusinessService UserDAO User Commands Consider a business lock with a UserLockedEvent
- 72. ? Do you REALLY need a full lock Most „classic architecture“ applications are already running fine with optimistic locks
- 73. Introduce a version field for the domain entity User Guid id Long version String email String password RegisterUserCommand ChangeEmailCommand UserRegisteredEvent Guid id Date timestamp String email String password EmailChangedEvent Guid userId Date timestamp String email Long version
- 74. Each writing event increases the version UserRegisteredEvent {guid: 12, version: 0, email: alicebob@xyz.com, password: werwe2343} EmailChangedEvent version: 0 EmailChangedEvent version: 1 EmailChangedEvent version: 1 {guid: 12, version: 1, email: alice_bob@xyz.com, password: werwe2343} {guid: 12, version: 2, email: alice@xyz.com, password: werwe2343} EmailChangeFailedEvent
- 75. Also here: you should be as consistent as your domain requires
- 76. Unless otherwise indicated, these slides are © 2013-2015 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ SPRINGONE2GX WASHINGTON, DC Thank You! Michael Ploed - innoQ Twitter: @bitboss Slides: http://slideshare.net/mploed