Event Sourcing: Introduction & Challenges
5 likes3,936 views
The document discusses event sourcing as an architectural pattern, emphasizing its ability to track state changes through a sequence of events rather than just storing the current state of applications. It highlights the advantages and challenges of implementing event sourcing and CQRS, including issues related to consistency, validation, and parallel updates. The author argues for the importance of aligning implementation with business domain needs and notes the necessity of careful consideration in managing application state and event storage.
Event Sourcing: Introduction & Challenges
- 1. Event SourcingIntro & Challenges
- 3. Most current systems only store the current state
- 4. IncidentRestController IncidentBusinessService IncidentDAO Incident ID USER_ID DATE TEXT 1 23423 11.03.2014 Maus defekt 2 67454 12.03.2014 EMail Empfang 3 93729 12.03.2014 Monitor defekt … … … … Classical Architecture
- 5. IncidentRestController IncidentBusinessService IncidentDAO Incident ID USER_ID DATE TEXT 1 23423 11.03.2014 Maus ist kaputt 2 67454 12.03.2014 EMail Empfang 3 93729 12.03.2014 Monitor defekt … … … … Update The dataset is directly being changed, no history
- 6. 1 Audit Log only with extra effort 2 No replay 3 Snapshots only with extra effort or via backups
- 7. ? Issues
- 8. Many applications are working well with the classic approach
- 9. However there are areas where this approach reaches it’s limitations
- 10. ?Reactive
- 11. ? Responsive ? Resilient ? Elastic ? Message driven
- 12. IncidentRestController IncidentBusinessService IncidentDAO Incident Not quite reactive… Database
- 13. Event Sourcing is an architectural pattern in which the state of the application is being determined by a sequence of events
- 14. Building Blocks Applications Event Queue Applications issues events to a queue Event Handler The Event handler is processing the events Event Store The event store is persisting the events
- 15. An event is something that happened in the past t jetzt EventEventEventEventEvent
- 16. The names of the Events are part of the Ubiquitous Language D D D
- 18. Code Example public class CustomerVerifiedEvent { private String eventId; private Date eventDate; private CustomerNumber customerNumber; private String comment; public CustomerVerifiedEvent(CustomerNumber custNum, String comment) { this.customerNumber = cusNum; this.comment = comment; this.eventDate = new Date(); } }
- 19. Scope your events on the basis of Aggregates D D D
- 20. An Event is always immutable !
- 21. The sequence of events in the queue is also called Event Stream tnow EventEventEventEventEvent
- 22. IncidentCreatedEvent incidentNumber: 1 userNumber: 23423 timestamp: 11.03.2014 12:23:23 text: „Maus defekt“ status: „offen“ Event Stream Example IncidentTextChangedEvent incidentNumber: 1 text: „Maus ist Kaputt“ IncidentClosedEvent incidentNumber: 1 solution: „Neue Maus“ status: „geschlossen“
- 23. Let’s reuse the ESB from the failed SOA project
- 24. NO
- 25. NO
- 26. NO
- 27. ! Prefer dumb pipes with smart endpoints as a suitable message broker architecture
- 28. 1 Complete rebuild is possible 2 Temporal Queries 3 Event Replay
- 29. Well known examples = Version Control Systems or Database Transaction Logs
- 30. The Event Store has a very high business value
- 31. There is no delete! !
- 32. A delete is just another event IncidentCreatedEvent incidentNumber: 1 userNumber: 23423 timestamp: 11.03.2014 12:23:23 text: „Maus defekt“ status: „offen“ IncidentChangedEvent incidentNumber: 1 text: „Maus ist Kaputt“ IncidentClosedEvent incidentNumber: 1 solution: „Neue Maus“ status: „geschlossen“ IncidentRemovedEvent incidentNumber: 1
- 33. Aren’t there performance issues attached to this kind of data storage?
- 34. YES!
- 35. Application State Think about applciation state Application Event Queue Event Handler Event Store Application State The application queries the pre-processed application state
- 36. CQRS
- 39. EventHandler EventsEvents Event Sourcing & CQRS IncidentCommandEndpoint IncidentCommandService IncidentCommandDAO IncidentQueryEndpoint IncidentQueryService IncidentQueryDAO Read Storage Events Select * from Event Store
- 40. Event Sourcing is an interesting architectural option. However there are various challanges, that have to be taken care of
- 41. 1 Consistency 2 Validation 3 Parallel Updates
- 42. YES
- 43. ! Systems based on CQRS and Event Sourcing are mostly eventually consistent
- 44. EventHandler EventsEvents Eventual Consistency IncidentCommandEndpoint IncidentCommandService IncidentCommandDAO IncidentQueryEndpoint IncidentQueryService IncidentQueryDAO Read Storage Events Select * from Event Store
- 45. BUT
- 46. ! You can build a fully consistent system which follows Event Sourcing principles
- 47. EventHandler EventsEvents Full Consistency IncidentCommandEndpoint IncidentCommandService IncidentCommandDAO IncidentQueryEndpoint IncidentQueryService IncidentQueryDAO Read Storage Events Select * from Event Store
- 48. Your business domain drives the level of consistency not technology Deeper Insight D D D
- 49. Increased (but still eventual) consistency EventHandler EventsEvents IncidentCommandEndpoint IncidentCommandService IncidentCommandDAO IncidentQueryEndpoint IncidentQueryService IncidentQueryDAO Read Storage Events Select * from Event Store async
- 50. ! There is no standard solution
- 51. 1 Consistency 2 Validation 3 Parallel Updates
- 52. Example Domain User Guid id String email String password RegisterUserCommand ChangeEmailCommand UserRegisteredEvent Guid id Date timestamp String email String password EmailChangedEvent Guid userId Date timestamp String email
- 53. We process 2 million+ registrations per day. A user can change her email address. However the emails address must be unique
- 54. ? How high is the probability that a validation fails Which data is required for the validation Where is the required data stored
- 55. $ What is the business impact of a failed validation that is not recognized due to eventual consistency and how high is the probability of failure
- 56. Your business domain drives the level of consistency Deeper Insight D D D
- 57. 1 Validate from Event Store 2 Validate from Read Store 3 Perform Validation in Event Handler
- 58. EventHandler EventsEvents Validation IncidentCommandEndpoint IncidentCommandService IncidentCommandDAO IncidentQueryEndpoint IncidentQueryService IncidentQueryDAO Read Storage Events Select * from Event Store
- 59. 1 Consistency 2 Validation 3 Parallel Updates
- 60. Example Domain User Guid id String email String password RegisterUserCommand ChangeEmailCommand UserRegisteredEvent Guid id Date timestamp String email String password EmailChangedEvent Guid userId Date timestamp String email
- 61. What happens when Alice and Bob share an account and both update the email address at the same time
- 62. ? What would we do in a „classic old school architecture“
- 63. UserRestController UserBusinessService UserDAO User ID EMAIL PASSWORD … … … 2341 alice_bob@xyz.com lksjdaslkdjas … … … Update Pessimistic or optimistic locking
- 64. Your business domain drives the locking quality Deeper Insight D D D
- 65. ! Pessimistic locking on a data-level will hardly work in event sourcing architectures
- 66. EventHandler EventsEvents Where to „pessimistically“ lock? Read Storage Events Event Store UserRestController UserBusinessService UserDAO User Commands Consider a business lock with a UserLockedEvent
- 67. ? Do you REALLY need a full lock Most „classic architecture“ applications are already running fine with optimistic locks
- 68. Introduce a version field for the domain entity User Guid id Long version String email String password RegisterUserCommand ChangeEmailCommand UserRegisteredEvent Guid id Date timestamp String email String password EmailChangedEvent Guid userId Date timestamp String email Long version
- 69. Each writing event increases the version UserRegisteredEvent {guid: 12, version: 0, email: alicebob@xyz.com, password: werwe2343} EmailChangedEvent version: 0 EmailChangedEvent version: 1 EmailChangedEvent version: 1 {guid: 12, version: 1, email: alice_bob@xyz.com, password: werwe2343} {guid: 12, version: 2, email: alice@xyz.com, password: werwe2343} EmailChangeFailedEvent
- 70. Also here: you should be as consistent as your domain requires