Mistakes - I’ve Made a Few. Blunders in Event-driven Architecture | Simon Aubury, ThoughtWorks Australia
1 like2,162 views
Simon Aubury discusses common mistakes in implementing event-driven architecture (EDA), emphasizing the importance of understanding the distinction between messages and events, and the necessity of adhering to EDA guardrails. He highlights the significance of event granularity, asynchronous communication, and the risks of over-engineering solutions. The presentation advocates for starting the adoption of EDA now to meet evolving system demands and improve architectural practices.
Mistakes - I’ve Made a Few. Blunders in Event-driven Architecture | Simon Aubury, ThoughtWorks Australia
- 1. @SimonAubury Event Driven Architecture Mistakes – I’ve made a few … A tale of confessions from Simon Aubury
- 2. @SimonAubury Why am I here? 2 I am Simon Aubury Principal Data Engineer @ ThoughtWorks
- 3. @SimonAubury Let’s help a younger me 3
- 4. @SimonAubury “ Event-driven architecture (EDA) is a software architecture paradigm promoting the production .. consumption and reaction to events. https://en.wikipedia.org/wiki/Event-driven_architecture
- 5. @SimonAubury Mistake #1 The “right size” for a Microservice .. 5
- 6. @SimonAubury Thinking of events and boundaries 6 BBQ Scout Cashier Stock Room Server
- 7. @SimonAubury Written in Scala Too obsessed with microservices 7 Single repository Team fed by 2 pizzas Written in Java Dev-ops for 6 services Cashier Stock Room Server BBQ Scout
- 8. @SimonAubury O rder 26 Boundary between services 8 Cashier Stock Room Server 2 Sausages with sauce Sausage for order 26 Sausages 2 Sausages with sauce & onion BBQ Scout
- 9. @SimonAubury Understand event boundaries 9 Scout Server Cashier Event Stream Order for #26 Stock Room 2 Sausages 2 Sausages for customer #26 2 Sausages 2 Sausages for customer #26 BBQ
- 10. @SimonAubury Event Granularity ◉ Context boundaries ○ How big is a microservice? ◉ Record events as received ○ Avoid early splitting ◉ Capture broad categories ○ Producers shouldn’t discard 10
- 11. @SimonAubury Lesson: the rules are different within vs across boundaries Favour asynchrony and eventual consistency at context boundaries, embrace the productive coupling of synchronous within the boundary 11
- 12. @SimonAubury Mistake #2 Messages != Events 12
- 13. @SimonAubury Messages are not events 13 “Charge Jane’s credit card $20” “Send an email to bob@example.com” Message “Jane ordered a burger” “Bob viewed the new customer web page” Event
- 14. @SimonAubury Why events? 14 Messages ◉ Persisted until consumed ◉ Targeted to the entity ◉ Couple producers and consumers Events ◉ Re-playable stream history ◉ Consume historic events ◉ Loose coupling consumers & producer Messages in Job Queue Events in an Event Log
- 15. @SimonAubury Lesson: Beware the passive aggressive events An event shouldn’t be used as a passive-aggressive command. It’s a “bad smell” if a source system expects the recipient to carry out an action yet styles the message as an event instead. 15
- 16. @SimonAubury Mistake #3 Event notification != Event sourcing 16
- 17. @SimonAubury Choreography vs. orchestration Which system decides that an action should be taken? ◉ Orchestration – system that will perform once told to do it by another system ◉ Choreography (event driven) - a system takes independent action 17
- 18. @SimonAubury Many Meanings of Event Driven ◉ Event Notification ◉ Event-Carried State Transfer ◉ Event-Sourcing ◉ Command Query Responsibility Segregation (CQRS) 18 https://martinfowler.com/articles/201701-event-driven.html
- 19. @SimonAubury Lesson: don’t over engineer Pick an event driven approach – and be consistent and simple. Don’t over-engineer an eventing solution using Event-Sourcing/CQRS 19
- 20. @SimonAubury Mistake #4 Building a message bus in Kafka 20
- 21. @SimonAubury History vs State 21 OCR Scan Start Document found Customer contact section identified Name extracted with 98% confidence Property land size found on page 4 OCR jam while scanning page 5 {"Cust": "Bob", "Property": "House"} Document Scanner
- 22. @SimonAubury History vs State 22 Projection of events into local state store Event notification: Document scanned OCR Scan Start Document found Customer contact section identified Name extracted with 98% confidence Property land size found on page 4 OCR jam while scanning page 5 {"Cust": "Bob", "Property": "House"} Document Scanner
- 23. @SimonAubury Lesson: Don’t build a message bus in Kafka It’s a slippery slope – remember to produce & be reactive to events … and not messages. 23
- 24. @SimonAubury Mistake #5 Producers are not children 24
- 25. @SimonAubury Producers are not children ◉ Basic rules for events ○ EDA guard-rails ○ Data Governance & Security ○ Good citizen rules ○ Naming scheme for topics ◉ Support & training ○ Opinioned framework approach 25
- 26. @SimonAubury Cross bounded context … 26 Direct API Access Event Streaming Coupled API’s Schema required
- 27. @SimonAubury Plan for schema evolution Support change - data domains need to evolve at their own rate … without breaking consumers. TL;DR - Use schema registry 27
- 28. @SimonAubury There is a process to find the events that matter ◉ Use everyone to identify the events that matter ◉ Understand the systems ◉ Start with broad categories 28
- 29. @SimonAubury Lesson: EDA guardrails Event must have’s ◉ Name – past tense ◉ Correlation ID ◉ Event production time ◉ Originating system ◉ Event creation system (may be different) ◉ A payload of stuff 29
- 30. @SimonAubury Mistake #6 Don’t reinvent the EDA wheel 30
- 31. @SimonAubury Event first thinking ◉ Capture facts & behaviour ◉ Represent the real world ◉ Model use cases of how we think ◉ Repeatability & scaling ◉ Common language 31 https://www.confluent.io/blog/journey-to-event- driven-part-1-why-event-first-thinking-changes- everything/
- 32. @SimonAubury DDD - existing practices ◉ Problem modelling ○ Contexts - delineate boundary of consistency ◉ Separate our business logic from other application concerns ◉ Reduce complexity ○ More effective software delivery ◉ Communicate better / A common language 32
- 33. @SimonAubury Lesson: know your events Modelling - discovery & integration … Use simple language; solicit everyone's input. Develop your system inside out, focus on the domain 33
- 34. @SimonAubury Lesson: it’s hard work without DDD Domain Driven Design gives us the tools to define our bounded contexts, which give us our services. Modelling the domain helps us identify the events that are important to the domain. 34
- 35. @SimonAubury 35 What did I learn? ◉ Messages != Events ◉ Rules are different within vs across boundaries ◉ Don’t over engineer ◉ Don’t build a message bus in Kafka ◉ EDA guardrails ◉ Know your events
- 36. @SimonAubury Lesson: start .. now Event Driven Architecture adoption should start now. Starting the first leads to the next transformational opportunity. Shift towards EDA is driven by increasing demands and heightened expectations and system modernisation Make (and share) your own mistakes … 36
- 37. @SimonAubury Any questions ? Thanks! 37 @SimonAubury linkedin.com/in/simonaubury Presentation template by SlidesCarnival