Opportunities and Pitfalls of Event-Driven Utopia
1 like307 views
The document discusses the opportunities and challenges of event-driven architecture, highlighting its potential benefits in facilitating scalable and resilient systems. It emphasizes the importance of understanding command versus event dynamics, as well as managing event chains to avoid complexities in system interactions. The content is derived from a presentation at QCon New York, focusing on empowering software development through knowledge sharing and innovation.
Opportunities and Pitfalls of Event-Driven Utopia
- 1. Opportunities & Pitfalls of Event-Driven Utopia @berndruecker
- 2. InfoQ.com: News & Community Site • 750,000 unique visitors/month • Published in 4 languages (English, Chinese, Japanese and Brazilian Portuguese) • Post content from our QCon conferences • News 15-20 / week • Articles 3-4 / week • Presentations (videos) 12-15 / week • Interviews 2-3 / week • Books 1 / month Watch the video with slide synchronization on InfoQ.com! https://www.infoq.com/presentations/ event-driven-benefits-pitfalls/
- 3. Presented at QCon New York www.qconnewyork.com Purpose of QCon - to empower software development by facilitating the spread of knowledge and innovation Strategy - practitioner-driven conference designed for YOU: influencers of change and innovation in your teams - speakers and topics driving the evolution and innovation - connecting and catalyzing the influencers and innovators Highlights - attended by more than 12,000 delegates since 2007 - held in 9 cities worldwide
- 6. Service 1 Agenda Service 2 Events on the inside Events on the outside 1 3
- 7. Service 1 Agenda Service 2 Events on the inside Events on the outside 1 3 Events inside out2
- 8. Service 1 Agenda Service 2 Events on the inside1 Events on the outside3 Events inside out2
- 9. Once upon a time… RDMS Application Client BBC architecture (box - arrow – box – arrow - cylinder) Every architecture diagram you'll ever need
- 10. The great thing about this architecture RDMS Application DB gurantees (e.g. ACID)
- 11. The problem not webscale resiliency is expensive RDMS Application RDMS
- 13. Append-only Log … bank account created +2,500 $ transfered -14.99$ paid by credit card … Persistent change RDMS Account # Balance 12345 2,500$ Persistent state Current Balance = 2,485.01 $
- 14. Append-only Log … bank account created +2,500 $ transfered -14.99$ paid by credit card … Persistent change Event Bank Account Created 2019/04/16 11:00 # 12345 Event Money Transfer Received 2,500$ 2019/04/16 11:00 # 12345
- 15. Event Sourcing in a nutshell Customer Business Logic Save event 2. Read events Replay … Internal State 3. Build internal state 5. Add Customer Created Event 1. Create Customer 4. Validation and Invariant Checks e.g. Customer Created, Customer Credit Limit Approved, …Customer Event Store 6. Async publish Customer Created Domain Event
- 16. Working without distributed transactions Customer Business Logic Save event 2. Read events Replay … Internal State 3. Build internal state 5. Add Customer Created Event 1. Create Customer 4. Validation and Invariant Checks e.g. Customer Created, Customer Credit Limit Approved, …Customer Event Store 6. Async publish Customer Created Domain Event This is the only atomic operation required
- 17. Traditional Architecture Customer 1. Create Customer Account RDMS Business Logic Open Account 2. Persist state changes 3. Remote Communication
- 18. Pat Helland “ Distributed Systems Guru Worked at Amazon, Microsoft & Salesforce @berndruecker
- 19. Pat Helland Grown-Ups Don’t Use Distributed Transactions “ Distributed Systems Guru Worked at Amazon, Microsoft & Salesforce
- 20. Open Account Outbox pattern in traditional architectures Customer 1. Create Customer Account RDMS Business Logic Job: „Open Account“ 2. Persist state changes 3. Remote CommunicationExecute: „Open Account“ TX 1 TX 2 Async after first transaction!
- 21. Outbox pattern – Implementation Approaches Scheduler Database Transaction Log Workflow Automation
- 22. Idempotency Customer 1. Create Customer Account RDMS Business Logic Job: „Open Account“ 2. Persist state changes 3. Remote Communication Async after transaction! Execute: „Open Account“ Capture Request TX 1 TX 2
- 23. Idempotency Customer 1. Create Customer Account RDMS Business Logic Job: „Open Account“ 2. Persist state changes 3. Remote Communication Async after transaction! Execute: „Open Account“ Capture Request TX 1 TX 3TX 2
- 24. Working without distributed transactions Customer Business Logic Save event 2. Read events Replay … Internal State 3. Build internal state 5. Add Customer Created Event 1. Create Customer 4. Validation and Invariant Checks e.g. Customer Created, Customer Credit Limit Approved, …Customer Event Store 6. Async publish Customer Created Domain Event This is the only atomic operation required
- 25. Events on the inside. An example from my world
- 26. mail@berndruecker.io @berndruecker Bernd Ruecker Co-founder and Chief Technologist of Camunda
- 27. We offer two different workflow engines. Why? Camunda ZeebePersistent State Persistent change
- 28. Workflow Instance Id Current Activity State 2 RetrievePayment running Workflow Instance Id Current Activity State 2 ShipGoods running Workflow Instance Id Current Activity State 2 OrderDelivered ended 2. 3. UPDATE 2. UPDATE 1. INSERT 3.1. RDMS Workflow Engine
- 30. Event Handling, Replication & Single Writer Follower Follower complete task command task completed event 1 send 2 append command Broker (Leader) Stream Processor 4 process 7 store & replicate event 6 append event 3 store & replicate command 5 respond Single Writer (single thread)
- 31. What we do different Follower Follower complete task command task completed event 1 send 2 append command Leader Stream Processor 4 process 7 store & replicate event 6 append event 3 store & replicate command 5 respond Single Writer (single thread) Store and replay commands Delete records that are fully processed Persist & replicate internal state
- 33. Zeebe is CP Follower Follower complete task command task completed event 1 send 2 append command Leader Stream Processor 4 process 7 store & replicate event 6 append event 3 store & replicate command 5 respond Single Writer (single thread)
- 34. Horizontal scalability by partitioning Partition 1 Partition 2 Partition 3 Partition 4 Every workflow instance is exactly handled by one partition instance id: 2-42 instance id: 3-66 Stream Processor Single Writer (single thread)
- 36. Recap 1 – Events on the inside # Natural mechanism to build scalable services in distributed systems (with Outbox & co included) But # You have to think about reads, queries & eventual consistency # Much less industry experience available @berndruecker
- 37. Service 1 Agenda Service 2 Events on the inside Events on the outside 1 3 Events inside out2
- 38. Event Store and Messaging Customer … 1. Create Customer Customer Event Store
- 39. Merge Messaging and Event Store Customer … 1. Create Customer Customer Event Store
- 40. Merge messaging and event store Customer … 1. Create Customer Shared Event Store
- 41. Enter the world of Kafka…
- 42. Merge messaging and event store Customer … 1. Create Customer Shared Event Store
- 43. Kafka as transport Customer … 1. Create Customer Used as queue (but persistent!)
- 44. Service 1 Agenda Service 2 Events on the inside Events on the outside 1 3 Events inside out2
- 45. Once upon a time Billing Customer Change Address
- 47. Event Notification Address changed Billing Customer Billing Customer Reverse direction of dependency direction of dependency Change Address
- 48. Event Notification Address changed Billing Customer AdressChanged { customerId: 42 } Ask for details
- 49. Event-carried State Transfer Address changed Billing Customer AddressChanged { customerId: 42, address: ... } CustomerChanged { customerId: 42, status: A, address: ..., } AddressChanged { customerId: 42, oldAddress: ... newAddress: ... } CustomerMoved { ...,
- 50. This decision is complex Address changed Billing Customer Billing Customer Reverse direction of dependency direction of dependency Change Address
- 51. Example Change Address Address Submit From bla@company.com Date 2019-04-23 09.05 To confirm your address change please click on this link: http://company.com/confirm?id=82e97d49-166c-4862- 9973-4db348e6225d Incoming Email
- 52. Example Customer Notification Address change confirmed Change Adress Address change requested http://company.com/confirm?id=82e97 d49-166c-4862-9973-4db348e6225d direction of dependency
- 55. It is NOTabout communication protocols Address changed Billing Customer Billing Customer Change Address It can be messaging, REST, whatever, ….
- 56. Manifold ways of transport …
- 57. Manifold ways of transport …
- 58. ? Event Command Query Message Record Event Fact, happened in the past, immutable Intend, Want s.th. to happen, The intention itself is a fact
- 59. ? Event Command Query Message Record Event
- 60. Commands in disguise The Customer Needs To Be Sent A Message To Confirm Address Change Event Send Message Wording of Sender Wording of recipient
- 61. Examples PaymentOrder Subscription Retrieve Payment More general, does not need to know who is retrieving payments Customer Order Address Changed Billing More general, does not need to care about who is interessted in address changes Notification Send Mail Global service Order Notification Order Placed Payment Received Goods Shipped Service that can handle notifications for orders autonomously
- 62. Distributed Monoliths Authorization Service Document Context Page Context Document attached Page created Document moved Page moved …
- 63. Define stable contract/API instead Authorization Service Document Context Page Context Add auth …
- 65. Event Chains Adress Check Credit Check Registration @berndruecker Customer Event Bus Registration requested Credit checked Address checked Customer registered
- 66. Event Chains Adress Check Credit Check Registration @berndruecker Customer Event Bus Registration requested Credit checked Address checked Customer registered How does customer registration work?
- 67. The danger is that it's very easy to make nicely decoupled systems with event notification, without realizing that you're losing sight of that larger-scale flow, and thus set yourself up for trouble in future years. https://martinfowler.com/articles/201701-event-driven.html @berndruecker
- 68. The danger is that it's very easy to make nicely decoupled systems with event notification, without realizing that you're losing sight of that larger-scale flow, and thus set yourself up for trouble in future years. https://martinfowler.com/articles/201701-event-driven.html @berndruecker
- 69. The danger is that it's very easy to make nicely decoupled systems with event notification, without realizing that you're losing sight of that larger-scale flow, and thus set yourself up for trouble in future years. https://martinfowler.com/articles/201701-event-driven.html @berndruecker
- 70. Monitoring Workflows Across Microservices https://www.infoq.com/articles/monitor-workflow-collaborating-microservices @berndruecker
- 71. Typical approachesDistributed Tracing Data Lake / Event Monitoring Process Mining Process Tracking @berndruecker
- 72. What we currently build with customers… Camunda Optimize Elastic Registration requested Credit checked Address checked Customer registered @berndruecker
- 73. All great – until you have to move…
- 74. Changes required for an additional check Adress Check Credit Check Registration Criminal Check @berndruecker Customer Event Bus Registration requested Credit checked Customer registered Address checked
- 75. Changes required for an additional check Adress Check Credit Check Registration Criminal Check @berndruecker Customer Event Bus Registration requested Credit checked Customer registered Address checked Criminal checked
- 77. Alternative flow Adress Check Credit Check Registration Criminal Check @berndruecker Customer Kafka Customer registered Registration requested Address checked Credit checked Criminal checked „Credit checks got more expensive, do that only if all other checks succeed“
- 78. Keep it stable, just move sticks with yellow color to the top. How hard can it be?
- 79. What we wanted Photo by Lijian Zhang, available under Creative Commons SA 2.0 License and Pedobear19 / CC BY-SA 4.0 @berndruecker
- 80. Orchestration Adress CheckCredit Check Registration Customer Kafka Registration requested Credit checked Address checked Customer registeredCheck credit Check address Customer checked Customer On-boarding Of course these two services could be merged
- 82. Comparison 2 changes criminal check can be deployed first 2 changes, criminal check can be deployed first See also https://www.infoworld.com/article/3391592/ how-to-tame-event-driven-microservices.html
- 83. In my world… Customer On-boarding Leverage Workflow Engine & BPMN within Service
- 86. Recap 2 # Commands vs. Events: Decide about the direction of dependencies # Beware of event-chains and avoid losing sight # Balance choreography and orchestration @berndruecker
- 87. Service 2Service 1 Recap Events on the inside Events on the outside 1 3 Persistent state vs persistent change Event sourcing & Event Store Consistency & CAP Read Models & CQRS Events as API Event vs Command Event chains & visibility Orchestration vs Choreography Shared Event Store Events inside out2
- 88. Want to see code?
- 89. Nothing for the faint of heart… Events on the inside Events on the outside
- 90. Nothing for the faint of heart… …but doable… …and worth it
- 93. Watch the video with slide synchronization on InfoQ.com! https://www.infoq.com/presentations/ event-driven-benefits-pitfalls/