Event Driven Microservices architecture
- 1. Patterns & Practices for Implementing Event-driven Microservices Nikhil Barthwal @nikhilbarthwal nikhilbarthwal@hotmail.com www.nikhilbarthwal.com
- 2. Agenda (1/2) • Motivations to move from Monolithic to Microservices • Understanding the Distributed Data in Microservices • Using Domain Driven Design for partitioning data • Problems with Distributed Data • Event Driven Microservices 2
- 3. Agenda (2/2) • Communication Patterns for Microservices • Event Sourcing • Command Query Responsibility Segregation • Introducing Sagas • Resources for further reading 3
- 4. Monolithic: The old school approach … • Large code base → Slower development • More time to test & deploy • Too much team interdependency Database Web Service Easy to Develop, Test and Deploy … But doesn’t scale! 4
- 5. Microservices: The Brave new world ... • Smaller code base to understand & change • Autonomous teams • Faster testing & deployment Assembly of Loosely coupled services! Image Source: https://www.slideshare.net/JontheBeach/distributed-sagas-a-protocol-for-coordinating-microservices 5
- 6. Loose Coupling → Distributed Data Database per Service! Faster Innovation! Loosely coupled services Reduces team interdependencies Image Source: http://microservices.io/patterns/microservices.html 6
- 7. How to create service boundaries? An approach to software development for complex needs by connecting the implementation to an evolving model. • Entities • Value Objects • Bounded Context • Aggregates Infrastructure Layer Domain Layer Application Layer User Interface Use Domain Driven Design (DDD)! 7
- 8. DDD: Aggregates • Cluster of domain objects that can be treated as a single unit • One of its component is root • All outside reference would only go to the aggregate root Domain Model = Collection of Aggregates 8
- 9. Data Partition Strategies: Use DDD Aggregates provide boundaries for partition 9
- 10. Problems with Distributed Data • How do we keep data consistent? • How do we query scattered data? 10
- 11. Cannot use ACID Transactions BEGIN TRANSACTION SELECT ADDRESS FROM CUSTOMERS WHERE CUSTOMER_ID = XXX SELECT PRICE FROM PRODUCTS WHERE CUSTOMER_ID = YYY INSERT INTO ORDERS ... COMMIT TRANSACTION Private to Customer service Private to Product service 11
- 12. Cannot use 2-Phase Commit • Coordinator is a single point of failure • Very chatty, with O(4n) messages, O(n2) with retries • Locks would decrease throughputs • Impacts availability (CAP theorem limitations) Guarantees consistency, but … 12
- 13. Consistency Eventual Consistency Availability Partition (Network) Eventual Consistency Use Event Driven Microservice Architecture! 13
- 14. Event Driven Architecture • Event occurs when a change happens in system • All listeners get notified of the event, may take action • Highly distributed/loosely coupled architecture • Often used for asynchronous flows of information 14
- 15. Communication: Direct Messaging Benefits: • Easier implementation • Better Error control Drawbacks: • Increased coupling • Difficult to scale Order Service Product Service Customer Service Fulfillment Service Payment Service Shipping Service 15
- 16. Communication: Via Message broker Benefits: • More scalable • Increased Decoupling Drawbacks: • Requires Broker as Highly Available component Order Service Product Service Customer Service Fulfillment Service Payment Service Shipping Service Message Broker 16
- 17. Communication Patterns Request/Response (synchronous) Event-Driven (asynchronous) Tier-3: Little to No impact for Customer Tier-2: Impacting Customers indirectly Tier-1: Directly Customer facing 17
- 18. Event Sourcing: Introduction • Modeling state changes as sequence of events • Storing the event that could trigger the state change • Enables rolling back to particular time in history Examples: 18
- 19. Event Sourcing: Snapshots Alternative to playing events: Snapshots 19
- 20. Event Sourcing: Multiple views Image Source: https://www.confluent.io/wp-content/uploads/2016/09/Event-sourced-based-architecture.jpeg Adding applications that process event … but create a different view! 20
- 21. Event Sourcing: Benefits & Drawbacks Benefits: • 100% accurate audit logging • Easy temporal queries • Applications can process same events but create views Drawbacks: • Adds Complexity • No Strict Consistency • Longer bootup times (Snapshots can help) 21
- 22. Command Query Responsibility Segregation CQRS Pattern Must for Event Sourcing! Read Write Image Source: https://martinfowler.com/bliki/CQRS.html 22
- 23. CQRS: Benefits & Drawbacks Benefits: • Needed for Event Sourcing • Improved separation of concerns • Supports scalable multiple denormalized views Drawbacks: • Increased complexity • Potential code duplication • Replication lag as No Strict Consistency 23
- 24. Sagas: Introduction • Based on a 1987 paper • Initially for a single database running on one node • Now adapted for distributed systems with asynchrony and partial failure 24
- 25. Transaction & Rollback Transaction • Every Transaction has a Rollback transaction • This logic must be included in the service T1 T2 T3 R1 R2 R3 Transactions → Rollback Transactions → 25
- 26. Sagas: Example This action initiates the saga Check Credit Service Withdrawal Service Deposit Service Deposit Check Long running transactions … use compensating actions to handle failures! Does Sender have enough money in his account? Withdraw the money (unless it is on hold)? Deposit the money (unless account is deactivated)? 26
- 27. Resources • Using DDD to develop Microservices: https://www.infoq.com/news/2016/02/ddd- microservices • Overview of Saga Pattern: http://microservices.io/patterns/data/saga.html • Distributed Sagas: https://www.slideshare.net/JontheBeach/distributed-sagas-a- protocol-for-coordinating-microservices 27
- 28. Summary • Microservices enable faster innovation • DDD can used to define service boundaries • Data is distributed which create challenges • Use Event Driven Architecture for eventual consistency • Understand Event Sourcing & CQRS • Use Sagas to maintain data consistency across services 28