Gluecon: Using sagas to maintain data consistency in a microservice architecture
4 likes2,612 views
The document discusses using sagas to maintain data consistency in a microservices architecture. It explains that traditional ACID transactions are not an option for microservices due to loose coupling of data and services. Sagas provide an alternative by executing a series of local transactions with compensating transactions to rollback any completed steps if errors occur. The document covers how sagas are coordinated either through a centralized orchestrator or by asynchronous messaging to sequence transaction steps reliably across services.
![@crichardson
Saga orchestrators are state
machines
RESERVING
CREDIT
APPROVED
REJECTED
/ customerService.
reserveCredit()
credit reserved
/ order.approve()
credit limit exceed
/ order.reject()
event [guard] / action
Initial action
reply and action](https://image.slidesharecdn.com/usingsagastomaintaindataconsistencyinamicroservicearchitecturegluecon2017-170530211600/85/Gluecon-Using-sagas-to-maintain-data-consistency-in-a-microservice-architecture-34-320.jpg)
Gluecon: Using sagas to maintain data consistency in a microservice architecture
- 1. @crichardson Using sagas to maintain data consistency in a microservice architecture Chris Richardson Founder of Eventuate.io Founder of the original CloudFoundry.com Author of POJOs in Action @crichardson chris@chrisrichardson.net http://eventuate.io
- 2. @crichardson Presentation goal Distributed data management challenges in a microservice architecture Why 2PC is not an option Sagas as the transaction model
- 4. @crichardson About Chris Consultant and trainer focusing on modern application architectures including microservices (http://www.chrisrichardson.net/)
- 5. @crichardson About Chris Founder of a startup that is creating an open-source/SaaS platform that simpliïŹes the development of transactional microservices (http://eventuate.io)
- 7. @crichardson Agenda ACID is not an option Overview of sagas Coordinating sagas
- 8. The microservice architecture structures an application as a set of loosely coupled services
- 9. @crichardson Microservices = modularity Tackles complexity: Division of knowledge Division of labor * might improve scalability too
- 10. @crichardson Microservices accelerate software development through parallelization Customer Service Order Service ⊠Customer Team Order Team ⊠Team
- 12. @crichardson Loose coupling = encapsulated data Order Service Customer Service Order Database Customer Database Order table Customer table orderTotal creditLimit
- 13. @crichardson How to maintain data consistency?!?!? Invariant: sum(open order.total) <= customer.creditLimit
- 14. @crichardson Cannot use ACID transactions BEGIN TRANSACTION ⊠SELECT ORDER_TOTAL FROM ORDERS WHERE CUSTOMER_ID = ? ⊠SELECT CREDIT_LIMIT FROM CUSTOMERS WHERE CUSTOMER_ID = ? ⊠INSERT INTO ORDERS ⊠⊠COMMIT TRANSACTION Private to the Order Service Private to the Customer Service Distributed transactions
- 15. @crichardson 2PC is not an option Guarantees consistency BUT 2PC coordinator is a single point of failure Chatty: at least O(4n) messages, with retries O(n^2) Reduced throughput due to locks Not supported by many NoSQL databases (or message brokers) CAP theorem 2PC impacts availability âŠ.
- 16. @crichardson Agenda ACID is not an option Overview of sagas Coordinating sagas
- 17. @crichardson From a 1987 paper
- 18. @crichardson Saga Use Sagas instead of 2PC Distributed transaction Service A Service B Service A Local transaction Service B Local transaction Service C Local transaction X Service C
- 19. @crichardson Order Service Create Order Saga Local transaction Order state=PENDING createOrder() Customer Service Local transaction Customer reserveCredit() Order Service Local transaction Order state=APPROVED approve order() createOrder()
- 20. @crichardson If only it were this easyâŠ
- 21. @crichardson Rollback using compensating transactions ACID transactions can simply rollback BUT Developer must write application logic to ârollbackâ eventually consistent transactions Careful design required!
- 22. @crichardson Saga: Every Ti has a Ci T1 T2 ⊠C1 C2 Compensating transactions T1 T2 C1 FAILS
- 23. @crichardson Order Service Create Order Saga - rollback Local transaction Order createOrder() Customer Service Local transaction Customer reserveCredit() Order Service Local transaction Order reject order() createOrder() FAIL InsufïŹcient credit
- 24. @crichardson Sagas complicate API design Request initiates the saga. When to send back the response? Option #1: Send response when saga completes: + Response speciïŹes the outcome - Reduced availability Option #2: Send response immediately after creating the saga (recommended): + Improved availability - Response does not specify the outcome. Client must poll or be notiïŹed
- 25. @crichardson Revised Create Order API createOrder() returns id of newly created order NOT fully validated getOrder(id) Called periodically by client to get outcome of validation
- 26. @crichardson Minimal impact on UI UI hides asynchronous API from the user Saga will usually appear instantaneous (<= 100ms) If it takes longer UI displays âprocessingâ popup Server can push notiïŹcation to UI
- 27. @crichardson Sagas complicate the business logic Changes are committed by each step of the saga Other transactions see âinconsistentâ data, e.g. Order.state = PENDING more complex logic Interaction between sagas and other operations e.g. what does it mean to cancel a PENDING Order? âInterruptâ the Create Order saga Wait for the Create Order saga to complete?
- 28. @crichardson Agenda ACID is not an option Overview of sagas Coordinating sagas
- 29. @crichardson How to sequence the saga transactions? After the completion of transaction Ti âsomethingâ must decide what step to execute next Success: which T(i+1) - branching Failure: C(i - 1)
- 30. @crichardson Order Service Orchestration-based saga coordination Local transaction Order state=PENDING createOrder() Customer Service Local transaction Customer reserveCredit() Order Service Local transaction Order state=APPROVED approve order() createOrder() CreateOrderSaga
- 31. @crichardson Complex coordination logic is centralized đ
- 32. @crichardson Services expose APIs that are invoked by saga đ
- 33. @crichardson Order Service CreateOrderSaga orchestrator Customer Service Create Order Customer creditLimit creditReservations ... Order state total⊠reserveCredit() CreateOrderSaga OrderService create() create() approve() creditReserved()
- 34. @crichardson Saga orchestrators are state machines RESERVING CREDIT APPROVED REJECTED / customerService. reserveCredit() credit reserved / order.approve() credit limit exceed / order.reject() event [guard] / action Initial action reply and action
- 35. @crichardson Create Order Saga code Enum Persistent data Stateless singleton: Behavior Eventuate Saga framework State
- 36. @crichardson Create Order Saga State machine deïŹnition Specify reply handlers
- 37. @crichardson Initializing the saga Invoke saga participant Initialize state Create order
- 38. @crichardson Handling a reply Change state Update Order
- 39. @crichardson Customer Service - command handling Reserve credit
- 40. @crichardson Saga Participant About Saga orchestrator participant communication Saga Orchestrator Saga Participant command reply Saga must complete even if there are transient failures
- 42. @crichardson Create Order Saga messaging Order Service Message Broker Customer Service Begin TXN Read data Send reply Commit TXN Credit Reserved Credit Limit Exceeded Create Order Saga Orchestrator reserve credit Customer Service Request Channel Customer Service Reply Channel Order create() approve() reject() Commands Replies
- 43. @crichardson Summary Microservices tackle complexity and accelerate development Database per service is essential for loose coupling Use sagas to maintain data consistency across services Use transactional messaging to make sagas reliable