microXchg: Managing data consistency in a microservice architecture using Sagas
8 likes11,553 views
The presentation discusses managing data consistency in microservice architectures through the use of sagas as an alternative to traditional ACID transactions, highlighting the challenges of distributed data management. It outlines two coordination strategies for sagas: orchestration and choreography, each with its own benefits and drawbacks. Additionally, it presents various countermeasures to address data anomalies and ensure reliable operations in microservices environments.
microXchg: Managing data consistency in a microservice architecture using Sagas
- 1. @crichardson Managing data consistency in a microservice architecture using Sagas Chris Richardson Founder of eventuate.io Author of Microservices Patterns Founder of the original CloudFoundry.com Author of POJOs in Action @crichardson chris@chrisrichardson.net http://eventuate.io http://learn.microservices.io
- 2. @crichardson Presentation goal Distributed data management challenges in a microservice architecture Sagas as the transaction model
- 4. @crichardson About Chris Consultant and trainer focusing on modern application architectures including microservices (http://www.chrisrichardson.net/)
- 5. @crichardson About Chris Founder of a startup that is creating an open-source/SaaS platform that simplifies the development of transactional microservices (http://eventuate.io)
- 6. @crichardson For more information http://learn.microservices.io 40% off ctwmicroxchg18
- 7. @crichardson Agenda ACID is not an option Overview of sagas Coordinating sagas Countermeasures for data anomalies
- 8. The microservice architecture structures an application as a set of loosely coupled services
- 9. @crichardson Microservices enable continuous delivery/deployment Process: Continuous delivery/deployment Organization: Small, agile, autonomous, cross functional teams Architecture: Microservice architecture Enables Enables Enables Successful Software Development Services = testability and deployability Teams own services
- 12. @crichardson Loose coupling = encapsulated data Order Service Customer Service Order Database Customer Database Order table Customer table orderTotal creditLimit
- 13. @crichardson How to maintain data consistency?!?!? Invariant: sum(open order.total) <= customer.creditLimit
- 14. @crichardson Cannot use ACID transactions that span services BEGIN TRANSACTION … SELECT ORDER_TOTAL FROM ORDERS WHERE CUSTOMER_ID = ? … SELECT CREDIT_LIMIT FROM CUSTOMERS WHERE CUSTOMER_ID = ? … INSERT INTO ORDERS … … COMMIT TRANSACTION Private to the Order Service Private to the Customer Service Distributed transactions
- 15. @crichardson 2PC is not an option Guarantees consistency BUT 2PC coordinator is a single point of failure Chatty: at least O(4n) messages, with retries O(n^2) Reduced throughput due to locks Not supported by many NoSQL databases (or message brokers) CAP theorem 2PC impacts availability ….
- 17. @crichardson Agenda ACID is not an option Overview of sagas Coordinating sagas Countermeasures for data anomalies
- 18. @crichardson From a 1987 paper
- 19. @crichardson Saga Use Sagas instead of 2PC Distributed transaction Service A Service B Service A Local transaction Service B Local transaction Service C Local transaction X Service C
- 20. @crichardson Saga step = a transaction local to a service Service Database Message Broker update publish message/event Transactional DB: BEGIN … COMMIT DDD: Aggregate NoSQL: single ‘record’ How to make atomic without 2PC?
- 21. @crichardson Order Service Create Order Saga Local transaction Order state=PENDING createOrder() Customer Service Local transaction Customer reserveCredit() Order Service Local transaction Order state=APPROVED approve order() createOrder() Initiates saga
- 22. @crichardson But what about rollback? BEGIN TRANSACTION … UPDATE … … INSERT …. … …. BUSINESS RULE VIOLATED!!!! … ROLLBACK TRANSACTION Really simple!
- 23. @crichardson Rolling back sagas Use compensating transactions Developer must write application logic to “rollback” eventually consistent transactions Careful design required!
- 24. @crichardson Saga: Every Ti has a Ci T1 T2 … C1 C2 Compensating transactions T1 T2 C1 FAILS
- 25. @crichardson Order Service Create Order Saga - rollback Local transaction Order createOrder() Customer Service Local transaction Customer reserveCredit() Order Service Local transaction Order reject order() createOrder() FAIL Insufficient credit
- 26. @crichardson Agenda ACID is not an option Overview of sagas Coordinating sagas Countermeasures for data anomalies
- 27. @crichardson How to sequence the saga transactions? After the completion of transaction Ti “something” must decide what step to execute next Success: which T(i+1) - branching Failure: C(i - 1)
- 28. @crichardson Choreography: distributed decision making vs. Orchestration: centralized decision making
- 29. @crichardson Option #1: Choreography-based coordination using events Order Service Customer Service Order created Credit Reserved Credit Limit Exceeded Create Order OR Customer creditLimit creditReservations Order state total create() reserveCredit() approve()/ reject() Order events channel Customer events channel
- 30. @crichardson Order Service: publishing domain events Publish event
- 31. @crichardson Customer Service: consuming domain events https://github.com/eventuate-tram/eventuate-tram-examples-customers-and-orders Subscribe to event
- 32. Benefits and drawbacks of choreography Benefits Simple, especially when using event sourcing Participants are loosely coupled Drawbacks Cyclic dependencies - services listen to each other’s events Overloads domain objects, e.g. Order and Customer know too much Events = indirect way to make something happen https://github.com/eventuate-examples/eventuate-examples-java-customers-and-orders
- 33. @crichardson Order Service Option #2: Orchestration-based saga coordination Local transaction Order state=PENDING createOrder() Customer Service Local transaction Customer reserveCredit() Order Service Local transaction Order state=APPROVED approve order() createOrder() CreateOrderSaga InvokesInvokesInvokes
- 34. @crichardson A saga (orchestrator) is a persistent object that tracks the state of the saga and invokes the participants
- 35. Saga behavior On create: Invokes a saga participant Wait for a reply On reply: Determine which saga participant to invoke next Invokes saga participant Updates its state Wait for a reply …
- 36. @crichardson Order Service CreateOrderSaga orchestrator Customer Service Create Order Customer creditLimit creditReservations ... Order state total… reserveCredit() CreateOrder Saga OrderService create() create() approve() creditReserved()
- 37. @crichardson CreateOrderSaga definition Sequence of steps step = (Ti, Ci) Build command to send Saga’s Data
- 38. @crichardson Customer Service command handler Route command to handler Reserve credit Make reply message
- 39. @crichardson Eventuate Tram Sagas Open-source Saga orchestration framework Currently for Java https://github.com/eventuate-tram/eventuate-tram-sagas https://github.com/eventuate-tram/eventuate-tram-sagas- examples-customers-and-orders
- 40. @crichardson Use asynchronous messaging Ensures sagas complete when participants are temporarily unavailable
- 41. @crichardson Create Order Saga - messaging Order Service Create Order Saga Message Broker Customer Service Customer Reserve Credit Reserve Credit Reply Customer Request Channel Saga Reply Channel Order Request Channel Approve Order
- 42. Benefits and drawbacks of orchestration Benefits Centralized coordination logic is easier to understand Reduced coupling, e.g. Customer knows less Reduces cyclic dependencies Drawbacks Risk of smart sagas directing dumb services
- 43. @crichardson Agenda ACID is not an option Overview of sagas Coordinating sagas Countermeasures for data anomalies
- 44. @crichardson Lack of isolation complicates business logic Order Service Local transaction Order state=PENDING createOrder() Customer Service Local transaction Customer reserveCredit() Order Service Local transaction cancelOrder() ? Time
- 45. @crichardson How to cancel a PENDING Order? Don’t throw an OrderNotCancellableException Questionable user experience “Interrupt” the Create Order saga? Cancel Order Saga: set order.state = CANCELLED Causes Create Order Saga to rollback But is that enough to cancel the order? Cancel Order saga waits for the Create Order saga to complete? Suspiciously like a distributed lock But perhaps that is ok
- 46. @crichardson Countermeasure Transaction Model http://bit.ly/semantic-acid-ctm - paywall 😥
- 47. @crichardson Sagas are ACD Atomicity Saga implementation ensures that all transactions are executed OR all are compensated Consistency Referential integrity within a service handled by local databases Referential integrity across services handled by application Durability Durability handled by local databases
- 48. @crichardson Lack of I anomalies
- 49. @crichardson Anomaly: Lost update Ti: Create Order Tj: Approve Order Ti: Cancel Order Saga 1 Saga 2 Time Overwrites cancelled order
- 50. @crichardson Anomaly: Dirty reads Ti: Reserve Credit Ci: Unreserve credit Ti: Reserve Credit Reads uncommitted changes Saga 1 Saga 2 Order rejected unnecessarily OR Credit limit exceeded Time
- 51. @crichardson Anomaly: non-repeatable/ fuzzy read Ti: Begin Revise Order Tj: Finalize Revise Order Ti: Update Order Time Order has changed since Ti Saga 1 Saga 2
- 52. @crichardson Countermeasures for reducing impact of isolation anomalies… * i.e. good enough, eventually consistent application
- 53. Saga structure T1 C1 … T2 C2 Tn+1 Tn+2 …. Compensatable transactions Pivot transaction = GO/NO GO Retriable transactions that can’t fail
- 54. Countermeasure: Semantic lock Compensatable transaction sets flag, retriable transaction releases it Indicates a possible dirty read Flag = lock: prevents other transactions from accessing it Require deadlock detection, e.g. timeout Flag = warning - treat the data differently, e.g. Order.state = PENDING a pending deposit … … … Approve Order Create Pending Order Reject Order Order.state = PENDING Order.state = REJECTED Order.state = APPROVED
- 55. @crichardson Countermeasure: Commutative updates Commutative: g(f(x)) = f(g(x)) For example: Account.debit() compensates for Account.credit() Account.credit() compensates for Account.debit() Avoids lost updates
- 56. @crichardson Countermeasure: Pessimistic view Increase avail. credit Reduce avail. credit Cancel Order Delivery Cancel Order Reorder saga to reduce risk Cancel Order Delivery Cancel Order Increase avail. credit … …… … Won’t be compensated, so no dirty read
- 57. @crichardson Countermeasure: Re-read value Ti: Read Order Tj: Re-read, then update Order Ti: Update Order Time Saga 1 Saga 2 Verify unchanged, possibly restart Prevents lost update ~Offline optimistic lock pattern
- 58. @crichardson Countermeasure: version file Ti: Create Order Tj: Authorize Credit Card Ti: Cancel Order Time Saga 1 Saga 2 Tj: Reserve Credit Tj: Reverse Credit Card Payment 1. reverse() 2. authorize() “log” of changes: Enables operations to commute
- 59. @crichardson Countermeasure: By value Business risk determines strategy Low risk => semantic ACID High risk => use 2PC/distributed transaction
- 60. @crichardson Summary Microservices tackle complexity and accelerate development Database per service is essential for loose coupling Use orchestration-based or choreography-based sagas to maintain data consistency across services Use countermeasures to reduce impact of anomalies caused by lack of isolation