Bernd Rücker - Lost in transaction? Strategies to manage consistency in serverless architectures
- 1. @berndruecker Lost in transaction? Strategies to manage consistency in serverless architectures
- 2. An example for today – the classical one λ Book Car λ Book Hotel λ Book Flight
- 4. Orchestration / coordinator function λ (Trip) λ (Car) $.post(carHireUrl, ... λ (Hotel) $.post(carHireUrl, ... // book har invokeCar() // book hotel invokeHotel()
- 6. Orchestration / coordinator function λ (Trip) λ (Car) $.post(carHireUrl, ... λ (Hotel) $.post(carHireUrl, ... // book har invokeCar() // book hotel invokeHotel()
- 7. There is two-phase commit (XA)!! TX Coordinator Resource Managers Prepare Phase Commit Phase @berndruecker
- 8. Pat Helland “ Distributed Systems Guru Worked at Amazon, Microsoft & Salesforce @berndruecker
- 9. Pat Helland Grown-Ups Don’t Use Distributed Transactions “ Distributed Systems Guru Worked at Amazon, Microsoft & Salesforce @berndruecker
- 11. That means Temporarily inconsistent Eventually consistent againt Consistent Local ACID Local ACID Violates „I“ of ACID @berndruecker λ Book Car λ Book Hotel
- 12. „Building on Quicksand“ Paper A C I D 2.0Pat Helland @berndruecker
- 13. Associative Commutative Idempotent Distributed 2.0 (a + b) + c = a + (b + c) a + b = b + a f(x) = f( f(x) ) „Building on Quicksand“ Paper Pat Helland @berndruecker
- 16. Orchestration / coordinator function λ (Trip) λ (Car) $.post(carHireUrl, ... λ (Hotel) $.post(carHireUrl, ... // book har invokeCar() // book hotel invokeHotel()
- 17. Orchestration / coordinator function λ (Trip) λ (Car) $.post(carHireUrl, ... λ (Hotel) $.post(carHireUrl, ... // book har invokeCar() // book hotel invokeHotel()
- 18. Orchestration / coordinator function λ (Trip) λ (Car) $.post(carHireUrl, ... λ (Hotel) $.post(carHireUrl, ... // book har invokeCar() // book hotel invokeHotel() Not idempotent Idempotent bookHotel(hotel, duration) bookHotel(tripId, hotel, duration)
- 19. Photo by pixabay, available under Creative Commons CC0 1.0 license. @berndruecker
- 20. Requirement: Idempotency of services! Photo by pixabay, available under Creative Commons CC0 1.0 license. @berndruecker
- 21. Requirement: Idempotency of services! Photo by Chr.Späth, available under Public Domain. @berndruecker
- 23. It is impossible to differentiate certain failure scenarios: Independant of communication style! @berndruecker
- 24. Orchestration / coordinator function λ (Trip) λ (Car) $.post(carHireUrl, ... λ (Hotel) $.post(carHireUrl, ... // book har invokeCar() // book hotel invokeHotel()
- 25. // book hotel try { bookHotel() } catch (err) { cancelHotel() } Cleanup! λ (Trip) λ (Car) $.post(carHireUrl, ... λ (Hotel) $.post(carHireUrl, ... PUT& DELETE // book har invokeCar()
- 26. Orchestration with workflow λ (Trip) // book har startTripWorkflow() λ (Car) $.post(carHireUrl, ... λ (Hotel) $.post(carHireUrl, ... Workflow Engine Car -> Hotel -> ...
- 28. Berlin, Germany bernd.ruecker@camunda.com @berndruecker Bernd Ruecker Co-founder and Chief Technologist of Camunda
- 31. Leverage stateful retrying λ (Car) λ (Hotel) Workflow Engine λ (Flight)λ (Trip)
- 32. But how to solve ACID?
- 33. @berndruecker
- 34. @berndruecker
- 35. Compensation – the classical example Saga book hotel book car book flight cancel hotel cancel car 1. 2. 3. 5.6. In case of failure trigger compensations book trip
- 36. 2 alterntive approaches: choreography & orchestration @berndruecker
- 38. Event-driven choreography Hotel Flight Car Trip Trip failed Trip requested Hotel booked Car booked Request trip Flight failed Car canceled Hotel canceled Perform undo (cancel car booking) Perform undo (cancel hotel) @berndruecker
- 39. The danger is that it's very easy to make nicely decoupled systems with event notification, without realizing that you're losing sight of that larger-scale flow, and thus set yourself up for trouble in future years. https://martinfowler.com/articles/201701-event-driven.html @berndruecker
- 40. The danger is that it's very easy to make nicely decoupled systems with event notification, without realizing that you're losing sight of that larger-scale flow, and thus set yourself up for trouble in future years. https://martinfowler.com/articles/201701-event-driven.html @berndruecker
- 41. The danger is that it's very easy to make nicely decoupled systems with event notification, without realizing that you're losing sight of that larger-scale flow, and thus set yourself up for trouble in future years. https://martinfowler.com/articles/201701-event-driven.html @berndruecker
- 42. Implementing changes in the process Hotel Flight Car Trip Trip failed Trip requested Hotel booked Car booked Request trip Flight failed Car canceled Hotel canceled We have a new basic agreement with the car rental agency and can cancel for free within 1 hour – do that first! @berndruecker
- 43. Implementing changes in the process Hotel Flight Car Trip Trip failed Trip requested Hotel booked Car booked Request trip Flight failed Car canceled Hotel canceled You have to adjust all services and redeploy at the same time! We have a new basic agreement with the car rental agency and can cancel for free within 1 hour – do that first! @berndruecker
- 44. What we wanted Photo by Lijian Zhang, available under Creative Commons SA 2.0 License and CC BY-SA 4.0 @berndruecker
- 46. Orchestration Hotel Flight Car Trip Trip booked Request trip Book hotel Hotel booked Car booked Flight booked Book car Book flight We have a new basic agreement with the car rental agency and can cancel for free within 1 hour – do that first! You have to adjust one service and redeploy only this one! @berndruecker
- 47. Compensation with workflow Workflow Engine λ (Trip) Saga Pattern (implemented by BPMN compensation)
- 48. Live hacking https://github.com/berndruecker/trip-booking-saga-serverless Workflow engine based on BPMN Cloud-native Source-available Available as managed service
- 49. Demo architecture with Zeebe & Camunda Cloud Camunda Cloud Your AWS account Zeebe Client BPMN Workflow Definition Trip Booking Function Hotel Booking Function … Zeebe API Gateway start Invoke function via HTTP Deployed onto Zeebe
- 51. Clemens Vasters Architect at Microsoft http://vasters.com/archive/Sagas.html @berndruecker
- 52. Clemens Vasters Architect at Microsoft http://vasters.com/archive/Sagas.html @berndruecker
- 53. Clemens Vasters Architect at Microsoft http://vasters.com/archive/Sagas.html @berndruecker
- 54. Living documentation for long-running behaviour @berndruecker
- 57. Think of more complicated scenarios…
- 58. Recap • Function coordination might include orchestration and workflow • Carefully decide about choreography vs. orchestration • Grown ups don‘t use distributed transactions but eventual consistency • Idempotency is super important! • Know Saga / Compensation as strategy