Felix Müller: Microservices Integration: Challenges and Solutions - Camunda Days Oslo, CPH, Amsterdam
1 like199 views
The document discusses challenges with microservices integration and distributed systems. It presents strategies like circuit breakers and workflow engines to handle failures and asynchronous communication in a resilient manner. Specifically, it recommends using a workflow engine like Camunda to implement stateful retry, timeouts, compensation to keep failures localized. It also advocates for using external tasks with Camunda to enable polyglot programming, scaling, decoupling and natural transactions in a microservices architecture.
Felix Müller: Microservices Integration: Challenges and Solutions - Camunda Days Oslo, CPH, Amsterdam
- 1. Microservices Integration Challenges and Solutions Felix Müller Technical Consultant
- 2. Who I am? Felix Müller @felixlmueller
- 6. Some Service Some Service Some Service Some Service Some Service Some Service Some Service Failure will happen. Accept it! But keep it local! Be resilient.
- 7. Photo by Tookapic, available under Creative Commons CC0 1.0 license.
- 8. Translation : „There was an error while sending your boarding pass“
- 12. Circuit Breaker Photo by CITYEDV, available under Creative Commons CC0 1.0 license.
- 16. Fail fast is important but not enough!
- 17. Check-in Barcode Generator Web-UI Output Mgmt Current situation – the bad part User
- 18. Check-in Barcode Generator Web-UI Output Mgmt Current situation – the bad part User
- 19. Check-in Barcode Generator Web-UI Output Mgmt Current situation – the bad part Stateful Retry User
- 21. We are having some technical difficulties and cannot present you your boarding pass right away. But we do actively retry ourselves, so lean back, relax and we will send it on time. …just made this up…
- 22. Check-in Barcode Generator Web-UI Output Mgmt Possible situation – much better! User
- 23. Check-in Barcode Generator Web-UI Output Mgmt Possible situation – much better! Stateful Retry User
- 24. Check-in Barcode Generator Web-UI Output Mgmt Stateful Retry Possible situation – much better! The failure never leaves this scope! User
- 25. Persist thing (Entity, Document, Actor, …) State machine or workflow engine Typical concerns DIY = effort, accidental complexity Complex, proprietary, heavyweight, slow, don‘t scale, developer adverse Scheduling, Versioning, operating, visibility, scalability, … Handling State
- 26. Let‘s try this with a simple example Credit Card Payment REST
- 28. Payment Now you have a state machine! Credit Card REST
- 29. Distributed systems introduce complexity you have to tackle! Credit Card Payment REST
- 31. It is impossible to differentiate certain failure scenarios. Independant of communication style! Service Provider Client
- 32. Distributed systems introduce complexity you have to tackle! Credit Card Payment REST
- 33. Distributed systems introduce complexity you have to tackle! Credit Card Payment REST Do it reliably
- 34. Workflows live within service boundaries Credit Card Payment REST
- 39. generateBoardingPass HTTP 200 OK HTTP 202 ACCEPTED Check-in A synchronous response is possible in the happy case, otherwise it is switched to asynchronous processing.
- 41. Check-in Barcode Generator Web-UI User Output Mgmt Asynchronous communication You need to monitor timeouts
- 42. Check-in Barcode Generator Web-UI User Output Mgmt Remember… The failure never leaves this scope!
- 44. Workflow…
- 45. has to implement Timeout, Retry has to implement Idempotence Client Service Provider
- 46. Manifold architecture options The image part with relationsh ip ID rId4 was not found in the file. https://blog.bernd-ruecker.com/architecture-options-to-run-a-workflow-engine-6c2419902d91
- 47. Manifold architecture options https://blog.bernd-ruecker.com/architecture-options-to-run-a-workflow-engine-6c2419902d91 The image part with relationsh ip ID rId4 was not found in the file.
- 48. Manifold architecture options https://blog.bernd-ruecker.com/architecture-options-to-run-a-workflow-engine-6c2419902d91 The image part with relationsh ip ID rId4 was not found in the file.
- 50. Distributed transactions using compensation * Compensation *aka Sagapattern
- 51. Eventual consistency Temporarily inconsistent state But only temporarily!
- 52. has to implement Timeout, Retry, Compensation has to offer Compensation has to implement Idempotence Client Service Provider
- 53. has to implement Timeout, Retry, Compensation has to offer Compensation has to implement Idempotency Client Service ProviderDon‘t forget about state
- 54. # Be aware of complexity of distributed systems # Know strategies and tools to handle it e.g. Circuit breaker (Hystrix) Workflow engine for stateful retry, waiting, timeout, compensation, and visibility (Camunda)
- 56. Remember this? External Tasks – The Pull Principle
- 57. Remember this? Let‘s implement it... Temporarily inconsistent state But only temporarily!
- 58. Live hacking
- 59. # External Tasks are the perfect fit for a decoupled Microservice architecture with Camunda
- 60. # External Tasks are the perfect fit for a decoupled Microservice architecture with Camunda … and help you to solve some other problems along the way: . Polyglot programming . Scaling and Clustering . Decoupling . Natural transactions for distributed systems
- 61. Thank you