[WSO2Con EU 2017] Resilience Patterns with Ballerina
1 like319 views
The document describes a developer, Bob, who faced challenges in creating a mobile self-care banking application due to issues with legacy services, particularly transient network failures. It discusses the concept of resilience in software systems, emphasizing the importance of designing applications to handle failures and maintain availability through various resilience patterns like bulkhead, retry, and circuit breaker. The use of Ballerina to implement these resilience patterns is also highlighted as a key component for improving the reliability of services in the banking application.
[WSO2Con EU 2017] Resilience Patterns with Ballerina
- 1. Senior Technical Lead, WSO2 Resilience Patterns with Ballerina Isuru Udana
- 2. Bob is a developer who works at the IT Department of a popular bank. Bob
- 3. Bob was asked to develop a mobile selfcare banking application.
- 4. This mobile application should be capable of showing account balances as well as details of past transactions.
- 5. Legacy services are already available.
- 6. But when he started to implement the application, bob found issues with some of the legacy services. Transient Network Failures Moderate Load Intermittent Failures
- 7. Bob found it very difficult to build a reliable application.
- 8. Resilient service Bob got an idea!
- 9. Resilience
- 10. Ability to return to the original form, position after being affected by a particular alteration What is Resilience?
- 11. In a software system, resilience means ... … the ability to recover to a working condition after being affected by a serious incident Resilience in Software Applications
- 12. “The probability of failure-free software operation for a specified period of time in a specified environment.” - The IEEE Reliability Society • 100% operational all the time Reliability and Resilience Reliability http://www.picserver.org/r/reliability.html
- 13. Focusing on Reliability is Enough...?
- 14. Distributed and complex systems with many interactions are prone to failures Why Focusing on Reliability is Not Enough Systems are Complex and Prone to Failures
- 15. • Untested corner cases • Minor mistakes can affect serious production incidents • Failures are unpredictable Why Focusing on Reliability is Not Enough Avoiding Failures is Not Practical
- 16. • Handle unexpected situations • When one feature is temporarily unavailable, the rest of the application still runs • Stop propagating errors happening at downstreams of a complex system into upstreams Resilience in Production
- 17. It’s All About Achieving Availability of a Production System!
- 18. Best case: • User get’s a 100% availability of the service Typical case: • User sees a graceful degradation of the service What Does it Mean to a User?
- 19. • Never expect systems to be 100% reliable • Design systems thinking about connection issues, down times, etc. What Does it Mean to a developer ?
- 20. • Bulkhead • Retry • Circuit Breaker • Timeout • … Resilience Patterns
- 21. Isolate components of an application into multiple pools. If one component fails, others will continue to service Bulkhead Isolation
- 22. • Transient failures are not uncommon • They recover by themselves • Can be handled by – Cancel – Retry – Retry with a delay Retry https://www.flickr.com/photos/markgregory/8184890333 https://creativecommons.org/licenses/by-nc-sa/2.0/legalcode
- 23. • Hide downstream latency and keep the responsiveness to upstream • Prevent waiting forever Timeout
- 24. • Some transient failures takes much longer to recover • Repeatedly retrying may hinder recoverability • Retry up to a certain degree and cut off Circuit Breaker
- 25. Circuit Breaker Fail/Keep Open Reset Timeout Fail Success Fail (threshold not reached) Fail (threshold exceeded) Success Open Half-OpenClosed States in circuit breaker
- 27. • Designed to implement resilient programs/services • Highly structured error and exception handling Resilience with Ballerina
- 28. Back to the Story
- 29. Banking Service Banking Service Account Balance Service Account History Service Mobile Application Can only handle moderate load Transient Failures Sometimes takes a long time to respond
- 30. Banking Service Account Balance Legacy Service Account History Legacy Service Account Balance Resource Account History Resource
- 32. Connectors, Connections and Endpoints Endpoint Connection Params Connector Options Struct
- 33. Handling Transient Failures Retry Banking Service Account Balance Service Mobile Application
- 36. Protect Services From Overload Circuit Breaker Retries Before Suspension Suspension Duration
- 37. Applying Multiple Patterns Circuit Breaker + Retry + Timeout Circuit Breaker Retry Timeout
- 38. Balancing Load Banking Service Account Balance Service 1 Mobile Application Account Balance Service 2
- 40. Story Continued ... For priority customers, application should provide zero downtime.
- 41. Offering Different Quality of Services Bulkhead Banking Service Reliable Service Service got Transient Failures Standard Customer Priority Customer
- 42. Offering Different Quality of Services Bulkhead Reliable Service Service with Transient Issues Priority Check Function
- 43. Conclusion ● What is Resilience ● Reliability and resilience ● Resilience patterns ● Building resilience patterns with Ballerina
- 44. wso2.com