2019 04-25-agile communitymeetup-essentialcapabilitiesbehindmicroservices
- 1. © 2019, Domain Driven Design Taiwan Community Kim Kao ( ) April 25, 2019 Essential Capabilities behind Microservices
- 2. © 2019, Domain Driven Design Taiwan Community A typical day for a customer new to AWS... Manager - âWe are going to run workload(s) on AWS. We have new sub-systems/module to develop with legacy services. Container is good, Lambda is awesome. Itâs great to have whole cloud native advantage if you guys migrate all service into microservice, serverless...â Developer - âNot a problem. Iâll make it âŠâ
- 3. © 2019, Domain Driven Design Taiwan Community Business Wants https://vaughnvernon.co/tag/event-storming/
- 4. © 2019, Domain Driven Design Taiwan Community But You Want https://vaughnvernon.co/tag/event-storming/
- 5. © 2019, Domain Driven Design Taiwan Community Challenge on migration to Microserivces ⹠Legacy looks like Big ball of mud(BBOM) ⹠Heavy dependency with external system ⹠No idea on splitting BBOM ⹠No idea to find out system boundary ⹠Which service(s) worth to do ⹠Human resources allocation ⹠Team, out sourcing, ISVs solution
- 6. © 2019, Domain Driven Design Taiwan Community What are Microservices?
- 7. © 2019, Domain Driven Design Taiwan Community = Microservices ?
- 8. © 2019, Domain Driven Design Taiwan Community 100/200/300/1000? LOC
- 9. © 2019, Domain Driven Design Taiwan Community © 2018, Amazon Web Services, Inc. or Its Affiliates. All rights reserved. What Are Microservices? âA software architecture style in which complex applications are composed of small, independent processes communicating with each other using language- agnostic APIs. These services are small, highly decoupled and focus on doing a small task, facilitating a modular approach to system-building.â - Wikipedia https://en.wikipedia.org/wiki/Microservices
- 10. © 2019, Domain Driven Design Taiwan Community Way to divide services from Monolith âą By noun(s) âą By Your Experience(s) âą By Teamâs decision âą Up to Leader/Manager/CxO ... âą No right or power to influence
- 11. © 2019, Domain Driven Design Taiwan Community Problem behind Microservices
- 12. © 2019, Domain Driven Design Taiwan Community Monolith Load Balancer Account Service Cart Service Shipping Service StoreFront UI Browser Database Data Access Service
- 13. © 2019, Domain Driven Design Taiwan Community Account Database Inventory Database Shipping Database Migrate to Microservices Load Balancer StoreFront UI Browser Account Service Cart Service Shipping Service Load Balancer Load Balancer Load Balancer
- 14. © 2019, Domain Driven Design Taiwan Community Challenge is Coming ⹠Authentication & Authorization ⹠Session mechanism is broken ⹠Centralized & Robust alternative one is required ⹠Service Discovery ⹠Write the traffic routing code(s) in logic? ⹠Re-try, Failover, Caching, ... ⹠Persistence ⹠Isolated Persistence for each Service
- 15. © 2019, Domain Driven Design Taiwan Community A Classic solution on AWS Load Balancer StoreFront UI Browser Account Database Account Service Cart Service Inventory Database Shipping Service API Gateway Load Balancer Load Balancer Load Balancer Shipping Database
- 16. © 2019, Domain Driven Design Taiwan Community Authentication & Authorization ⹠Token based solution fit in ⹠Cognito ⹠3rd party Authentication Federation ⹠Amazon, Google, Apple, Facebook... ⹠Self developed/hosted centralize A&A services
- 17. © 2019, Domain Driven Design Taiwan Community Service Discovery ⹠Client Side-Service Discovery ⹠Application Load Balancer-based Service Discovery ⹠DNS-Based Service Discovery ⹠Service Discovery using ECS Event Stream ⹠Configuration Management ⹠New * App Service Mesh (preview) ⹠Cloud Map
- 18. © 2019, Domain Driven Design Taiwan Community Microservices For Persistence ? Stateful Persistent always be the top issue !!!
- 19. © 2019, Domain Driven Design Taiwan Community âą Accept all API calls to one RDS Cluster âą RDS Instance Sizing predict by services scaling ability âą DBA(s) : How to manage Connection Pool ? âą Read replica canât help on Write intention âą RDS I/O be the bottleneck when highly Scale up âą All Services impacted while any failure occurred âą Option : Upgrade Read Replica to Master Launching 1 RDS cluster
- 20. © 2019, Domain Driven Design Taiwan Community Launching N * RDS for Independency?
- 21. © 2019, Domain Driven Design Taiwan Community ⹠DynamoDB ⹠Perfect deal with high transaction (write) ⹠Serve each table for only one transaction/service intention ⹠De-Normalize schema is required ⹠Prevent revision issue ⹠Hard to do complexity Join Query ⹠Cost High ⹠Low Performance ⹠Coding for iteration & aggregation ⹠Apply all RDS tables into DynamoDB? Using NoSQL Solution?
- 22. © 2019, Domain Driven Design Taiwan Community Itâs about capability âą Are you ready to deal with M:N transaction compensation ? âą Are you ready to embrace the rapidly change by contract ?
- 23. © 2019, Domain Driven Design Taiwan Community Transaction Dependencies between Microservices ⹠Upstream-Downstream co-relationship ⹠One Transaction invoke local API and Remote API Book Rental Service Book Flight Service Trip Service Book Hotel Service Exception / Error ? Exception / Error ? Exception / Error ? !" # !$ # !# # + + Transaction Compensate
- 24. © 2019, Domain Driven Design Taiwan Community Saga : Alternative for 2 Phase -commit
- 25. © 2019, Domain Driven Design Taiwan Community Context ⹠You have applied the Database per Service pattern. Each service has its own database. ⹠Some business transactions, however, span multiple service so you need a mechanism to ensure data consistency across services.
- 26. © 2019, Domain Driven Design Taiwan Community Problem How to maintain data consistency across services?
- 27. © 2019, Domain Driven Design Taiwan Community Forces ⹠2 Phase-Commit(PC) is a well-known solution for years ⹠The 2PC coordinator also represents a Single Point of Failure, which is unacceptable for critical systems
- 28. © 2019, Domain Driven Design Taiwan Community Solutions ⹠Each local transaction updates self and publishes a message/event to trigger the next local transaction in the saga. ⹠If a local transaction fails because it violates a business rule then the saga executes a series of compensating transactions that undo the changes that were made by the preceding local transactions. https://microservices.io/patterns/data/saga.html
- 29. © 2019, Domain Driven Design Taiwan Community Two ways of Saga Pattern ⹠Choreography - each local transaction publishes domain events that trigger local transactions in other services ⹠Orchestration - an orchestrator (object) tells the participants what local transactions to execute https://microservices.io/patterns/data/saga.html
- 30. © 2019, Domain Driven Design Taiwan Community Choreography-based Saga https://microservices.io/patterns/data/saga.html 2 1 3.b 3.a 4.a 4.b ⹠Logic in code ⹠Compensate chain ⹠Multiple states present in transaction
- 31. © 2019, Domain Driven Design Taiwan Community Orchestration -based Saga https://microservices.io/patterns/data/saga.html ⹠Coordinator stand alone ⹠Each service provide a pair function for Success or Fail ⹠Abstract compensate logic out of Codes1 1.1 2 3 45
- 32. © 2019, Domain Driven Design Taiwan Community Resulting Context ⹠This pattern has the following benefits: ⹠It enables an application to maintain data consistency across multiple services without using distributed transactions ⹠This solution has the following drawbacks: ⹠The programming model is more complex. For example, a developer must design compensating transactions that explicitly undo changes made earlier in a saga. ⹠There are also the following issues to address: ⹠In order to be reliable, a service must atomically update its database and publish a message/event. It cannot use the traditional mechanism of a distributed transaction that spans the database and the message broker. Instead, it must use one of the patterns listed below.
- 33. © 2019, Domain Driven Design Taiwan Community Implementing Saga
- 34. © 2019, Domain Driven Design Taiwan Community ⹠Each Service provide cancel operation ⹠State change trigger cancel operation ⹠Simplify Complexity rules ⹠Just a JSON file
- 35. © 2019, Domain Driven Design Taiwan Community Code snippet https://github.com/humank/lambda-saga-pattern
- 36. © 2019, Domain Driven Design Taiwan Community Demo
- 37. © 2019, Domain Driven Design Taiwan Community
- 38. © 2019, Domain Driven Design Taiwan Community How to adopt Microservices?
- 39. © 2019, Domain Driven Design Taiwan Community
- 40. © 2019, Domain Driven Design Taiwan Community
- 41. © 2019, Domain Driven Design Taiwan Community
- 42. © 2019, Domain Driven Design Taiwan Community
- 43. © 2019, Domain Driven Design Taiwan Community
- 44. © 2019, Domain Driven Design Taiwan Community Business operation without whole picture The Blind Men and the Elephant Is It correct to all in microservices or serverless ?
- 45. © 2019, Domain Driven Design Taiwan Community Precondition to do microservices Rapid Provisioning Basic monitoring Rapid application deployment Martin Fowler
- 46. © 2019, Domain Driven Design Taiwan Community Better way to decompose Monolith Domain Expert Matters
- 47. &
- 48. © 2019, Domain Driven Design Taiwan Community How to break Monolith?
- 49. © 2019, Domain Driven Design Taiwan Community Way to collaborate ⹠Point out the events ⹠Who send the command ⹠Find the Noun(s) ⹠Have all team voices Commands Events Aggregate
- 50. © 2019, Domain Driven Design Taiwan Community Coffee shop experience DDD by EventStorming
- 51. © 2019, Domain Driven Design Taiwan Community
- 52. © 2019, Domain Driven Design Taiwan Community
- 53. © 2019, Domain Driven Design Taiwan Community
- 54. © 2019, Domain Driven Design Taiwan Community
- 55. © 2019, Domain Driven Design Taiwan Community
- 56. © 2019, Domain Driven Design Taiwan Community
- 57. © 2019, Domain Driven Design Taiwan Community
- 58. © 2019, Domain Driven Design Taiwan Community Go through Event Storming approach Donât tell tech only Donât sell tech partially Aim for Core value Figure out trigger and result
- 59. © 2019, Domain Driven Design Taiwan Community Seat occupied Menu offered Ordered 2 cups of Americano Paid Order received Coffee made up Customers Left Table Cleaned *Key Business Events in Coffeeshop
- 60. © 2019, Domain Driven Design Taiwan Community Event Trigger ⹠Client ⹠Server ⹠Counter ⹠Barista ACTORS
- 61. © 2019, Domain Driven Design Taiwan Community Most Valuable / Risky Events
- 62. © 2019, Domain Driven Design Taiwan Community © 2018, Amazon Web Services, Inc. or Its Affiliates. All rights reserved. Order Make Up Payment Event Bus (pub/sub) Put event Event Event CloudWatch Event . . . Any other messaging technology Coffee shop Domain implementation Core Domain Sub Domain (Messaging) Support Domain Core Domain
- 63. © 2019, Domain Driven Design Taiwan Community When you should dive in Microservices Team Partners Business Operation Coding Value
- 64. © 2019, Domain Driven Design Taiwan Community How DDD can help you ⹠Business strategy on resource allocation ⹠Best resources should be put in most key/core domain ⹠Buy or out-sourcing common domain and sub domain ⹠Service re-architecture ⹠Form up the system context boundary ⹠Knowing the upstream-downstream relationship between domains ⹠Meaningful to do microservice (separate computing/persist, and API communication ) ⹠Service Migration ⹠Good to re-architecture
- 65. © 2019, Domain Driven Design Taiwan Community Take Away Know Why/What/How ⹠Do you really need Microservices? ⹠Leverage Business Events to aggregate context and form up Service Boundary ⹠There is no C4 to solve distributed persistence issue ⹠State Machine to do Transaction Compensate (Step Functions way to go) ⹠DDD is good to collaborate Business and Technology guys by speaking Ubiquitous Language ⹠Crunch Problem, then design solution
- 66. © 2019, Domain Driven Design Taiwan Community Implementing DDD on AWS Commounty : DDD Taiwan@FB Telegram : YikaiKao WeChat : YikaiKao Twitter : @YikaiKao GitHub Repos