Events and microservices
2 likes1,099 views
The document discusses using events and asynchronous messaging to coordinate activities across microservices. It describes how events can be used for loose coupling between services, avoiding the need for centralized orchestration. Events are proposed as a way to handle data synchronization and business processes across bounded contexts. Various options for transporting events such as syndication, web hooks, and message queues are also examined.
Events and microservices
- 1. Events & Microservices Saul Caganoff, CTO Sixtree saul@sixtree.com.au @scaganoff
- 2. Introduction • Asynchrony is considered a microservice characteristic – But not often discussed • In the top 20 Google results – 7 mentioned async/choreography or events – 3 of those by Chris Richardson or Martin Fowler • Mostly mentioned in passing • Honourable Mentions: – http://www.infoq.com/articles/microservices-‐intro – http://highscalability.com/blog/2014/4/8/microservices-‐not-‐a-‐free-‐ lunch.html
- 3. WHY EVENTS?
- 4. Bounded Contexts Finance Sales Fulfilment Product Online bookstore is the “Hello World” of enterprise systems http://maribyrnong.com.au
- 6. Finance Context Wishlist Book ISBN Title Author Price Order Status Timestamp Sub-‐total GST Total Item Quantity History Customer Name Age Loyalty Status Credit Card Holder Name Number Expiry Code
- 7. Fulfilment Context Book ISBN Title Customer Name Receipt Order Number Sub-‐total GST Total Address Street Number Street Name Suburb Postcode Courier Depot Location Driver Shipment Dispatch Time ETA Location
- 9. Coupling & Cohesion • Cohesion – how well a subsystem forms a uniting whole – How well a bounded context does its job • Coupling – the propensity for change in one part of a system to disrupt another – how badly “stuff” leaks across the boundaries • Coupling is evil – it sneaks into solutions in all kinds of insidious ways
- 11. Coupling & Cohesion • Microservices strive to eliminate coupling • Insidious sources of coupling – Shared databases – Single Common Data Model (vs bounded contexts) – DRY – shared code – APIs generated from code – Autogenerated client SDKs – Projects – funding and scope – Assumptions about business processes – Organizational structure
- 12. Business Processes • End-‐to-‐end business processes require coordination across bounded contexts Finance Sales Ful-‐ filment Product
- 13. Business Processes • Browse the inventory • Create order • Enter Shipping Details • Quote shipping charge • Make the payment • Update inventory • Create Shipment • Pick & Pack • Courier • Accept delivery GET /products POST /order PUT /order/:id/address GET /3pl/charges?zip=… POST /finance/payment?order= POST /products POST /3pl/shipment GET /3pl/shipment/:id GET /3pl/shipment/:id/address PUT /3pl/shipment/:id?status=
- 14. Business Processes • Orchestration – Centralized coordination – E.g. the user-‐agent, BPM engine, service facade – the “god” object • Hypermedia – HATEOAS – But what about at the edges? – How much should one BC know about another BC? “microservices architecture will never be SOA done right unless it means building hypermedia systems of microservices” – Jason Bloomberg http://intellyx.com/2015/07/20/are-‐microservices-‐soa-‐done-‐right/
- 15. Some Perspective 0 2 4 6 8 10 12 DB Triggers Monolith Orchestration Hypermedia EDA Coupling
- 16. Data Synchronization • How to handle data “owned” by multiple domains – Glibness: “don’t” have multiple data owners – …in reality it is impossible to eliminate – …especially if legacy or COTS systems involved. • Some data synchronization is naturally conveyed by the business process • …some is not.
- 17. Choreography • Use events to coordinate activity across bounded contexts • Publishers & Subscribers are highly decoupled – Event subscribers can come & go minimal or no code or configuration changes • Publishers & Subscribers are more resilient to change – either planned or unplanned – E.g. events redelivered in the event of subscriber failure – either to the restored subscriber, or a replacement
- 18. Choreography • But trade-‐offs • Asynchronous events lie at the heart of some of the microservices challenges – Centralised orchestration is harder to change, but easier to control/monitor – Event-‐based choreography is easier to change but harder to control/monitor • Distributed logging/monitoring systems are needed to provide a view into asynchronous interactions
- 19. Coordination • Think about hypermedia inside boundaries • Events outside the boundaries Finance Sales Ful-‐ filment Product
- 20. MODELLING EVENTS
- 21. Commands & Events • Duality between commands & events – Commands cause state transitions – Events reflect state transitions • An external observer can infer the state of a resource from its event-‐stream – This is the essence of event-‐sourcing & CQRS
- 22. State Modelling Empty Cart Full Cart Quoted Paid POST /cart PUT /cart/:id { book: … } PUT /cart/:id { book: … } DELETE /cart/:id/books/:id PUT /cart/:id { address: … } POST /payments { cart: …, card: {…}} Commands (HATEOAS) {create: { cart: … } … } {add: { cart: …, book: {…}} ..} {add: { cart: …, book: {…}} ..} {del: { cart: …, book: {…}} ..} {quoted: { cart: … } … } {paid: { cart: … } … } Events (EATNOAS)
- 24. Event Structure • Immutable structures • Typical properties – Timestamp – Source – Entity URI – Action (state transition) • The MVP is to notify that a resource has changed and allow subscribers to pull the update if needed.
- 25. Concurrency • Race Conditions • The importance of order • Single Master vs Multi-‐Master • Deltas vs Copies • Action indicators • CRDTs – conflict-‐free replicated data types • …think in terms of events rather than copies
- 26. EVENT TRANSPORTS
- 27. Syndication • Service exposes an RSS/atom feed • Consumers do all the hard work: – Subscription – State Management (where am I up to?) – Retries • Positives – Very easy…well-‐known pattern – Web-‐scale • Negatives – Polling – High latency
- 28. Web Hooks • Service POSTs events to subscribers • Publishers do all the hard work: – Manage subscribers – Managed guaranteed delivery/ retries • Positives – Not polling, relatively lower latency • Negatives – Subscription management & guaranteed delivery must be done yourself…can be tricky.
- 29. Message Queues • Events published to queues/topics – E.g JMS, AMQP, ZMQ, SQS/SNS, IronMQ • Positives – Mature, well-‐known technologies – Good open-‐source implementations available – Competing consumer pattern • Negatives – Extra infrastructure – Reliability depends on reliable file-‐systems • SANs, Distributed file-‐lock managers
- 30. Apache Kafka • Distributed log • Similar model to syndication – Single node shares data by writing to logfiles – Consumers tail the logs • Consumers control their position in the log • Low latency • Reliability handled at the application layer – whether Zookeeper is less burdensome than clustered file system is up to you
- 31. Event Processing • Simple events are very useful for data synchronization or process coordination • Continuum of complexity • Some tools: Drools, Riemann.io, Apache Storm • This is a whole ‘nother topic Cardinality State Tools Single event Simple Event processor Multiple events/stream Simple Stream processor Multiple events/stream Complex / Rules Complex Event Processor
- 32. Wrap-Up • Bounded contexts are important – they promote cohesion and reduce coupling • You need to decide how to coordinate data & processes across boundaries – Orchestration (relatively higher coupling) – Hypermedia (couples across boundaries) – Events • Prefer Hypermedia within bounded contexts & events across bounded contexts Events have many benefits but also some downsides -‐ they are an important tool in your toolbox.