CQRS and Event-Sourcing in Magento2 by examples of MSI
0 likes177 views
This document discusses CQRS (Command Query Responsibility Segregation) and Event Sourcing principles and how they are applied in Magento's Multi-Source Inventory (MSI) module. It provides an overview of how MSI uses a CQRS architecture with event sourcing to allow inventory to be managed across multiple sources. It describes how MSI uses event sourcing to store inventory data and reservations and allows querying inventory availability. It also discusses how MSI supports microservices and provides APIs for integration with other systems.
CQRS and Event-Sourcing in Magento2 by examples of MSI
- 1. CQRS and Event Sourcing by example of Multi-Source Inventory Igor Miniailo Software Architect | Magento Commerce @iminyaylo atTwitter #MM18DE
- 3. #MM18DE Classical N-layered architecture • Presentation (Block classes) • Application processing (Controllers and Model classes) • Data management functions (Resource Model classes) are physically separated
- 4. #MM18DE
- 5. #MM18DE
- 6. Magento 2 + Dependency Injection- nothing stays buried forever
- 7. Big Ball of Mud
- 8. SOLID comes to rescue
- 11. #MM18DE Command–query separation (CQS) is a principle of imperative computer programming. It was devised by Bertrand Meyer as part of his pioneering work on the Eiffel programming language. It states that every method should either be a command that performs an action, or a query that returns data to the caller, but not both. In other words, Asking a question should not change the answer. CQS
- 12. #MM18DE class UserAccount { /** * @var boolean */ private $isActive; /** * @return boolean Return true if active; otherwise return false */ public function isActive() { return $this->isActive; } /** * @return void */ public function activate() { $this->isActive = true; } } • Command methods change the state • Commands do not return value (only void) • Query methods read the state • Queries do not change the state of data being used • One object in a code for state changing and querying data
- 15. #MM18DE CQRS “Two objects where there previously was just one” (c) Greg Young
- 16. #MM18DE
- 17. #MM18DE
- 18. #MM18DE
- 19. #MM18DE CQRS as simple as to draw an owl
- 20. #MM18DE Actually, not so easy Applying CQRS you should think about design of Domain Model first (Command)
- 21. #MM18DE CQRS and Domain Driven Design
- 22. #MM18DE And that’s not all
- 23. #MM18DE
- 24. #MM18DE User Interfaces (с) Cayetano de Arquer Buigas
- 25. #MM18DE CRUD Based UI CQRS does not fit for CRUD applications (“forms over data”)
- 27. #MM18DE Task Based UI / Inductive User Interface
- 28. CAPTheorem
- 30. CQRS over HTTP •GET (safe) – Query •POST/PUT/DELETE/PATCH (not safe) – Command
- 32. CQRS and MVC Correct (c) https://antonkril.github.io/mvc-http-cqrs
- 33. CQRS + Event Sourcing
- 35. • History • Rebuild the State for any given point of time • High Scalability • Append Only operations • Low-latency writes • Allows asynchronous processing
- 36. Micro services
- 37. Microservice architecture as an alternative to monolith
- 38. Monoliths vs Microservices. Scalability
- 39. Conway’s Law Any organization that designs a system (defined broadly) will produce a design whose structure is a copy of the organization's communication structure. -- Melvyn Conway, 1967
- 40. Conway’s Law. Cross-functional teams
- 42. MSI
- 43. #MM18DE Where did we start? West Central East Requests from merchants and partners • Need Multiple warehouses • Can’t split inventory for a single product Current state • Single stock only • Must customize or extend • Can be complex to manage and upgrade Multiple types needed • Warehouses • Stores • Distribution Centers • Drop Shipping Increasingly common • Even for smaller merchants • Applies to both B2C and B2B
- 44. #MM18DE The MSI “Story” Original artwork ©2012 Seth Wolfshornd
- 45. #MM18DE A little bit of history
- 46. #MM18DE Original internal Epic MAGETWO-14308 Progress! Source management Assign products and quantity to sources Priority-based selection algorithm for shipment Original internal Epic MAGETWO-14308
- 47. #MM18DE Performance Degradation on order placement which can’t be controlled, as computation of Source Selection could be very time consuming
- 48. #MM18DE And let’s get back to nowadays
- 50. 600 601 15 16 69 70 Submitted Pull Requests Individual Contributors Agencies Represented MSI Project Contributions interjar #MM18DE
- 51. #MM18DE Technical Overview • MSI is a brand new way to handle inventory • Event Sourcing + CQRS architecture • Scalable Multi-Dimensional indexes for Stocks • Usage of PHP 7.1 features • Highly modular design
- 52. #MM18DE Size of MSI Project 33 modules created in the scope of the project Current Codebase of MSI Modules: 57 518 lines of PHP code It is ~12% of the whole Magento Modules codebase
- 53. #MM18DE
- 54. #MM18DE Inventory Reservations in MSI • When an order is placed, a reservation is made to ensure there will be enough quantity available to fulfill the order • Reservations are append only operations to prevent blocking operations and race conditions at checkout • The reservations table is periodically cleaned of reservations that sum to zero Merchant Benefits Performant checkout even with high concurrent sessions Prevents overselling available inventory
- 55. #MM18DE Reservation Example – Step 1 France Warehouse SKU-1: 5 qty EU Stock SKU-1: 15 qty Italy Warehouse SKU-1: 10 qty Raw data Index Reservations Salable Qty: 15 (data from index, empty reservations)
- 56. #MM18DE Reservation Example – Step 2 Action: Customer buys 5 products on frontend (SKU-1)
- 57. #MM18DE Reservation Example – Step 3 France Warehouse SKU-1: 5 qty EU Stock SKU-1: 15 qty Italy Warehouse SKU-1: 10 qty Raw data Index Reservations Salable Qty: 10 (data from index: 15, apply all reservations: -5) Data is NOT changed SKU-1: -5 Order #001 Reservation has been added
- 58. #MM18DE Reservation Example – Step 4 Actions: From the original order of 5 products, 3 are returned
- 59. #MM18DE Reservation Example – Step 5 France Warehouse SKU-1: 5 qty EU Stock SKU-1: 15 qty Italy Warehouse SKU-1: 10 qty Raw data Index Reservations Salable Qty: 13 (data from index: 15, apply all reservations: -5+3) Data is NOT changed Reservation has been added SKU-1: -5 SKU-1: +3 Order #001 Return #001
- 60. #MM18DE Reservation Example – Step 6 Actions: • Admin ships remaining items in order (qty 2) • Reindex on shipment of order
- 62. #MM18DE Reservation Example – Step 7 France Warehouse SKU-1: 3 qty EU Stock SKU-1: 13 qty Italy Warehouse SKU-1: 10 qty Raw data Index Reservations Salable Qty: 13 (data from index: 13, apply all reservations: -5+3+2=0) Data is CHANGED SKU-1: -5 SKU-1: +3 SKU-1: +2 Order #001 Return #001 Order #001 Shipment Reservation has been added
- 63. #MM18DE Reservation_id Stock_id sku quantity metadata 1 2 SKU-1 -5.000 order_placed:order:1 2 2 SKU-1 3.000 order_cancelled:order:1 3 2 SKU-1 2.000 shipment_created:order:1 Reservation Example – Step 8 Action: • Reservation cleaning • Looping through these reservations we can find reservations which in sum return 0 (Zero) and remove them
- 64. #MM18DE Reservation Example – Step 9 (like Step 1) France Warehouse SKU-1: 3 qty EU Stock SKU-1: 13 qty Italy Warehouse SKU-1: 10 qty Raw data Index Reservations Salable Qty: 13 (data from index, empty reservations table) Data is NOT changed Reservations have been removed
- 65. #MM18DE Extension points and APIs in MSI • For integration with ERP and PIM systems • Inventory Sales API (Reservations placement) • For extension developers • Source Selection AlgorithmAPIs to provide own more efficient and business oriented choice of Sources for order fulfillment
- 66. Microservices and MSI • InventoryApi (APIs) • Inventory (Implementation for APIs) • InventoryAdminUi • InventoryFrontendUi
- 67. Microservices and Multi-Source Inventory
- 68. Q & A iminiailo@magento.com @iminyaylo atTwitter