Microservices + Events + Docker = A Perfect Trio (dockercon)
25 likes28,022 views
The document outlines how microservices, event-driven architecture, and Docker can enhance application development and deployment. It compares monolithic and microservice architectures, discussing the benefits and complexities of each, particularly in terms of data management and service communication. The author emphasizes using event sourcing and Docker for improving data consistency and simplifying development processes.
Microservices + Events + Docker = A Perfect Trio (dockercon)
- 1. @crichardson Microservices + Events + Docker = A Perfect Trio Chris Richardson Founder of Eventuate.io Founder of the original CloudFoundry.com Author of POJOs in Action @crichardson chris@chrisrichardson.net http://eventuate.io
- 2. @crichardson Presentation goal http://muppet.wikia.com/wiki/Brought_to_You_by_the_Number_3 Microservices, Events, and Docker are a great way to develop and deploy applications
- 4. @crichardson About Chris Consultant and trainer focusing on modern application architectures including microservices (http://www.chrisrichardson.net/)
- 5. @crichardson About Chris Founder of a startup that is creating a platform that makes it easier for developers to write transactional microservices (http://eventuate.io)
- 7. @crichardson Agenda Monolith vs. microservices Event-driven microservices Developing and deploying microservices using Docker
- 8. @crichardson Let’s imagine you are building a large, complex application, e.g. an online store
- 9. @crichardson Successful software development Architecture Process Organization Agile Continuous delivery … Small, autonomous, teams 3
- 11. @crichardson
- 12. @crichardson The monolithic architecture Tomcat Browser WAR SQL database HTML REST/JSON Client App Simple to …. Develop Test Deploy Scale Catalog Module Reviews Module Orders Module StoreFront UI Module
- 14. @crichardson Monolithic architecture Process Organization Agile Continuous delivery … Small, autonomous, teams
- 15. @crichardson Apply functional decomposition X axis - horizontal duplication Z axis -data partitioning Y axis - functional decomposition Scale by splitting sim ilar things Scale by splitting different things 3
- 17. @crichardson Microservice architecture Process Organization Agile Continuous delivery … Small, autonomous, teams✔ ✔
- 19. @crichardson Drawbacks Complexity of developing a distributed system Implementing inter-process communication Handling partial failures Complexity of implementing business transactions that span multiple databases (without 2PC) Complexity of testing a distributed system Complexity of deploying and operating a distributed system Managing the development and deployment of features that span multiple services Fortunately solutions exists
- 20. @crichardson The benefits typically outweigh the drawbacks for large, complex applications
- 21. @crichardson Issues to address How to deploy the services? How do the services communicate? How do clients of the application communicate with the services? How to partition the system into services? How to deal with distributed data management problems? ….
- 22. @crichardson Agenda Monolith vs. microservices Event-driven microservices Developing and deploying microservices using Docker
- 24. @crichardson The Database Shared database Order Service Customer Service … Service Order table Customer table … orderTotal creditLimit Tight coupling Simple and ACID
- 25. @crichardson Database per service Order Service Customer Service Order Database Customer Database Order table Customer table orderTotal creditLimit Loose coupling 😀 but more complex 😓 and….
- 26. @crichardson 2PC (aka. distributed transactions) is not viable choice for most modern applications
- 27. @crichardson Customer management How to maintain data consistency without 2PC? Order management Order Service placeOrder() Customer Service updateCreditLimit() Customer creditLimit ... has ordersbelongs toOrder total Invariant: sum(open order.total) <= customer.creditLimit ?
- 29. @crichardson Use event-driven, eventually consistent order processing Order Service Customer Service Order created Credit Reserved Credit Check Failed Place Order OR
- 30. @crichardson How atomically update database and publish an event Order Service Order Database Message Broker insert Order publish OrderCreatedEvent dual write problem ?
- 31. @crichardson Reliably publish events when state changes
- 32. @crichardson Use event-sourcing Event table Aggregate type Event id Aggregate id Event data Order 902101 …OrderApproved Order 903101 …OrderShipped Event type Order 901101 …OrderCreated
- 33. @crichardson Replay events to recreate state Order state OrderCreated(…) OrderAccepted(…) OrderShipped(…) Events Periodically snapshot to avoid loading all events
- 34. @crichardson Benefits of event sourcing Solves data consistency issues in a Microservice/NoSQL based architecture Reliable event publishing: publishes events needed by predictive analytics etc, user notifications,… Eliminates O/R mapping problem (mostly) Reifies state changes: Built in, reliable audit log temporal queries Preserved history More easily implement future requirements
- 35. @crichardson Drawbacks of event sourcing Requires application rewrite Weird and unfamiliar style of programming Events = a historical record of your bad design decisions Must handle duplicate events: idempotent handlers or duplicate detection Querying the event store can be challenging
- 36. But what about queries?
- 37. @crichardson Find recent, valuable customers SELECT * FROM CUSTOMER c, ORDER o WHERE c.id = o.ID AND o.ORDER_TOTAL > 100000 AND o.STATE = 'SHIPPED' AND c.CREATION_DATE > ? Customer Service Order Service What if event sourcing is used?…. is no longer easy
- 38. @crichardson Command Query Responsibility Segregation (CQRS) Command side Commands Aggregate Event Store Events Query side Queries Materialized View Events POST PUT DELETE GET MongoDB Redis Neo4j SQL ElasticSearch … More complex 😓 but high performance, scalable views 😀
- 39. @crichardson Agenda Monolith vs. microservices Event-driven microservices Developing and deploying microservices using Docker
- 40. @crichardson We have applied the microservices pattern: How to deploy the 10s or 100s of services?
- 41. @crichardson Forces Services are written using a variety of languages, frameworks, and framework versions Each service consists of multiple service instances for throughput and availability Building and deploying a service must be fast Services must be deployed and scaled independently Service instances need to be isolated Deployment must be reliable and cost-effective
- 42. @crichardson
- 43. @crichardson
- 44. @crichardson VM VM Pattern: Service per Container host Service Container image Container Service Container Service Container Service packaged as deployed as
- 45. @crichardson Benefits of containers Great isolation Great manageability Container encapsulates implementation technology Efficient resource utilization Fast deployment
- 47. Running infrastructure services on development machines Typical services needs a database, message broker, … Making sure every developer installs the correctly version = painful rabbitmq: image: rabbitmq:3.5.3 ports: - "5672:5672" - "15672:15672" mongodb: image: mongo:3.0.4 ports: - "27017:27017" command: mongod --smallfiles
- 48. @crichardson Deploying microservices for end-to-end testing restfulservice: image: java:openjdk-8u45-jdk working_dir: /app volumes: - ./spring-boot-restful-service/build/libs:/app command: java -jar /app/spring-boot-restful-service.jar ports: - "8081:8080" links: - rabbitmq - mongodb environment: SPRING_DATA_MONGODB_URI: mongodb://mongodb/userregistration SPRING_RABBITMQ_HOST: rabbitmq
- 49. @crichardson Jenkins-based deployment pipeline Build & Test microservice Build & Test Docker image Deploy Docker image to registry One pipeline per microservice
- 50. @crichardson Smoke testing docker images Smoke test Docker daemon Service containerGET /health POST /containers/create creates POST /containers/{id}/start Docker daemon must listen on TCP port
- 51. @crichardson Running on Docker! EC2 Instance Jenkins Container Artifactory container EBS volume /jenkins- home /gradle-home /artifactory- home
- 52. @crichardson Summary Use microservices to accelerate development Use an event-driven architecture to maintain data consistency Use Docker to simplify development and deployment