Microservices in Clojure
11 likes4,632 views
The document discusses the implementation of microservices using Clojure, emphasizing concepts like immutability, pure functions, and the principles of ports and adapters architecture. It illustrates how Clojure’s immutability features enhance data handling and outlines the structure of microservice components for better organization and testing. It also stresses the importance of simplicity in design and architecture within the context of microservices development.
![Components
https://github.com/stuartsierra/component
(defprotocol Database
(query [this query-str]))
(defrecord SomeDatabase [config-1 config-2 other-components]
component/Lifecycle
(start [this]
(assoc this :connection (connect! config-1 config-2 other-components)))
(stop [this]
(release! (:connection this))
(dissoc this :connection))
Database
(query [this query-str] (do-query! (:connection this) query-str)))](https://image.slidesharecdn.com/microservicesinclojure-170209181649/85/Microservices-in-Clojure-14-320.jpg)
Microservices in Clojure
- 2. Context Microservices ~80 Clojure services ~60 engineers ~10 teams 3.5 years old
- 3. OOP Objects, the mainstream abstraction Image @ http://www.eduardopires.net.br/2015/01/solid-teoria-e-pratica/
- 4. What about Functional Programming? SÃO PAULO, BRASIL
- 5. TABLE OF CONTENTS Immutability Components Pure Functions Schemas Ports and Adapters SÃO PAULO, BRASIL
- 6. Immutability SOUTHEAST BRAZIL REGION FROM SPACE
- 7. Immutability Definition “If I’m given a value, it’s guaranteed that it won’t ever change”
- 9. Clojure Immutability All default data structures are immutable: -Maps, Lists, Sets -Records Mutability is explicit: atoms/refs @, dynamic vars *…*
- 10. Datomic Immutability Datomic stores the changes/transactions, not just the data -append only -db as a value -everything is data (transaction, schema, entities)
- 11. Kafka Immutability Persistent Queues/Topics -each consumer has its offset -ability to replay messages
- 12. AWS + Docker Immutability Ability to spin machines with a given image/ configuration -Each build generates a docker image -Each deploy spins a new machine with the new version -As soon as the new version is healthy, old version is killed. (blue-green deployment)
- 13. Components SOUTHEAST BRAZIL REGION FROM SPACE
- 14. Components https://github.com/stuartsierra/component (defprotocol Database (query [this query-str])) (defrecord SomeDatabase [config-1 config-2 other-components] component/Lifecycle (start [this] (assoc this :connection (connect! config-1 config-2 other-components))) (stop [this] (release! (:connection this)) (dissoc this :connection)) Database (query [this query-str] (do-query! (:connection this) query-str)))
- 15. System map Components {:database #SomeDatabase{...} :http-client #HttpClient{...} :kafka #Kafka{...} :auth #AuthCredentials{...} ...} -Created at startup -Entrypoints (e.g http server or kafka consumers) have access to all components the business flows need -dependencies of a given flow are threaded from the entry point until the end, one by one if possible -Thus no static access to system map! (e.g via a global atom) -Any resemblance to objects and classes is just coincidence ;)
- 16. Pure functions SOUTHEAST BRAZIL REGION FROM SPACE
- 17. Pure functions Definition "Given the same inputs, it will always produce the same output"
- 18. Simplicity Pure functions -easier to reason about, fewer moving pieces -easier to test, less need for mocking values -parallelizable by default, no need for locks or STMs
- 19. Datomic Pure functions -Datomic’s db as a value allows us to consider a function that queries the database as a pure function -db is a snapshot of the database at a certain point in time. -So, querying the same db instance will always produce the same result
- 20. Impure functions Pure functions -functions that produce side effects should be marked as such. We use `!` at the end. -split code which handles and transforms data from code that handles side effects -should be moved to the borders of the flow, if possible -Consider returning a future/promise like value, so side effect results can be composed (e.g with manifold or finagle) https://github.com/ztellman/manifold https://github.com/twitter/finagle
- 21. Schema/Spec SOUTHEAST BRAZIL REGION FROM SPACE
- 22. Schema Legacy Majority of our code base was written before clojure.spec existed, so I’ll be talking about the Schema library instead. Most principles apply to clojure.spec as well.
- 23. Schema/Spec Documentation -Clojure doesn’t force you to write types -parameter names are not enough -declaring types helps a lot when glancing at the function -values can be verified against a schema
- 24. Function declaration Schema/spec -All pure functions declare schemas for parameters and return value -All impure functions declare for parameters and don’t declare output type if it’s not relevant. -Validated at runtime in dev/test environments, on every function call -Validation is off on production.
- 25. Wire formats Schema/Spec -Internal schemas are your domain models -Wire schemas are how you expose data to other services/ clients -If they are different, you can evolve internal schemas without breaking clients -Need an adapter layer -wire schemas are always validated on entry/exit points, specially in production -single repository for all wire schemas (for all 60+ services) -caveat: this repository has a really high churn. Beware
- 26. Growing Schemas Spec-ulation Please watch Rich Hickey’s talk at Clojure Conj 2016 Spec-ulation: https://www.youtube.com/watch?v=oyLBGkS5ICk
- 27. Ports and Adapters (a.k.a Hexagonal Architecture) SOUTHEAST BRAZIL REGION FROM SPACE
- 28. Ports and Adapters Definition Core logic is independent to how we can call it (yellow) A port is an entry-point of the application (blue) An adapter is the bridge between a port and the core logic (red) http://www.dossier-andreas.net/software_architecture/ports_and_adapters.html http://alistair.cockburn.us/Hexagonal+architecture
- 29. Ports and Adapters (Nubank version) Extended Definition Pure business logic (green) Controller logic wires the flow between the ports (yellow) A port is an entry-point of the application (blue) An adapter is the bridge between a port and the core logic (red)
- 30. Ports (Components) Ports and Adapters -Ports are initialised at startup -Each port has a corresponding component -Serializes data to a transport format (e.g JSON, Transit) -Usually library code shared by all services -Tested via integration tests HTTP Kafka Datomic File Storage Metrics E-mail
- 31. Adapters (Diplomat) Ports and Adapters -Adapters are the interface to ports -Contain HTTP and Kafka consumer handlers -Adapt wire schema to internal schema -Calls and is called by controller functions -Tested with fake versions of the port components, or mocks HTTP Kafka Datomic File Storage Metrics E-mail
- 32. Controllers Ports and Adapters -Controllers wires the flow between entry-point and the side effects -Only deals with internal schemas -Delegates business logic to pure functions -Composes side effect results -Tested mostly with mocks HTTP Kafka Datomic File Storage Metrics E-mail
- 33. Business Logic Ports and Adapters -Handles and transforms immutable data -Pure functions -Best place to enforce invariants and type checks (e.g using clojure.spec) -Can be tested using generative testing -Should be the largest part of the application HTTP Kafka Datomic File Storage Metrics E-mail
- 34. Microservices Ports and Adapters -Each service follows about the same design -Services communicate with each other using one of the ports (e.g HTTP or Kafka) -Services DON’T share databases -HTTP responses contain hypermedia, so we can replace a service without having to change clients -Tested with end to end tests, with all services deployed
- 35. Clojure is simple Keep your design simple Keep your architecture simple SÃO PAULO, BRASIL