Designing scalable application: from umbrella project to distributed system - Anton Mishchuk
1 like325 views
This document discusses designing scalable applications from an umbrella project to a distributed system. It presents a machine learning tools demo project with four applications organized under an umbrella project. It describes using interfaces modules to encapsulate applications and enable inter-application communication as the system scales to multiple nodes. Example communication methods covered include RPC for synchronous calls, distributed tasks for asynchronous work, and HTTP for a third-party API. The document also discusses limiting concurrency using a poolboy worker pool.
![Main application
defp deps do
[
{:datasets, in_umbrella: true},
{:models, in_umbrella: true},
{:utils, in_umbrella: true}
]
end
13](https://image.slidesharecdn.com/designingscalableapplication-fromumbrellaprojecttodistributedsystem-171025152806/85/Designing-scalable-application-from-umbrella-project-to-distributed-system-Anton-Mishchuk-13-320.jpg)
![ModelsInterface.Rf
defmodule ModelsInterface.Rf do
def fit_model(data) do
ModelsInterface.remote_call(
Models.Interfaces.Rf,
:fit_model,
[data]
)
end
end
26](https://image.slidesharecdn.com/designingscalableapplication-fromumbrellaprojecttodistributedsystem-171025152806/85/Designing-scalable-application-from-umbrella-project-to-distributed-system-Anton-Mishchuk-26-320.jpg)
![What about tests
defp deps do
[
...
{:models, in_umbrella: true, only: [:test]},
{:models_interface, in_umbrella: true},
...
]
end
30](https://image.slidesharecdn.com/designingscalableapplication-fromumbrellaprojecttodistributedsystem-171025152806/85/Designing-scalable-application-from-umbrella-project-to-distributed-system-Anton-Mishchuk-30-320.jpg)
![Start Supervisor
defmodule Datasets.Application do
...
def start(_type, _args) do
children = [
supervisor(Task.Supervisor,
[[name: Datasets.Task.Supervisor]],
[restart: :temporary, shutdown: 10000])
]
...
end
end 33](https://image.slidesharecdn.com/designingscalableapplication-fromumbrellaprojecttodistributedsystem-171025152806/85/Designing-scalable-application-from-umbrella-project-to-distributed-system-Anton-Mishchuk-33-320.jpg)
![def start(_type, _args) do
pool_opts = [
name: {:local, Models.Interface},
worker_module: Models.Interfaces.Worker,
size: 5, max_overflow: 5]
children = [
:poolboy.child_spec(Models.Interface, pool_opts, []),
]
end
poolboy in
Models.Application
39](https://image.slidesharecdn.com/designingscalableapplication-fromumbrellaprojecttodistributedsystem-171025152806/85/Designing-scalable-application-from-umbrella-project-to-distributed-system-Anton-Mishchuk-39-320.jpg)
![defmodule Models.Interfaces.Rf do
def fit_model(data) do
with_poolboy({Models.Rf, :fit_model, [data]})
end
def with_poolboy(args) do
worker = :poolboy.checkout(Models.Interface)
result = GenServer.call(worker, args, :infinity)
:poolboy.checkin(Models.Interface, worker)
result
end
end
Models.Interfaces.Rf
41](https://image.slidesharecdn.com/designingscalableapplication-fromumbrellaprojecttodistributedsystem-171025152806/85/Designing-scalable-application-from-umbrella-project-to-distributed-system-Anton-Mishchuk-41-320.jpg)
Designing scalable application: from umbrella project to distributed system - Anton Mishchuk
- 1. Designing scalable application: from umbrella project to distributed system Elixir Club 8, Kyiv, October 21, 2017
- 2. HELLO! I am Anton Mishchuk - Ruby/Elixir developer at Matic Insurance Services we are hiring! - A big fan of Elixir programming language - Author and maintainer of: ESpec and Flowex libraries github: antonmi 2
- 3. BEAM Applications BEAM VM Application GS S GS GS GS Application GS S GS GS GS Application GS S GS GS GS 3
- 4. Intro ○ Monolith or micro-services ○ BEAM approach ○ Umbrella project ○ Inter-service communication 4
- 5. Contents ○ “ML Tools” demo project ○ Three levels of code organization ○ Interfaces modules ○ Scaling to distributed system ○ Limiting concurrency 5
- 6. ML Tools ○ Stands for “Machine Learning Tools” ○ One can find it here: □ https://github.com/antonmi/ml_tools ○ Umbrella project with 4 applications: □ Datasets □ Models □ Utils □ Main 6
- 7. Umbrella project Monolith and microservices at the same time: ○ All applications are in one place ○ Easy to develop and test ○ Service-oriented architecture ○ Ready to scale 7
- 8. Code organization ○ Service level ○ Context level ○ Implementation level 8
- 9. Service level Main Datasets Models Utils 9
- 10. Context and implementation (datasets) datasets/ lib/ datasets/ fetchers/ fetchers.ex aws.ex kaggle.ex collections/ ○ Datasets.Fetchers ○ Datasets.Fetchers.Aws ○ Datasets.Fetchers.Kaggle 10
- 11. Context and implementation (models) models/ lib/ models/ lm/ lm.ex engine.ex rf/ rf.ex engine.ex ○ Models.Lm ○ Models.Lm.Engine ○ Models.Rf ○ Models.Rf.Engine 11
- 12. Context and implementation (utils) utils/ lib/ utils/ pre_processing/ pre_processing.ex filter.ex normalizer.ex visualization/ ... ○ Utils.PreProcessing ○ Utils.PreProcessing.Filter ○ Utils.PreProcessing.Normalizer 12
- 13. Main application defp deps do [ {:datasets, in_umbrella: true}, {:models, in_umbrella: true}, {:utils, in_umbrella: true} ] end 13
- 14. Main application defmodule Main.Zillow do def rf_fit do Datasets.Fetchers.zillow_data |> Utils.PrePorcessing.normalize_data |> Models.Rf.fit_model end end 14
- 15. What’s wrong with this code? defmodule Main.Zillow do def rf_fit do Datasets.Fetchers.zillow_data |> Utils.PreProcessing.normalize_data |> Models.Rf.fit_model end end 15
- 16. Encapsulation problem ○ Modules, functions and that’s it! ○ Main application has access to all public functions in all the applications ○ Micro-services -> tightly coupled monolith 16
- 17. Interfaces Modules ○ Specific modules in the application that implements public interface ○ Grouped by context ○ Other application must use only functions from this modules 17
- 18. Just delegate defmodule Datasets.Interfaces.Fetchers do alias Datasets.Fetchers defdelegate zillow_data, to: Fetchers defdelegate landsat_data, to: Fetchers end 18
- 19. Use Interfaces only! def rf_fit do Datasets.Interfaces.Fetchers.zillow_data |> Utils.Interfaces.PreProcessing.normalize_data |> Models.Interfaces.Rf.fit_model end 19
- 20. Interfaces Modules ○ It’s just a convention ○ But very important convention ○ It creates a good basis for future scaling 20
- 21. Let’s scale! ○ We decide to run each of the applications on a different node ○ Modules from one application will be not accessible in another one ○ But we still want keep all the things in one place 21
- 22. Interface applications Main Datasets Models Utils DatasetsInterface ModelsInterface UtilsInterface 22
- 24. Inter-Application Communication ○ RPC (for ‘models’ app) ○ Distributed tasks (for ‘datasets’ app) ○ HTTP (for ‘utils’ app) 24
- 25. RPC ○ Remote Procedure Call ○ :rpc.call(node, module, fun, args) ○ BEAM handles all the serialization and deserialization stuff 25
- 26. ModelsInterface.Rf defmodule ModelsInterface.Rf do def fit_model(data) do ModelsInterface.remote_call( Models.Interfaces.Rf, :fit_model, [data] ) end end 26
- 27. ModelsInterface defmodule ModelsInterface do def remote_call(module, fun, args, env Mix.env) do do_remote_call({module, fun, args}, env) end defp do_remote_call({module, fun, args}, :test) do apply(module, fun, args) end defp do_remote_call({module, fun, args}, _) do :rpc.call(remote_node(), module, fun, args) end end 27
- 28. Quick refactoring ○ Replace Models.Interfaces with ModelsInterface and that’s it ○ You continue develop with the same speed ○ Nothing to change in tests 28
- 29. What about tests defp do_remote_call({module, fun, args}, :test) do apply(module, fun, args) end defp do_remote_call({module, fun, args}, _) do :rpc.call(remote_node(), module, fun, args) end 29
- 30. What about tests defp deps do [ ... {:models, in_umbrella: true, only: [:test]}, {:models_interface, in_umbrella: true}, ... ] end 30
- 31. RPC cons ○ Synchronous calls from “local” process ○ Needs additional logic for asynchronous execution 31
- 32. Distributed tasks ○ Build on top of Elixir Task ○ Task are spawned on remote node ○ Supervisor controls the tasks 32
- 33. Start Supervisor defmodule Datasets.Application do ... def start(_type, _args) do children = [ supervisor(Task.Supervisor, [[name: Datasets.Task.Supervisor]], [restart: :temporary, shutdown: 10000]) ] ... end end 33
- 34. DatasetsInterface defmodule DatasetsInterface do ... defp do_spawn_task({module, fun, args}, _) do Task.Supervisor.async(remote_supervisor(), module, fun, args) |> Task.await end defp remote_supervisor, do: #config data end 34
- 35. Erlang Distribution Protocol Pros: ○ Battle-tested over decades ○ Free serialization / deserialization Cons: ○ Not very secure ○ No very fast ○ BEAM specific 35
- 36. HTTP UtilsInterface defmodule UtilsInterface do ... defp do_remote_call({module, fun, args}, _) do {:ok, resp} = HTTPoison.post(remote_url(), serialize({module, fun, args})) deserialize(resp.body) end ... end 36
- 37. Plug in Utils app defmodule Utils.Interfaces.Plug do use Plug.Router ... post "/remote" do {:ok, body, conn} = Plug.Conn.read_body(conn) {module, fun, args} = deserialize(body) result = apply(module, fun, args) send_resp(conn, 200, serialize(result)) end end 37
- 38. Limiting concurrency When do you need this? ○ Third-party API rate limits ○ Heavy calculations per request 38
- 39. def start(_type, _args) do pool_opts = [ name: {:local, Models.Interface}, worker_module: Models.Interfaces.Worker, size: 5, max_overflow: 5] children = [ :poolboy.child_spec(Models.Interface, pool_opts, []), ] end poolboy in Models.Application 39
- 40. defmodule Models.Interfaces.Worker do use GenServer ... def handle_call({module, fun, args}, _from, state) do result = apply(module, fun, args) {:reply, result, state} end end Models.Interfaces.Worker 40
- 41. defmodule Models.Interfaces.Rf do def fit_model(data) do with_poolboy({Models.Rf, :fit_model, [data]}) end def with_poolboy(args) do worker = :poolboy.checkout(Models.Interface) result = GenServer.call(worker, args, :infinity) :poolboy.checkin(Models.Interface, worker) result end end Models.Interfaces.Rf 41
- 42. Conclusion ○ Start with micro-services from the very beginning ○ Think carefully about app interface ○ Use EDP for communication when scale ○ Keep “communication” code in separate app 42