Practical Fault Tolerance in Elixir - Alexei Sholik
1 like308 views
This document provides an overview of practical fault tolerance in Elixir. It discusses why fault tolerance is important, and how Elixir and Erlang communities focus on it. It demonstrates fault tolerance techniques using processes, links, monitors, tasks, supervisors, and durable message queues. Other tools like remote error tracking, exponential backoff, and alternative supervisors are also covered. The key messages are to trust OTP patterns for fault tolerance but also leverage other tools, anticipate failures, isolate failures, and follow the principles of failing fast and restarting.
![def call(process, request, timeout) do
monitor = Process.monitor(process)xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
send(process, {:"$gen_call", {self(), monitor}, request})
receive
{^monitor, reply} ->
Process.demonitor(monitor, [:flush])
{:ok, reply}
{:DOWN, ^monitor, _, _, reason} ->xxxxxxxxxxxxxxxxxxxxxxxxxxxx
exit(reason)
after timeout ->
Process.demonitor(monitor, [:flush])
exit(:timeout)
end
end](https://image.slidesharecdn.com/practicalfaulttoleranceinelixirupdated-180323134737/85/Practical-Fault-Tolerance-in-Elixir-Alexei-Sholik-20-320.jpg)
Practical Fault Tolerance in Elixir - Alexei Sholik
- 1. Practical Fault Tolerance in Elixir Alexei Sholik Elixir Club Kyiv, 17 Mar 2018
- 2. About me Backend engineer at Contractbook.co. Co-host at BeamEaters podcast. Contributor to Elixir. github.com/alco
- 3. What is fault tolerance?
- 6. Why do only Erlang/Elixir communities seem to care about it?
- 8. “Let it crash” is not the full story
- 9. Fail fast → restart → try again
- 11. Building blocks of fault tolerance
- 12. Process
- 15. Error
- 16. Link
- 18. Monitor
- 20. def call(process, request, timeout) do monitor = Process.monitor(process)xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx send(process, {:"$gen_call", {self(), monitor}, request}) receive {^monitor, reply} -> Process.demonitor(monitor, [:flush]) {:ok, reply} {:DOWN, ^monitor, _, _, reason} ->xxxxxxxxxxxxxxxxxxxxxxxxxxxx exit(reason) after timeout -> Process.demonitor(monitor, [:flush]) exit(:timeout) end end
- 21. Task
- 22. Supervisor
- 23. Bohrbug
- 24. Heisenbug
- 26. ... DB SuperDocs.Web.Endpoint <request process> <db_connection process> Ecto’s connection pool supervisor
- 29. DB SuperDocs.Web.Endpoint <request process> <db_connection process> RabbitMQ SuperDocs.TaskSup ImportService.import_documents_for AMQPClient AMQPWorker AMQPWorker ...
- 30. Other tools and techniques
- 31. Remote error tracking Sentry, Rollbar, log aggregators
- 34. And so on...
- 35. Recap ● Trust in OTP… ● ...but don’t dismiss other tools ● Anticipate failures ● Isolate failures ● Fail fast, restart, try again
- 36. ● Why do computers fail and what can be done about it? ● Making reliable distributed systems in the presence of software errors ● It's About the Guarantees ● On Erlang, State and Crashes ● Error Kernels Reading material