Increasing performance with Elixir Tasks
- 1. Increasing performance with Elixir Tasks Jeffrey Chan Alembic Twitter: @jgmchan Github: https://github.com/jgmchan Email: jgmchan@gmail.com
- 2. Background ✤ Client product - Places students into classes ✤ Algorithm time took > 4 minutes (sometimes 10) ✤ Sequential number crunching to work out best solution ✤ Users were willing to wait but… ✤ Cloud service LB has a hard limit of 230s
- 3. Option 1 ✤ Scale vertically ✤ Stronger CPU ✤ But already had the largest CPU’s
- 4. Option 2 ✤ Scale Horizontally ✤ Efficiently use more of the CPU’s available ✤ Elixir Tasks
- 5. Option 3 (Bonus) ✤ Change the algorithm ✤ Actually the “real” correct solution ✤ Can’t get faster than not doing it
- 6. ElixirTask Module ✤ Module for spawning and awaiting tasks ✤ https://hexdocs.pm/elixir/Task.html ✤ Really easy to use (Erlang)
- 7. How they work ✤ Spawn separate Erlang processes to do work ✤ Wait for child process to send a message back with result ✤ Processes are linked
- 8. How they work ✤ Single Process ✤ Input →⛏ → ⛏ → ⛏ → 💰
- 9. How they work ✤ With Tasks ✤ P1: Input →🤰 → ☕ → 😴 → 📞 →💰 ⇣ P2: ⇢ 👶 →⛏ → ⇡ ⇣ P3: ⇢ 👶 → ⛏→ ⇡ ⇣ P4: ⇢ 👶 →⛏ → ⇡
- 10. Pre-requisites ✤ Tasks are short lived - usually isolated (no communication) ✤ Pure functions are best
- 11. Demo!
- 13. Massive Performance Improvement May as well use it everywhere 🤑 🤑 🤑
- 14. Massive Performance Improvement Or is it?
- 15. A Better Demo
- 16. Limitations ✤ Number of CPU’s (can use multiple machines) ✤ BEAM overhead of switching tasks
- 17. Conclusion ✤ Task module can be very handy and easy to parallelise your program ✤ Can use all the cores available in your machine (and other machines) ✤ Not the answer to all problems ✤ Always measure!
- 18. Thanks!