Modern Messaging for Distributed Systems
- 1. Modern Messaging for Distributed Systems Andrea Rabbaglietti
- 3. Agenda ● Who funk is Andrea Rabbaglietti ● Disclaimer ● Pre-requisites ○ Agentful vs. Agentless ○ Scale Up Vs. Scale Out ○ Sync Vs. Async ● IPC / Language interoperability ● Channel ○ HTTP RESTful ○ Custom ● Use Case: Facebook ● One (working & in-production) example
- 4. Who funk is Andrea Rabbaglietti ● Computer Science @ DIB alumnus ● ex-CTO Bauzaar.it ● co-founder Serverable.io ● founder JetDeploy.io ● Django Framework contributor ● Software/System Engineer ● Business coach & tutor
- 5. Context
- 8. Scalability
- 9. Scale up ● Make a single node as fast as possible ● Increase clock speed ● Add RAM ● Make disk I/O go faster
- 10. Scale out ● Make many nodes work together ● Manage the distributed nodes ● Let the system be fault-tolerant ● Learn how to divide your problems into independent subproblems But is that possible?
- 11. Stateful Vs. Stateless class MyClass: i = 12345 def f(self): return 'hello world: ' + self.i class MyClass: def f(self): i = 1234 return 'hello world: ' + self.i
- 13. Stateful
- 14. Sync Vs. Async
- 15. programs have an inherent attribute: each line executes in order Sync
- 16. Async
- 17. Async could reveal HELL
- 19. Inter-process Communication (IPC) ● File (or memory-mapped file) ● Signal (kill) ● Socket (unix socket) ● Message queue ● Pipe ● Shared Memory ● Message passing Legenda: Most operating-systems POSIX only
- 21. ○ RabbitMQ ○ ZeroMQ ○ Apache Thrift
- 22. Facebook
- 23. Actor Model Erlang VM User (you..) a2 a2 a3 a4 a5 a1 Java
- 24. Actor Model (2) ● No shared state ● Lightweight processes ● Asynchronous message-passing ● Mailboxes to buffer incoming messages
- 25. JetDeploy.io
- 26. Q & A
- 27. thank you so much guys Andrea Rabbaglietti rabbagliettiandrea@gmail.com GIT repo @ https://github.com/silverfix/gdgipc