Distributed computing for new bloods
3 likes887 views
The document discusses the fundamentals of distributed systems, emphasizing their importance in overcoming challenges related to scalability and availability. It highlights key theoretical concepts such as the FLP impossibility result and the CAP theorem, which inform design decisions in distributed computing. Additionally, it critiques common fallacies and past attempts at distributed computing, urging the need for understanding failures and designing systems that can tolerate them.
![Partitioning Replication
A B C
A B A B
[0..100] [0..100] [101..200] [101..200]
C C
Node 1 Node 2](https://image.slidesharecdn.com/s2y8ffo3s8stjgdqww1l-signature-8a5b546a629363c3d21f39adc8ab96ff43c2443daec5953a386943a01ef9356b-poli-141018230159-conversion-gate01/85/Distributed-computing-for-new-bloods-51-320.jpg)
Distributed computing for new bloods
- 1. DISTRIBUTED COMPUTING FOR NEW BLOODS RAYMOND TAY I WORK IN HEWLETT-PACKARD LABS SINGAPORE @RAYMONDTAYBL
- 2. What is a Distributed System?
- 3. What is a Distributed System? It’s been there for a long time
- 4. What is a Distributed System? It’s been there for a long time You did not realize it
- 5. DISTRIBUTED SYSTEMS ARE THE NEW FRONTIER whether YOU like it or not
- 6. DISTRIBUTED SYSTEMS ARE THE NEW FRONTIER whether YOU like it or not LAN Games Mobile Databases ATMs Social Media
- 7. WHAT IS THE ESSENCE OF DISTRIBUTED SYSTEMS?
- 8. WHAT IS THE ESSENCE OF DISTRIBUTED SYSTEMS? IT IS ABOUT THE ATTEMPT TO OVERCOME INFORMATION TRAVEL WHEN INDEPENDENT PROCESSES FAIL
- 9. Information flows at speed of light ! MESSAGE X Y
- 10. When independent things DONT fail. I’ve sent it. I’ve received it. X Y I’ve received it. I’ve sent it.
- 11. When independent things fail, independently. X Y NETWORK FAILURE
- 12. When independent things fail, independently. X Y NODE FAILURE
- 13. There is no difference between a slow node and a dead node
- 14. Why do we NEED it? Scalability when the needs of the system outgrows what a single node can provide
- 15. Why do we NEED it? Scalability when the needs of the system outgrows what a single node can provide Availability Enabling resilience when a node fails
- 16. IS IT REALLY THAT HARD?
- 17. IS IT REALLY THAT HARD? The answer lies in knowing what is possible and what is not possible…
- 18. IS IT REALLY THAT HARD? Even widely accepted facts are challenged…
- 19. THE FALLACIES OF DISTRIBUTED COMPUTING The network is reliable Peter Deutsch and other fellows at Sun Microsystems
- 20. THE FALLACIES OF DISTRIBUTED COMPUTING The network is reliable
- 21. THE FALLACIES OF DISTRIBUTED COMPUTING The network is reliable The network is secure Peter Deutsch and other fellows at Sun Microsystems
- 22. THE FALLACIES OF DISTRIBUTED COMPUTING The network is reliable The network is secure 2013 cost of cyber crime study: United States
- 23. THE FALLACIES OF DISTRIBUTED COMPUTING The network is reliable Peter Deutsch and The network is secure other fellows at Sun The network is Microsystems homogeneous
- 24. THE FALLACIES OF DISTRIBUTED COMPUTING The network is reliable Peter Deutsch and The network is secure other fellows at Sun The network is Microsystems homogeneous Latency is zero Ingo Rammer on latency vs bandwidth
- 25. THE FALLACIES OF DISTRIBUTED COMPUTING The network is reliable Peter Deutsch and The network is secure other fellows at Sun The network is Microsystems homogeneous Latency is zero Bandwidth is infinite
- 26. THE FALLACIES OF DISTRIBUTED COMPUTING The network is reliable Peter Deutsch and The network is secure other fellows The network is at Sun homogeneous Microsystems Latency is zero Bandwidth is infinite Topology doesn’t change
- 27. THE FALLACIES OF DISTRIBUTED COMPUTING The network is reliable The network is secure Peter Deutsch and The network is other fellows homogeneous at Sun Microsystems Latency is zero Bandwidth is infinite Topology doesn’t change There is one administrator
- 28. THE FALLACIES OF DISTRIBUTED COMPUTING The network is reliable The network is secure Peter Deutsch and The network is homogeneous other fellows at Sun Latency is zero Microsystems Bandwidth is infinite Topology doesn’t change There is one administrator Transport cost is zero
- 29. THE FALLACIES OF DISTRIBUTED COMPUTING The network is reliable The network is secure Peter Deutsch and The network is homogeneous other fellows at Sun Latency is zero Microsystems Bandwidth is infinite Topology doesn’t change There is one administrator Transport cost is zero
- 30. WHAT WAS TRIED IN THE PAST? DISTRIBUTED OBJECTS REMOTE PROCEDURE CALL DISTRIBUTED SHARED MUTABLE STATE
- 31. WHAT WAS ATTEMPTED? DISTRIBUTED OBJECTS REMOTE PROCEDURE CALL DISTRIBUTED SHARED MUTABLE STATE • Typically, programming constructs to “abstract” the fact that there are local and distributed objects • Ignores latencies • Handles failures with this attitude → “i dunno what to do…YOU (i.e. invoker) handle it”.
- 32. WHAT WAS ATTEMPTED? DISTRIBUTED OBJECTS REMOTE PROCEDURE CALL DISTRIBUTED SHARED MUTABLE STATE • Assumes the synchronous processing model • Asynchronous RPC tries to model after synchronous RPC…
- 33. WHAT WAS ATTEMPTED? DISTRIBUTED OBJECTS REMOTE PROCEDURE CALL DISTRIBUTED SHARED MUTABLE STATE • Found in Distributed Shared Memory Systems i.e. “1” address space partitioned into “x” address spaces for “x” nodes • e.g. JavaSpaces ⇒ Danger of 2 independent processes to commit successfully.
- 34. YOU CAN APPRECIATE THAT IT IS VERY HARD TO SOLVE IN ITS ENTIRETY
- 35. The question really is “How can we do better?”
- 36. The question really is “How can we do better?” To start,we need to understand 2 results
- 37. FLP IMPOSSIBILITY RESULT The FLP result shows that in an asynchronous model, where one processor might crash, there is no distributed algorithm to solve the consensus problem.
- 38. Consensus is a fundamental problem in fault-tolerant distributed systems. Consensus involves multiple servers agreeing on values. Once they reach a decision on a value, that decision is final. Typical consensus algorithms make progress when any majority of their servers are available; for example, a cluster of 5 servers can continue to operate even if 2 servers fail. If more servers fail, they stop making progress (but will never return an incorrect result). Paxos (1989), Raft (2013)
- 39. CAP THEOREM CAP CONJECTURE (2000) established as a theorem in 2002 Consistency Availability Partition Tolerance
- 40. CAP THEOREM CAP CONJECTURE (2000) established as a theorem in 2002 Consistency Availability Partition Tolerance • Consistency ⇒ all nodes should see the same data, eventually
- 41. CAP THEOREM CAP CONJECTURE (2000) established as a theorem in 2002 Consistency Availability Partition Tolerance • Consistency ⇒ all nodes should see the same data, eventually • Availability ⇒ System is still working when node(s) fails
- 42. CAP THEOREM CAP CONJECTURE (2000) established as a theorem in 2002 Consistency Availability Partition Tolerance • Consistency ⇒ all nodes should see the same data, eventually • Availability ⇒ System is still working when node(s) fails
- 43. CAP THEOREM CAP CONJECTURE (2000) established as a theorem in 2002 Consistency Availability Partition Tolerance • Consistency ⇒ all nodes should see the same data, eventually • Availability ⇒ System is still working when node(s) fails • Partition-Tolerance ⇒ System is still working on arbitrary message loss
- 44. How does knowing FLP and CAP help me?
- 45. How does knowing FLP and CAP help me? It’s really about asking which would you sacrifice when a system fails? Consistency or Availability?
- 46. You can choose C over A It needs to preserve reads and writes i.e. linearizability i.e. A read will return the last completed write (on ANY replica) e.g. Two-Phase-Commit
- 47. You can choose A over C It might return stale reads …
- 48. You can choose A over C It might return stale reads … DynamoDB uses vector clocks; Cassandra uses a clever form of last-write-wins
- 49. You cannot NOT choose P Partition Tolerance is mandatory in distributed systems
- 50. To scale, partition To resilient, replicate
- 51. Partitioning Replication A B C A B A B [0..100] [0..100] [101..200] [101..200] C C Node 1 Node 2
- 52. Replication is strongly related to fault-tolerance Fault-tolerance relies on reaching consensus Paxos (1989), ZAB, Raft (2013)
- 53. Consistent Hashing is used often PARTITIONING and REPLICATION C-H commonly used in load-balancing web objects. In DynamoDB Cassandra, Riak and Memcached
- 54. • If your problem can fit into memory of a single machine, you don’t need a distributed system • If you really need to design / build a distributed system, the following helps: • Design for failure • Use the FLP and CAP to critique systems • Algorithms that work for single-node may not work in distributed mode • Avoid coordination of nodes • Learn to estimate your capacity
- 55. Jeff Dean - Google
- 56. REFERENCES • http://queue.acm.org/detail.cfm?id=2655736 “The network is reliable” • http://cacm.acm.org/blogs/blog-cacm/83396-errors-in-database-systems- eventual-consistency-and-the-cap-theorem/fulltext • http://mvdirona.com/jrh/talksAndPapers/JamesRH_Lisa.pdf • “Towards Robust Distributed Systems,” http:// www.cs.berkeley.edu/~brewer/cs262b-2004/PODC-keynote.pdf • “Why Do Computers Stop and What Can be Done About It,” Tandem Computers Technical Report 85.7, Cupertino, Ca., 1985. http://www.hpl.hp.com/techreports/tandem/TR-85.7.pdf • http://www.cs.cornell.edu/projects/ladis2009/talks/dean-keynote- ladis2009.pdf • http://codahale.com/you-cant-sacrifice-partition-tolerance/ • http://dl.acm.org/citation.cfm?id=258660 “Consistent hashing and random trees: distributed caching protocols for relieving hot spots on the World Wide Web”