Consistency without Consensus: CRDTs in Production at SoundCloud
5 likes2,230 views
The document discusses the implementation of Conflict-free Replicated Data Types (CRDTs) in production at SoundCloud, emphasizing the balance between consistency and availability in distributed systems. It highlights challenges in distributed programming and introduces various CRDTs, explaining their operations such as insertion and deletion in detail. The conclusion suggests adopting CRDTs to embrace invariants while adapting to the unique challenges of distributed systems.
![Fan in on read
༼⍢༽ •••••
(•ི̛ᴗ•̛)ྀ
༼•͟ ͜•༽
(ಠ_ಠ)
[¬º-°] •••
••••••](https://image.slidesharecdn.com/untitled-150223094458-conversion-gate01/85/Consistency-without-Consensus-CRDTs-in-Production-at-SoundCloud-49-320.jpg)
Consistency without Consensus: CRDTs in Production at SoundCloud
- 1. Consistency without consensus: CRDTs in production at SoundCloud
- 2. InfoQ.com: News & Community Site • 750,000 unique visitors/month • Published in 4 languages (English, Chinese, Japanese and Brazilian Portuguese) • Post content from our QCon conferences • News 15-20 / week • Articles 3-4 / week • Presentations (videos) 12-15 / week • Interviews 2-3 / week • Books 1 / month Watch the video with slide synchronization on InfoQ.com! http://www.infoq.com/presentations /crdt-soundcloud
- 3. Purpose of QCon - to empower software development by facilitating the spread of knowledge and innovation Strategy - practitioner-driven conference designed for YOU: influencers of change and innovation in your teams - speakers and topics driving the evolution and innovation - connecting and catalyzing the influencers and innovators Highlights - attended by more than 12,000 delegates since 2007 - held in 9 cities worldwide Presented at QCon San Francisco www.qconsf.com
- 4. Consistency without consensus: CRDTs in production at SoundCloud
- 5. Me Some guy Embedded, sensor networks Distributed systems SoundCloud infrastructure
- 6. Theory
- 7. Distributed programming “The art of solving the same problem that you can solve on a single computer using multiple computers.” —book.mixu.net
- 8. Distributed programming “Generally a bad idea, best avoided.” —me
- 9. >>> x = 1 >>> print x 1
- 10. $ curl -XPOST -d'{"val": 1}' http://db/vars/x HTTP 502 Bad Gateway $ curl -XGET http://db/vars/x HTTP 503 Service Unavailable
- 11. Idioms
- 12. 1980s — RPC
- 13. 1990s — CORBA
- 14. 2000 — CAP
- 15. Partition tolerance “The system continues to operate despite message loss due to network and/or node failure.” —book.mixu.net
- 17. CP Chubby, Doozer — Paxos ZooKeeper — Zab Consul, etcd — Raft ? — Viewstamped Replication
- 24. A' A → A'
- 25. + (1 + 2) + 3 = 1 + (2 + 3) 1 + 2 = 2 + 1 1 + 1 ≠ 1
- 26. ∪ ( 1 ∪ 2 ) ∪ 3 = 1 ∪ ( 2 ∪ 3 ) 1 ∪ 2 = 2 ∪ 1 1 ∪ 1 = 1
- 27. { } { } { }
- 28. { } { } { } 123
- 29. { } { } { } 123 123
- 30. { } { } { } 123 123
- 31. { } { } { } 123 123 123
- 32. { } { } { } 123 123 123
- 33. { } { } { } 123 123 123 456
- 34. { } { } { } 123 123 123, 456 456
- 35. { } { } { } 123 123 123, 456 ✘
- 36. { } { } { } 123 123, 456 123, 456
- 37. { } { } { } 123 123, 456 123, 456
- 38. Read {123, 456} ∪ {123} ∪ {123, 456} = {123, 456} {123, 456} ∆ {123} ∆ {123, 456} = {456}
- 39. { } { } { } 123 123, 456 123, 456 456
- 40. { } { } { } 123, 456 123, 456 123, 456
- 41. { } { } { } 123, 456 123, 456 123, 456
- 42. { } { } { } 123, 456 123, 456 123, 456
- 43. Interlude — Bending the problem
- 48. Fan out on write ༼⍢༽ ● ●●●● (•ི̛ᴗ•̛)ྀ ●●● ༼•͟ ͜•༽ ●●●●● (ಠ_ಠ) ● ● ● ●
- 49. Fan in on read ༼⍢༽ ••••• (•ི̛ᴗ•̛)ྀ ༼•͟ ͜•༽ (ಠ_ಠ) [¬º-°] ••• ••••••
- 50. Unique events — use a set G-set — can’t delete 2P-set — add, remove once OR-set — storage overhead
- 51. A wild set appears
- 52. Roshi set S+ { A/1 B/2 C/3 } S– { D/4 } S { A B C }
- 53. Roshi set S = actor’s outbox key snoopdogg·outbox A/B/C/D = actor+verb+thing snoopdogg·repost·theeconomist/election-day 1/2/3 = timestamp 2014-04-01T15:16:17.187Z
- 54. Reading is easy
- 56. Insert • If either key+ or key– already contains element, and the existing score >= score, no-op and exit. • Insert (element, score) into add set key+. • Delete (element) from remove set key–.
- 57. Delete • If either key+ or key– already contains element, and the existing score >= score, no-op and exit. • Insert (element, score) into add set key–. • Delete (element) from remove set key+.
- 58. Example
- 59. S+ { A/1 B/2 } S– { C/3 }
- 60. S+ { A/1 B/2 } S– { C/3 } Insert D/4
- 61. S+ { A/1 B/2 D/4 } S– { C/3 } Insert D/4
- 62. S+ { A/1 B/2 D/4 } S– { C/3 }
- 63. S+ { A/1 B/2 D/4 } S– { C/3 } Insert D/4
- 64. S+ { A/1 B/2 D/4 } S– { C/3 } Insert D/4
- 65. S+ { A/1 B/2 D/4 } S– { C/3 }
- 66. S+ { A/1 B/2 D/4 } S– { C/3 } Delete D/3
- 67. S+ { A/1 B/2 D/4 } S– { C/3 } Delete D/3
- 68. S+ { A/1 B/2 D/4 } S– { C/3 }
- 69. S+ { A/1 B/2 D/4 } S– { C/3 } Delete D/5
- 70. S+ { A/1 B/2 D/4 } S– { C/3 D/5 } Delete D/5
- 71. S+ { A/1 B/2 } S– { C/3 D/5 }
- 72. S+ { A/1 B/2 } S– { C/3 D/5 } Delete D/6
- 73. S+ { A/1 B/2 } S– { C/3 D/6 } Delete D/6
- 74. S+ { A/1 B/2 } S– { C/3 D/6 }
- 75. Making it real
- 77. Pool Cluster
- 79. Writing is easy
- 83. ClusterCluster Cluster {A B C} {A C} {A B C} ∪ = {A B C} ∆ = {B}
- 86. In conclusion,
- 87. Consistency without consensus = CRDT. Embrace your invariants. Maybe bend your problem, not your solution.
- 89. Watch the video with slide synchronization on InfoQ.com! http://www.infoq.com/presentations/crdt- soundcloud