![Research: Operational Transformation
“Unfortunately, implementing OT sucks. There's a
million algorithms with different tradeoffs, mostly
trapped in academic papers. The algorithms are really
hard and time consuming to implement correctly. […]
Wave took 2 years to write and if we rewrote it today, it
would take almost as long to write a second time.”
- Joseph Gentle
(Google Wave/ShareJS engineer)](https://image.slidesharecdn.com/sunnyconclavepresentruby-200524024028/85/Building-Conclave-a-decentralized-real-time-collaborative-text-editor-16-320.jpg)
![How to generate globally unique character?
A
Character Object
siteId “3kj7”
value “A”
position [8, 5]](https://image.slidesharecdn.com/sunnyconclavepresentruby-200524024028/85/Building-Conclave-a-decentralized-real-time-collaborative-text-editor-18-320.jpg)
![Relative and Unchanging
[ , , ]
Character
Object
siteId “abc4”
value “H”
position [8]
Character
Object
siteId “3kj7”
value “A”
position [8, 5]
Character
Object
siteId “h6f9”
value “T”
position [9]
insert(“A”, 1)](https://image.slidesharecdn.com/sunnyconclavepresentruby-200524024028/85/Building-Conclave-a-decentralized-real-time-collaborative-text-editor-19-320.jpg)
![H T
A
8
5
9
H T
8 9
A
8.5
Fractional Indices
H A T
[8] [8, 5] [9]](https://image.slidesharecdn.com/sunnyconclavepresentruby-200524024028/85/Building-Conclave-a-decentralized-real-time-collaborative-text-editor-23-320.jpg)
Building Conclave: a decentralized, real-time collaborative text editor
- 1. Building Conclave: a decentralized, real-time collaborative text editor Using WebRTC and CRDTs Presenter: Sun-Li Beatteay
- 2. Elise Olivares Nitin Savant Sun-Li Beatteay The Team
- 3. Overview 1. Text Editor Basics 2. Primary Challenges and Solutions 3. System Architecture 4. Optimizations 5. Future Plans
- 4. What is a text editor? ● List of characters ● Character ○ Value ○ Absolute Index ● Basic operations ○ Insert ○ Delete
- 5. HAT CHAT insert(“C”, 0) delete(1) CAT H A T 0 1 2 C H A T 0 1 2 3 C A T 0 1 2
- 6. Collaborative Text Editor ● Users in different locations editing a document at the same time ● Document replicated at each user’s location ● Changes applied locally immediately, then broadcasted to other users
- 7. Primary Challenges ● Merging conflicts to create a consistent document ● Reduce latency to achieve real time effect ● Scaling the network to house many users
- 8. Scenario: Two Users and Central Server User User Central Relay Server
- 9. User1 User2 HAT HAT CHAT AT insert(“C”, 0) delete(0) HAT CAT
- 10. User1 User2 HAT HAT AT AT delete(0) delete(0) T T
- 11. ● Commutativity ○ Insert + Delete = Delete + Insert ○ Operations converge regardless of order ● Idempotency ○ Repeated operations produce same result ● Can’t use absolute indices Consistency Requirements
- 12. ● Algorithm heavy ● Modifies operations based on the previous changes to the document. ● Usually requires a central server and single source of truth. Possible Solution: Operational Transformation
- 13. Possible Solution: Operational Transformation User1 User2 HAT HAT CHAT AT CAT CAT delete(1) insert(“C”, 0) insert(“C”, 0) delete(0) Commutativity OT
- 14. Possible Solution: Operational Transformation User1 User2 HAT HAT AT AT AT AT Already applied delete(0) delete(0) Idempotency Already applied OT
- 15. Possible Solution: Operational Transformation Google Wave EtherPadGoogle Docs
- 16. Research: Operational Transformation “Unfortunately, implementing OT sucks. There's a million algorithms with different tradeoffs, mostly trapped in academic papers. The algorithms are really hard and time consuming to implement correctly. […] Wave took 2 years to write and if we rewrote it today, it would take almost as long to write a second time.” - Joseph Gentle (Google Wave/ShareJS engineer)
- 17. Conflict-Free Replicated Data Type (CRDT) ● Created by researchers trying to simplify OT ● Complicated data structure with simpler algorithms ● Each user has a copy of the CRDT ○ No central server bottleneck ● Each character is treated as a unique object
- 18. How to generate globally unique character? A Character Object siteId “3kj7” value “A” position [8, 5]
- 19. Relative and Unchanging [ , , ] Character Object siteId “abc4” value “H” position [8] Character Object siteId “3kj7” value “A” position [8, 5] Character Object siteId “h6f9” value “T” position [9] insert(“A”, 1)
- 20. H T 8 9 Characters as Nodes in Tree
- 21. H T A 8 5 9 Characters as Nodes in Tree
- 22. H T A 8 5 9 H T 8 9 A 8.5 Fractional Indices
- 23. H T A 8 5 9 H T 8 9 A 8.5 Fractional Indices H A T [8] [8, 5] [9]
- 25. C A T 1 2 3 CRDT User1 User2 delete(“C”, 1) C A T 1 2 3 C H A T 1 1.5 2 3 A T 2 3 Insert(”H”, 1.5)delete(“C”, 1) H A T 1.5 2 3 H A T 1.5 2 3 Insert(”H”, 1.5) Commutativity
- 26. H A T 1 2 3 CRDT User1 User2 delete(”H”, 1) delete(“H”, 1) Idempotency H A T 1 2 3 A T 2 3 A T 2 3 A T 2 3 A T 2 3
- 27. Primary Challenges ● Merging conflicts to create a consistent document ● Reduce latency to achieve real time effect ● Scaling the network to house many users
- 29. Limitations of Central Server ● Potentially high latency between users
- 31. Limitations of Central Server ● Potentially high latency between users ● Scaling requires additional resources and money
- 32. Limitations of Central Server ● Potentially high latency between users ● Scaling requires additional resources and money ● Single point-of-failure
- 33. Peer-to-Peer Architecture Peer 1 Peer 2 Peer 4 Peer 3 Peer 5 Peer 6 Peer 7
- 34. WebRTC ● Protocol designed for real-time communication
- 36. WebRTC ● Protocol designed for real-time communication ● Requires signaling server to initiate connections between users ● UDP does not guarantee in-order delivery
- 37. Peer 1 Peer 2 Peer 3
- 38. Peer 1 Peer 2 Peer 3 Insert “A”
- 39. Insert “A” Peer 1 Peer 2 Peer 3 Delete “A”
- 40. Insert Op. Delete Op. Peer 1 Peer 2 Peer 3
- 41. Insert Op. Delete Op. What if the delete arrives first?Peer 1 Peer 2 Peer 3
- 42. ● Strategy to track operations received from each user and maintain order ● Each operation includes “version” (siteId, counter) Version Vector and Deletion Buffer
- 43. Peer 1 Peer 2 Peer 3 Peer Op # 1 23 2 5 Version Vector Insert Op - SiteId: 1 - Counter: 24 Delete Op - SiteId: 2 - Counter: 6
- 44. Delete Op - SiteId: 2 - Counter: 6 Peer 1 Peer 2 Peer 3 Peer Op # 1 23 2 5 Version Vector Insert Char - SiteId: 1 - Counter: 24 Deletion Buffer
- 45. Delete Op - SiteId: 2 - Counter: 6 Peer 1 Peer 2 Peer 3 Peer Op # 1 23 2 5 Version Vector Insert Char - SiteId: 1 - Counter: 24 Deletion Buffer Char - SiteId: 1 - Counter: 24 Process Buffer
- 46. Delete Op - SiteId: 2 - Counter: 6 Peer 1 Peer 2 Peer 3 Peer Op # 1 24 2 5 Version Vector Insert Char - SiteId: 1 - Counter: 24 Deletion Buffer Char - SiteId: 1 - Counter: 24 Process Buffer
- 47. Delete Op - SiteId: 2 - Counter: 6 Peer 1 Peer 2 Peer 3 Peer Op # 1 24 2 6 Version Vector Insert Char - SiteId: 1 - Counter: 24 Deletion Buffer Char - SiteId: 1 - Counter: 24 Process Buffer
- 48. Version Vector and Deletion Buffer
- 50. Optimizations
- 51. CRDT Structure and Performance
- 52. Initial CRDT (Linear Array) ● Inefficient for big documents ● Slow communication between CRDT and Editor { line: 2, ch: 8} 30
- 53. CRDT as a Two-Dimensional Array
- 54. Search Insert/Delete Find Index Linear Array O(log N) O(N) O(N) Two Dimensional Array O(log L + log C) O(C) O(1) Linear Array vs. Two-Dimensional Arrays N: # of Characters in Document L: # of Lines in Document C: # of Characters in line
- 55. Linear array vs. Array of arrays Total Time (ms) # of Operations Linear Array Two-Dimen Array
- 57. Peer 1 Peer 2 Peer 3
- 58. Peer 2 Peer 3
- 59. ● Each peer maintains a list of all peers in the network ○ List is updated when peers join/leave the network ● Searches for new peer when a connection closes Network Array and Peer Discovery
- 60. Peer 2 Peer 3 Network Peer 1 Peer 3 Network Peer 1 Peer 2
- 61. Peer 2 Peer 3 New peer is found! Network Peer 3 Network Peer 2
- 62. Can we proactively avoid a single point of failure?
- 63. Peer 2 Peer 1 Peer 3 Peer 4 Load Balancing via Connection Forwarding
- 64. Peer 2 Peer 1 Peer 3 Peer 4 Peer 5 Connection Request
- 65. Peer 2 Peer 1 Peer 3 Peer 4 Peer 5
- 66. Future Plans ● Improve connection distribution ● Optimize mass insertions and deletions ● Automated tests for P2P network Feel free to contribute!
- 67. ● Project Page: https://bit.ly/conclave-site/ ● Conclavity: https://bit.ly/conclavity ● Demo: https://bit.ly/conclave-meetup3 My Information ● Name: Sun-Li Beatteay ● E-mail: sjbeatteay@gmail.com ● Website: http://sunli.co Project Information