O(1) DHT
Download as PPTX, PDF0 likes1,458 views
This document discusses O(1) DHT designs that allow lookups in a distributed hash table to complete in one hop. It describes two approaches: D1HT, which achieves this by having each node know all other nodes and maintain a full routing table, creating bandwidth issues during joins and leaves. Kelips, which addresses this by having nodes periodically exchange partial routing tables to gradually build knowledge of the network, reducing bandwidth consumption but introducing some lookup latency. Both aim to improve on the typical O(logN) lookup time in Kademlia-based DHTs by trading off bandwidth for speed, with potential applications in large-scale databases like Cassandra.
O(1) DHT
- 1. O(1) DHT lanning
- 2. Reference: • Kelips - Building an Efficient and Stable P2P DHT Through Increased Memory and Background Overhead (2003) • D1HT A Distributed One Hop Hash Table (2006) • Cassandra(DHT) (2009)
- 3. About DHT - Every node has an ID - Every object has and ID - Every node has a routing table - Everything is <key, Value> - Lookup: O(logN) Reference: Kademlia - A Peer-to-Peer Information System Based on the XOR Metric (2002)
- 4. O(1) Lookup • Search a key takes only one hop to get its value • How ?
- 5. O(1) Lookup • Key concept: each node knows all nodes Routing table has exactly N elements • Every join or leave must let every node know
- 6. D1HT ‘s idea • One Hop P1 P’s routing table P2 P3 P4 P5 P6 P7 P8 P9 . . P found Pp is dead, so it broadcasts this event
- 8. Cost Bandwidth consumption (to inform every nodes) is the major cost: • D1HT: O(N) messages for each join or leave • Kelips: 100,000 nodes, 10,000,000 files inserted takes 1.93MB
- 9. Application ex. Cassandra DB - noSQL concept: shift the burder of search to update • DB for Cloud Application e.g. Facebook, Twitter, Plurk 20,000,000,000 events per second Traditional SQL DB not work ! solutions: • HBase/Map Reduce/BigTable • Fully Distributed DB (noSQL) record query request
- 10. Conclusion O(1) DHT - From Usesless To Useful, if in the right place