Redis Day TLV 2018 - Graph Distribution
0 likes216 views
This document discusses graph databases and how they distribute data. It provides examples of graph queries and how they are executed by first traversing indexes to find relevant entities, then retrieving and joining entity properties. The document also compares different graph database solutions like RedisGraph, JanusGraph, DGraph and ArangoDB in terms of their architectures, distribution methods, and consistency models.
![Documents
ID: 1,
Name: ‘Roi’,
Age: 33,
Height: 187,
Visited: [6]
ID: 6,
Name: ‘Japan’,
Population: 127M](https://image.slidesharecdn.com/graphdistribution-180327223433/85/Redis-Day-TLV-2018-Graph-Distribution-7-320.jpg)
![Adjacency matrix
1 0 1 0 1 0 0 0
0 1 0 0 1 1 0 1
0 0 1 0 1 0 0 0
1 1 0 1 1 0 1 1
1 0 1 0 1 1 0 0
Node i is connected to node j If A[i,j] = 1](https://image.slidesharecdn.com/graphdistribution-180327223433/85/Redis-Day-TLV-2018-Graph-Distribution-10-320.jpg)
![Query
Find friends of mine who’ve visited places I’ve been to and are
older than me.
Match (ME:person)-[friend]->(F:person)-[visited]->(C:country)<-[visit]-(ME)
WHERE ME.ID = 33 AND F.age > ME.age
RETURN F.name, C.name](https://image.slidesharecdn.com/graphdistribution-180327223433/85/Redis-Day-TLV-2018-Graph-Distribution-18-320.jpg)
![Graph traversal
(ME:person)-[friend]->(F:person)
Graph index](https://image.slidesharecdn.com/graphdistribution-180327223433/85/Redis-Day-TLV-2018-Graph-Distribution-20-320.jpg)
![(F:person)-[visited]->(C:country)
Graph traversal
Graph index](https://image.slidesharecdn.com/graphdistribution-180327223433/85/Redis-Day-TLV-2018-Graph-Distribution-21-320.jpg)
![(C:country)<-[visit]-(ME)
Graph traversal
Graph index](https://image.slidesharecdn.com/graphdistribution-180327223433/85/Redis-Day-TLV-2018-Graph-Distribution-22-320.jpg)
![Query
Find all posts liked by friends of friends of mine, written by author X.
MATCH
(ME:person)-[friend]->(:person)-[friend]->(F:person)-[like]->(post)<-[author]-(A:author)
WHERE ME.ID=46 AND A.ID=71070
RETURN A.name, F.name](https://image.slidesharecdn.com/graphdistribution-180327223433/85/Redis-Day-TLV-2018-Graph-Distribution-28-320.jpg)
![1. Node X contains FRIEND relations.
2. Seek to my ID in Node X (1 RPC). Retrieve a list of friend uids.
3. Do multiple seeks for each of the friend uids, to generate a list of friends of
friends uids. result set 1
Query
Friend
Index
Query
executor
(ME:person)-[friend]->(:person)-[friend]->(F:person)](https://image.slidesharecdn.com/graphdistribution-180327223433/85/Redis-Day-TLV-2018-Graph-Distribution-29-320.jpg)
![1. Node Y contains posting list for predicate LIKE.
2. Ship result set 1 to Node Y (1 RPC), and do seeks to generate a list of all posts
liked by result set 1. result set 2
Query
Like
Index
Query
executor
(F:person)-[like]->(post)
Resultset 1](https://image.slidesharecdn.com/graphdistribution-180327223433/85/Redis-Day-TLV-2018-Graph-Distribution-31-320.jpg)
![Query
Node Z contains relations for predicate AUTHOR.
Ship result set 2 to Node Z (1 RPC).
Seek to author X, and generate a list of posts authored by X. result set 3
Author
Index
Query
executor
(post)<-[author]-(A:author)
Resultset 2](https://image.slidesharecdn.com/graphdistribution-180327223433/85/Redis-Day-TLV-2018-Graph-Distribution-33-320.jpg)
![(you)-[ask]->(question)
@roilipman](https://image.slidesharecdn.com/graphdistribution-180327223433/85/Redis-Day-TLV-2018-Graph-Distribution-38-320.jpg)
Redis Day TLV 2018 - Graph Distribution
- 3. Graph Database Use cases: Fraud detection Recommendation engine Social networks...
- 4. ● Property graph ● Labeled entities ● Schema less ● Cypher query language ● Aggregations, Arithmetic expressions, Sort... ● Tabular resultset RedisGraph
- 5. Structure
- 6. Tables Name Age Height Roi 33 187 Hila 33 170 Shany 23 167 Amit 31 180 Name Population Israel 8.5M Japan 127M Italy 60M SRC DEST 1 2 2 2 2 3 4 1 4 3 Person CountryVisit
- 7. Documents ID: 1, Name: ‘Roi’, Age: 33, Height: 187, Visited: [6] ID: 6, Name: ‘Japan’, Population: 127M
- 9. Adjacency list 12 3 4 1 2 3 4
- 10. Adjacency matrix 1 0 1 0 1 0 0 0 0 1 0 0 1 1 0 1 0 0 1 0 1 0 0 0 1 1 0 1 1 0 1 1 1 0 1 0 1 1 0 0 Node i is connected to node j If A[i,j] = 1
- 11. Hexastore SPO OSP SOP PSO OPS POS S Subject P Predicate O Object 6
- 13. Node property set Entities - Key value store. Person node with attributes: { ‘name’: ‘Bruce Buffer’, ‘age’: 60, ‘gender’: ‘male’ }
- 14. 2 billion users 338 average friends for user 676 billion edges 152 terabytes ~= 1024*32 bytes per user + 64 * 2 bytes per edge Problem
- 16. Partitioning
- 17. Entities distribution Property set 1 Property set 2 Graph index
- 18. Query Find friends of mine who’ve visited places I’ve been to and are older than me. Match (ME:person)-[friend]->(F:person)-[visited]->(C:country)<-[visit]-(ME) WHERE ME.ID = 33 AND F.age > ME.age RETURN F.name, C.name
- 19. (ME:person) ME.ID = 33 Graph traversal Graph index
- 23. Resultset Friend ID Friend name Country ID Country name 70 ? 25 ? 92 ? 55 ? 56 ? 4 ?
- 24. Query WHERE F.age > ME.age RETURN F.name, C.name NETWORK! Index Entities Fetch age for ID 33
- 25. Query example continued WHERE F.age > ME.age RETURN F.name, C.name NETWORK! Index Entities Fetch name of every entity in (IDs) Entity’s age > 29
- 26. Resultset Friend ID Friend name Country ID Country name 70 Noam 25 Japan
- 27. Index distribution Friend relation Visit relation Graph index
- 28. Query Find all posts liked by friends of friends of mine, written by author X. MATCH (ME:person)-[friend]->(:person)-[friend]->(F:person)-[like]->(post)<-[author]-(A:author) WHERE ME.ID=46 AND A.ID=71070 RETURN A.name, F.name
- 29. 1. Node X contains FRIEND relations. 2. Seek to my ID in Node X (1 RPC). Retrieve a list of friend uids. 3. Do multiple seeks for each of the friend uids, to generate a list of friends of friends uids. result set 1 Query Friend Index Query executor (ME:person)-[friend]->(:person)-[friend]->(F:person)
- 30. Resultset 1 Friends of friends Friend ID Friend name 70 ? 92 ? 56 ?
- 31. 1. Node Y contains posting list for predicate LIKE. 2. Ship result set 1 to Node Y (1 RPC), and do seeks to generate a list of all posts liked by result set 1. result set 2 Query Like Index Query executor (F:person)-[like]->(post) Resultset 1
- 32. Resultset 2 Liked posts Friend ID Friend name Post ID 70 ? 534 70 ? 431 92 ? 8964 56 ? 12 56 ? 5356
- 33. Query Node Z contains relations for predicate AUTHOR. Ship result set 2 to Node Z (1 RPC). Seek to author X, and generate a list of posts authored by X. result set 3 Author Index Query executor (post)<-[author]-(A:author) Resultset 2
- 34. Resultset 4 Intersected resultset 2 and 3 Friend ID Friend name Post ID Author ID Author name 70 ? 534 71070 ? 92 ? 8964 71070 ?
- 35. Node N contains names for all uids, ship result set 4 to Node N (1 RPC), and convert uids to names by doing multiple seeks. Query Author Index Query executor RETURN A.name, F.name Resultset 4
- 36. Resultset 4 Intersected resultset 2 and 3 Friend ID Friend name Post ID Author ID Author name 70 Ailon 534 71070 Omri 92 Boaz 8964 71070 Omri
- 37. RedisGraph Not distributed, Yet, Work in progress: Compact distributed index Concurrent fast independent traversals
- 39. JanusGraph successor of Titan ● Relays on a storage backend e.g. Casandar. ● Provides a graph interface on top of a table. ● Delegates storing, replicating, distributing and persisting a graph to the underline storage backend. Takes a mature application from a similar domain and introduce a new data type API on top of existing data structure. (not optimal) Solutions
- 40. Solutions DGraph Uses the concept of RDF NQuad to represents connections and badger as its key value store. Both the graph index and the entities are distributed.
- 41. Solutions Arangodb From my understanding this multi model database uses documents to represent all three data types: Documents, key value store and graph. Not sure about how it distributes its data but it’s using RAFT to ensure consistency It is ACID.