![Random Walk with Restarts (RWR) sampling
The algorithm:
Take random walks in the graph, but keep restarting from the
same node root intermittently
When enough of the graph visited, output visited graph
Shown by Leskovec et al. [*] to produce structurally
representative subgraphs
[*]: Sampling from Large Graphs, Leskovec et al, 2006, Proceedings of the 12th ACM SIGKDD international conference on Knowledge discovery and data mining
14](https://image.slidesharecdn.com/008gnnsatscalewithgraphdatasciencesamplingandpythonclientintegration-nodes2022americasadvanced2-adam-250419060228-a4ad4175/85/Apple-Logic-Pro-X-for-MacOS-Free-Download-14-320.jpg)
Apple Logic Pro X for MacOS Free Download
- 1. 1 GNNs at Scale With Graph Data Science Sampling and GDS Python Client Integration Adam Schill Collberg Senior Software Engineer, Graph Data Science Team at Neo4j
- 2. Overview ⹠Problem Statement ⊠Cora Dataset ⊠Node Classification ⹠Using Graph Neural Networks (GNN) ⊠A Very Brief Introduction ⊠PyTorch and PyG ⊠GNN Pros and Cons ⹠Graph Sampling ⊠Random Walk with Restarts (RWR) ⊠Neo4j Graph Data Science (GDS) ⹠Demo ⹠Summary and Further Learning 2
- 3. 3 Problem Statement Classifying subjects of research papers in a citation network 3
- 4. The Cora Dataset âą A research paper citation network âą 2708 research Paper nodes âą 5429 CITES relationships Paper Paper CITES Schema: Paper nodes color coded by subject 4 âequationâ âturingâ âgraphâ ⊠Paper X 1 0 0 ⊠Feature vector âą Each paper node has a length 1433 binary feature vector ⊠Each dimension represents a keyword ⊠Value 1 if paper has keyword, 0 otherwise âą Each paper node belongs to a subject
- 5. The Node Classification Problem Computer Science Mathematics Paper 1 Paper 2 Feature vectors 5
- 6. Supervised Machine Learning Approach ⊠Supervised ML model Eg. neural network Paper subject Feature Vector Representation But what about all the relationship information? âequationâ âturingâ âgraphâ ⊠Paper 1 1 0 0 ⊠Paper 2 0 1 1 ⊠6
- 7. Degree could be interesting? Maybe interesting, but can we do better? ⊠Supervised ML model Eg. neural network Paper subject Feature Vector Representation âequationâ âturingâ âgraphâ ⊠Node degree Paper 1 1 0 0 ⊠82 Paper 2 0 1 1 ⊠3 7
- 8. 8 Using Graph Neural Networks Graph topology-aware deep machine learning 8
- 9. A Very Brief Intro to GNNs A neural network architecture based on message passing, where: â Inputs are node feature vectors (same as regular ML) â Layer connectivity is defined by relationships Example: Each target nodes gets its own computation graph, but node vectors and weights are shared Layer 0: Node feature vectors Layer 1, outputs transformed vectors Layer 2 GNN computation graph for A 9
- 10. The GNN layer The GNN layer abstractly consists of two parts: 1. Aggregating neighbor output vectors from previous layer 2. Updating target node vector with neural network Graph Convolution Network (GCN) example: Layer i xC(i - 1) xA(i - 1) xB(i) 10 where Ï is non-linear activation and W(i) weight matrix
- 11. PyTorch and PyG â An optimized tensor library for deep learning using GPUs and CPUs â Originally developed by Meta AI, now part of the Linux Foundation umbrella â Rich ecosystem â Interfaces in both Python and C++ â PyTorch Geometric (PyG): â Extension for graph learning â Supports lots of GNN architectures 11
- 12. 12 GNN Cons GNN Pros âą Requires a lot of memory âą Are slow to compute ⊠Even with GPUs âą Hard to interpret âą Fairly complicated to implement âą Leverage topology through message passing âą Can capture lots of information in weights âą Are inductive: ⊠Can be trained on one graph and applied for prediction on another (similar) graph Letâs train a GNN on a graph subsample!
- 13. 13 Graph Sampling Random walk with restarts and Neo4j Graph Data Science 13
- 14. Random Walk with Restarts (RWR) sampling The algorithm: Take random walks in the graph, but keep restarting from the same node root intermittently When enough of the graph visited, output visited graph Shown by Leskovec et al. [*] to produce structurally representative subgraphs [*]: Sampling from Large Graphs, Leskovec et al, 2006, Proceedings of the 12th ACM SIGKDD international conference on Knowledge discovery and data mining 14
- 15. Neo4j Graph Data Science (GDS) Library â A Neo4j DB plugin for analytics â Part of the DBMS (server) process â Projects database into memory for analysis â Provides high performance graph algorithms â Running at scale (100s of billions of nodes) â Has a nice implementation of RWR sampling â Supports rapid transfer to and from the library â Using the Apache Arrow memory format and libraries 15 Client side Server side Neo4j GDS Neo4j Driver JVM Bolt Arrow Client Arrow But there is an easier wayâŠ
- 16. Neo4j Graph Data Science Client â A Pythonic surface for GDS â Wraps Neo4j Python driver â Looks very similar to the GDS Cypher API â Adds additional convenience functionality â Uses Arrow client for fast data transfer seamlessly under the hood â pip install graphdatascience 16 Client side Server side Neo4j GDS GDS Client JVM Bolt Arrow
- 18. 18 Summary and Further Learning What did we learn? And whatâs next? 18
- 19. What did we learn? â Node classification is an interesting graph ML problem â Itâs useful to leverage topology when learning on graphs â GNNs can do this well, but are slow â Since GNNs are inductive we can train them on a subsample â We can use PyG for GNN training â We can subsample using RWR â We can use GDS (including its Python client) for RWR â GDSâs rapid transfer capabilities enable a fast workflow 19
- 20. Also at NODES If you liked this presentation, you might also like âą Fundamentals of Neo4j Graph Data Science Series 2.x â Pipelines and more ⊠Mats Rydberg ⊠Thursday, November 17, 9:40 â 10:25 CET âą Link Prediction With Graph Data Science at Scale ⊠Florentin Dörre ⊠Thursday, November 17, 13:40 â 13:55 CET
- 21. A Bunch of Links â The DEMO notebook â Neo4j Graph Data Science Manual â Neo4j Graph Data Science Client Manual â PyTorch Docs â PyTorch Geometric Docs â The Cora dataset â GCN paper â Sampling from Large Graphs paper â Graph Representation Learning book â Apache Arrow 21
- 22. 22 Thank you! Contact me at adam.schill-collberg@neo4j.com adamnsch@Github