Creating Community at WeWork through Graph Embeddings with node2vec - Karry Lu
3 likes110,085 views
This document discusses how WeWork is using graph embeddings and the node2vec algorithm to power member recommendations. It first describes WeWork's member knowledge graph that contains data on members' profiles, interactions, interests and skills. It then explains how node2vec can learn vector representations of each member node that capture similarities, which can be used for recommendations. WeWork runs node2vec on the social graph of each location to map members to vectors and identify the most similar members to power recommendations like onboarding suggestions and introductions between members.
Creating Community at WeWork through Graph Embeddings with node2vec - Karry Lu
- 1. Deep Learning World | Munich 2019 Machine Learning at WeWork: Creating Community Through Graph Embeddings
- 2. PHYSICAL + DIGITAL DATA
- 3. Physical Space: Use data to optimize how we construct, maintain and design our spaces On the Physical side, we use data to optimize how we construct, maintain and design our spaces -reduce extraneous costs from construction and building logistics -encourage physical connections through intelligent interior design -maximize utilization
- 4. Digital Space: Focus on helping people make connections On the Digital, we focus mostly on helping people make connections: -promote their business -request help from others -simply make new friends and connections
- 5. In order to better facilitate connections, WeWork started an ML team to focus on rec systems
- 6. In order to better facilitate connections, WeWork started an ML team to focus on rec systems
- 7. In order to better facilitate connections, WeWork started an ML team to focus on rec systems -personalized newsfeed -text classifiers -entity extraction -onboarding skills suggestion -conference rooms recommender -experiment platform
- 8. Graph Embeddings with node2vec Or: How to recommend members for fun and profit
- 9. Graph Embeddings with node2vec 1) Business Use Case 2) Data 3) Model
- 10. Use Case: Member Needs Our members have broadly expressed the desire to: -meet other members in the community -make social and professional connections -feel a sense of “belonging”
- 11. Use Case: Member Needs We are thus interested in facilitating the meeting of members
- 12. Use Case: Member Needs How do we create an intelligent member-to-member recommendation service?
- 13. Data Structure: Member Knowledge Graph
- 14. Member Knowledge Graph A central repository of information about our members
- 15. Member Knowledge Graph A central repository of information about our members
- 16. Member Knowledge Graph A central repository of information about our members
- 17. Member Knowledge Graph A central repository of information about our members
- 18. Member Knowledge Graph We collect data from: -member profiles -member interactions on the app -posts you’ve liked -events you’ve bookmarked -community team notes -external data sources that we match to our internal data
- 20. Member Knowledge Graph All this information is expressed in graph form
- 21. Member Knowledge Graph All this information is expressed in graph form Sleeveless shirts Laundry The color blue Motorbikes
- 22. Member Knowledge Graph All this information is expressed in graph form Cell phones Jackets The color blue
- 23. Member Knowledge Graph Connecting members through shared skills or interests The color blue
- 24. Member Knowledge Graph Connecting members through shared skills or interests The color blue
- 25. Member Knowledge Graph Connecting members through shared skills or interests Laundry The color blue The color blue
- 26. Member Knowledge Graph We assume that people who have lots of shared skills or interests may be good recommendations for each other Laundry The color blue The color blue
- 27. Member Knowledge Graph How do we calculate how member similarity using common skills or interests? Laundry The color blue The color blue
- 28. Embeddings
- 29. Word Embeddings for Text Learn vector representations of words that capture semantic meanings
- 30. Word Embeddings for Text Process: train a neural network Outcome: each word can be represented by an N-dimensional vector
- 31. Word Embeddings for Text Ultimately the results of this model can be used for many NLP tasks such as word similarity, clustering, classification
- 32. Node Embeddings for Graphs Can we do something similar for graph networks?
- 33. Node Embeddings for Graphs Instead of words we have nodes Each node can be represented with a vector after training a neural network model
- 34. Node Embeddings for Graphs Node = person Edge = Social interaction between 2 people OR Edge = Similarity between 2 people’s skills OR Edge = Similarity between 2 people’s interests OR Edge = Some other function…
- 35. Node Embeddings for Graphs Nodes with similar vectors (measured by something like cosine similarity) should be clustered close together
- 36. Node Embeddings for Graphs Luckily, someone’s already written a paper about this…
- 37. word2vec vs Node2vec word2vec: Given a corpus, maximize P(next word | context words)
- 38. word2vec vs Node2vec word2vec: Given a corpus, maximize P(next word | context words) CBOW
- 39. word2vec vs Node2vec word2vec: Given a corpus, maximize P(next word | context words) CBOW Skip-gram
- 40. word2vec vs Node2vec node2vec: For node2vec, we want to maximize the log-probability of observing a network neighborhood Ns(u) for a node u conditioned on its feature representation
- 41. word2vec vs Node2vec node2vec: In order to make this problem tractable, we assume: - Conditional independence of neighborhood nodes
- 42. word2vec vs Node2vec node2vec: In order to make this problem tractable, we assume: - Symmetry in feature space (a source node and neighborhood node have a symmetric effect over each other) - Conditional likelihood of every node pair written as a softmax function
- 43. word2vec vs Node2vec node2vec: With these two assumptions, original function simplifies to:
- 44. word2vec vs Node2vec node2vec: If we know the representation for node v (the gray node), can we predict its neighborhood (x1, x2, x3)?
- 45. node2vec Algorithm Problem: There’s no obvious way to identify separate neighborhoods in a graph like you can identify sentences in a document
- 46. node2vec Algorithm Solution: Simulate many random walks around each node to define possible “neighborhoods” around the node
- 47. node2vec Algorithm Each walk is a directed subgraph analogous to a “sentence” in a text corpus
- 48. node2vec Algorithm From the graph on the left we can create multiple potential neighborhoods and use them as “training data”
- 49. node2vec Algorithm In order to efficiently explore many possible neighborhoods of node u, the random walk algorithm takes two parameters P and Q Determines how ”locally” or “globally” the walk explores the neighborhood of node u High P à less likely to revisit an already-visited node High Q à more likely to visit close-by nodes
- 50. node2vec Algorithm Combinations of P, Q can emulate different sampling strategies: Breadth First Search: Neighborhood is restricted to immediate neighbors of the node u Depth First Search: Neighborhood consists of nodes sampled at increasing distances from u
- 51. node2vec The node2vec algorithm can be decomposed into 3 steps: a) Precompute transition probabilities for the random walk simulation b) For every node, simulate r random walks of fixed length l c) Feed random walks into a word2vec model and solve with SGD
- 52. node2vec Once we train a model and find embeddings for each node we can do: - Clustering using node embeddings - Node classification - Link prediction - Community Detection
- 53. Clustering off node2vec For each WeWork location we can run node2vec on its social graph…
- 54. Clustering off node2vec Using the data in the Member Knowledge Graph… Laundry The color blue The color blue
- 55. Clustering off node2vec Map every member to a vector…
- 56. Clustering off node2vec Then retrieve the most similar members for each member…
- 57. Clustering off node2vec And use this to power different kinds of member recommendations
- 58. Clustering off node2vec And use this to power different kinds of member recommendations -General member recommendations during onboarding
- 59. Clustering off node2vec And use this to power different kinds of member recommendations -General member recommendations during onboarding -Facilitated introductions between WeWork Labs members