The ways of node embedding
- 2. Node Embedding • Goal : Efficient feature learning for ML • Learned vectors can be used for • Classification : SVM, Logistic Regression • Clustering : K-means • Link Prediction
- 3. Node Embedding 𝑆𝑖𝑚 𝑢, 𝑣 ≈ 𝑆𝑖𝑚(𝑧!, 𝑧")
- 4. Node Embedding • How to define Node Similarity? Common Neighbors : |𝑁 𝑥 ∩ 𝑁 𝑦 | Adamic-Adar Index : ∑ !∈ # $ ∩# & ' ()* |# ! | Jaccard Similarity : # $ ∩# & # $ ∪# &
- 5. Node Embedding • Why not?? • Unseen node -> how to embed? • 𝑂(|𝑉|) parameters needed • Cannot incorporate node features • GCN
- 7. Neighborhood Aggregation ℎ! " = 𝑥! ℎ! # = 𝜎(𝑊# ' $∈& ! ℎ$ #'( |𝑁 𝑣 | + 𝐵#ℎ! #'( )
- 8. Node embedding We now introduce the ways of node embedding using idea of word embedding
- 9. DeepWalk : Online Learning of Social Representations 나는 점심에 중국집에서 ____ 을 먹었다 짬뽕 짜장면 탕수육 감자탕 축구독서 주변 단어는 중심단어를 표현한다.(CBOW) 𝑥 = 𝑎𝑟𝑔𝑚𝑎𝑥$ 𝑠𝑖𝑚 점심, 𝑥 , 𝑠𝑖𝑚 중국집, 𝑥 , 𝑠𝑖𝑚(먹었다, 𝑥)
- 10. DeepWalk : Online Learning of Social Representations ___ 짜장면 ___ 먹었다 중국집 점심 감자탕 축구독서 중심단어는 주변단어를 표현한다.(Skip-Gram) 𝑥 = 𝑎𝑟𝑔𝑚𝑎𝑥$ 𝑠𝑖𝑚 짜장면, 𝑥
- 11. DeepWalk : Online Learning of Social Representations How can we make a sentence of nodes? Random Walk Neighborhood preserving likelihood Encoder-decoder structure
- 12. DeepWalk : Online Learning of Social Representations
- 13. DeepWalk : Online Learning of Social Representations 하지만, SkipGram에서 분류모델의 레이블 수는 V개라서 매우 비효율적 Hierarchical Softmax h에 W를 곱하고 softmax : O(N) -> infeasible O(logN) 아직도 많다면??
- 14. DeepWalk : Online Learning of Social Representations Negative Sampling 전체를 보지 말자 : |V| -> K
- 15. DeepWalk : Online Learning of Social Representations Pros : 1. Unseen data에 대해서 해결못함 2. Node/Edge feature 사용 안함. 3. Node label 사용 안함 Cons : 1. Unseen edge는 해결가능 2. Local information 사용 3. 병렬처리 가능
- 16. Node2vec: Scalable Feature Learning for Networks