NS-CUK Seminar: H.B.Kim, Review on "metapath2vec: Scalable representation learning for heterogeneous networks", KDD 2017
- 1. Ho-Beom Kim Network Science Lab Dept. of Mathematics The Catholic University of Korea E-mail: hobeom2001@catholic.ac.kr 2023 / 07 / 24 DONG, Yuxiao; CHAWLA, Nitesh V.; SWAMI, Ananthram. ACM SIGKDD
- 2. 2 Introduction Problem Statements • Neural network-based learning models can represent latent embeddings that capture the internal relations of rich, complex data across various modalities, such as image, audio, and language. • Social and information networks are similarly rich and complex data that encode the dynamics and types of human interactions, and are similarly amenable to representation learning using neural networks. • Recent research publications have proposed word2vec-based network representation learning frameworks • DeepWalk, LINE, node2vec → These work has far focused on representation learning for homogeneous networks-representative of singular type of nodes and relationships. • A large number of social and information networks are heterogeneous in nature, involving diversity of node types and/or relationships between nodes
- 3. 3 Introduction Problem Statements • The latent heterogeneous network embeddings can be further applied to various network mining tasks, such as node classification, clustering, and similarity search • In contrast to conventional meta-path-based methods, the advantage of latent-space representation learning lies in its ability to model similarities between nodes without connected meta-paths. • If there is a problem such that similarity is zero, will be naturally overcome by network representation learning
- 4. 4 Introduction Contributions 1. Formalizes the problem of heterogeneous network representation learning and identifies its unique challenges resulting from network heterogeneity 2. Develops effective and efficient network embedding frameworks, metapath2vec & metapath2vec++, for preserving both structural and semantic correlations of heterogeneous networks 3. Through extensive experiments, demonstrates the efficacy and scalability of the presented methods in various heterogeneous network mining tasks, such as node classification and node clustering. 4. Demonstrates the automatic discovery of internal semantic relationships between different types of nodes in heterogeneous networks by metapath2vec & metapath2vec++, not discoverable by existing work.
- 5. 5 Related Works PTE • Deep neural networks fully leverage labeled information that is available for a task when they learn the representations of the data • Most text embedding methods are not able to consider labeled information when learning the representations • Previous embedding models using unsupervised text embeddings → Generalizable for different tasks but have a weaker predictive power for a particular task • Text embedding methods are much more efficient, are much easier to tune, and naturally accommodate unlabeled data
- 6. 6 Related Works PTE • Fist learns a low dimensional embedding for words through a heterogeneous text network. • The network encodes different levels of co-occurrence information between words and words, words and documents, and words and labels. • The network is embedded into a low dimensional vector space that preserves the second-order proximity between the vertices in the network. • Proposed to learn predictive text embeddings in a semi-supervised manner. • Unlabeled data and labeled information are integrated into a heterogeneous text network which incorporates different levels of cooccurrence information in text • Propose PTE, which learns a distributed representation of text through embedding the heterogeneous text network into a low dimensional space • PTE uses the average value of word embeddings • PTE considers the linear relationship between words and labels
- 7. 7 Problem Definition Definition A Heterogeneous Network defined as a graph G = 𝑉, 𝐸, 𝑇 node v is associated mapping functions 𝜙 𝑣 : 𝑉 → 𝑇𝑉 Link e is associated mapping functions 𝜑 𝑒 : 𝐸 → 𝑇𝐸 𝑇𝑉, 𝑇𝐸 denote the sets of object and relation types where |𝑇𝑉| + 𝑇𝐸 > 2
- 8. 8 Problem Definition Definition A : authors P : papers V : venues O : organizations → Nodes A-A : coauthor A-P,P-V : publish O-A : affiliation
- 9. 9 Problem Definition Problem1 : Heterogeneous Network Representation Learning • The task is to learn the d-dimensional latent representations X ∈ ℝ 𝑉 X𝑑 , 𝑑 ≪ 𝑉 that are able to capture the structural and semantic relations among them • The output : the low-dimensional matrix X with the vth row-a d-dimensional vector 𝑋𝑣-corresponding to the representation of node v • There are different types of nodes in V their representations are mapped into the same latent space • The learned node representations can benefit embedding vector of each node can be used as the feature input of node classification, clustering, and similarity search tasks • The main challenge of this problem comes from the network heterogeneity, wherein it is difficult to directly apply homogeneous language and network embedding methods • The premise of network embedding models is to preserve the proximity between a anode and its neighborhood.