NS-CUK Seminar: S.T.Nguyen, Review on "Hierarchical Graph Convolutional Networks for Semi-supervised Node Classification", IJCAI 2019
- 1. Nguyen Thanh Sang Network Science Lab Dept. of Artificial Intelligence The Catholic University of Korea E-mail: sang.ngt99@gmail.com 2023-04-07
- 2. 1 Paper Introduction Problem Contributions Framework Experiment Conclusion
- 3. 2 Hierarchical graph structure • Hierarchical structure is pervasive across complex networks with examples spanning from neuroscience, economics, social organizations, urban systems, communications, pharmaceuticals and biology, particularly metabolic and gene networks.
- 4. 3 Problems Many embedding methods can be used in the node classification task by converting the graph structure into sequences by performing random walks on the graph and computing co- occurrence statistics. => unsupervised algorithms and cannot perform node classification tasks in an end-to-end applications.
- 5. 4 Problems Graph convolutional networks (GCNs) generates node embedding by combining information from neighborhoods. lack the “graph pooling” mechanism, which restricts the scale of the receptive field. difficulty in obtaining adequate global information. adding too many convolutional layers will result in the output features over-smoothed and make them indistinguishable
- 6. 5 Problems Some recent methods try to get the global information through deeper models. they are either unsupervised models or need many training examples. they are still not capable of solving the semi-supervised node classification task directly.
- 7. 6 Contributions • First work to design a deep hierarchical model for the semi-supervised node classification task which consists of more layers with larger receptive fields. obtain more global information through the coarsening and refining procedures. • Applying deep architectures and the pooling mechanism into classification tasks. • The proposed model outperforms other state-of-the-art approaches and gains a considerable improvement over other approaches with very few labeled samples provided for each class.
- 8. 7 Overview • For each coarsening layer, the GCN is conducted to learn node representations. • A coarsening operation is performed to aggregate structurally similar nodes into hyper-nodes. each hyper-node represents a local structure of the original graph, which can facilitate exploiting global structures on the graph. • Following coarsening layers, a symmetric graph refining layers is applied to restore the original graph structure for node classification tasks. comprehensively capture nodes’ information from local to global perspectives, leading to better node representations.
- 9. 8 Graph Convolutional Networks • Update node representation by using neighbor features.
- 10. 9 Graph Coarsening Layer Two steps: • Structural equivalence grouping (SEG). If two nodes share the same set of neighbors, they are considered to be structurally equivalent. assign these two nodes to be a hyper-node. • Structural similarity grouping (SSG). Then, we calculate the structural similarity between the unmarked node pairs. => form a new hyper-node and mark the two nodes by largest structural similarity. The largest structural similarity is defined by comparing the normalized connection strength:
- 11. 10 Graph Coarsening Layer • The hidden node embedding matrix is determined as: • Update adjacency matrix: The hidden representation is fed into the next layer as input. The resulting node embedding to generate in each coarsening layer will then be of lower resolution.
- 12. 11 Graph Refining Layer • To restore the original topological structure of the graph and further facilitate node classification stacking the same numbers of graph refining layers as coarsening layers. • Each refining layer contains two steps: o generating node embedding vectors o restoring node representations. • A residual connections between the two corresponding coarsening and refining layers.
- 13. 12 Node Weight Embedding and Multiple Channels • Multi-channel mechanisms help explore features in different subspaces and H-GCN employs multiple channels on GCN to obtain rich information jointly at each layer
- 14. 13 The Output Layer • Softmax classifier: • The loss function is defined as the cross-entropy of predictions over the labeled nodes:
- 15. 14 Datasets • Four widely-used datasets including three citation networks and one knowledge graph.
- 16. 15 Experiment • The proposed method consistently outperforms other state-of-the-art methods, which verify the effectiveness of the proposed coarsening and refining mechanisms. • DeepWalk cannot model the attribute information, which heavily restricts its performance. • The proposed H-GCN manages to capture global information through different levels of convolutional layers and achieves the best results among all four datasets.
- 17. 16 Impact of Scale of Training Data • The proposed method outperforms other baselines in all cases. • With the number of labeled data decreasing, it obtains a more considerable margin over these baseline algorithms. • The proposed H-GCN with increased receptive fields is well-suited when training data is extremely scarce and thereby is of significant practical values.
- 18. 17 Coarsening, refining layers and embedding weights • The proposed H-GCN has better performance compared to H-GCN without coarsening mechanisms on all datasets. => The coarsening and refining mechanisms contribute to performance improvements since they can obtain global information with larger receptive fields. • Model with node weight embeddings performs better, the necessity to add this embedding vector in the node embeddings.
- 19. 18 Comparing with number of coarsening layers and channels • Since fewer labeled nodes are supplied on NELL than others, deeper layers and larger receptive fields are needed. • Adding too many coarsening layers, the performance drops due to overfitting. • The performance improves with the number of channels increasing until four channels => help capture accurate node features. • Too many channels will inevitably introduce redundant parameters to the model, leading to overfitting as well.
- 20. 19 Conclusions • A novel hierarchical graph convolutional networks for the semi-supervised node classification task. • The H-GCN model consists of coarsening layers and symmetric refining layers. • By grouping structurally similar nodes to hyper-nodes, this model can get a larger receptive field and enable sufficient information propagation. • Compared with other previous work, H-GCN is deeper and can fully utilize both local and global information. • The has achieved substantial gains over them in the case that labeled data is extremely scarce.
- 21. 20