![6
METHODOLOGY
Graph Construction and Problem Definition
• A dynamic spatial-temporal graph 𝐺 = (𝐺1, 𝐺2, … , 𝐺𝜏)
o 𝐺𝜏 = {𝑉
𝜏, ℰ𝜏, 𝐴𝜏} represents the graph during the 𝜏-th month.
o 𝑋𝜏 = [𝑋𝜏
𝑡
∈ ℝ𝑁𝜏×𝐹
|𝑡 = 0, … , 𝑇ℎ] is the node feature matrix with feature dimension F of 𝑁𝜏 nodes in
the past 𝑇ℎ time steps.
• Problem: learn a model function ℱ to predict future graph signals
o Strategies for retraining method: requires collected data from 𝐺𝜏 for training to encompass period (𝜏 − 1) months
o Strategies for online learning method: collect half of the month’s data in 𝜏-th month to fine-tune the archive
model with parameters 𝜔](https://image.slidesharecdn.com/20240624labseminarhuytowardst-240624082308-89113683/85/20240624_LabSeminar_Huy-Towards-Dynamic-Spatial-Temporal-Graph-Learning-A-Decoupled-Perspective-pptx-6-320.jpg)
![11
EXPERIMENT AND RESULT
EXPERIMENT SETTINGs
• Dataset:
o Traffic dataset PeMS in California.
• Baselines:
o Retraining learning: TCN, STGCN[1], STNN[2], ST-GAM[3], and DSTG+AD (DSTG with all available data).
o Continuous learning: DSTG+SK(STKEC)[4], DSTG+TS(Traffic Stream)[5], and DSTG+NN(new nodes with N-
hop neighbors to fine-tune all parameters.
o Online learning: DSTG-Static(no fine-tuning).
[1] Yu, B., Yin, H., & Zhu, Z. (2017). Spatio-temporal graph convolutional networks: A deep learning framework for traffic forecasting. arXiv preprint arXiv:1709.04875.
[2] Yang, S., Liu, J., & Zhao, K. (2021, December). Space meets time: Local spacetime neural network for traffic flow forecasting. In 2021 IEEE International Conference on Data Mining (ICDM) (pp. 817-826). IEEE.
[3] Wang, P., Zhu, C., Wang, X., Zhou, Z., Wang, G., & Wang, Y. (2022). Inferring intersection traffic patterns with sparse video surveillance information: An st-gan method. IEEE Transactions on Vehicular Technology, 71(9), 9840-9852.
[4] Wang, B., Zhang, Y., Wang, P., Wang, X., Bai, L., & Wang, Y. (2023, April). A Knowledge-Driven Memory System for Traffic Flow Prediction. In International Conference on Database Systems for Advanced Applications (pp. 192-207). Cham: Springer Nature Switzerland.
[5] Chen, X., Wang, J., & Xie, K. (2021). TrafficStream: A streaming traffic flow forecasting framework based on graph neural networks and continual learning. arXiv preprint arXiv:2106.06273.
• Measurement:
o Mean Absolute Error(MAE), Mean Absolute Percentage Error (MAPE), and Root Mean Square Error
(RMSE).](https://image.slidesharecdn.com/20240624labseminarhuytowardst-240624082308-89113683/85/20240624_LabSeminar_Huy-Towards-Dynamic-Spatial-Temporal-Graph-Learning-A-Decoupled-Perspective-pptx-11-320.jpg)
![[20240624_LabSeminar_Huy]Towards Dynamic Spatial-Temporal Graph Learning: A Decoupled Perspective.pptx](https://image.slidesharecdn.com/20240624labseminarhuytowardst-240624082308-89113683/85/20240624_LabSeminar_Huy-Towards-Dynamic-Spatial-Temporal-Graph-Learning-A-Decoupled-Perspective-pptx-15-320.jpg)
![[20240624_LabSeminar_Huy]Towards Dynamic Spatial-Temporal Graph Learning: A Decoupled Perspective.pptx](https://image.slidesharecdn.com/20240624labseminarhuytowardst-240624082308-89113683/85/20240624_LabSeminar_Huy-Towards-Dynamic-Spatial-Temporal-Graph-Learning-A-Decoupled-Perspective-pptx-16-320.jpg)
[20240624_LabSeminar_Huy]Towards Dynamic Spatial-Temporal Graph Learning: A Decoupled Perspective.pptx
- 1. Quang-Huy Tran Network Science Lab Dept. of Artificial Intelligence The Catholic University of Korea E-mail: huytran1126@gmail.com 2024-06-24 Towards Dynamic Spatial-Temporal Graph Learning: A Decoupled Perspective Binwu Wang et al. AAAI-2024: Proceedings of the Thirty-Eight Conference on Artificial Intelligence
- 2. 2 OUTLINE • MOTIVATION • INTRODUCTION • METHODOLOGY • EXPERIMENT & RESULT • CONCLUSION
- 3. 3 MOTIVATION • Spatial-temporal graph prediction has emerged as an essential task in the intelligent transportation systems. • The prevailing approaches: o GCN for spatial correlation and sequence modules for temporal correlation. Overview • Most of them are evaluated using short-term dataset and portray the underlying graph as static and unchanging. • When considering a longer time frame, the distribution of the graph can undergo substantial evolution over time - a dynamic spatial-temporal graph: o the underlying structure of the graph would change over time. o the distribution feature of the original nodes in the graph would also evolve over time.
- 4. 4 MOTIVATION Overview: Previous Approach and Limitation • Existing works rely on two main approaches: o the retraining method. o continuous learning strategies. • Limitation of these methods. o assume that there is sufficient data available from the updated graph for at least one month. o very ideal and oversimplifies the dynamic graph scenario.
- 5. 5 INTRODUCTION • reframe the problem of spatial-temporal prediction within the context of data streaming and tackle this problem using dynamic spatial-temporal graph learning. • proposed a decoupled learning framework (DLF): a disentangled spatial-temporal graph network (DSTG) and a decoupled training strategy o STG decouples temporal correlation into seasonal and trend patterns. o employ a decoupled training strategy that alternately updates these two patterns, facilitating efficient and effective dynamic graph learning.
- 6. 6 METHODOLOGY Graph Construction and Problem Definition • A dynamic spatial-temporal graph 𝐺 = (𝐺1, 𝐺2, … , 𝐺𝜏) o 𝐺𝜏 = {𝑉 𝜏, ℰ𝜏, 𝐴𝜏} represents the graph during the 𝜏-th month. o 𝑋𝜏 = [𝑋𝜏 𝑡 ∈ ℝ𝑁𝜏×𝐹 |𝑡 = 0, … , 𝑇ℎ] is the node feature matrix with feature dimension F of 𝑁𝜏 nodes in the past 𝑇ℎ time steps. • Problem: learn a model function ℱ to predict future graph signals o Strategies for retraining method: requires collected data from 𝐺𝜏 for training to encompass period (𝜏 − 1) months o Strategies for online learning method: collect half of the month’s data in 𝜏-th month to fine-tune the archive model with parameters 𝜔
- 8. 8 METHODOLOGY Spatial-Temporal Graph Learning (DSTG) • Input module: input signal is separated into seasonal factors and trend factors o incorporate time position information into the model. • Disentangled spatial-temporal module: two GCNs and a disentangled temporal layer. o GCN: Given the input at 𝑙-th layer 𝐻𝑙 ∈ ℝ𝑁×𝑑𝑙. o Disentangled temporal module: given a node representation 𝐻 ∈ ℝ𝑇ℎ×𝑑ℎ, decompose into a trend 𝐻𝑡 and a seasonal 𝐻𝑟 by a moving average kernel in the input layer. seasonal 𝐻𝑟 is applied self-attention and a position-wise feedforward layer: trend 𝐻𝑡 is input into Temporal Convolutional Network to capture short-term patterns. Combine both. • Output module: CNNs is used to generate predictions.
- 9. 9 METHODOLOGY Decoupled Training Strategy • involves updating two patterns in an alternating manner. • Training for seasonal pattern: utilize three-month data to thoroughly train the model for seasonal patterns o masked autoencoder mechanism: start with a continuous long-term series as the input. divide this input sequence into P patches where P is set to a large value (shape P: 𝐿 × 𝑁𝜏 × 𝐹 , where 𝐿 is length of the input sequence). create a challenging self-supervised task by randomly masking a subset of patches with a masking ratio up to 75% - reduce computational complexity while providing sufficient long-term information.
- 10. 10 METHODOLOGY Decoupled Training Strategy • Fine-tuning for new knowledge: select a subset of each graph to continuously fine-tune the weights of trend. o Detect evolved nodes whose patterns have changed significantly for expanding unseen patterns and replay nodes whose patterns are consistent for reinforcing learned patterns. select the half-month data from training archived model 𝑍𝑟 and current update data 𝑍𝑐. sum these two sequences along the time dimension and obtain daily average flow vectors 𝑍𝑟 ∈ ℝ𝑁𝑟×𝐿ℎ×𝐹 and 𝑍𝑐 ∈ ℝ𝑁𝑐×𝐿ℎ×𝐹 , 𝐿ℎ is time-steps of one-day data and 𝑁𝑟 and 𝑁𝑐 represent the number of nodes. calculate the similarity between 𝑍𝑟 𝑖 and 𝑍𝑐 𝑖 of node 𝑣𝑖 based on Wasserstein distance. sample the top 𝑘% of nodes with high distances, as well as newly added nodes that appear in the archived data, along with their N-hop neighbors.
- 11. 11 EXPERIMENT AND RESULT EXPERIMENT SETTINGs • Dataset: o Traffic dataset PeMS in California. • Baselines: o Retraining learning: TCN, STGCN[1], STNN[2], ST-GAM[3], and DSTG+AD (DSTG with all available data). o Continuous learning: DSTG+SK(STKEC)[4], DSTG+TS(Traffic Stream)[5], and DSTG+NN(new nodes with N- hop neighbors to fine-tune all parameters. o Online learning: DSTG-Static(no fine-tuning). [1] Yu, B., Yin, H., & Zhu, Z. (2017). Spatio-temporal graph convolutional networks: A deep learning framework for traffic forecasting. arXiv preprint arXiv:1709.04875. [2] Yang, S., Liu, J., & Zhao, K. (2021, December). Space meets time: Local spacetime neural network for traffic flow forecasting. In 2021 IEEE International Conference on Data Mining (ICDM) (pp. 817-826). IEEE. [3] Wang, P., Zhu, C., Wang, X., Zhou, Z., Wang, G., & Wang, Y. (2022). Inferring intersection traffic patterns with sparse video surveillance information: An st-gan method. IEEE Transactions on Vehicular Technology, 71(9), 9840-9852. [4] Wang, B., Zhang, Y., Wang, P., Wang, X., Bai, L., & Wang, Y. (2023, April). A Knowledge-Driven Memory System for Traffic Flow Prediction. In International Conference on Database Systems for Advanced Applications (pp. 192-207). Cham: Springer Nature Switzerland. [5] Chen, X., Wang, J., & Xie, K. (2021). TrafficStream: A streaming traffic flow forecasting framework based on graph neural networks and continual learning. arXiv preprint arXiv:2106.06273. • Measurement: o Mean Absolute Error(MAE), Mean Absolute Percentage Error (MAPE), and Root Mean Square Error (RMSE).
- 12. 12 EXPERIMENT AND RESULT RESULT – Overall Performance
- 13. 13 EXPERIMENT AND RESULT Result: Generalization of Models • Evaluate it on the Knowair dataset (different domain): o the air pollution feature PM2.5 and 18 meteorological features.
- 14. 14 CONCLUSION • proposing a decoupled learning framework on spatial-temporal graph prediction with a dynamic scenario. o a disentangled spatial-temporal graph convolutional network (DSTG) and a decoupled training strategy. o DSTG decomposes temporal correlations into seasonal and trend patterns. o The training strategy updates these patterns alternately to facilitate dynamic spatial-temporal graph learning.