NS-CUK Seminar: S.T.Nguyen, Review on "Multi-modal Trajectory Prediction for Autonomous Driving with Semantic Map and Dynamic Graph Attention Network", NIPS 2020
- 1. Nguyen Thanh Sang Network Science Lab Dept. of Artificial Intelligence The Catholic University of Korea E-mail: sang.ngt99@gmail.com 2023-03-24
- 2. 1 Paper Introduction Problem Contributions Framework Experiment Conclusion
- 3. 2 Autonomous driving cars • Predicting future trajectories of surrounding obstacles is a crucial task for autonomous driving cars to achieve a high degree of road safety.
- 4. 3 Problems Challenges in trajectory prediction in real-world traffic scenarios: • Considering surrounding traffic environments. • Dealing with social interactions. • Handling traffic of multi-class movement, • Predicting multi-modal trajectories with probability. • Probability awareness.
- 5. 4 Contributions • Multi-modal predictions with considering traffic environments, dealing with social interactions, and predicting multi-class movement patterns with probability values, simultaneously. • Dynamic Graph Attention Network (DGAN), Dynamic Attention Zone and GAT are combined to model the intention and habit of human driving in heterogeneous traffic scenarios. • To capture complex social interactions among road agents, the authors combine a semantic HD map, observed trajectories of road agents, and the current status of the traffic.
- 6. 5 Dynamic Attention Zone and Graph Modelling • A dynamic attention zone is designed to capture the normal ability of people when interacting with others in traffic. • Human choose which surrounding moving agents: distances, headings, velocities, and sizes. an attention circle
- 7. 6 Overview
- 8. 7 Feature Extraction • Semantic Map: semantic HD map contains valuable traffic rule information. • The middle-layer output estimated by the CNN is extracted as the visual feature: • Observed Trajectory: An LSTM is used to extract joint features from the observed trajectories of all involved agents. • Traffic state: S is very important for capturing extra information to predict the future trajectories: • The final embedding feature:
- 9. 8 Graph Attention Network • GAT: multiple stacked graph attention layers. • Hierarchical classification to calculate the probabilities belonging to class c and anchor 𝑘𝑐:
- 10. 9 Multi-modal Trajectory Prediction • Predicting multiple possible future trajectories with corresponding probability using pre-defined anchor trajectories. • The final loss consists of anchor classification loss and trajectory offset loss: • represents the single-mode loss L of the 𝑖𝑡ℎ agent’s 𝑘𝑐 anchor, where: • Hierarchical classification loss:
- 11. 10 Logistic Delivery Dataset • Left: Logistic delivery dataset example, consisting of three-dimensional cloud points with manually labeled information, front camera image, and semantic map. • Middle: observed in dashed yellow and future ground truth trajectories in red. • Right: Prediction results using our proposed DGAN method showing up the two most likely future trajectories, and corresponding probabilities encoded in a color map to the right. handling complex situations at traffic intersections. the predicted trajectory with the maximum probability value is more likely to follow center lines of lanes guiding by the semantic map.
- 12. 11 Stanford Drone Dataset • The model achieves state-of-the-art performance.
- 13. 12 ETH and UCY Datasets • Datasets for pedestrian trajectory prediction only, include 5 scenes in total, including ETH, HOTEL, ZARA1, ZARA2, and UNIV.
- 14. 13 Conclusions • A dynamic social interaction-aware model that predicts the future trajectories of agents in real-world settings to solve several challenges. • An encoded semantic map, the observed history trajectories, and the current status of agents as the input of the GAT. • To generate the graph at the current time step, we use the dynamic attention zone to simulate the intuitive ability of people to navigate roads in real-world traffic. • The experiment results demonstrate the potential ability of the proposed method for trajectory prediction in a real-world setting and achieves good performance.
- 15. 14