Temporal Action Localization in Untrimmed Videos via Multi Stage CNNs
2 likes791 views
The document presents a multi-stage CNN framework for temporal action localization in untrimmed videos, detailing the network architecture, training strategies, and segment generation methods. Experiments were conducted on datasets such as thumos 2014, demonstrating the effectiveness of the proposed approach in identifying action categories and improving detection accuracy. Results indicate the benefit of integrating dense trajectory features with SVM for enhanced performance.
![Temporal Action Localization in
Untrimmed Videos via Multi-
Stage CNNs
Slides by Alberto Montes
Computer Vision Group Reading Group,
[arXiv] [code]
Zheng Shou, Dongang Wang and Shih-Fu Chang](https://image.slidesharecdn.com/13temporalactionlocalizationinuntrimmedvideosviamulti-stagecnns1-160614103500/85/Temporal-Action-Localization-in-Untrimmed-Videos-via-Multi-Stage-CNNs-1-320.jpg)
Temporal Action Localization in Untrimmed Videos via Multi Stage CNNs
- 1. Temporal Action Localization in Untrimmed Videos via Multi- Stage CNNs Slides by Alberto Montes Computer Vision Group Reading Group, [arXiv] [code] Zheng Shou, Dongang Wang and Shih-Fu Chang
- 2. Introduction
- 3. Previous Work Improved Dense Trajectory (iDT) Fisher Vector 2D Convolution
- 4. Segment-CNN
- 5. Segment-CNN
- 6. Segment-CNN
- 7. Segment-CNN
- 8. Problem Definition Video: frame # frames Annotations: Candidates: action category action category start and ending frame
- 9. Multi-Scale Segment Generation ◉ Each frame resized to 171x128 pixels ◉ Temporal sliding windows: ○ 16, 32, 64, 128, 256, 512 frames ○ 75% overlap ◉ Construct segment s by uniformly sampling 16 frames
- 11. Training Proposal and Classification Network ◉ lr=0.0001 except fc8 lr=0.01, momentum=0.9, weight decay factor=0.0005 ◉ Drop lr by factor of 2 every 10K iterations Proposal Network: ● fc8: 2 nodes Classification Network: ● fc8: K+1 nodes
- 12. Localization Network Add Custom Loss function
- 13. Localization Network true class label overlap sensitivity Try to boost segments with high overlap Works best with: λ = 1, α = 0.25
- 16. Prediction and Post- processing ◉ Keep segments with Ppro > 0.7 ◉ Remove background segments ◉ Ploc multiply with class-specific frequency of occurrence for each window length in the training data to leverage window length distribution patterns ◉ NMS based on Ploc to remove redundancy. (θ - 0.1)
- 17. Experiments
- 18. MEXaction2 “Bull Charge Cape” and “Horse Riding” videos 77 hours of videos Training set: 1336 instances Validation set: 310 instances Test set: 329 instances Datasets THUMOS 2014 Temporal Action Detection Task 20 categories Training set: 2755 videos Validation set: 1010 videos and 3007 instances Test set: 1574 videos and 3358 instances
- 19. Results MEXaction2 DFT: Dense Trajectory Features + SVM
- 22. Evaluation
- 23. Evaluation
- 24. Evaluation Impact of individual networks:
- 25. Conclusions Propose a multi-stage framework Semgent-CNN to address temporal action location