"Object Trackers: Approaches and Applications," a Presentation from Intel
0 likes499 views
The document discusses various approaches and applications of object tracking, highlighting the efficiency of trackers in visual analysis and the challenges of detection stability. It covers different types of trackers, such as short-term, long-term, and zero-term trackers, and emphasizes the importance of temporal and spatial scheduling for optimizing performance. Additionally, it notes how deep learning can enhance tracking algorithms by improving image patch comparison and overall accuracy.
![© 2019 Intel
Improving Tracking Algorithms with Deep Learning
The core of tracking is how we can compare two different image patches.
Short/Long/Multi-target trackers all use this basic functionality in its core,
and deep learning can boost the accuracy of image patch comparison.
• Using deeply-learned features (e.g. ImageNet)
• Using metric learning (e.g. Siamese network)
22
[Bertinetto et al. 2017]](https://image.slidesharecdn.com/t1t6parkintel2019-190702190540/85/Object-Trackers-Approaches-and-Applications-a-Presentation-from-Intel-22-320.jpg){kind=tracking}
![© 2019 Intel
Improving Tracking Algorithms with Reinforcement
Learning
Deep reinforcement learning can be used for formulating object tracking as
a Markov decision process with a set of “tracking actions”.
23
[Yun et al. 2017]](https://image.slidesharecdn.com/t1t6parkintel2019-190702190540/85/Object-Trackers-Approaches-and-Applications-a-Presentation-from-Intel-23-320.jpg){kind=tracking}
"Object Trackers: Approaches and Applications," a Presentation from Intel
- 1. © 2019 Intel Object Trackers : Approaches and Applications Minje Park, Sean Pyo, Songki Choi, Jinwook Lee, Peter Roh Intel May 2019
- 2. © 2019 Intel Outline Why we need trackers Trackers with different temporal coverages Trackers with temporal and spatial scheduling Enhancing trackers with deep learning 2
- 3. © 2019 Intel Object Trackers Efficient visual analysis exploiting temporal coherence • Detection is generally slow • Detection is generally unstable across frames Identity-preserving visual analysis • Detection doesn’t know correspondence between frames • Detection is class-specific not instance-specific 3
- 4. © 2019 Intel Object Trackers (from Algorithm Perspective) 4 Particle Filter Kalman Filter Mean Shift Template Matching Kernelized Correlation Filter Tracking by Detection MOSSE Tracker ...
- 5. © 2019 Intel Object Trackers from Temporal Coverage Perspective Short-term trackers Long-term trackers Zero-term trackers (Multi-target trackers) 5
- 6. © 2019 Intel Short-term Trackers Detection is fast enough for regular/quick recovery • Assume no self-recovery (recovery is done by detection) • Very straightforward and applicable for many cases • Drifting is handled by detector reset 6 D T T T D T T T D T T T
- 7. © 2019 Intel Long-term Trackers Detection is slow or unstable across frames • Assume tracker has self-recovery mechanism (keep updating the internal representation of the current target) • Drifting is handled by model update (of tracker) 7 D T T T T T T T R T T T track fails but recovered by tracker itself
- 8. © 2019 Intel State Transition with Detector 8 Full detector Tracker Partial detector tracker successes (w/ confidence threshold) tracker fails and auto- recovery also fails partial detection successes target lost target initialized
- 9. © 2019 Intel State Transition with Detector (An Example Implementation) 9 Faster RCNN* (full image) KCF** RCNN* (cropped region) tracker successes (w/ confidence threshold) tracker fails and auto- recovery also fails partial detection successes target lost target initialized *RCNN: Region CNN **KCF : Kernelized Correlation Filter
- 10. © 2019 Intel Difficult to Scale to Many Objects Single target trackers are fast (few ms) but it can’t scale to many concurrent targets. 10 detection tracking detection tracking 30 ms 60 ms 30 ms
- 11. © 2019 Intel Zero-term Trackers (Multi-target Trackers) Let’s assume that our main concern is identity-preservation • Fixed compute resource consumption agnostic to the number of targets (this is desirable for robust system design) • Globally-optimal identity assignment for all concurrent targets 11 D D, MTT D , MTT D , MTT D , MTT D , MTT D , MTT D , MTT D , MTT D , MTT D , MTT D , MTT 0.3 0.9 0.1 0.8 0.2 0.1 0.3 0.0 0.7 s1 s2 s3 ... ... current frame
- 12. © 2019 Intel Zero-term Trackers (Multi-target Trackers) Spaced detection + Short/Long-term tracking • Classic way of combining detection and tracking • Overall system performance is dominated by trackers • Useful for small number of concurrent targets Per-frame detection + Zero-term tracking • Overall performance is dominated by detector • Useful when the number of concurrent targets is changing (or large) 12
- 13. © 2019 Intel Temporal Scheduling In both cases full detection is still too slow However, we can trade-off between latency and fps by distributing detection computation into several frames Keep system resource utilization as flat as possible 13 1/4 2/4 3/4 4/4 1/4 2/4 3/4 T4/4 1/4 2/4 D1 T2 T3 T4 D5 T6 T7 T8 D9 T10 ... ... An example of distributing the processing of a single frame into four frames by allowing a latency of four frames
- 14. © 2019 Intel Spatial Scheduling Tradeoff between latency and fps can be also achieved by processing image sub-regions 14 1 2 43 5 1/5 2/5 3/5 4/5 5/5 1/5 2/5 ... Tracker handles targets until the next call for detection An example of distributing the processing of a single frame into five frames by allowing a latency of five frames. Unlike the previous temporal scheduling, each frame only processes a partial region of the entire frame.
- 15. © 2019 Intel Partial Detection RCNN shows good regression results on already cropped regions if there is temporal coherency 15 Run RCNN on slightly increased cropped ROI frame (t-1) frame t (before partial detection) frame t (after partial detection) updated detection old detection
- 16. © 2019 Intel Putting It All Together 16 An example implementation with spatial scheduling, full/partial detectors, and multi-target tracker for a scene with dense subjects 1 2 43 5 1/5 2/5 3/5 4/5 5/5 1/5 2/5 ... - Spatial scheduling : Sub-region detection, 10 ms (vs. full-region detection, 30 ms) - Partial detection, <1 ms/subject - Multi-target tracking, 5 ms MTT keeps subject IDs across frames Partial detection handles targets until the next call for sub-region detection
- 17. © 2019 Intel Application: Face recognition (short-term) 17
- 18. © 2019 Intel Application: Target Tracking (long-term) 18
- 19. © 2019 Intel Application: Target Tracking (long-term) 19
- 20. © 2019 Intel Application: People Counting (zero-term) 20
- 21. © 2019 Intel Application: People Counting (zero-term) 21
- 22. © 2019 Intel Improving Tracking Algorithms with Deep Learning The core of tracking is how we can compare two different image patches. Short/Long/Multi-target trackers all use this basic functionality in its core, and deep learning can boost the accuracy of image patch comparison. • Using deeply-learned features (e.g. ImageNet) • Using metric learning (e.g. Siamese network) 22 [Bertinetto et al. 2017]
- 23. © 2019 Intel Improving Tracking Algorithms with Reinforcement Learning Deep reinforcement learning can be used for formulating object tracking as a Markov decision process with a set of “tracking actions”. 23 [Yun et al. 2017]
- 24. © 2019 Intel Summary Trackers provide you with design flexibility. Recent advances in detectors alleviate the need for short/long term trackers from a performance perspective, but a tracker is a good choice for identity preservation. Temporal or spatial scheduling with proper implementation of detectors and trackers enables robust and scalable applications. Deep learning makes the tracking algorithms more accurate and robust 24
- 25. © 2019 Intel Resource 25 Intel OpenVINO Toolkit https://software.intel.com/en-us/openvino- toolkit OpenCV Tracking API https://docs.opencv.org/3.4/d9/df8/group_ _tracking.html OT BMT & Datasets OTB Challenge https://github.com/foolwood/benchmark_re sults MOT Challenge https://motchallenge.net/results/MOT17