Object Detection Beyond Mask R-CNN and RetinaNet I
3 likes3,643 views
This document summarizes a tutorial on object detection beyond RetinaNet and Mask R-CNN. It discusses challenges in object detection including the backbone network, detection head, pretraining, handling scale variations, large batch sizes, detecting objects in crowds, and neural architecture search. It also introduces recent works that aim to address these challenges, such as DetNet, Light Head R-CNN, Objects365 pretraining, SFace for scale, MegDet for batch size, CrowdHuman benchmark for crowds, and NAS approaches. The document concludes that further improving object detection requires focusing on details and that continued progress will significantly benefit computer vision applications.
Object Detection Beyond Mask R-CNN and RetinaNet I
- 1. Gang Yu ζ· θ§ η η©Ά ι’ Object Detection in Recent 3 Years Beyond RetinaNet and Mask R-CNN
- 2. Schedule of Tutorial β’ Lecture 1: Beyond RetinaNet and Mask R-CNN (Gang Yu) β’ Lecture 2: AutoML for Object Detection (Xiangyu Zhang) β’ Lecture 3: Finegrained Visual Analysis (Xiu-shen Wei)
- 3. Outline β’ Introduction to Object Detection β’ Modern Object detectors β’ One Stage detector vs Two-stage detector β’ Challenges β’ Backbone β’ Head β’ Pretraining β’ Scale β’ Batch Size β’ Crowd β’ NAS β’ Fine-Grained β’ Conclusion
- 4. Outline β’ Introduction to Object Detection β’ Modern Object detectors β’ One Stage detector vs Two-stage detector β’ Challenges β’ Backbone β’ Head β’ Pretraining β’ Scale β’ Batch Size β’ Crowd β’ NAS β’ Fine-Grained β’ Conclusion
- 5. What is object detection?
- 6. What is object detection?
- 7. Detection - Evaluation Criteria Average Precision (AP) and mAP Figures are from wikipedia
- 8. Detection - Evaluation Criteria mmAP Figures are from http://cocodataset.org
- 9. How to perform a detection? β’ Sliding window: enumerate all the windows (up to millions of windows) β’ VJ detector: cascade chain β’ Fully Convolutional network β’ shared computation Robust Real-time Object Detection; Viola, Jones; IJCV 2001 http://www.vision.caltech.edu/html-files/EE148-2005-Spring/pprs/viola04ijcv.pdf
- 10. General Detection Before Deep Learning β’ Feature + classifier β’ Feature β’ Haar Feature β’ HOG (Histogram of Gradient) β’ LBP (Local Binary Pattern) β’ ACF (Aggregated Channel Feature) β’ β¦ β’ Classifier β’ SVM β’ Bootsing β’ Random Forest
- 13. General Detection Before Deep Learning Traditional Methods β’ Pros β’ Efficient to compute (e.g., HAAR, ACF) on CPU β’ Easy to debug, analyze the bad cases β’ reasonable performance on limited training data β’ Cons β’ Limited performance on large dataset β’ Hard to be accelerated by GPU
- 14. Deep Learning for Object Detection Based on the whether following the βproposal and refineβ β’ One Stage β’ Example: Densebox, YOLO (YOLO v2), SSD, Retina Net β’ Keyword: Anchor, Divide and conquer, loss sampling β’ Two Stage β’ Example: RCNN (Fast RCNN, Faster RCNN), RFCN, FPN, MaskRCNN β’ Keyword: speed, performance
- 15. A bit of History Image Feature Extractor classification localization (bbox) One stage detector Densebox (2015) UnitBox (2016) EAST (2017) YOLO (2015) Anchor Free Anchor importedYOLOv2 (2016) SSD (2015) RON(2017) RetinaNet(2017) DSSD (2017) two stages detector Image Feature Extractor classification localization (bbox) Proposal classification localization (bbox) Refine RCNN (2014) Fast RCNN(2015) Faster RCNN (2015) RFCN (2016) MultiBox(2014) RFCN++ (2017) FPN (2017) Mask RCNN (2017) OverFeat(2013)
- 16. Outline β’ Introduction to Object Detection β’ Modern Object detectors β’ One Stage detector vs Two-stage detector β’ Challenges β’ Backbone β’ Head β’ Pretraining β’ Scale β’ Batch Size β’ Crowd β’ NAS β’ Fine-Grained β’ Conclusion
- 17. Modern Object detectors Backbone Head β’ Modern object detectors β’ RetinaNet β’ f1-f7 for backbone, f3-f7 with 4 convs for head β’ FPN with ROIAlign β’ f1-f6 for backbone, two fcs for head β’ Recall vs localization β’ One stage detector: Recall is high but compromising the localization ability β’ Two stage detector: Strong localization ability Postprocess NMS
- 18. One Stage detector: RetinaNet β’ FPN Structure β’ Focal loss Focal Loss for Dense Object DetectionοΌ Lin etc, ICCV 2017 Best student paper
- 19. One Stage detector: RetinaNet β’ FPN Structure β’ Focal loss Focal Loss for Dense Object DetectionοΌ Lin etc, ICCV 2017 Best student paper
- 20. Two-Stage detector: FPN/Mask R-CNN β’ FPN Structure β’ ROIAlign Mask R-CNNοΌ He etc, ICCV 2017 Best paper
- 21. What is next for object detection? β’ The pipeline seems to be mature β’ There still exists a large gap between existing state-of-arts and product requirements β’ The devil is in the detail
- 22. Outline β’ Introduction to Object Detection β’ Modern Object detectors β’ One Stage detector vs Two-stage detector β’ Challenges β’ Backbone β’ Head β’ Pretraining β’ Scale β’ Batch Size β’ Crowd β’ NAS β’ Fine-Grained β’ Conclusion
- 23. Challenges Overview β’ Backbone β’ Head β’ Pretraining β’ Scale β’ Batch Size β’ Crowd β’ NAS β’ Fine-grained Backbone Head Postprocess NMS
- 24. Challenges - Backbone β’ Backbone network is designed for classification task but not for localization task β’ Receptive Field vs Spatial resolution β’ Only f1-f5 is pretrained but randomly initializing f6 and f7 (if applicable)
- 25. Backbone - DetNet β’ DetNet: A Backbone network for Object Detection, Li etc, 2018, https://arxiv.org/pdf/1804.06215.pdf
- 31. Challenges - Head β’ Speed is significantly improved for the two-stage detector β’ RCNN - > Fast RCNN -> Faster RCNN - > RFCN β’ How to obtain efficient speed as one stage detector like YOLO, SSD? β’ Small Backbone β’ Light Head
- 32. Head β Light head RCNN β’ Light-Head R-CNN: In Defense of Two-Stage Object Detector, 2017, https://arxiv.org/pdf/1711.07264.pdf Code: https://github.com/zengarden/light_head_rcnn
- 33. Head β Light head RCNN β’ Backbone β’ L: Resnet101 β’ S: Xception145 β’ Thin Feature map β’ L:C_{mid} = 256 β’ S: C_{mid} =64 β’ C_{out} = 10 * 7 * 7 β’ R-CNN subnet β’ A fc layer is connected to the PS ROI pool/Align
- 34. Head β Light head RCNN
- 35. Head β Light head RCNN
- 36. Head β Light head RCNN β’ Mobile Version β’ ThunderNet: Towards Real-time Generic Object Detection, Qin etc, Arxiv 2019 β’ https://arxiv.org/abs/1903.11752
- 37. Pretraining β Objects365 β’ ImageNet pretraining is usually employed for backbone training β’ Training from Scratch β’ Scratch Det claims GN/BN is important β’ Rethinking ImageNet Pretraining validates that training time is important
- 38. Pretraining β Objects365 β’ Objects365 Dataset
- 39. Pretraining β Objects365 β’ Pretraining with Objects365 vs ImageNet vs from Sctratch
- 40. Pretraining β Objects365 β’ Pretraining on Backbone or Pretraining on both backbone and head
- 41. Pretraining β Objects365 β’ Results on VOC Detection & VOC Segmentation
- 42. Pretraining β Objects365 β’ Summary β’ Pretraining is important to reduce the training time β’ Pretraining with a large dataset is beneficial for the performance
- 43. Challenges - Scale β’ Scale variations is extremely large for object detection
- 44. Challenges - Scale β’ Scale variations is extremely large for object detection β’ Previous works β’ Divide and Conquer: SSD, DSSD, RON, FPN, β¦ β’ Limited Scale variation β’ Scale Normalization for Image Pyramids, Singh etc, CVPR2018 β’ Slow inference speed β’ How to address extremely large scale variation without compromising inference speed?
- 45. Scale - SFace β’ SFace: An Efficient Network for Face Detection in Large Scale Variations, 2018, http://cn.arxiv.org/pdf/1804.06559.pdf β’ Anchor-based: β’ Good localization for the scales which are covered by anchors β’ Difficult to address all the scale ranges of faces β’ Anchor-free: β’ Able to cover various face scales β’ Not good for the localization ability
- 46. Scale - SFace
- 47. Scale - SFace
- 48. Scale - SFace
- 49. Scale - SFace β’ Summary: β’ Integrate anchor-based and anchor-free for the scale issue β’ A new benchmark for face detection with large scale variations: 4K Face
- 50. Challenges - Batchsize β’ Small mini-batchsize for general object detection β’ 2 for R-CNN, Faster RCNN β’ 16 for RetinaNet, Mask RCNN β’ Problem with small mini-batchsize β’ Long training time β’ Insufficient BN statistics β’ Inbalanced pos/neg ratio
- 51. Batchsize β MegDet β’ MegDet: A Large Mini-Batch Object Detector, CVPR2018, https://arxiv.org/pdf/1711.07240.pdf
- 52. Batchsize β MegDet β’ Techniques β’ Learning rate warmup β’ Cross-GPU Batch Normalization
- 53. Challenges - Crowd β’ NMS is a post-processing step to eliminate multiple responses on one object instance β’ Reasonable for mild crowdness like COCO and VOC β’ Will Fail in the case when the objects are in a crowd
- 54. Challenges - Crowd β’ A few works have been devoted to this topic β’ Softnms, Bodla etc, ICCV 2017, http://www.cs.umd.edu/~bharat/snms.pdf β’ Relation Networks, Hu etc, CVPR 2018, https://arxiv.org/pdf/1711.11575.pdf β’ Lacking a good benchmark for evaluation in the literature
- 55. Crowd - CrowdHuman β’ CrowdHuman: A Benchmark for Detecting Human in a Crowd, 2018, https://arxiv.org/pdf/1805.00123.pdf, http://www.crowdhuman.org/ β’ A benchmark with Head, Visible Human, Full body bounding-box β’ Generalization ability for other head/pedestrian datasets β’ Crowdness
- 57. Crowd-CrowdHuman
- 58. Crowd-CrowdHuman β’ Generalization β’ Head β’ Pedestrian β’ COCO
- 59. Conclusion β’ The task of object detection is still far from solved β’ Details are important to further improve the performance β’ Backbone β’ Head β’ Pretraining β’ Scale β’ Batchsize β’ Crowd β’ The improvement of object detection will be a significantly boost for the computer vision industry
- 60. εΉΏει¨ε β’ Megvii Detection η₯δΉδΈζ Email: yugang@megvii.com