Backbone can not be trained at once rolling back to pre trained network for person re-identification
2 likes1,558 views
This document discusses a technique called "rolling back" to pre-trained networks for improving person re-identification (ReID) in deep learning models. ReID aims to match images of the same person across non-overlapping camera views. The technique involves fine-tuning a pre-trained convolutional neural network on a ReID dataset, but periodically rolling back higher-level layers to their original pre-trained weights to allow lower-level layers to train more. This incremental rolling back approach leads to better generalization performance compared to standard fine-tuning, achieving state-of-the-art results on ReID benchmarks without using additional data or model structures.
![• Low-level layers contain detail information.
• High-level layers contain semantic information
Perception and Intelligence Lab., Copyright © 2016 9
The roll of the conv filter
conv
conv
conv
Block1 Block2 Block3
conv
conv
Block4
conv
conv
FC
Block5
conv
conv
Feature extractor Classifier
low level layers
[edge, blob/texture]
high level layers
[object parts]
FC layers
[object classes]](https://image.slidesharecdn.com/backbonecannotbetrainedatoncerollingbacktopre-trainednetworkforpersonre-identification-190416055809/85/Backbone-can-not-be-trained-at-once-rolling-back-to-pre-trained-network-for-person-re-identification-9-320.jpg)
Backbone can not be trained at once rolling back to pre trained network for person re-identification
- 1. “ ” Perception and Intelligence Lab, School of ECE, ASRI, Seoul National University, Seoul, Korea Presenter : 2019.03.15 Backbone Can Not be Trained at Once: Rolling Back to Pre-trained Network for Person Re-identification AAAI-19 Youngmin Ro , Jongwon Choi , Dae Ung Jo , Byeongho Heo , Jongin Lim and Jin Young Choi Youngmin Ro
- 2. • From the non-overlapped cameras, • Find who is the same person. • Simply call it ReID Perception and Intelligence Lab., Copyright © 2016 2 What is Person Re-identification http://www.liangzheng.com.cn/Project/project_prw.html Query ReID dataset Gallery
- 3. • There are two strategies for ReID • Triplet loss • Cross entropy loss Perception and Intelligence Lab., Copyright © 2016 3 How to solve ReID CNN Anchor Positive Negative Anchor Positive Negative Negative pair Positive pair CNN Class: 1 2 3 4 5 010000 labelprediction § Consider each identities to classes Person Re-identification: Past, Present and Future(2016 arXiv Liang Zheng et al.) −
- 4. • Using Pre-trained on the large dataset (e.g., ImageNet à 1000class) Re-define Classifier to target dataset (e.g., Market-1501 à 751class) 1. Only train Classifier with freezing Feature extractor 2. Train Classifier and Feature extractor together. Perception and Intelligence Lab., Copyright © 2016 4 Fine tuning in ReID conv conv conv Block1 Block2 Block3 conv conv Block4 conv conv GAP Block5 conv conv FC Feature à 1D vector Classify target classes Feature extractor Classifier
- 5. • The superficial issue of ReID is to solve the following problems; pose variation, misalignment, background clutter, occlusion, missing body parts, illumination change Perception and Intelligence Lab., Copyright © 2016 5 Issues of ReID Market-1501 dataset
- 6. Perception and Intelligence Lab., Copyright © 2016 6 Another issue of ReID • In ReID, all dataset suffer from the insufficient training set • Insufficient training set can yield overfitting problem. • In Market-1501 dataset, training set has 751 classes(id) and about 10 images per class(id). (MNIST 10 class /5000 per class) (CIFAR 100 class/ 500 per class) Unlabeled samples generated by gan improve the person re-identification baseline in vitro (ICCV2017 Zhedong Zheng et al)
- 7. • An effective way to solve overfitting problems during network training Perception and Intelligence Lab., Copyright © 2016 7 What is the Rollback? Drop learning rate
- 8. • Generally, ReID uses the ImageNet pre-trained network. • We need to sufficiently fine-tune the low-level layers to improve the discriminant power for the specific class ‘person’ Perception and Intelligence Lab., Copyright © 2016 8 Why Rollback? “cherry”“container ship” “mite” “grille” “mushroom” “person”“leopard” Fine tune
- 9. • Low-level layers contain detail information. • High-level layers contain semantic information Perception and Intelligence Lab., Copyright © 2016 9 The roll of the conv filter conv conv conv Block1 Block2 Block3 conv conv Block4 conv conv FC Block5 conv conv Feature extractor Classifier low level layers [edge, blob/texture] high level layers [object parts] FC layers [object classes]
- 10. • Rollback to the pre-trained weight. • Rollback gives a chance for low-level layers to trained more. Perception and Intelligence Lab., Copyright © 2016 10 What is the Rollback? Block1 Block2 Block3 Block4 FC Block5 conv conv conv conv conv conv conv conv conv Feature extractor Classifier conv conv conv conv conv conv conv FC conv conv Fine-tuned Network (once train starts from pre-trained network) Pre-trained Network From ImageNet High-level blocks are rollbacked to weights of pre-trained one
- 11. • Rollback to the pre-trained weight. • Rollback gives a chance for low-level layers to trained more. Perception and Intelligence Lab., Copyright © 2016 11 What is the Rollback? Block1 Block2 Block3 Block4 FC Block5 conv conv conv conv conv conv conv conv conv conv conv conv Block1 Block2 Block3 conv conv Block4 conv conv FC Block5 conv conv Fine-tuned Network (once train starts from pre-trained network) Pre-trained Network From ImageNet Strengthened!!
- 12. Perception and Intelligence Lab., Copyright © 2016 12 Analysis of Rollback (Rollback w/o one block) 0.0005 0.005 0.05 0 10 20 30 baseline Block1 Block2 Block3 Block4 Block5 epochs trainloss Remaining low-level blocks (Block1, Block2, Block3) shows slower convergence, but better generalization performance
- 13. • When train loss does not decrease anymore, rollback high-level layers Perception and Intelligence Lab., Copyright © 2016 13 When/Where the Rollback? Remain blocks Rollback blocks 1st B1 B2+B3+B4+B5 2nd B1+B2 B3+B4+B5 3rd B1+B2+B3 B4+B5 • Where (In the incremental manner) 0.0003 0.003 0.03 0.3 0 10 20 30 40 epochs trainloss learning rate: 0.01 Does not decrease learning rate: 0.001
- 14. Perception and Intelligence Lab., Copyright © 2016 14 Power of the Rollback • Rollback and Rollback again, then performance grows up!! Red line(our) Blue line(base)
- 15. Perception and Intelligence Lab., Copyright © 2016 15 Generalization of Rollback • Rollback can be adopted to various architecture and various task.
- 16. Perception and Intelligence Lab., Copyright © 2016 16 This Simple Rollback achieved S-O-T-A • No additional information! • No additional structure! • No data augmentation! (only horizontal flip) CVPR2018
- 17. Discrimination ability • Rollback can generate a more distinguishable feature than the baseline.
- 18. Perception and Intelligence Lab., Copyright © 2016 18 Conclusion • In this paper, we proposed a refine tuning method with a rolling-back scheme which further enhances the backbone network. • The key idea of the rolling-back scheme is to restore the weights in a part of the backbone network to the pre-trained weights when the fine-tuning converges at a premature state. • To escape from the premature state, we adopt an incremental refine tuning strategy by applying the fine tuning repeatedly, along with the rolling-back. • According to the experimental results, the rolling-back scheme makes a meaningful contribution to enhancement of the backbone network where it derives the convergence to a local basin of a good generalization performance. • As a result, our method without any add-on scheme could outperform the state-of-the-arts with help of add-on scheme.