Deep learning for person re-identification
- 1. Deep learning for person re-identification Jingya Wang jingya.wang@usyd.edu.au
- 3. Person re-identification (re-id) aims at matching people across non-overlapping camera views distributed at distinct locations. Camera A Camera B
- 4. Presentation Outline • Supervised Person Re-Identification • Unsupervised Person Re-Identification • Active Learning for Person Re-Identification
- 5. Supervised Person Re-Identification • Training and testing data are from same domain Lavi, B., Serj, M.F. and Ullah, I., 2018. Survey on deep learning techniques for person re-identification task Contrastive loss Triplet lossClassification loss
- 10. Presentation Outline • Supervised Person Re-Identification • Unsupervised Person Re-Identification • Active Learning for Person Re-Identification
- 11. Unsupervised Person Re-Identification Training and testing data are from different domain -> learned on the source domain and transfer the knowledge to target domain (unsupervised domain adaptation) Train: Market1501 Test: DUKE
- 12. Challenges: • Source and target domains have unknown camera viewing conditions • The identity/class between source and target domains are non-overlapping therefore presents a more challenging open-set recognition problem -> Transferring knowledge of the source domain to target domain in attribute space Wang J, Zhu X, Gong S, Li W. Transferable joint attribute-identity deep learning for unsupervised person re-identification. CVPR,2018 Unsupervised Person Re-Identification
- 13. Wang J, Zhu X, Gong S, Li W. Transferable joint attribute-identity deep learning for unsupervised person re-identification. CVPR,2018 Unsupervised Person Re-Identification
- 14. Unsupervised Target Domain Adaptation Wang J, Zhu X, Gong S, Li W. Transferable joint attribute-identity deep learning for unsupervised person re-identification. CVPR,2018 Unsupervised Person Re-Identification
- 15. Unsupervised Person Re-Identification Image-to-image translation method: SPGAN Deng et al., Image-image domain adaptation with preserved self-similarity and domain-dissimilarity for person re-identification. CVPR 2018 preserved self-similarity and domain dissimilarity
- 16. Unsupervised Person Re-Identification Image-to-image translation method: CamStyle GAN Zhong, Zhun, et al. "Camera style adaptation for person re-identification." CVPR. 2018.
- 17. 10 300 600 900 1200 1500 1800 2100 2400 2700 3000 3300 3600 3900 4200 4500 4800 5100 5400 5700 6000 6300 6600 6900 7200 7500 7800 8100 8400 8700 9000 9300 9600 9900102001050010800 AnnotationCost Data Size Challenge For Re-ID rapid increase
- 18. Presentation Outline • Supervised Person Re-Identification • Unsupervised Person Re-Identification • Active Learning for Person Re-Identification
- 19. Make AI work in the real world: Human-In-The-Loop Human-in-the-Loop (HITL) explores human feedback in an incremental learning cycle of the machine for rapid model domain adaptation
- 20. Active learning is a special case of machine learning in which a learning algorithm is able to interactively query the user (or some other information source) to obtain the desired outputs at new data points.
- 21. There are three scenarios for Active learning : 1. Membership Query Synthesis: the learner generates/constructs an instance (from some underlying natural distribution). 2. Stream-Based selective sampling, i.e, each sample is considered separately in our case for label-querying or rejection. Similarly to online-learning, the data is not saved, there are no assumptions on data distribution, and therefore it is adaptive to change.
- 22. 3. Pool-Based sampling, i.e., sampled are chosen from a pool of unlabeled data for the purpose of labeling
- 23. Training Pool Agent request label model query Training Pool selection strategy request label model Active Learning Person Re-Identification Liu, Z.*, Wang, J *., Gong, S., Lu, H. and Tao, D. Deep Reinforcement Active Learning for Human-in-the-Loop Person Re-Identification. ICCV, 2019,Oral
- 24. Concept A user annotates few informative pedestrian pairs recommended by an adaptive agent in a human-in-the-loop learning mechanism Re-ID Model Sample Selection (agent) annotator Pairwise Data human-in-the-loop Agent action!" query ancho r query for label unlabeled gallery pool state reward Goal: Sample Informative Pair Action: Select One Sample at Each Step State: Reflect Sample Correlation Reward: Uncertainty Liu, Z.*, Wang, J *., Gong, S., Lu, H. and Tao, D. Deep Reinforcement Active Learning for Human-in-the-Loop Person Re-Identification. ICCV, 2019,Oral
- 25. STATE annotator Re-ID Loss(Triplet) REWARD ACTION Sample Selection Strategy gallery pool query q … g 1 g 2 g N 0 0.83 0.71 0.66 0.47 0.36 0.83 0 0.85 0 0.87 0 0.71 0.85 0 0 0 0 0.66 0 0 0 0 0 0.47 0.87 0 0 0 0.77 0.36 0 0 0 0.77 0 gKq Methodology Joint Reinforcement Active Learning in A Deep Network false CNN ! Agent " Liu, Z.*, Wang, J *., Gong, S., Lu, H. and Tao, D. Deep Reinforcement Active Learning for Human-in-the-Loop Person Re-Identification. ICCV, 2019,Oral
- 26. Action: Select One Sample at Each Step State: We construct a sparse similarity graph among query and gallery samples and take it as the state value (Reflect Sample Correlation) 1. Base CNN Network 2. A Deep Reinforced Active Learner - An Agent Methodology
- 27. An example of state updating with different human feedback Reward: we perform a similar hard triplet loss to measure the uncertainty of data.
- 28. Dataset & Result (Market-1501) Supervised Transfer unsupervised 87.95 84.2 42.79 Ours 0.15% annotated 100% annotated 0% annotated 73.25 66.26 20.04 R-1 mAP
- 30. Presentation Outline Link Person Re-Identification with …. • Attribute Learning • Detection (Person Search ) • Tracking (Multi-target multi-camera tracking)
- 31. Attribute recognition usually denotes local structures of a person Person Re-Identification and Attribute Learning Ø How do human brain match person? Long hair bag 31 Attribute recognition usually denotes local structures of a person Person Re-Iden3fica3on and A6ribute Learning Ø How do human brain match person? Long hair bag
- 32. Attribute Recognition in in Surveillance ØChallenges: • Poor image quality • Complex background clutter • Uncontrolled viewing conditions • Small number of labelled training
- 33. ØMain idea: •Discover the interdependency and correlation among attributes •Explore visual context as an extra information source to assist attribute recognition ØContributions: •A novel end-to-end encoder-decoder architecture capable of jointly learning image level context and attribute level sequential correlation •Exploit more latent and richer higher-order dependency among attributes Wang, J, et al. "Attribute recognition by joint recurrent learning of context and correlation." ICCV. 2017 Attribute Recognition in in Surveillance
- 34. Person Re-Identification and Attribute Recognition Lin, Yutian, et al. "Improving person re-identification by attribute and identity learning." Pattern Recognition (2019).
- 35. Attribute-based Person Re-Identification 35 •Teenager •Backpack •Pants •Short bottom wear •Short top wear •Long hair •Female •Top white •Bottom blue Ranked retrieval results Query attribute descriptions Gallery images
- 36. Yin, Zhou, et al. "Adversarial attribute-image person re-identification." IJCAI, 2018. Attribute-based Person Re-Identification
- 37. Presentation Outline Link Person Re-Identification with …. • Attribute Learning • Detection (Person Search ) • Tracking (Multi-target multi-camera tracking)
- 38. Person Re-Identification and Detection Zheng, L., Yang, Y., & Hauptmann, A. G. (2016). Person re-identification: Past, present and future.
- 40. Detection Xiao, Tong, et al. "Joint detection and identification feature learning for person search." CVPR. 2017 Person Re-Identification and Detection
- 41. Liu, Hao, et al. "Neural person search machines." ICCV. 2017. Person Re-Identification and Detection
- 42. Presentation Outline Link Person Re-Identification with …. • Attribute Learning • Detection (Person Search ) • Tracking (Multi-target multi-camera tracking)
- 43. Multi-target multi-camera tracking 1st MTMCT and ReID workshop CVPR 2017 2nd MTMCT and ReID workshop CVPR 2019 Duke MTMC (Multi-Target, Multi-Camera) dataset
- 44. Multi-target multi-camera tracking Ristani, Ergys, and Carlo Tomasi. "Features for multi-target multi- camera tracking and re-identification." CVPR. 2018.
- 45. Conclusion • Supervised Person Re-Identification • Unsupervised Person Re-Identification • Active Learning for Person Re-Identification Link Person Re-Identification with …. Ø Attribute Learning Ø Detection (Person Search ) Ø Tracking (Multi-target multi-camera tracking)
- 46. Thank you