[DL輪読会]Semi-convolutional Operators for Instance Segmentation
3 likes1,278 views
1. The document discusses techniques for instance segmentation, including semi-convolutional operations, instance coloring, and Mask R-CNN. 2. Semi-convolutional operations are proposed to address the position invariance of CNNs which causes different instances to have identical embeddings. 3. Instance coloring methods aim to distinguish instances by assigning unique colors, using techniques such as deep metric learning or recurrent pixel embedding.
![DEEP LEARNING JP
[DL Papers]
“Semi-convolutional Operators for Instance Segmentation”
,
http://deeplearning.jp/](https://image.slidesharecdn.com/semiconv-181005004455/85/DL-Semi-convolutional-Operators-for-Instance-Segmentation-1-320.jpg)
![Propose & Verify
5
• &
– SM N
⭕ P P
❌ VR
SM C
[2017, K. He] Mask R-CNN
[2014 R. Girshick] Rich feature hierarchies for accurate object detection and semantic segmentation](https://image.slidesharecdn.com/semiconv-181005004455/85/DL-Semi-convolutional-Operators-for-Instance-Segmentation-5-320.jpg)
![P & V (RCNN)
M CVR
C a VR
VR , e
Ø C 232 . ,.
Ø N & & . 1 F S P 6
[2014 R. Girshick] Rich feature hierarchies for accurate object detection and semantic segmentation](https://image.slidesharecdn.com/semiconv-181005004455/85/DL-Semi-convolutional-Operators-for-Instance-Segmentation-6-320.jpg)
![P & V (Mask RCNN)
. 1
Ø 2 3 7
[2017 K. He] Mask R-CNN](https://image.slidesharecdn.com/semiconv-181005004455/85/DL-Semi-convolutional-Operators-for-Instance-Segmentation-7-320.jpg)
![Instance Coloring
•
– I C
⭕ (
⭕ ) D(
8
[2017 A. Kendall] Multi-Task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semantics](https://image.slidesharecdn.com/semiconv-181005004455/85/DL-Semi-convolutional-Operators-for-Instance-Segmentation-8-320.jpg)
![IC
•
•
9
[2017 A. Fathi] Semantic Instance Segmentation via Deep Metric Learning](https://image.slidesharecdn.com/semiconv-181005004455/85/DL-Semi-convolutional-Operators-for-Instance-Segmentation-9-320.jpg)
![IC
• N
• N
– R - -
• E
10
[2018 S. Kong] Recurrent Pixel Embedding for Instance Grouping
(参考) arXiveTimes (https://github.com/arXivTimes/arXivTimes/issues/628)](https://image.slidesharecdn.com/semiconv-181005004455/85/DL-Semi-convolutional-Operators-for-Instance-Segmentation-10-320.jpg)
![Mask RCNN
• M -
• + - C R
N a S
• + a
20
[2017 K. He] Mask R-CNN](https://image.slidesharecdn.com/semiconv-181005004455/85/DL-Semi-convolutional-Operators-for-Instance-Segmentation-20-320.jpg)
![• a e v R
• 2 1 P h w R
– . / & 0 1 0 0 A 8 0 2
2
– a c v
– dk R t ML C
– ] C C L h wR N C
• f g[ v R ML
– 0 8 P Vr[ uo
– s nl i
25](https://image.slidesharecdn.com/semiconv-181005004455/85/DL-Semi-convolutional-Operators-for-Instance-Segmentation-25-320.jpg)
[DL輪読会]Semi-convolutional Operators for Instance Segmentation
- 1. DEEP LEARNING JP [DL Papers] “Semi-convolutional Operators for Instance Segmentation” , http://deeplearning.jp/
- 2. • – )2 - • -0 2182 21 ( - 2 iP – O O O O t E – npeE a r iP • – s o luE – V rCmv S 2
- 4. • • 2) &) & – V P – • 2 () – C – I 4
- 5. Propose & Verify 5 • & – SM N ⭕ P P ❌ VR SM C [2017, K. He] Mask R-CNN [2014 R. Girshick] Rich feature hierarchies for accurate object detection and semantic segmentation
- 6. P & V (RCNN) M CVR C a VR VR , e Ø C 232 . ,. Ø N & & . 1 F S P 6 [2014 R. Girshick] Rich feature hierarchies for accurate object detection and semantic segmentation
- 7. P & V (Mask RCNN) . 1 Ø 2 3 7 [2017 K. He] Mask R-CNN
- 8. Instance Coloring • – I C ⭕ ( ⭕ ) D( 8 [2017 A. Kendall] Multi-Task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semantics
- 9. IC • • 9 [2017 A. Fathi] Semantic Instance Segmentation via Deep Metric Learning
- 10. IC • N • N – R - - • E 10 [2018 S. Kong] Recurrent Pixel Embedding for Instance Grouping (参考) arXiveTimes (https://github.com/arXivTimes/arXivTimes/issues/628)
- 11. IC 11 2つの羊はほ ぼ同じ見た目 CNNの演算の位置不変性により同 じ埋め込み表現となってしまいイ ンスタンスを区別できない 位置不変でない演算方法を提案 (Semi-Convolutional Operation) • • I C & V – N – P& I
- 12. • P V S l O I – & ( f vm t – ) - I rp C • o g i au I ceO O y • au i ) ) O Ø P n Os C 12
- 13. Semi-Convolutional Operation for IC (1) • • • • C 13 : X ! L⌦ (L 2 Rd )<latexit sha1_base64="EonYQ5G0ZK07MkJ3/smomCeY4Dg=">AAACu3ichVFNTxRBEH0MKDCorHox8TJxA8EL6SUkGI0JkYsHE5fFXTZhYNPTNLsNPR+Z6d0EJvMH+AMePGFijPEvePPCH/DA1ZvBGyRePFAzOwkqUavT3a9e1auu7vYirRLD2MmINTp27fr4xKQ9dePmrenK7TutJOzHQjZFqMO47fFEahXIplFGy3YUS+57Wq57eyt5fH0g40SFwSuzH8lNn3cDtaMEN0R1Ki233lOOmxcKHNfnpie4TtuZ45rw0n+RbaXuS192eWa7T/Jhz/0SdFxVij0vbWRb2w87lSqbZ4U5V0GtBFWUVg8r7+FiGyEE+vAhEcAQ1uBIaGygBoaIuE2kxMWEVBGXyGCTtk9ZkjI4sXu0dsnbKNmA/LxmUqgFnaJpxqR0MMO+sA/sjB2zj+wb+/nXWmlRI+9ln3ZvqJVRZ/rw3tqP/6p82g16l6p/9mywg0dFr4p6jwomv4UY6gcHr8/WHjdm0ln2lp1S/0fshH2mGwSDc/FuVTbewKYPqP353FdBa2G+Rnh1sbr8rPyKCdzHA8zRey9hGc9RR5PO/YSvOMV366klrF1LD1OtkVJzF7+Z1b8ApxWrPw==</latexit><latexit sha1_base64="EonYQ5G0ZK07MkJ3/smomCeY4Dg=">AAACu3ichVFNTxRBEH0MKDCorHox8TJxA8EL6SUkGI0JkYsHE5fFXTZhYNPTNLsNPR+Z6d0EJvMH+AMePGFijPEvePPCH/DA1ZvBGyRePFAzOwkqUavT3a9e1auu7vYirRLD2MmINTp27fr4xKQ9dePmrenK7TutJOzHQjZFqMO47fFEahXIplFGy3YUS+57Wq57eyt5fH0g40SFwSuzH8lNn3cDtaMEN0R1Ki233lOOmxcKHNfnpie4TtuZ45rw0n+RbaXuS192eWa7T/Jhz/0SdFxVij0vbWRb2w87lSqbZ4U5V0GtBFWUVg8r7+FiGyEE+vAhEcAQ1uBIaGygBoaIuE2kxMWEVBGXyGCTtk9ZkjI4sXu0dsnbKNmA/LxmUqgFnaJpxqR0MMO+sA/sjB2zj+wb+/nXWmlRI+9ln3ZvqJVRZ/rw3tqP/6p82g16l6p/9mywg0dFr4p6jwomv4UY6gcHr8/WHjdm0ln2lp1S/0fshH2mGwSDc/FuVTbewKYPqP353FdBa2G+Rnh1sbr8rPyKCdzHA8zRey9hGc9RR5PO/YSvOMV366klrF1LD1OtkVJzF7+Z1b8ApxWrPw==</latexit><latexit sha1_base64="EonYQ5G0ZK07MkJ3/smomCeY4Dg=">AAACu3ichVFNTxRBEH0MKDCorHox8TJxA8EL6SUkGI0JkYsHE5fFXTZhYNPTNLsNPR+Z6d0EJvMH+AMePGFijPEvePPCH/DA1ZvBGyRePFAzOwkqUavT3a9e1auu7vYirRLD2MmINTp27fr4xKQ9dePmrenK7TutJOzHQjZFqMO47fFEahXIplFGy3YUS+57Wq57eyt5fH0g40SFwSuzH8lNn3cDtaMEN0R1Ki233lOOmxcKHNfnpie4TtuZ45rw0n+RbaXuS192eWa7T/Jhz/0SdFxVij0vbWRb2w87lSqbZ4U5V0GtBFWUVg8r7+FiGyEE+vAhEcAQ1uBIaGygBoaIuE2kxMWEVBGXyGCTtk9ZkjI4sXu0dsnbKNmA/LxmUqgFnaJpxqR0MMO+sA/sjB2zj+wb+/nXWmlRI+9ln3ZvqJVRZ/rw3tqP/6p82g16l6p/9mywg0dFr4p6jwomv4UY6gcHr8/WHjdm0ln2lp1S/0fshH2mGwSDc/FuVTbewKYPqP353FdBa2G+Rnh1sbr8rPyKCdzHA8zRey9hGc9RR5PO/YSvOMV366klrF1LD1OtkVJzF7+Z1b8ApxWrPw==</latexit><latexit sha1_base64="EonYQ5G0ZK07MkJ3/smomCeY4Dg=">AAACu3ichVFNTxRBEH0MKDCorHox8TJxA8EL6SUkGI0JkYsHE5fFXTZhYNPTNLsNPR+Z6d0EJvMH+AMePGFijPEvePPCH/DA1ZvBGyRePFAzOwkqUavT3a9e1auu7vYirRLD2MmINTp27fr4xKQ9dePmrenK7TutJOzHQjZFqMO47fFEahXIplFGy3YUS+57Wq57eyt5fH0g40SFwSuzH8lNn3cDtaMEN0R1Ki233lOOmxcKHNfnpie4TtuZ45rw0n+RbaXuS192eWa7T/Jhz/0SdFxVij0vbWRb2w87lSqbZ4U5V0GtBFWUVg8r7+FiGyEE+vAhEcAQ1uBIaGygBoaIuE2kxMWEVBGXyGCTtk9ZkjI4sXu0dsnbKNmA/LxmUqgFnaJpxqR0MMO+sA/sjB2zj+wb+/nXWmlRI+9ln3ZvqJVRZ/rw3tqP/6p82g16l6p/9mywg0dFr4p6jwomv4UY6gcHr8/WHjdm0ln2lp1S/0fshH2mGwSDc/FuVTbewKYPqP353FdBa2G+Rnh1sbr8rPyKCdzHA8zRey9hGc9RR5PO/YSvOMV366klrF1LD1OtkVJzF7+Z1b8ApxWrPw==</latexit>
- 14. Semi-Convolutional Operation for IC (2) • a nvxC kN – r l r a u C C • c ( - - (- - - – a u n i – n N a u • r lt a u C – ) l muoCpNu – l mupNe S 14
- 15. Semi-Convolutional Operation for IC (3) • • () 15
- 16. Semi-Convolutional Operation for IC (4) • – – • 2 – 2 16
- 17. Steered bilateral filter • 2 2 ) – – 2 ( – 2 – ) ) 17
- 18. 1 .2 3 18
- 19. • O i • 5 8 10 a S • e • c e R • 10 1 5-10 2 51 C - 10 1 5-10 2 51 N 19
- 20. Mask RCNN • M - • + - C R N a S • + a 20 [2017 K. He] Mask R-CNN
- 21. • C C • M • R • ) ( N 21
- 22. • 2 • A2 10 22
- 23. 23
- 24. ( ) • a v – & ( o – ) - V c y C • p g eil r • g I ) ) f – p n V cg – p s S c • um OP fv t 24
- 25. • a e v R • 2 1 P h w R – . / & 0 1 0 0 A 8 0 2 2 – a c v – dk R t ML C – ] C C L h wR N C • f g[ v R ML – 0 8 P Vr[ uo – s nl i 25
- 26. • K KO S 7AIE K KHPOEK H 4LAM OKMN KM NO A 7ACIA O OEK • -EMNDE G E D A OPMA DEAM M DEAN KM PM OA K FA O AOA OEK NAI OE NACIA O OEK • 0 .A 2 NG • 0A HH 2PHOE 8 NG 1A M E C NE C AMO E OS OK AECD 1KNNAN KM 7 A A -AKIAOMS 7AI OE N • , ODE 7AI OE NO A 7ACIA O OEK E AAL 2AOME 1A M E C • 7 0K C A PMMA O ERAH I A E C KM NO A -MKPLE C 26
- 27. Appendix: Hough Voting • dn o u u H – 1 . . a – 1 l i 3 m s • 2 u o u s V o u h et g p 27 参考) 和歌山大学講義資料 http://www.wakayama-u.ac.jp/~wuhy/CV/CV2010/CV06.pdf