![𝐸 𝑥 =
𝑖
𝜓𝑖(𝑥𝑖) +
𝑖,𝑗
𝜓𝑖,𝑗(𝑥𝑖, 𝑥𝑗)
𝜓𝑖 𝑥𝑖 = −log P(𝑥𝑖)
𝜓𝑖,𝑗 𝑥𝑖, 𝑥𝑗 = 𝜇 𝑥𝑖, 𝑥𝑗 [𝑤1 exp −
𝑝𝑖 − 𝑝𝑗
2
2𝜎 𝛼
2 −
𝐼𝑖 − 𝐼𝑗
2
2𝜎𝛽
2 + 𝑤2 exp −
𝑝𝑖 − 𝑝𝑗
2
2𝜎 𝛾
2 ]
𝜇 𝑥𝑖, 𝑥𝑗 = ቊ
1 𝑥𝑖 ≠ 𝑥𝑗
0 𝑂𝑡ℎ𝑒𝑟𝑤𝑖𝑠𝑒
𝑝𝑖, 𝑝𝑗: Pixel 위치
𝐼𝑖, 𝐼𝑗: Pixel RGB값
𝑤1, 𝑤2: Kernel Weights
𝜎 𝛼, 𝜎 𝛼, 𝜎 𝛼: Hyper-parameter
Efficient inference infully connectedcrfswithgaussianedgepotentials, Krähenbühl etal,NIPS2011
Unary Term (from Classifier)
Pairwise Term
의미?
Pixel이 서로 비슷한데
(위치적으로, RGB상으로)
Label이 서로 다르면
Energy 증가하여 Penalty](https://image.slidesharecdn.com/pr045deeplabsemanticsegmentation-171105113047/85/Pr045-deep-lab_semantic_segmentation-41-320.jpg)
![계산량이 많으니까 Mean Field Approximation
Efficient inference infully connectedcrfswithgaussianedgepotentials, Krähenbühl etal,NIPS2011
𝑃 𝑋 𝐼 =
1
𝑍(𝐼)
exp −𝐸(𝑋)
𝜓𝑖,𝑗 𝑥𝑖, 𝑥𝑗 = 𝜇 𝑥𝑖, 𝑥𝑗 [𝑤1 exp −
𝑝𝑖 − 𝑝𝑗
2
2𝜎 𝛼
2 −
𝐼𝑖 − 𝐼𝑗
2
2𝜎𝛽
2 + 𝑤2 exp −
𝑝𝑖 − 𝑝𝑗
2
2𝜎𝛾
2 ]
𝑄𝑖 𝑥𝑖 = 𝑙 =
1
𝑍𝑖
exp −𝜓 𝑢 𝑥𝑖 −
𝑙′
𝜇 𝑙, 𝑙′
𝑚
𝑤 𝑚
𝑗
𝑘 𝑚 𝑓𝑖, 𝑓𝑗 𝑄𝑗(𝑙′)
대신에 𝑄 𝑋 = ς𝑖 𝑄𝑖(𝑋𝑖) 를 정의하고 𝐷 𝐾𝐿(𝑄||𝑃)를 최소화하도록 만들면
아래와 같은 Update 식을 얻을 수 있음
• Efficient Inference in Fully Connected CRFs with Gaussian Edge Potentials: Supplementary Material
• Chapter 11.5 of Koller and Friedman “Probabilistic Graphical Models: Principles and Techniques”, 2009](https://image.slidesharecdn.com/pr045deeplabsemanticsegmentation-171105113047/85/Pr045-deep-lab_semantic_segmentation-42-320.jpg)
![CRF Learning with Validation (DeepLab v2)
𝜓𝑖,𝑗 𝑥𝑖, 𝑥𝑗 = 𝜇 𝑥𝑖, 𝑥𝑗 [𝑤1 exp −
𝑝𝑖 − 𝑝𝑗
2
2𝜎 𝛼
2 −
𝐼𝑖 − 𝐼𝑗
2
2𝜎𝛽
2 + 𝑤2 exp −
𝑝𝑖 − 𝑝𝑗
2
2𝜎 𝛾
2 ]](https://image.slidesharecdn.com/pr045deeplabsemanticsegmentation-171105113047/85/Pr045-deep-lab_semantic_segmentation-44-320.jpg)
![𝐼 Network
𝜓 𝑢 𝑥𝑖
𝑄𝑖
𝑄𝑖
෨𝑄𝑖
𝑚
𝑙 =
𝑗
𝑘 𝑚
𝑓𝑖, 𝑓𝑗 𝑄𝑗 𝑙
ෘ𝑄𝑖(𝑙) =
𝑚
𝑤 𝑚 ෨𝑄𝑖
𝑚
𝑙
𝑄𝑖 𝑙 =
𝑙′
𝜇 𝑙, 𝑙′ ෘ𝑄𝑖 𝑙
ሖ𝑄𝑖 𝑙 = −𝜓 𝑢 𝑥𝑖 − 𝑄𝑖 𝑙
𝑄𝑖 =
1
𝑍𝑖
exp( ሖ𝑄𝑖 𝑙 )
𝑄𝑖
𝐼
𝜓𝑖,𝑗 𝑥𝑖, 𝑥𝑗 = 𝜇 𝑥𝑖, 𝑥𝑗 [𝑤1 exp −
𝑝𝑖 − 𝑝𝑗
2
2𝜎 𝛼
2 −
𝐼𝑖 − 𝐼𝑗
2
2𝜎𝛽
2 + 𝑤2 exp −
𝑝𝑖 − 𝑝𝑗
2
2𝜎𝛾
2 ]
Message Passing
Weighting
Compatibility Trans.
𝑄𝑖 𝑙
𝜓 𝑢 𝑥𝑖
Addition
Normalization
𝑄𝑖
Conditional RandomFieldsasRecurrent NeuralNetworks, Zhengetal,ICCV2015](https://image.slidesharecdn.com/pr045deeplabsemanticsegmentation-171105113047/85/Pr045-deep-lab_semantic_segmentation-46-320.jpg)
Pr045 deep lab_semantic_segmentation
- 1. PR-045, 5th Nov, 2017 MVPLAB @ Yonsei Univ.
- 2. 1. Fully Convolutional Networks for Semantic Segmentation Arxiv 2014, CVPR 2015, TPAMI 2017 2. Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs Arxiv 2014, ICLR 2015 3. Multi Scale Context Aggregation by Dilated Convolution Arxiv 2015, ICLR 2016 4. DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution and Fully Connected CRFs Arxiv 2016, Accepted to TPAMI 5. Pyramid Scene Parsing Network Arxiv 2016, CVPR 2017 6. Rethinking Atrous Convolution for Semantic Image Segmentation Arxiv 2017
- 3. 1. Fully Convolutional Networks for Semantic Segmentation Arxiv 2014, CVPR 2015, TPAMI 2017 2. Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs Arxiv 2014, ICLR 2015 3. Multi Scale Context Aggregation by Dilated Convolution Arxiv 2015, ICLR 2016 4. DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution and Fully Connected CRFs Arxiv 2016, Accepted to TPAMI 5. Pyramid Scene Parsing Network Arxiv 2016, CVPR 2017 6. Rethinking Atrous Convolution for Semantic Image Segmentation Arxiv 2017 FCN DeepLab DilatedConv DeepLab v2 PSPNet DeepLab v3
- 5. Microsoft COCO: Common Objects in Context, Arxiv 2015 Pixel-level Dense Prediction Instance-level Object Detection Today
- 6. Slides from ICCV 17 COCO Challenge Workshop by FAIR Today
- 7. • IoU (Intersection Over Union) = TP / (TP+FP+FN) GT Prediction True Positive False Negative False Positive IoU =
- 9. Cityscapes
- 10. Cityscapes • Pixel Level Segmentation • Instance Level Segmentation
- 11. “DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution and Fully Connected CRFs”, 2016 • Networks • DilatedConv • DeepLab • PSPNet • DeepLab v3
- 12. Baseline “Fully Convolutional NetworksforSemantic Segmentation”, 2014
- 17. VGG16 (Classification) 224x224x3 Conv1 Pool1 112x112x64 Conv2 Pool2 56x56x128 Conv3 Pool3 28x28x256 Conv4 Pool4 14x14x512 Conv5 Pool5 7x7x512 Fully Connected Softmax 1x1 Result 위치 정보 분실
- 19. VGG16 (Segmentation) 224x224x3 Conv1 Pool1 112x112x64 Conv2 Pool2 56x56x128 Conv3 Pool3 28x28x256 Conv4 Pool4 14x14x512 Conv5 Pool5 7x7x512 1x1x512 Conv 7x7 Heatmap x32 Upsample Softmax 위치 정보 활용가능: Conv의 특징
- 20. VGG16 (Segmentation) 32x32x3 Conv1 Pool1 16x16x64 Conv2 Pool2 8x8x128 Conv3 Pool3 4x4x256 Conv4 Pool4 2x2x512 Conv5 Pool5 1x1x512 1x1x512 Conv 1x1 Heatmap x32 Upsample Softmax 임의의 32x32 Image 또는 Patch 다양한 Size 사용가능 Conv Filter를 학습하기 때문
- 21. VGG16 (Segmentation) 32x32x3 Conv1 Pool1 16x16x64 Conv2 Pool2 8x8x128 Conv3 Pool3 4x4x256 Conv4 Pool4 2x2x512 Conv5 Pool5 1x1x512 1x1x512 Conv 1x1 Heatmap x32 Upsample Softmax 임의의 32x32 Image 또는 Patch
- 23. 32x32x3 Conv1 Pool1 16x16x64 Conv2 Pool2 8x8x128 Conv3 Pool3 4x4x256 Conv4 Pool4 2x2x512 Conv5 Pool5 1x1x512 1x1 Heatmap x2 Upsample x16 Upsample Softmax Every 32x32 Patch + 1x1 Conv
- 24. 32x32x3 Conv1 Pool1 16x16x64 Conv2 Pool2 8x8x128 Conv3 Pool3 4x4x256 Conv4 Pool4 2x2x512 Conv5 Pool5 1x1x512 1x1 Heatmap x2 Upsample x8 Upsample Softmax Every 32x32 Patch + x2 Upsample +x2 Upsample 1x1 Conv
- 25. PASCAL VOC 2012 Cityscapes (IoU / iIoU) FCN-8s-CVPR15 62.2% 65.3% / 41.7% FCN-8s-PAMI17 67.2%
- 30. 1. LearningDeconvolution NetworkforSemanticSegmentation, 2015,Nohetal. 2. Attention toScale:Scale-AwareSemanticImageSegmentation, 2015,Chenetal. • 굳이 Downsample할 필요가 있는가? • 굳이 여러 Scale의 Input을 별개로 처리할 필요가 있는가?
- 31. • 굳이 Downsample할 필요가 있는가? • 굳이 여러 Scale의 Input을 별개로 처리할 필요가 있는가?
- 32. Figure from DeepLab Figure from DeepLab-v2
- 33. Figures from DilatedConv 3x3 Conv r=1 3x3 Range
- 34. Figures from DilatedConv 3x3 Conv r=1 3x3 Conv r=1 3x3 Conv r=2 3x3 Range 7x7 Range
- 35. Figures from DilatedConv 3x3 Conv r=1 3x3 Conv r=1 3x3 Conv r=2 3x3 Range 7x7 Range 3x3 Conv r=1 15x15 Range 3x3 Conv r=4 3x3 Conv r=2
- 38. FromMcCallum's introduction toCRFs ,( )p y x |( )p y x
- 39. • Graph Model • 각 Node = Label of Pixel • 각 Node의 Latent Variable = Pixel • 각 Node 사이를 확률 Modeling • Posterior를 최대화 하도록 확률 Model을 학습
- 40. Maximize Posterior 𝑃 𝑋 𝐼 = 1 𝑍(𝐼) exp −∑𝜙𝑐 𝑋 𝐶 𝐼 Minimize Energy Efficient inference infully connectedcrfswithgaussianedgepotentials, Krähenbühl etal,NIPS2011 𝐸 𝑋 𝐼 = ∑𝜙𝑐 𝑋 𝐶 𝐼 𝐸(𝑋) = ∑𝜓 𝐶 𝑋 𝐶 Normalization Image Label 𝐸 𝑥 = 𝑖 𝜓𝑖(𝑥𝑖) + 𝑖,𝑗 𝜓𝑖,𝑗(𝑥𝑖, 𝑥𝑗) FullyConnected Unary
- 41. 𝐸 𝑥 = 𝑖 𝜓𝑖(𝑥𝑖) + 𝑖,𝑗 𝜓𝑖,𝑗(𝑥𝑖, 𝑥𝑗) 𝜓𝑖 𝑥𝑖 = −log P(𝑥𝑖) 𝜓𝑖,𝑗 𝑥𝑖, 𝑥𝑗 = 𝜇 𝑥𝑖, 𝑥𝑗 [𝑤1 exp − 𝑝𝑖 − 𝑝𝑗 2 2𝜎 𝛼 2 − 𝐼𝑖 − 𝐼𝑗 2 2𝜎𝛽 2 + 𝑤2 exp − 𝑝𝑖 − 𝑝𝑗 2 2𝜎 𝛾 2 ] 𝜇 𝑥𝑖, 𝑥𝑗 = ቊ 1 𝑥𝑖 ≠ 𝑥𝑗 0 𝑂𝑡ℎ𝑒𝑟𝑤𝑖𝑠𝑒 𝑝𝑖, 𝑝𝑗: Pixel 위치 𝐼𝑖, 𝐼𝑗: Pixel RGB값 𝑤1, 𝑤2: Kernel Weights 𝜎 𝛼, 𝜎 𝛼, 𝜎 𝛼: Hyper-parameter Efficient inference infully connectedcrfswithgaussianedgepotentials, Krähenbühl etal,NIPS2011 Unary Term (from Classifier) Pairwise Term 의미? Pixel이 서로 비슷한데 (위치적으로, RGB상으로) Label이 서로 다르면 Energy 증가하여 Penalty
- 42. 계산량이 많으니까 Mean Field Approximation Efficient inference infully connectedcrfswithgaussianedgepotentials, Krähenbühl etal,NIPS2011 𝑃 𝑋 𝐼 = 1 𝑍(𝐼) exp −𝐸(𝑋) 𝜓𝑖,𝑗 𝑥𝑖, 𝑥𝑗 = 𝜇 𝑥𝑖, 𝑥𝑗 [𝑤1 exp − 𝑝𝑖 − 𝑝𝑗 2 2𝜎 𝛼 2 − 𝐼𝑖 − 𝐼𝑗 2 2𝜎𝛽 2 + 𝑤2 exp − 𝑝𝑖 − 𝑝𝑗 2 2𝜎𝛾 2 ] 𝑄𝑖 𝑥𝑖 = 𝑙 = 1 𝑍𝑖 exp −𝜓 𝑢 𝑥𝑖 − 𝑙′ 𝜇 𝑙, 𝑙′ 𝑚 𝑤 𝑚 𝑗 𝑘 𝑚 𝑓𝑖, 𝑓𝑗 𝑄𝑗(𝑙′) 대신에 𝑄 𝑋 = ς𝑖 𝑄𝑖(𝑋𝑖) 를 정의하고 𝐷 𝐾𝐿(𝑄||𝑃)를 최소화하도록 만들면 아래와 같은 Update 식을 얻을 수 있음 • Efficient Inference in Fully Connected CRFs with Gaussian Edge Potentials: Supplementary Material • Chapter 11.5 of Koller and Friedman “Probabilistic Graphical Models: Principles and Techniques”, 2009
- 43. Update Rule Efficient inference infully connectedcrfswithgaussianedgepotentials, Krähenbühl etal,NIPS2011 𝑄𝑖 𝑥𝑖 = 𝑙 = 1 𝑍𝑖 exp −𝜓 𝑢 𝑥𝑖 ෨𝑄𝑖 𝑚 𝑙 = 𝑗 𝑘 𝑚 𝑓𝑖, 𝑓𝑗 𝑄𝑗 𝑙 ෘ𝑄𝑖 𝑙 = 𝑚 𝑤 𝑚 ෨𝑄𝑖 𝑚 𝑙 𝑄𝑖 𝑙 = 𝑙′ 𝜇 𝑙, 𝑙′ ෘ𝑄𝑖 𝑙 ሖ𝑄𝑖 𝑙 = −𝜓 𝑢 𝑥𝑖 − 𝑄𝑖 𝑙 𝑄𝑖 = 1 𝑍𝑖 exp( ሖ𝑄𝑖 𝑙 ) 𝑄𝑖 𝑥𝑖 = 𝑙 = 1 𝑍𝑖 exp −𝜓 𝑢 𝑥𝑖 − 𝑙′ 𝜇 𝑙, 𝑙′ 𝑚 𝑤 𝑚 𝑗 𝑘 𝑚 𝑓𝑖, 𝑓𝑗 𝑄𝑗(𝑙′ ) 초기화 수렴할때 까지 Message Passing Weighting Compatibility Transform Adding Unary (Local Update) Normalization (Softmax)
- 44. CRF Learning with Validation (DeepLab v2) 𝜓𝑖,𝑗 𝑥𝑖, 𝑥𝑗 = 𝜇 𝑥𝑖, 𝑥𝑗 [𝑤1 exp − 𝑝𝑖 − 𝑝𝑗 2 2𝜎 𝛼 2 − 𝐼𝑖 − 𝐼𝑗 2 2𝜎𝛽 2 + 𝑤2 exp − 𝑝𝑖 − 𝑝𝑗 2 2𝜎 𝛾 2 ]
- 45. CRF를 Classification Network 뒤의 Post-processing으로써 사용하여 Detail을 높임
- 46. 𝐼 Network 𝜓 𝑢 𝑥𝑖 𝑄𝑖 𝑄𝑖 ෨𝑄𝑖 𝑚 𝑙 = 𝑗 𝑘 𝑚 𝑓𝑖, 𝑓𝑗 𝑄𝑗 𝑙 ෘ𝑄𝑖(𝑙) = 𝑚 𝑤 𝑚 ෨𝑄𝑖 𝑚 𝑙 𝑄𝑖 𝑙 = 𝑙′ 𝜇 𝑙, 𝑙′ ෘ𝑄𝑖 𝑙 ሖ𝑄𝑖 𝑙 = −𝜓 𝑢 𝑥𝑖 − 𝑄𝑖 𝑙 𝑄𝑖 = 1 𝑍𝑖 exp( ሖ𝑄𝑖 𝑙 ) 𝑄𝑖 𝐼 𝜓𝑖,𝑗 𝑥𝑖, 𝑥𝑗 = 𝜇 𝑥𝑖, 𝑥𝑗 [𝑤1 exp − 𝑝𝑖 − 𝑝𝑗 2 2𝜎 𝛼 2 − 𝐼𝑖 − 𝐼𝑗 2 2𝜎𝛽 2 + 𝑤2 exp − 𝑝𝑖 − 𝑝𝑗 2 2𝜎𝛾 2 ] Message Passing Weighting Compatibility Trans. 𝑄𝑖 𝑙 𝜓 𝑢 𝑥𝑖 Addition Normalization 𝑄𝑖 Conditional RandomFieldsasRecurrent NeuralNetworks, Zhengetal,ICCV2015
- 47. Iteration RNN End-to-End 𝐼 Network 𝜓 𝑢 𝑥𝑖 𝑄𝑖 𝑄𝑖 𝑄𝑖 𝐼 Message Passing Weighting Compatibility Trans. 𝑄𝑖 𝑙 𝜓 𝑢 𝑥𝑖 Addition Normalization 𝑄𝑖 Conditional RandomFieldsasRecurrent NeuralNetworks, Zhengetal,ICCV2015
- 50. 8 Layer Context Module of DilatedConv Multi Scale Context Aggregation by Dilated Convolution 3x3 Conv r=1 3x3 C 3x3 Conv r=1 5x5 C 3x3 Conv r=2 9x9 C 3x3 Conv r=4 17x17 C 3x3 Conv r=8 33x33 C 3x3 Conv r=16 65x65 C 3x3 Conv r=1 67x67 C 1x1 Conv r=1 67x67 C 2C 2C 4C 8C 16C 32C 32C C Basic LargeInput: 64x64xC OutputChannels 새로운Network CascadeDilatedConv.
- 51. 8 Layer Context Module of DilatedConv 3x3 Conv r=1 3x3 C 3x3 Conv r=1 5x5 C 3x3 Conv r=2 9x9 C 3x3 Conv r=4 17x17 C 3x3 Conv r=8 33x33 C 3x3 Conv r=16 65x65 C 3x3 Conv r=1 67x67 C 1x1 Conv r=1 67x67 C 2C 2C 4C 8C 16C 32C 32C C Basic LargeInput: 64x64xC OutputChannels ReceptiveFields 새로운Network CascadeDilatedConv.
- 53. VGG16 Front-End Module of DilatedConv 32x32x3 Conv1 Pool1 16x16x64 Conv2 Pool2 8x8x128 Conv3 Pool3 4x4x256 Conv4 4x4x512 Conv r=2 4x4x512 Conv r=4 4x4 Heatmap x8 Upsample Softmax Pooling 제거 Pooling 제거
- 54. Front-End Module for Context Module Input Conv1 Pool1 Conv2 Pool2 Conv3 Pool3 Conv4 Conv r=2 Conv r=4 64x64xC Heatmap Context Module Heatmap Size에맞게InputPadding C=21(Class개수)
- 55. • Context Module 제안: Dilated Conv Layer로만 구성된 새로운 Network • Front-End Module 제안: VGG16을 Pooling을 제거하고 Dilated Conv로 구성
- 57. PASCAL VOC 2012 Cityscapes (IoU / iIoU) FCN-8s-CVPR15 62.2% 65.3% / 41.7% FCN-8s-PAMI17 67.2% DeepLab v1 71.6% 63.1% / 34.5% CRF-RNN 72.0% 62.5% / 34.4% Dilated Conv FrontEnd 71.3% 10-Layer Context 67.1% / 42.0% Dilated Conv+ Context 73.5% Dilated Conv+ CRFRNN 75.3%
- 58. “Semantic ImageSegmentation withDeepConvolutional NetsandFullyConnected CRFs”2014 asDeepLabv1 “DeepLab:Semantic ImageSegmentation withDeepConvolutional Nets,AtrousConvolution andFully Connected CRFs”,2015 asDeepLabv2
- 59. CRFs를 강조 • CRF 사용 • Hole Algorithm 제안
- 60. • Atrous Conv. (Dilated Conv.) • CRFs • ASPP 제안 • 추가적인 학습 방법 등 DeepLab v1 DeepLab v2
- 61. 32x32x3 Conv1 Pool1 16x16x64 Conv2 Pool2 8x8x128 Conv3 Pool3 4x4x256 Conv4 4x4x512 Conv r=2 4x4x512 fc6 r=4 x8 Upsample Softmax fc7 1x1 fc8 1x1 DilatedConv와 비슷 (Front-End) Fc6 Layer의 rate가 다름
- 62. ASPP 제안: Atrous Conv. (Dilated Conv.) • Dilated Conv만으로는 Multi-scale을 본다고 하기 어렵다 • Dilated Conv는 Pooling 대신 Resolution 유지를 위해 사용하는 것 • 물론 Multi-scale로 Input을 넣어주면 성능은 무조건 향상 • 대신 ASPP를 제안하여 간단하게 그 효과를 본다!
- 63. Fc6 Layer에서 Parallel하게 Atrous Convolution한 뒤 Fusion K. He의 Spatial Pyramid Pooling에서 영감
- 64. • Poly Learning Rate • ASPP • CRF • VGG16 ResNet • Multi-scale Inputs • Pretrained on MS-COCO • Data Augmentation
- 66. PASCAL VOC 2012 Cityscapes (IoU / iIoU) FCN-8s-CVPR15 62.2% 65.3% / 41.7% FCN-8s-PAMI17 67.2% DeepLab v1 71.6% 63.1% / 34.5% CRF-RNN 72.0% 62.5% / 34.4% Dilated Conv FrontEnd 71.3% 10-Layer Context 67.1% / 42.0% Dilated Conv Context 73.5% Dilated Conv+ CRFRNN 75.3% DeepLab v2 79.7% 70.4% / 42.6%
- 68. Deep Network with a Suitable Global-scene-level Prior can much Improve the Performance of Scene Parsing 주변이 강이라면 Car보단 Boat Building? Skyscraper 비슷한 Texture
- 69. • Pyramid Pooling Module + Concat • Auxiliary Loss (ResNet) ResNet DilatedConv 1/8 1x1 중간에서 Loss Check
- 70. Average Pooling이 좋음 1x1 Convolution 효과 깊을수록 좋다 Auxiliary Loss의 효과
- 73. • Cascade Atrous Convolutions • MultiGrid ex) Output Stride = 16 Feature Map이 원본의 1/16 Resolution 한 Block에 3개의 Conv Layer 각 Layer의 Conv Rate 조절
- 74. • Modified ASSP + Batch Normalization • Inference Strategy on Val Set • Pretrained on COCO • Bootstrapping • Pretrained on JFT-300M Block 4에만 적용! Iteration마다 어려운 Label의 Data양을 늘려서 학습
- 75. Output Stride=16에 학습 Output Stride=8로 Test ASPP가 추가되면 Multigrid = (1,2,4)가 좋음 +Image Pooling 효과 ASPP가 추가된 경우 Output Stride=8로 Test 하면 좋음 깊을수록 좋다
- 76. PASCAL VOC 2012 Cityscapes (IoU / iIoU) FCN-8s-CVPR15 62.2% 65.3% / 41.7% FCN-8s-PAMI17 67.2% DeepLab v1 71.6% 63.1% / 34.5% CRF-RNN 72.0% 62.5% / 34.4% Dilated Conv FrontEnd 71.3% 10-Layer Context 67.1% / 42.0% Dilated Conv Context 73.5% Dilated Conv+ CRFRNN 75.3% DeepLab v2 79.7% 70.4% / 42.6% PSPNet 85.4% 81.2% / 59.6% DeepLab v3 85.7% 81.3% / 62.1% DeepLab v3-JFT 86.9%
- 77. PASCAL VOC 2012 Cityscapes (IoU / iIoU) Contribution FCN-8s-CVPR15 62.2% 65.3% / 41.7% FCN FCN-8s-PAMI17 67.2% DeepLab v1 71.6% 63.1% / 34.5% Dilated + CRF CRF-RNN 72.0% 62.5% / 34.4% CRF (End-to-End) Dilated Conv FrontEnd 71.3% 10-Layer Context 67.1% / 42.0% Cascade DilatedDilated Conv Context 73.5% Dilated Conv+ CRFRNN 75.3% DeepLab v2 79.7% 70.4% / 42.6% Dilated+ASPP+CRFs+ResNet PSPNet 85.4% 81.2% / 59.6% Pyramid Pooling + Aux. Loss DeepLab v3 85.7% 81.3% / 62.1% Modified Layer & ASPP + BatchNorm + Traning Strategies DeepLab v3-JFT 86.9%
- 78. Q&A?