![MobileNet
Computational cost = # Filter params x # Filter positions x # of filters
432 = 3x3 x 4x4 x 3
Computational cost = # Filter params x # Filter positions x # of filters
240 = 1x1x3 x 4x4 x 5
Total Computational cost of Normal Convolution = 2160
Total Computational cost of MobileNet = 432 + 240 = 672
MobileNet v1 and Mobilenet v2
FC layer
or
softmax
layer
FC layer
or
softmax
layer
17 times
13 times
[2017]
[2018]
6](https://image.slidesharecdn.com/cnnmodelseminar-231116124712-9fdac676/85/Introduction-to-CNN-Models-DenseNet-MobileNet-6-320.jpg)
Introduction to CNN Models: DenseNet & MobileNet
- 1. Introduction to CNN Models: DenseNet and MobileNet Deep Learning (CS258) By Krishnakoumar C M.Tech - Data Science (II – year) Puducherry Technological University 1
- 2. DenseNet What is DenseNet? • Each layer receives feature maps from all the preceding layers, and passes its output to all subsequent layers. • Feature maps received from other layers are fused through concatenation, and not through summation. Why DenseNet? • To improve declined accuracy caused by vanishing gradient. 2 ResNet DenseNet Key Advantage • Reduces vanishing gradient problem. • Stronger feature propagation. • Feature reuse. • Reduced parameter count.
- 3. DenseNet Dense Connections 3 Why introduce Growth Rate? • Growth rate (K) adds K features for each layers on top of the global state. Transition Layer (Convolution + Pooling) • The layers between the dense blocks are called transition layers. • The growth rate k is the additional number of channels for each layer. • It consists of a batch-norm layer, 1x1 convolution followed by a 2x2 average pooling layer. Resnet A DenseNet Block A DenseNet Architecture with 3 dense blocks Drawbacks • Data is replicated multiple times. • It also suffers slight overfitting problem.
- 4. DenseNet Dense Connections 4 Transition Layer (Convolution + Pooling) • The layers between the dense blocks are called transition layers. • The growth rate k is the additional number of channels for each layer. • It consists of a batch-norm layer, 1x1 convolution followed by a 2x2 average pooling layer. Resnet A DenseNet Block A Densenet Architecture with 3 dense blocks Why introduce Growth Rate? • Growth rate (K) adds K features for each layers on top of the global state. Drawbacks • Data is replicated multiple times. • It also suffers slight overfitting problem.
- 5. MobileNet Why MobileNet? • Low computation cost • Useful for mobile and embedded vision applications Key Concepts • Depth-wise separable Convolution • Useful for mobile and embedded vision applications Computational cost = # Filter params x # Filter positions x # of filters 2160 = 3x3x3 x 4x4 x 5 Depthwise Separable Convolution 13 times 5 Drawbacks • Performance is low. • Eg: ResNet-50 has accuracy 81% in 30 epochs and MobileNet has accuracy 65% in 100 epochs
- 6. MobileNet Computational cost = # Filter params x # Filter positions x # of filters 432 = 3x3 x 4x4 x 3 Computational cost = # Filter params x # Filter positions x # of filters 240 = 1x1x3 x 4x4 x 5 Total Computational cost of Normal Convolution = 2160 Total Computational cost of MobileNet = 432 + 240 = 672 MobileNet v1 and Mobilenet v2 FC layer or softmax layer FC layer or softmax layer 17 times 13 times [2017] [2018] 6
- 7. 7 References 1. Howard, A. G., Zhu, M., Chen, B., Kalenichenko, D., Wang, W., Weyand, T., ... & Adam, H. (2017). Mobilenets: Efficient convolutional neural networks for mobile vision applications. arXiv preprint arXiv:1704.04861. 2. Huang, G., Liu, Z., Van Der Maaten, L., & Weinberger, K. Q. (2017). Densely connected convolutional networks. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 4700-4708). 3. https://www.coursera.org/learn/convolutional-neural-networks/lecture/B1kPZ/mobilenet 4. https://www.pluralsight.com/guides/introduction-to-densenet-with-tensorflow 5. https://www.pluralsight.com/guides/introduction-to-resnet 6. https://medium.com/@godeep48/an-overview-on-mobilenet-an-efficient-mobile-vision-cnn-f301141db94d 7. https://d2l.ai/chapter_convolutional-modern/densenet.html 8. https://blog.paperspace.com/popular-deep-learning-architectures-densenet-mnasnet-shufflenet/ 9. https://amaarora.github.io/2020/08/02/densenets.html 10. https://towardsdatascience.com/architecture-comparison-of-alexnet-vggnet-resnet-inception-densenet- beb8b116866d 11. https://machinelearningmastery.com/introduction-to-1x1-convolutions-to-reduce-the-complexity-of- convolutional-neural-networks/ 12. https://medium.com/swlh/resnets-densenets-unets-6bbdbcfdf010 13. https://towardsdatascience.com/review-densenet-image-classification-b6631a8ef803