OpenCV DNN module vs. Ours method
2 likes2,083 views
The document discusses the development of caffe2c, a tool that converts deep neural network models trained in Caffe into a single C code suitable for mobile devices, achieving faster execution than existing methods like OpenCV DNN. It highlights the challenges in deploying convolutional neural networks (CNNs) on mobile due to high computational and memory requirements, and outlines a deployment procedure for integrating the model into mobile applications. Additionally, it mentions the implementation of mobile CNN-based image recognition apps on iOS using the caffe2c framework.
![ⓒ 2016 UEC Tokyo.
• Caffe2C achieves faster execution speed in comparison
to the existing OpenCV DNN module
Caffe2C
Caffe2C OpenCV DNN
AlexNet
iPhone 7 Plus 106.9 1663.8
iPad Pro 141.5 1900.1
iPhone SE 141.5 2239.8
Runtime[ms] Caffe2C vs. OpenCV DNN(Input size: 227x227)
Speedup Rate:
About 15X〜](https://image.slidesharecdn.com/161128tanno51-161208092318/85/OpenCV-DNN-module-vs-Ours-method-14-320.jpg)
OpenCV DNN module vs. Ours method
- 1. ⓒ 2016 UEC Tokyo. 1. Caffe2C directly converts the Deep Neural Network to a C source code Reasons for Fast Execution Caffe2C OpenCV DNN ･Network ･Mean ･Label ･Model Caffe2C Single C code Execution like Compiler Execution like Interpreter
- 3. ⓒ 2016 UEC Tokyo. • Deep Learning achieved remarkable progress – E.g. Audio Recognition, Natural Language Processing, • Especially, in Image Recognition, Deep Learning gave the best performance – Outperform even humans such as recognition of 1000 object(He+, Delving deep into rectifier, 2015) Deep Learning(DNN,DCNN,CNN) 0 20 40 60 80 100 2010 2011 2012 2013 2014 2015 Human Trained 72% 75% 85% 88.3% 93.3% 96.4% 94.9% SIFT+BOF Deep Learning Deeeeeeeep Outperform Human !
- 4. ⓒ 2016 UEC Tokyo. • Many Deep Learning Framework have emerged – E.g. Caffe, TensorFlow, Chainer Deep Learning Framework
- 5. ⓒ 2016 UEC Tokyo. Convolution Architecture For Feature Extraction(CAFFE) Open Framework, models and examples for Deep Learning • Focus on Compuer Vision • Pure C++/CUDA architecture for deep learning • Command line, Python MATLAB interface • Fastest processing speed • Caffe is the most popular framework in the world What is Caffe?
- 6. ⓒ 2016 UEC Tokyo. • There are many attempts to archive CNN on the mobile – Require a high computational power and memory Bring to CNN to Mobile High Computational Power and Memory are Bottleneck!!
- 7. ⓒ 2016 UEC Tokyo. Files • 3 files are required for Training -> Output: Model – 3 files: Network definition, Mean, Label How to train a model by caffe? Training ･Network ･Mean ･Label 3 files Dataset Output ･Caffemodel Use these 4 files on mobile
- 8. ⓒ 2016 UEC Tokyo. • We currently need to use OpenCV DNN module – not optimized for the mobile devices – their execution speed is relatively slow Use the 4 Files by Caffe on the Mobile ･Network ･Mean ･Label ･Model 4 files
- 9. ⓒ 2016 UEC Tokyo. • We create a Caffe2C which converts the CNN model definition files and the parameter files trained by Caffe to a single C language code that can run on mobile devices • Caffe2C makes it easy to use deep learning on the C language operating environment • Caffe2C achieves faster runtime in comparison to the existing OpenCV DNN module Objective ･Network ･Mean ･Label ･Model 4 files Caffe2C Single C code
- 10. ⓒ 2016 UEC Tokyo. • In order to demonstrate the utilization of the Caffe2C, we have implemented 4 kinds of mobile CNN-based image recognition apps on iOS. Objective
- 11. ⓒ 2016 UEC Tokyo. 1. We create a Caffe2C which converts the model definition files and the parameter files of Caffe into a single C code that can run on mobile devices 2. We explain the flow of construction of recognition app using Caffe2C 3. We have implemented 4 kinds of mobile CNN-based image recognition apps on iOS. Contributions
- 12. ⓒ 2016 UEC Tokyo. 2. CONSTRUCTION OF CNN- BASED MOBILE RECOGNITION SYSTEM
- 13. ⓒ 2016 UEC Tokyo. • In order to use the learned parameters by Caffe on mobile devices, it is necessary to currently use the OpenCV DNN module not optimized, relatively slow • We create a Caffe2C which converts the CNN model definition files and the parameter files trained by Caffe to a single C language code – We can use parameter files trained by Caffe on mobile devices Caffe2C
- 14. ⓒ 2016 UEC Tokyo. • Caffe2C achieves faster execution speed in comparison to the existing OpenCV DNN module Caffe2C Caffe2C OpenCV DNN AlexNet iPhone 7 Plus 106.9 1663.8 iPad Pro 141.5 1900.1 iPhone SE 141.5 2239.8 Runtime[ms] Caffe2C vs. OpenCV DNN(Input size: 227x227) Speedup Rate: About 15X〜
- 15. ⓒ 2016 UEC Tokyo. 1. Caffe2C directly converts the Deep Neural Network to a C source code Reasons for Fast Execution Caffe2C OpenCV DNN ･Network ･Mean ･Label ･Model Caffe2C Single C code Execution like Compiler Execution like Interpreter
- 16. ⓒ 2016 UEC Tokyo. 2. Caffe2C performs the pre-processing of the CNN as much as possible to reduce the amount of online computation – Compute batch normalization in advance for conv weight. 3. Caffe2C effectively uses NEON/BLAS by multi-threading Reasons for Fast Execution ･Network ･Mean ･Label ･Model 4 files Caffe2C Single C code
- 17. ⓒ 2016 UEC Tokyo. Deployment Procedure 1. Train Deep CNN model by Caffe 2. Prepare model files 3. Generate a C source code by Caffe2C automatically 4. Implement C code on mobile with GUI code Trained Deep CNN Model Deep CNN Train Phase 1 ・Caffemodel ・Network ・Mean ・Label Model Preparation 2 Convert C code 3 Caffe2C Implement on Mobile 4
- 18. ⓒ 2016 UEC Tokyo. • We implemented apply our mobile framework into real-time CNN-based mobile image processing – such as Neural Style Transfer Additional work
- 19. ⓒ 2016 UEC Tokyo. Thank you for listening Object Recognition Neural Style Transfer iOS App is Available ! “DeepFoodCam“ iOS App is Available ! “RealTimeMultiStyleTransfer”