Using Deep Learning for Computer Vision Applications
Download as PPTX, PDF3 likes413 views
The document discusses the expanding market for computer vision applications using deep learning, projecting growth from $3.62 billion in 2018 to $25.32 billion by 2023, driven by sectors such as automotive, healthcare, and security. It also provides detailed guidance on utilizing OpenCV 4, including compiling the library, configuring projects in Visual Studio, and implementing models in Caffe and TensorFlow. Additionally, it references various resources, communities, and previous works related to deep learning in computer vision.
Using Deep Learning for Computer Vision Applications
- 1. Using Deep Learning for Computer Vision Applications Farshid PirahanSiah, PhD August 2018
- 2. Computer Vision with Deep Learning market AI PLATFORM and AI SOLUTION AUTOMOTIVE, ADAS (Advanced driver-assistance systems) & INFOTAINMENT, AUTONOMOUS & SEMI AUTONOMOUS VEHICLES, SPORTS & ENTERTAINMENT, AR / VR, BALL & PLAYER TRACKING, COMMERCIAL INSERTION, CONSUMER, GAMING, CAMERAS, WEARABLES, SMART PHONES, ROBOTICS & MACHINE VISION, QUALITY ASSURANCE & INSPECTION, POSITIONING & GUIDANCE, HEALTHCARE, RADIOLOGY, MEDICAL ADHERENCE, SECURITY & SURVEILLANCE, BIOMETRICS, IMAGE & VIDEO ANALYTICS, AI in GUIDED DRONE is BASED SURVEILLANCE, AGRICULTURE,...
- 3. Computer Vision with Deep Learning market “ The AI market for computer vision is expected to expand from US $ 3.62 billion in 2018 to US $ 25.32 billion by 2023. Market is expected to grow by 47.54% at CAGR (Composite Annual Growth Rate) from 2018 to 2023.” Industry Sector ● Automobile ● Sports, Entertainment ● Consumer ● Robot, machine vision ● Health care ● Security, monitoring ● Agriculture Main Innovator ● LEAP MOTION ● ALGOLUX ● CLARIFAI ● DITTO LABS ● MAD STREET DEN
- 4. Computer Vision with Deep Learning market Deep Learning Market Analysis By Application (Image Recognition, Voice Recognition, Video Surveillance) 2018 - 2025 “ The global deep learning market size was valued at USD 272.0 million in 2016. Increasing applicability in the autonomous vehicles and healthcare industries is expected to contribute to the industry growth significantly. This technology is gaining prominence on account of its complex data driven applications including voice and image recognition. It offers a huge investment opportunity as it can be leveraged over other technologies to overcome the challenges of high data volumes, high computing power, and improvement in data storage.”
- 5. Computer Vision with Deep Learning market Deep Learning Market Analysis By Application (Image Recognition, Voice Recognition, Video Surveillance) 2018 - 2025 Industry Sector ● Automobile ● Sports, Entertainment ● Consumer ● Robot, machine vision ● Health care ● Security, monitoring ● Agriculture Main Innovator ● LEAP MOTION ● ALGOLUX ● CLARIFAI ● DITTO LABS ● MAD STREET DEN
- 6. What's new in OpenCV 4 (August 2018) C++11 Deep Learning in OpenCV 4: Caffe TensorFlow Torch Darknet Some updates: Hardware-accelerated Video I/O module HighGUI module Graph API module Point Cloud module Tracking, Calibration, and Stereo Modules
- 7. Compile OpenCV 4 - part 1 Download (Cmake & Visual studio 2017 & Windows 10 (64 bit)) https://github.com/opencv/opencv https://github.com/opencv/opencv_contrib Create build folder Using Cmake to generate visual studio project Use the path of … opencv_contrib-mastermodules Open “OpenCV.sln” project
- 8. Compile OpenCV 4 - part 2 Build Debug Release Install Set the environment variable Add “C:opencv4buildinstallx64vc15bin” to windows path
- 9. Setup and configure your project (debug mode (64 bit)) Setup visual studio project to use OpenCV 4 (you only need install folder inside the build folder of compiled OpenCV 4) For the debug mode (64 bit) Add the include folder of OpenCV 4 to include path of your visual studio project C:opencv4buildinstallinclude Add the lib folder of OpenCV 4 to lib path of your visual studio project C:opencv4buildinstallx64vc15lib Put list of lib files for the debug mode (64 bit) (linker->input->additional dependencies) opencv_aruco400d.lib;opencv_bgsegm400d.lib;opencv_bioinspired400d.lib;opencv_calib3d400d.lib;opencv_ccalib400d.lib;o pencv_core400d.lib;opencv_datasets400d.lib;opencv_dnn400d.lib;opencv_dnn_objdetect400d.lib;opencv_dpm400d.lib;ope ncv_face400d.lib;opencv_features2d400d.lib;opencv_flann400d.lib;opencv_fuzzy400d.lib;opencv_hfs400d.lib;opencv_highg ui400d.lib;opencv_imgcodecs400d.lib;opencv_imgproc400d.lib;opencv_img_hash400d.lib;opencv_line_descriptor400d.lib;o pencv_ml400d.lib;opencv_objdetect400d.lib;opencv_optflow400d.lib;opencv_phase_unwrapping400d.lib;opencv_photo400d .lib;opencv_plot400d.lib;opencv_reg400d.lib;opencv_rgbd400d.lib;opencv_saliency400d.lib;opencv_shape400d.lib;opencv_ stereo400d.lib;opencv_stitching400d.lib;opencv_structured_light400d.lib;opencv_superres400d.lib;opencv_surface_matchin g400d.lib;opencv_text400d.lib;opencv_tracking400d.lib;opencv_video400d.lib;opencv_videoio400d.lib;opencv_videostab400 d.lib;opencv_xfeatures2d400d.lib;opencv_ximgproc400d.lib;opencv_xobjdetect400d.lib;opencv_xphoto400d.lib
- 10. Setup and configure your project (release mode (64 bit)) Setup visual studio project to use OpenCV 4 (you only need install folder inside the build folder of compiled OpenCV 4) For the released mode (64 bit) Add the include folder of OpenCV 4 to include path of your visual studio project C:opencv4buildinstallinclude Add the lib folder of OpenCV 4 to lib path of your visual studio project C:opencv4buildinstallx64vc15lib Put list of lib files for the released mode (64 bit) (linker->input->additional dependencies) opencv_aruco400.lib;opencv_bgsegm400.lib;opencv_bioinspired400.lib;opencv_calib3d400.lib;opencv_ccalib400.lib;opencv _core400.lib;opencv_datasets400.lib;opencv_dnn400.lib;opencv_dnn_objdetect400.lib;opencv_dpm400.lib;opencv_face400 .lib;opencv_features2d400.lib;opencv_flann400.lib;opencv_fuzzy400.lib;opencv_hfs400.lib;opencv_highgui400.lib;opencv_i mgcodecs400.lib;opencv_imgproc400.lib;opencv_img_hash400.lib;opencv_line_descriptor400.lib;opencv_ml400.lib;opencv _objdetect400.lib;opencv_optflow400.lib;opencv_phase_unwrapping400.lib;opencv_photo400.lib;opencv_plot400.lib;opencv _reg400.lib;opencv_rgbd400.lib;opencv_saliency400.lib;opencv_shape400.lib;opencv_stereo400.lib;opencv_stitching400.lib ;opencv_structured_light400.lib;opencv_superres400.lib;opencv_surface_matching400.lib;opencv_text400.lib;opencv_tracki ng400.lib;opencv_video400.lib;opencv_videoio400.lib;opencv_videostab400.lib;opencv_xfeatures2d400.lib;opencv_ximgpro c400.lib;opencv_xobjdetect400.lib;opencv_xphoto400.lib
- 11. Simple program to test #include "stdafx.h" #include <opencv2/core.hpp> #include <opencv2/highgui.hpp> int main() { cv::String filePath = "C:/opencv4/OpenCV4VS2017/OpenCV4VS2017/opencv-logo.png"; cv::Mat src = cv::imread(filePath); cv::imshow("test", src); cv::waitKey(10000); return 0; }
- 12. Using Caffe model in OpenCV 4 - part 1 Download 1. Synset_words.txt https://raw.githubusercontent.com/HoldenCaulfieldRye/caffe/master/ data/ilsvrc12/synset_words.txt 1. Bvlc_googlenet.caffemodel http://dl.caffe.berkeleyvision.org/bvlc_googlenet.caffemodel 1. Bvlc_googlenet.prototxt https://raw.githubusercontent.com/opencv/opencv_extra/master/test data/dnn/bvlc_googlenet.prototxt
- 13. Using caffe model in OpenCV 4 - part 2 static void getMaxClass(const cv::Mat &probBlob, int *classId, double *classProb) { cv::Mat probMat = probBlob.reshape(1, 1); //reshape the blob to 1x1000 matrix cv::Point classNumber; cv::minMaxLoc(probMat, NULL, classProb, NULL, &classNumber); *classId = classNumber.x; } static std::vector<cv::String> readClassNames(const char *filename = "C:/opencv4/OpenCV4VS2017/DeepLearningFiles/synset_words.txt") { std::vector<cv::String> classNames; std::ifstream fp(filename); if (!fp.is_open()) { std::cerr << "File with classes labels not found: " << filename << std::endl; exit(-1); } std::string name; while (!fp.eof()) { std::getline(fp, name); if (name.length()) classNames.push_back(name.substr(name.find(' ') + 1)); } fp.close(); return classNames; }
- 14. Using caffe model in OpenCV 4 - part 3 ● float scale = 1.0; ● cv::Scalar meanG = cv::Scalar(104, 117, 123); ● bool swapRB = true; ● int inpWidth = 224; ● int inpHeight = 224; ● cv::String model = path + "bvlc_googlenet.caffemodel"; ● cv::String config = path + "bvlc_googlenet.prototxt"; ● cv::String nameClass = path + "synset_words.txt"; ● cv::String nameImage = path + "space_shuttle.jpg"; ● cv::String framework; ● int backendId = 0; ● int targetId = 0; ● cv::dnn::Net net; ● cv::Mat blob; ● cv::Mat img = imread(nameImage); ● net = cv::dnn::readNet(model, config); ● net.setPreferableBackend(backendId); ● net.setPreferableTarget(targetId); ● cv::dnn::blobFromImage(img, blob, scale, cv::Size(inpWidth, inpHeight), meanG, swapRB, false); ● net.setInput(blob); ● cv::Mat prob = net.forward(); ● ● cv::Point classIdPoint; ● double confidence; ● minMaxLoc(prob.reshape(1, 1), 0, &confidence, 0, &classIdPoint); ● int classId = classIdPoint.x; ● std::vector<double> layersTimes; ● double freq = cv::getTickFrequency() / 1000; ● double t = net.getPerfProfile(layersTimes) / freq; ● std::string label = cv::format("Inference time: %.2f ms", t); ● putText(img, label, cv::Point(0, 15), cv::FONT_HERSHEY_SIMPLEX, 0.5, cv::Scalar(0, 255, 0)); ● cv::imshow("https://www.youtube.com/tiziran", img); ● std::vector<cv::String> classNames = readClassNames(); ● std::cout << "Best class: #" << classId << " '" << classNames.at(classId) << "'" << std::endl; ● std::cout << "Probability: " << confidence * 100 << "%" << std::endl; ● cv::waitKey(10000);
- 15. Using TensorFlow model in OpenCV 4 - part 1 TensorFlow API only for OpenCV 4 Download the inception model based on TensorFlow ● https://storage.googleapis.com/download.tensorflow.org/mo dels/inception5h.zip
- 16. Using TensorFlow model in OpenCV 4 - part 2 ● float scale = 1.0; ● cv::Scalar meanG = cv::Scalar(104, 117, 123); ● bool swapRB = true; ● int inpWidth = 224; ● int inpHeight = 224; ● cv::String model = path + "tensorflow_inception_graph.pb"; ● cv::String nameImage = path + "space_shuttle.jpg"; ● cv::String framework; ● int backendId = 0; ● int targetId = 0; ● cv::dnn::Net net; ● cv::Mat blob; ● cv::Mat img = imread(nameImage); ● net = cv::dnn::readNetFromTensorflow(model); ● net.setPreferableBackend(backendId); ● net.setPreferableTarget(targetId); ● cv::dnn::blobFromImage(img, blob, scale, cv::Size(inpWidth, inpHeight), meanG, swapRB, false); ● net.setInput(blob); ● cv::Mat prob = net.forward(); ● cv::Point classIdPoint; ● double confidence; ● minMaxLoc(prob.reshape(1, 1), 0, &confidence, 0, &classIdPoint); ● int classId = classIdPoint.x; ● std::vector<double> layersTimes; ● double freq = cv::getTickFrequency() / 1000; ● double t = net.getPerfProfile(layersTimes) / freq; ● std::string label = cv::format("Inference time: %.2f ms", t); ● putText(img, label, cv::Point(0, 15), cv::FONT_HERSHEY_SIMPLEX, 0.5, cv::Scalar(0, 255, 0)); ● cv::imshow("https://www.youtube.com/tiziran", img); ● std::vector<cv::String> classNames = readClassNamesTensorFlow(); ● std::cout << "Best class: #" << classId << " '" << classNames.at(classId) << "'" << std::endl; ● std::cout << "Probability: " << confidence * 100 << "%" << std::endl; ● cv::waitKey(10000);
- 17. Previous Videos, Courses, Classes, Training, Tutorials ❖ Compile full OpenCV 4 (main and contribute) from source ➢ https://www.youtube.com/watc h?v=VK70YdaMD44 ❖ Setup and configure Visual Studio 2017 project (C++) for using OpenCV 4 library ➢ https://www.youtube.com/watc h?v=J0Exttz4_m4 ❖ Using Caffe model in OpenCV 4 ➢ https://www.youtube.com/watc h?v=lj7AxKDoP9I ❖ Using TensorFlow model in OpenCV 4 ➢ https://www.youtube.com/watc h?v=5JqX0CbxtNk
- 18. Linkedin groups Computer Vision, Deep Learning, OpenCV 3, C++, DIGITS, Caffe, TensorFlow, Theano, Torch https://www.linkedin.com/groups/10320678
- 19. Telegram Deep Learning in Computer Vision (Channel) https://t.me/Dl4cv (Group) https://t.me/joinchat/CZ8vFgt1IdhUXkG615lsWA
- 20. WhatsApp Group WhatsApp Group Practical Deep Learning https://chat.whatsapp.com/LE5FYddBYl68iAaTZnTuYq
- 21. Facebook group Deep Learning, Computer Vision, Robotics https://www.facebook.com/groups/185926728115336/
- 24. GitHub Download source code from github https://github.com/pirahansiah/opencv4
- 25. Outline 1 Compile full OpenCV 4 (main and contribute) from source https://www.youtube.com/watch?v=VK70YdaMD44 3 Using Caffe model in OpenCV 4 https://www.youtube.com/watch?v=lj7AxKDoP9I 2 Setup and configure Visual Studio 2017 project (C++) for using OpenCV 4 library https://www.youtube.com/watch?v=J0Exttz4_m4 4 Using TensorFlow model in OpenCV 4 https://www.youtube.com/watch?v=5JqX0CbxtNk
- 26. Reference - I ● Effect of Augmented Datasets to Train Convolutional Neural Networks ● Pattern image significance for camera calibration ● https://doi.org/10.1109/SCORED.2017.830544 0 ● using an ant colony optimization algorithm for image edge detection as a threshold segmentation for ocr system ● http://www.jatit.org/volumes/Vol95No21/1Vol9 5No21.pdf ● GSFT-PSNR: Global Single Fuzzy Threshold Based on PSNR for OCR Systems ● http://oldver.ijcsns.com/June.2016-Volume.4- No.6/Article01.pdf ● Camera calibration for multi-modal robot vision based on image quality assessment ● https://doi.org/10.1109/ASCC.2015.7360336 ● Adaptive Image Thresholding Based on the Peak Signal-to-noise Ratio ● http://www.academia.edu/download/44161592 /Adaptive_Image_Thresholding_Based_on_th e20160328-31366-1wyb1jc.pdf ● peak signal-to-noise ratio based on threshold method for image segmentation ● www.jatit.org/volumes/Vol57No2/4Vol57No2.p df ● Simultaneous Localization And Mapping Trends And Humanoid Robot Linkages ● http://ejournal.ukm.my/apjitm/article/download/ 4164/2308
- 27. Reference - II ● character and object recognition based on global feature extraction ● https://pdfs.semanticscholar.org/2d22/66bfb2f 5ba5cb40b5946b82c620dc713d793.pdf ● License plate recognition with multi-threshold based on entropy ● https://doi.org/10.1109/ICEEI.2011.6021627 ● Character recognition based on global feature extraction ● https://doi.org/10.1109/ICEEI.2011.6021649 ● Comparison single thresholding method for handwritten images segmentation ● https://doi.org/10.1109/ICPAIR.2011.5976918 ● TafreshGrid: Grid computing in Tafresh university ● https://doi.org/10.1109/ICCSN.2011.6014009 ● Multi-threshold approach for license plate recognition system ● http://waset.org/publications/3636 ● Adaptive image segmentation based on peak signal-to-noise ratio for a license plate recognition system ● https://doi.org/10.1109/ICCAIE.2010.5735125 ● An evaluation of classification techniques using enhanced Geometrical Topological Feature Analysis ● https://scholar.google.com/scholar?oi=bibs&cl uster=3038184255311332521&btnI=1&hl=en ● Augmented Optical Flow Methods for Video ● http://www.academia.edu/download/45596617 /farshid.pdf