Deep Learning - system view
0 likes145 views
The document provides an overview of deep learning, detailing its tools, types of models, and the processes involved in training and deploying these models. It highlights classifiers, box detectors, segmentors, and the importance of data gathering and optimization for effective deep learning applications. Additionally, it discusses various platforms for deployment and challenges in obtaining suitable datasets.
Deep Learning - system view
- 1. 1 Deep Learning From black magic to structured programming
- 2. 2 Agenda ▪Introduction ▪Tools / Model Types ▪Classifiers to instance segmentors ▪Process ▪Program? Train? ▪Data ▪Where to get it? ▪Platforms ▪Optimization ▪Video Analytics
- 3. 3 Introduction What is Deep Learning?
- 4. 4 What is Deep Learning? ▪Deep Learning is not “Black Magic” ▪Deep Learning is using common statistics tools ▪Deep Learning is using iterative convergence process to find a local minimum point which minimize the loss (“best local solution”)
- 5. 5 What is Deep Learning ? Gradient Descent ! Neural Networks Convolutional Neural Networks Back Propagation!? X Impossible Math? X
- 6. 6 Deep Learning for system eng. & Architects ▪Set of Tools ▪Simple library ▪Development process ▪Non-standard programming
- 8. 8 The tools ▪Most common tools: ▪Classifier ▪Detector (Box Detectors) ▪Segmentor ▪Instance Segmentor ▪Time series networks
- 9. 9 Classifiers - 2 ▪Answer on the existence of X (in image?) ▪Number classifier 0-9 (first CNN) ▪Common objects (Car, Chair, Cat…) ▪Cancer / No Cancer ▪Gun shot / broken glass ▪Most classifiers use some form of CNNs
- 10. 10 Classifiers ▪Single classifier could classify 100 objects ▪Most classifiers use COCO (~80 classes) as default ▪Example of La-net ▪First CNN for digits ▪Over 20 years old!!!
- 11. 11 BOX Detectors ▪Box detectors detects the location of the object ▪Used to detect: ▪People, face, corona mask ▪car, road signs ▪cats
- 12. 12 BOX Detectors ▪Box detectors could detect more than one class ▪Box detectors could detect overlapping objects ▪Known box detectors: ▪YOLO3 (most used) YOLO5 (Latest common model) ▪Yolo2PP, YOLOR (Fast) ▪YOLACT-EDGE (fast and accurate) ▪SSD / SSD2 / Mobile-SSD
- 13. 13 Segmentor and Multi-Instance Segmentor ▪Segmentor ▪defines class for each pixel ▪Pixel(X,Y) belongs to cat ▪FCN-8 basic implementation ▪Does not separate overlapping objects!!! ▪Common Segmentor U-Net
- 14. 14 Multi-Instance Segmentor ▪Multi-instance segmentor ▪Separate pixel to classes ▪Separate a class to instances ▪Pixel(X,Y) belongs to cat3
- 15. 15 Multi-Instance Segmentor - Implementation ▪Common / Simple implementation – MRCNN ▪Detectoron2 - Good implementation by Facebook
- 16. 16 (Time) series network ▪Runs on sequential data ▪Number of visitors in site ▪ECG data ▪Solar spot data ▪Single variable Data ▪Multi Variable ▪Example: LSTM, RNN
- 17. 17 Landmarks extraction ▪Landmark extraction is done for: ▪Face recognition ▪Skeleton Pose estimation ▪Hand Gestures
- 20. 20 Data ▪Data gathering (Initial) ▪Cameras / datasets ▪Data generation ▪Graphic engine (Unity / Omnviverse) ▪GAN ▪Data exploration ▪Anomaly detection ▪Class bias ▪Errored data ▪Data cleaning ▪Setting data pipes
- 21. 21 Training ▪Model selection Consideration ▪Performance constraints (run under < 1 TFLOPS) ▪Accuracy requirements ▪Model ▪Our own network ▪Existing network with/out weights ▪Training ▪No weights - from scratch ▪ Weights – Transfer learning ▪Hyper-param optimization ▪learning rate, decay, batch size, optimizer
- 22. 22 KPI – Measuring the results ▪Classifier accuracy ▪Percentage detected out of total objects ▪False detection ▪Detector accuracy ▪Classifier accuracy + ▪Box Accuracy ▪Segmentor Accuracy ▪Class accuracy ▪Segmentation accuracy – IoU (Intersection of Union) ▪Instance segmentor accuracy…..
- 23. 23 Model Optimization ▪For which platform? CPU, GPU, TPU? ▪What type of GPU/TPU? ▪How many cores? ▪What type of cores (FP32, FP16, Int 16, Int8) ▪What batch to use?
- 24. 24 Deployment and Automation ▪How to install the environment? ▪Python ▪Libraries ▪SDKs? ▪In Docker ? ▪How to automate the training and deployment platforms?
- 26. 26 Data sources ▪Open-source data sets ▪Kaggle ▪Github ▪Buy tagged data ▪Tagging ourselves ▪Tagging service provider
- 27. 27 Dataset Challenges ▪Existence – does the data exists somewhere ▪Faces, numbers, clothing, animals, cars, fonts ▪Can we use the data to train freely? ▪Datasets Has license like code ▪Does it fit my needs? ▪Data is RGB we need IR ▪Data is from close range we need distant range ▪Good plants we need sick plants ▪Cancer non-cancer image ration is 1:4.3*10^5 ▪Data fine but too little
- 28. 28 Datasets ▪Know used datasets ▪MNIST – numbers ▪MNIST fashion – clothing ▪COCO – most used Dataset ▪Kaggle ▪Many dataset exists in Kaggle ▪Some closed for competition many open ▪Buy dataset ▪Create your own ▪Take images (10000? automate?) ▪Tag on your own / External service
- 30. 30 Deployment and Automation ▪For code we have Git and Git lab for CI/CD ▪Can we use it for Deep Learning ▪Requirements ▪Manage data gathering ▪Store data versioning ▪Create and store model versioning ▪Automatic training + KPI Measurements ▪Automatic deployment
- 31. 31 Deep Learning Automation - 2 ▪MLOP Platforms ▪ClearML (Allegro) ▪Converge.IO (Intel) ▪Igazio ▪Neptune ▪SageMaker
- 32. 32 Optimization ▪CNN Models are require TOO much processing ▪Optimization process ▪Dev platform to single representation ▪Pruning ▪Convert type
- 33. 33 Accelerator software platforms ▪Model porting and optimization ▪Nvidia TensorRT ▪Intel OpenVino ▪XILINX tools ▪Video processing and analytics platforms ▪NVIDIA DeepStream ▪Intel OpenVino ▪XILINX ▪All video analytics SDKs are based on GStreamer ▪Robotics ▪NVIDIA ISAAC (ROS / GEMS integration)