Azure Cognitive Services - Custom Vision
2 likes1,647 views
The document outlines an Azure workshop led by software engineers at Microsoft, focusing on Azure Cognitive Services and its applications in AI, such as image classification and object detection. It provides details on preparing for the workshop, utilizing the Custom Vision service, and best practices for training classifiers with various images. Key concepts include model evaluation, precision, recall, and REST-based API access for prediction services.
![Computer Vision API
Content of Image:
Categories v0: [{ “name”: “animal”, “score”: 0.9765625 }]
V1: [{ "name": "grass", "confidence": 0.9999992847442627 },
{ "name": "outdoor", "confidence": 0.9999072551727295 },
{ "name": "cow", "confidence": 0.99954754114151 },
{ "name": "field", "confidence": 0.9976195693016052 },
{ "name": "brown", "confidence": 0.988935649394989 },
{ "name": "animal", "confidence": 0.97904372215271 },
{ "name": "standing", "confidence": 0.9632768630981445 },
{ "name": "mammal", "confidence": 0.9366017580032349,
"hint": "animal" },
{ "name": "wire", "confidence": 0.8946959376335144 },
{ "name": "green", "confidence": 0.8844101428985596 },
{ "name": "pasture", "confidence": 0.8332059383392334 },
{ "name": "bovine", "confidence": 0.5618471503257751,
"hint": "animal" },
{ "name": "grassy", "confidence": 0.48627158999443054 },
{ "name": "lush", "confidence": 0.1874018907546997 },
{ "name": "staring", "confidence": 0.165890634059906 }]
Describe
0.975 "a brown cow standing on top of a lush green field“
0.974 “a cow standing on top of a lush green field”
0.965 “a large brown cow standing on top of a lush green field”](https://image.slidesharecdn.com/azurecognitiveservices-customvision-190922125243/85/Azure-Cognitive-Services-Custom-Vision-17-320.jpg)
Azure Cognitive Services - Custom Vision
- 1. Please check that you have your Azure subscription ready for the workshop ☺ azure.microsoft.com/free Students: aka.ms/azure4students
- 3. Introduction to Azure Cognitive Services Roman Sudarikov Software Engineer @Microsoft Luis Beltran Microsoft MVP AI, Developer Technologies Łukasz Foks Technologies CEE Azure Developer Product Marketing Manager @ Microsoft Microsoft Prague, September 22nd, 2019 roman-sudarikov-1508a358 luisantoniobeltran luis@luisbeltran.mx lukaszfoks
- 6. Azure Cognitive Services Applying AI to your business Computer Vision Face/Emotion Recognition OCR/Handwriting Custom Vision Video Indexer Content Moderator Text-to-Speech Speech-to-Text Translator Custom Speech Language Understanding PII Detection Text Translator Text Analytics QnA Maker Bing Custom Search Bing Visual Search
- 13. Face API
- 14. Face Detection
- 17. Computer Vision API Content of Image: Categories v0: [{ “name”: “animal”, “score”: 0.9765625 }] V1: [{ "name": "grass", "confidence": 0.9999992847442627 }, { "name": "outdoor", "confidence": 0.9999072551727295 }, { "name": "cow", "confidence": 0.99954754114151 }, { "name": "field", "confidence": 0.9976195693016052 }, { "name": "brown", "confidence": 0.988935649394989 }, { "name": "animal", "confidence": 0.97904372215271 }, { "name": "standing", "confidence": 0.9632768630981445 }, { "name": "mammal", "confidence": 0.9366017580032349, "hint": "animal" }, { "name": "wire", "confidence": 0.8946959376335144 }, { "name": "green", "confidence": 0.8844101428985596 }, { "name": "pasture", "confidence": 0.8332059383392334 }, { "name": "bovine", "confidence": 0.5618471503257751, "hint": "animal" }, { "name": "grassy", "confidence": 0.48627158999443054 }, { "name": "lush", "confidence": 0.1874018907546997 }, { "name": "staring", "confidence": 0.165890634059906 }] Describe 0.975 "a brown cow standing on top of a lush green field“ 0.974 “a cow standing on top of a lush green field” 0.965 “a large brown cow standing on top of a lush green field”
- 18. Old OCR New OCR OCR
- 20. Vision Speech Language Natural Language Processing Intent: PlayCall Knowledge Here are the top results: The purpose of Customer Life-cycle Management (CLM) is to maximize both customer retention and .... Predictive trend analysis provides business visibility. Oct 28, 2015 – Here are FIVE key trends in 2014 that would help marketers in rolling ... Of late, marketers are looking at customer lifecycle management (CLM) Jan 5, 2016 – The top 10 customer service trends for 2016 that .... North American Consumer Search Here is what I found: It also investigates the top three expected Fraud Detection and Prevention programs, in terms of demand in key markets… First, let’s point out that there is not one absolute answer—there are “pros” and “cons” to each. Those who favor in-house… Michael heads fraud prevention tool. Online and mobile shopping are expected to continue growing apace… A variety of real-world applications
- 21. Apps powered by Azure Cognitive Services
- 24. Call to Action • Azure Cognitive Services aka.ms/cognitive-services • Documentation: https://docs.microsoft.com/en-us/azure/cognitive-services/ • GitHub Samples: https://github.com/Azure-Samples/cognitive-services-REST-api-samples • Container Support: https://docs.microsoft.com/en-us/azure/cognitive-services/cognitive-services- container-support
- 25. Thank you for your attention! Roman Sudarikov Software Engineer @Microsoft Luis Beltran Microsoft MVP AI, Developer Technologies Łukasz Foks Technologies CEE Azure Developer Product Marketing Manager @ Microsoft Microsoft Prague, September 22nd, 2019 roman-sudarikov-1508a358 luisantoniobeltran luis@luisbeltran.mx lukaszfoks
- 26. Last call: Please check that you have your Azure subscription ready for the workshop ☺ azure.microsoft.com/free Students: aka.ms/azure4students
- 27. Deep Dive into Azure Custom Vision Roman Sudarikov Software Engineer @Microsoft Luis Beltran Microsoft MVP AI, Developer Technologies Łukasz Foks Technologies CEE Azure Developer Product Marketing Manager @ Microsoft Microsoft Prague, September 22nd, 2019 roman-sudarikov-1508a358 luisantoniobeltran luis@luisbeltran.mx lukaszfoks
- 28. 28 Prepare Data Image Classification Build & Train Run Model definition & training Model Evaluation Deploy the model - web service, Dockers Container or IoT EdgeScore the model
- 30. What is it? Custom Vision Service is an easy-to-use tool for prototyping, improving, and deploying a custom image classifier to a cloud service, without any background in computer vision or deep learning required. Model
- 31. Object Detection
- 32. Building a Classifier • Create a project • Select a domain • Add images • Assign tags to images • Train the classifier • Evaluate the classifier
- 33. Classifiers and Projects A classifier is a model you build using Custom Vision Service, by using a few training images. Each classifier you build is in its own project. Classifier = Project
- 34. Domains When you create a project, you select a domain for that project. The domain optimizes a classifier for a specific type of object in your images. • Food Optimized for dishes you would see on a restaurant menu. • Landmark Optimized for recognizable landmarks, both natural and artificial. • Retail Optimized for classifying images in a shopping catalog or shopping website. • Adult Optimized to better define between adult content and non-adult content.
- 35. Training Images To create a high precision classifier, Custom Vision Service needs several training images. A training image is a photograph of the image you want Custom Vision Service to classify.
- 36. Iteration Every time you Train or re-train your classifier, you create a new iteration of your model.
- 37. Testing a Model After you train your model, you can quickly test it using a locally stored image or an online image. The test uses the most recently trained iteration.
- 38. Important Terms Precision When you classify an image, how likely is your classifier to correctly classify the image? Recall Out of all images that should have been classified correctly, how many did your classifier identify correctly?
- 39. Using the Prediction API After a successful training, the Custom Vision Service can be accessed via an endpoint that references the Project Identifier, as long as the appropriate Prediction Key is passed in the request header.
- 40. Prediction API REST Concepts All actions related to the Custom Vision Service are accessed via standard REST- based methods, such as GET and POST against an API endpoint, making it simple to use the Prediction API on any platform or with any programming language.
- 41. Train in the Cloud, Run Anywhere Train in Custom Vision Service Deploy & Run Anywhere
- 42. Commonly Used APIs PredictionTraining • Createimages • Tagimages • Createprojects • Manageprojects • Manageiterations • Createtags • Getaccountinformation • Trainaproject • Predictimages • Predictandsaveimages • PredictimageURLs • PredictandsaveimageURLs
- 43. Improving a Classifier The best way to have a quality classifier is to add more varied tagged images (different backgrounds, angles, object size, groups of photos, and variants of types.) Always to train your classifier after you have added more images. Include images that are representative of what your classifier will encounter in the real world. Photos in context are better than photos of objects in front of neutral backgrounds, for example.
- 44. Best Practices for using Custom Vision • Use at least 30 images for each tag • Images should be the focus of the picture • Use sufficiently diverse images and backgrounds (ex: cats with red background and dogs with blue background) • Train with images that are similar in {quality, resolution, lighting, etc.} to the images that will be used in prod • Supports Microsoft accounts (MSA) and AAD
- 45. Computer Vision Scenario Examples ➢ Additional Scenarios ➢ Classify user submitted images to website ➢ Identifying elements – object counting, animal identification and lots more. ➢ Hazard detection/industrial safety – adding custom rules to videos
- 47. Call to Action • Azure Custom Vision https://www.customvision.ai/ • Documentation: https://docs.microsoft.com/en-us/azure/cognitive-services/custom-vision- service/
- 48. Thank you for your attention! Roman Sudarikov Software Engineer @Microsoft Luis Beltran Microsoft MVP AI, Developer Technologies Łukasz Foks Technologies CEE Azure Developer Product Marketing Manager @ Microsoft Microsoft Prague, September 22nd, 2019 roman-sudarikov-1508a358 luisantoniobeltran luis@luisbeltran.mx lukaszfoks