Why Don’t You Understand Me? Build Intelligence into Your Apps
2 likes93 views
The document discusses Microsoft Azure Cognitive Services and how they can be integrated into applications to enhance intelligence and functionality across various domains including vision, language, search, and speech. It outlines several APIs and services, provides an overview of a sample application called the Intelligent Kiosk, and emphasizes the flexibility and security of deploying these services on-premises or at the edge. The presentation is led by Eran Stiller, Chief Technology Officer at CodeValue, highlighting the ease of use and transformative potential of these AI technologies.
![Computer Vision API
17
Description { "tags": [ "person", "indoor", "woman", "sitting", "holding", "grass", "looking",
"young", "hair", "man", "smiling", "black", "laptop", "wearing", "standing",
"room", "phone", "shirt", "water", "red", "living", "white", "blue", "cat", "night",
"umbrella" ], "captions": [ { "text": "Mona Lisa looking at the camera",
"confidence": 0.8883329 } ] }
Tags [ { "name": "person", "confidence": 0.98936075 } ]
Image format "Jpeg"
Image dimensions 1797 x 1191
Clip art type 0
Line drawing type 0
Black and white false
Adult content false
Adult score 0.0318996236
Racy false
Racy score 0.0194663126
Categories null
Faces [ { "age": 22, "gender": "Female", "faceRectangle": { "top": 335, "left": 389,
"width": 290, "height": 290 } } ]
Dominant color
background "Black"
Dominant color
foreground "Black"
Accent Color
#516936](https://image.slidesharecdn.com/buildintelligence-200101182740/85/Why-Don-t-You-Understand-Me-Build-Intelligence-into-Your-Apps-16-320.jpg)
![Emotion API
[
{
"faceRectangle": {
"top": 335,
"left": 389,
"width": 290,
"height": 290
},
"scores": {
"anger": 6.88452974E-06,
"contempt": 0.00261615869,
"disgust": 1.97378677E-06,
"fear": 1.35348373E-07,
"happiness": 0.214705735,
"neutral": 0.7825558,
"sadness": 8.04257361E-05,
"surprise": 3.28936549E-05
}
}
]
21](https://image.slidesharecdn.com/buildintelligence-200101182740/85/Why-Don-t-You-Understand-Me-Build-Intelligence-into-Your-Apps-20-320.jpg)
Why Don’t You Understand Me? Build Intelligence into Your Apps
- 1. Why Don’t You Understand Me? Build Intelligence into Your Apps Eran Stiller Chief Technology Officer, CodeValue erans@codevalue.net @eranstiller https://stiller.blog https://codevalue.net
- 2. 2
- 3. 3
- 4. 4
- 5. Agenda ▪ Microsoft Cognitive Services ▪ Add Intelligence to a Sample App ▪ Vision ▪ Language ▪ Search ▪ Speech 5
- 6. About Eran Eran Stiller ▪ @eranstiller ▪ CTO & Founder at CodeValue ▪ Software architect, consultant and instructor ▪ Microsoft Regional Director & Azure MVP ▪ Founder of Azure Israel Meetup 6
- 7. About CodeValue ▪ Awesome software company! ▪ Technology experts ▪ High quality software development solutions 7
- 9. MSR Beijing MSR Cambridge MSR Redmond MSR Montreal MSR New EnglandMSR New York Turning ideas into reality for 27 years MSR India MSR Shanghai MSR Labs Researchers WW Patents PapersFields Source: https://mybuild.techcommunity.microsoft.com/sessions/76966
- 10. Pre-trained AI with various services Language Vision Speech Decision Web search Bing Spell Check Custom Vision Personalizer Form Recognizer Neural Text-to-Speech Anomaly Detector Content Moderator Content Moderator Custom transcription Speech transcription Text-to-Speech Conversation transcription capability Face Video Indexer Ink Recognizer Computer Vision Language Understanding QnA Maker Text Analytics Translator Text Bing Web Search Bing Custom Search Bing Video Search Bing Image Search Bing Local Business Search Bing Visual Search Bing Entity Search Bing News Search Bing Autosuggest Source: https://mybuild.techcommunity.microsoft.com/sessions/76966
- 11. Bring AI to every application Azure Cognitive Services Vision Speech DecisionLanguage Azure Bot Service Source: https://mybuild.techcommunity.microsoft.com/sessions/76966
- 12. Productive Any language, any developer, any skill level Unified API Key Built for enterprises Optimized to work at any scale with existing systems & processes 17 GA services across 25 regions worldwide Trusted You own and control your data 66+ certifications (SOC, ISO, HIPAA/BAA, HITRUST PCI & FedRAMP compliance) Investment principles Source: https://mybuild.techcommunity.microsoft.com/sessions/76966
- 13. Sample Application ▪ The Intelligent Kiosk ▪ https://github.com/microsoft/Cognitive-Samples-IntelligentKiosk 14
- 14. Vision 15
- 15. Computer Vision API ▪ General Purpose Vision API ▪ Description ▪ Tags ▪ Adult/Racy content classification ▪ Face detection ▪ Dominant colors ▪ Read text ▪ Including handwritten text ▪ General purpose face recognition ▪ Recognize celebrities and landmarks 16
- 16. Computer Vision API 17 Description { "tags": [ "person", "indoor", "woman", "sitting", "holding", "grass", "looking", "young", "hair", "man", "smiling", "black", "laptop", "wearing", "standing", "room", "phone", "shirt", "water", "red", "living", "white", "blue", "cat", "night", "umbrella" ], "captions": [ { "text": "Mona Lisa looking at the camera", "confidence": 0.8883329 } ] } Tags [ { "name": "person", "confidence": 0.98936075 } ] Image format "Jpeg" Image dimensions 1797 x 1191 Clip art type 0 Line drawing type 0 Black and white false Adult content false Adult score 0.0318996236 Racy false Racy score 0.0194663126 Categories null Faces [ { "age": 22, "gender": "Female", "faceRectangle": { "top": 335, "left": 389, "width": 290, "height": 290 } } ] Dominant color background "Black" Dominant color foreground "Black" Accent Color #516936
- 17. Custom Vision Service ▪ Build your own classification model! ▪ https://customvision.ai/ 18 Flexibility Complexity Computer Vision API Custom AI Custom Vision Service
- 18. Custom Vision Service ▪ Simple cycle ▪ Upload ▪ Train ▪ Evaluate ▪ Active learning 19
- 19. Emotion API ▪ Recognize emotions in images ▪ React to the emotions of your users 20
- 20. Emotion API [ { "faceRectangle": { "top": 335, "left": 389, "width": 290, "height": 290 }, "scores": { "anger": 6.88452974E-06, "contempt": 0.00261615869, "disgust": 1.97378677E-06, "fear": 1.35348373E-07, "happiness": 0.214705735, "neutral": 0.7825558, "sadness": 8.04257361E-05, "surprise": 3.28936549E-05 } } ] 21
- 21. Face Detection API ▪ Detect faces in an image ▪ Highly detailed information ▪ Face attributes ▪ Emotions ▪ Pose ▪ Gender ▪ Age ▪ Glasses ▪ Makeup ▪ Etc. 22
- 22. Other Vision Services ▪ Video API ▪ Stabilize shaky videos ▪ Detect and track faces ▪ Detect motion ▪ Generate video thumbnails ▪ Video Indexer ▪ Unlock video insights ▪ Ink Recognizer ▪ Form Recognizer 23
- 24. Language 25
- 25. Text Analytics API ▪ Analyze various features of text ▪ Language ▪ Sentiment ▪ Key Phrases ▪ Topic ▪ Varying number of supported languages ▪ https://docs.microsoft.com/en-us/azure/cognitive-services/text- analytics/overview#supported-languages 26
- 26. Text Analytics API Eran Stiller was awesome. He had the best lecture. I just wish he would have shown more code on-stage. 27
- 27. Other Language Services ▪ Language Understanding Intelligent Service (LUIS) ▪ Bing Spell Check ▪ Translator Text ▪ QnA Maker 28
- 29. Search 30
- 30. Bing Autosuggest API ▪ Empower users to type less and do more 31
- 31. Other Search Services ▪ Bing Visual Search ▪ Bing Image Search ▪ Bing News Search ▪ Bing Video Search ▪ Bing Web Search ▪ Bing Entity Search ▪ Bing Custom Search 32
- 33. Speech 34
- 34. RecordedSynthesized Nostalgia could still be evoked after only twenty years , and many remembered when. You can't know how long Tara would have been mad , because that was taken away from her. In Georgia , governor Zell Miller has declared a state of emergency in four counties. Recorded Synthesized Recorded Synthesized A palpable chill passes through the room at this maladroit refusal . RecordedSynthesized The forecast for next week shows sunshine with highs in the mid forties. Synthesized Recorded Azure Neural Text-to-Speech Samples Source: https://mybuild.techcommunity.microsoft.com/sessions/76966
- 35. Speech Services ▪ Text-to-Speech ▪ Neural Text-to-Speech ▪ Speech Transcription ▪ Custom Speech Transcription ▪ Conversation Transcription ▪ Speaker Recognition 36
- 37. Decision 38
- 38. Decision Services ▪ Content Moderator ▪ Image, text & video ▪ Human review tool ▪ Anomaly Detection ▪ Personalizer 39
- 40. Bringing AI to the Edge Enterprise pain-points/benefits 41 Unable to upload all audio/text data to the cloud Data security & privacy concerns Low bandwidth or intermittent connect On-premises deploy with high throughput / flexibility At the edge and control over data Meet every regulatory requirement Source: https://mybuild.techcommunity.microsoft.com/sessions/76966
- 41. Azure: Consistent Hybrid Cloud 42 Source: https://mybuild.techcommunity.microsoft.com/sessions/76966
- 42. Intelligent Cloud to the Intelligent Edge 43 … Azure Cognitive Services Deploy wherever your data resides Vision Speech Language Intelligent Cloud Intelligent Edge … Containers Source: https://mybuild.techcommunity.microsoft.com/sessions/76966
- 43. Container Workflow 44 Docker Containers Azure Kubernetes Service (AKS) Azure Stack Azure IoT EdgeAI Model On-premises, Edge devices.. Source: https://mybuild.techcommunity.microsoft.com/sessions/76966
- 44. Azure Cognitive Services at the Edge 45 Model ContainerCode Repository Build + Test Deploy Monitoring/Analytics • .NET Core + portable C++ (+) Linux • .NET Core + Windows C++ (+) SQLPAL • .NET Fx + windows C++ (+) SQLPAL • Python AI Model Code Source: https://mybuild.techcommunity.microsoft.com/sessions/76966
- 46. Cognitive Services Containers ▪ Face & Emotion Detection ▪ Recognize Text (OCR) ▪ Language Understanding ▪ Key Phrase Extraction ▪ Language Detection ▪ Sentiment Analysis ▪ Anomaly Detector ▪ Form Recognizer ▪ Speech to Text ▪ Text to Speech 47
- 47. Takeaways 48
- 48. Takeaways 49 ▪ Azure Cognitive Services are easy to use ▪ Intelligence as a Service ▪ Features which were very hard before are suddenly easy to add ▪ The next generation of applications will take this to the extreme ▪ Execute trained models in the cloud or on the edge
- 49. Takeaways 50
- 50. Eran Stiller Chief Technology Officer erans@codevalue.net @eranstiller https://stiller.blog https://codevalue.net