Unity and Microsoft Azure Cognitive Services - DIGITREK21 Workshop
0 likes326 views
The document presents an introductory workshop on utilizing AI through Microsoft Azure Cognitive Services in Unity applications and games. It covers key concepts such as the differences between traditional AI and machine learning, the integration of Azure services into apps, and practical challenges for participants to implement various AI functionalities. It also provides a structured roadmap including lectures, demonstrations, basic and in-depth implementation guidance, and a Q&A session.
Unity and Microsoft Azure Cognitive Services - DIGITREK21 Workshop
- 1. Intro to Cloud AI with Azure and Unity Pablo Farias Navarro Zenva founder
- 3. Workshop outcomes What is AI and Microsoft Azure Cognitive Services Using cloud AI in Unity apps & games Implementation details
- 4. Why? Impact in daily life Opportunities In-demand skills
- 6. Roadmap 1. ✅ Introduction 2. ➡ What is AI 3. Microsoft Azure Cognitive Services 4. Unity demos 5. Unity basics 6. Unity in-depth implementation 7. Q&A
- 7. What is AI?
- 8. GOFAI • Explicit definition of symbols and algorithms • There is no actual “learning” • No capacity to abstract information into new concepts
- 9. Machine Learning • Statistic approach • Train model with large amounts of data • Model is able to learn and generalize • Predict, classify, generate
- 10. Real-world applications • Computer vision • Text and speech recognition • Recommendation algorithms • Pattern detection • Includes all industry verticals
- 11. Using ML in your apps Creating and training a model Training an existing model Using a third-party ML service
- 12. Creation of models • Advanced Math, Statistics and Computer Science knowledge • Often done at postgraduate level • Really cutting-edge and exciting field
- 13. Training of existing models • Find a suitable model and data • Free libraries like Tensorflow • Deployment
- 14. Third-party ML services • Can be integrated into any app • You don’t have to train it, host it or maintain it • Microsoft, AWS, Google, and others • Black blox, cost $$$
- 15. Overview Creating and training a model Training an existing model Using a third-party ML service
- 16. Roadmap 1. ✅ Introduction 2. ✅ What is AI 3. ➡ Microsoft Azure Cognitive Services 4. Unity demos 5. Unity basics 6. Unity in-depth implementation 7. Q&A
- 18. • Set of APIs for common AI needs • Hosted on Microsoft Azure • Can be plugged into any app or game Overview
- 19. Supported languages • C# • Python • Java • JavaScript (Node.js) • Go • Direct API calls (language agnostic)
- 20. Main areas Decision Language Vision Speech
- 21. Workflow App takes photo App sends a request to Azure Azure processes image Azure sends a response back to the App App receives image data Example: https://azure.microsoft.com/en-au/services/cognitive-services/computer-vision/#features
- 22. JSON • Data exchange format • Commonly used when using APIs • Easy for humans to write and read • Example beautifier: https://jsonparseronline.com/
- 23. Creating the service • Create Azure subscription • Create Resource Group • Create Computer Vision service • Obtain keys and domain
- 24. Authentication • How does Azure know who is sending the request? • Keep your credentials safe! • You can use mine during this workshop
- 25. Challenge 1: Run an API call 1. Open a text or code editor (or go to pastebin.com) 2. Go to to the Image Analysis docs, REST API (link in Zoom) 3. Copy the command into the text editor 4. Make the following changes: • Replace <subscriptionKey> by what’s added on Zoom • Replace https://westcentralus.api.cognitive.microsoft.com by 5. Open a terminal, paste command and press ENTER: • Windows: Command Prompt (cmd), NOT Power Shell • Mac: Terminal 5. Copy response into a JSON beautifier (https://jsonparseronline.com/) 6. Extra credits: replace the image URL by another image
- 26. Challenge 2: Run an API call 1. Create an Azure Computer Vision Service 2. Obtain service keys 3. Modify the API call from Challenge 1, to utilize
- 27. Roadmap 1. ✅ Introduction 2. ✅ What is AI 3. ✅ Microsoft Azure Cognitive Services 4. ➡ Unity demos 5. Unity basics 6. Unity in-depth implementation 7. Q&A
- 28. Unity Demos
- 29. Overview Azure Cognitive Services APIS: • Object detection (image analysis) • Text recognition (OCR) • Text to speech Image sources: • Webcam • In-game camera
- 30. Roadmap 1. ✅ Introduction 2. ✅ What is AI 3. ✅ Microsoft Azure Cognitive Services 4. ✅ Unity demos 5. ➡ Unity basics 6. Unity in-depth implementation 7. Q&A
- 31. Unity Basics
- 32. Challenge 3: Re-create Webcam Demo 1. Create a new Scene in Unity 2. Add the following prefabs: • UICanvas • WebcamCanvas • Object Detection • Text Recognition • Text to Speech 3. Add subscription keys to Object Detection, Text Recognition and Text to Speech (select “Webcam” as the source) 4. Setup the events like in Demo 2 5. Make 1 unique change
- 33. Challenge 4: Expand In-Game Demo - Add new items to Demo 1 (use the Asset Store) Look for realistic, free packages. - Make 1 logic change
- 34. Roadmap 1. ✅ Introduction 2. ✅ What is AI 3. ✅ Microsoft Azure Cognitive Services 4. ✅ Unity demos 5. ✅ Unity basics 6. ➡ Unity in-depth implementation 7. Q&A
- 36. Modules • UI canvas • Webcam • In-game screenshots • Computer Vision • Speech
- 37. Roadmap 1. ✅ Introduction 2. ✅ What is AI 3. ✅ Microsoft Azure Cognitive Services 4. ✅ Unity demos 5. ✅ Unity basics 6. ✅ Unity in-depth implementation 7. ➡ Q&A
- 38. Q&A
- 39. Roadmap 1. ✅ Introduction 2. ✅ What is AI 3. ✅ Microsoft Azure Cognitive Services 4. ✅ Unity demos 5. ✅ Unity basics 6. ✅ Unity in-depth implementation 7. ✅ Q&A
- 40. Thank You Pablo Farias Navarro Zenva founder