AI Builder - Object Detection
Download as DOCX, PDF1 like184 views
This document provides instructions for building and training an object detection model in PowerApps to identify three varieties of tea from images. It then describes how to create a canvas app that uses the trained model to detect tea items in an uploaded image and allow adjusting the on-hand inventory counts. Key steps include uploading and tagging training images, training the model, testing the model accuracy, publishing the model, and building an app with controls linked to the model and a Common Data Service entity to display and update inventory data.
AI Builder - Object Detection
- 1. Object detection Objectdetectionletsyoucount,locate,andidentifyselectedobjectswithinanyimage. Youcan use this model inPowerAppstoextractinformationfrompicturesyoutake withthe camera,orloadintoan app. In thislab,we will buildandtraina detectionmodelandbuildanappthat usesthe detectionmodel to identifyobjectsfromavailable images. Note: If you are buildingthe firstmodel inanenvironment,clickonExplore Templatestogetstarted. Exercise 1 In thisexercise we will buildandtrainthe ObjectDetectionmodel forthree varietiesof tea. 1. In PowerAppsmaker,expandAIBuilderandselectBuild. SelectObjectDetection. 2. Name yourmodel GreenTeaProduct Detection andbecause youare workingina shared environmentalsomake sure toinclude yourname aspart of the model name. Thiswill make it easiertofindlater. Clickcreate.
- 2. 3. Your screenshouldnowlooklike the image here. 4. Notice the progressindicatoronthe left. Those are the stepswe will follow now tobuildand trainour model.
- 3. 5. We are now goingto define the objectswe are tracking. Clickonthe Selectobjectnames. 6. From the entitylist,selectObjectDetection Product.
- 4. 7. Selectthe Name field andclickSelectfield. 8. Selectthe teaitems andclickNext. 9. Notice the progressindicatorhasmovedforwardtothe Add imagesstep.
- 5. 10. Clickadd images. 11. Selectimagesfromthe setprovided. Youwill needenoughimagestoprovide 15samplesfor each type of tea we are tracking.
- 6. 12. Approve the uploadof images.ClickUploadimages.Afterthe uploadcompletes,clickClose. 13. Clicknexttobegintaggingthe images. 14. Selectthe firstimage tobegintagging.
- 7. 15. Hoveroverthe image,nearanitemyouwishto tag. A dotted-linedbox shouldappeararound the item. It has beendetectedasasingle itemthatcan be tagged. 16. Clickon the itemandselectthe matchingobjectname.
- 8. 17. If the pre-definedselectorisnotaccurate,as inthe below example,youcandrag the container to draw itto accuratelytag the item. 18. Do thisfor eachiteminthe image andfor eachimage inyour set. Whenyouhave taggedall of the imagesyouuploadedclickDone Tagginginthe topright of the screen.
- 9. 19. Once youhave completedtagging,youwillgetasummaryof the tags. If you haven’ttagged enoughforanalysis,youwill needtoloadandtag more examples. 20. Once youhave definedenoughtagsfortrainingthe model,youwill be allowedtoinitiate the training. ClickNext.
- 10. 21. ClickTrain. 22. The trainingtakesa fewmoments.
- 11. 23. Navigate tothe savedmodel view andconfirmyourmodel hascompletedtraining. 24. Selectthe model youjustmade. 25. SelectQuicktest.
- 12. 26. Upload or drag and dropone of your test imagestobe analyzed. 27. You will see the analysisandlevel of confidence forthe match.
- 13. 28. Upload an image youknowwill notmatch. You will see the analysisandlevel of confidence for the match. 29. Clickclose.
- 14. 30. Publishyourmodel. Exercise 2 We will nowcreate acanvas app youcan use fordetectingthe itemsthathave beentrainedinour model. The productwill be detectedfromthe image andyouwill be able toadjuston-handinventory for the item. 1. Navigate toApps,andselectCreate anapp, thenselectCanvas. If asked,grantpermissionforthe app to use youractive CDS credentials. 2. SelectBlankappwithPhone layout.
- 15. 3. On the makercanvas,selectthe Inserttab inthe ribbonand expandAIBuilder. SelectObject detectortoplace thiscontrol onyour app. 4. Selectthe AImodel youbuilt. 5. Resize the control tobetteruse the space.
- 16. 6. Make sure to leave roomformore itemswe will be placingsoon. 7. Playyourapp.
- 17. 8. Clickon Detect. 9. Choose one of yourtest imagesandclickOpen.
- 18. 10. The image will nowbe analyzed. 11. Our model hasdetected eachteainthe image.
- 19. 12. Exitthe app player. Bonus exercise- build out the data in your canvas app 1. We will nowselectourdatasource. SelectView fromthe ribbonand selectDataSources. 2. Click+ Add Data Source.
- 20. 3. Addthe CommonData Service datasource. Do not use CommonData Service (current environment). 4. Selectthe ObjectDetection Products entityandclickConnect. 5. Close the Data pane.
- 21. 6. WithScreen1selectedinthe Tree view,navigatetothe Insertribbontab,expandGalleryandselect Blankvertical gallery. 7. Rename the GalleryproductGallery. Youare re-namingthe gallerysoyoucanreference itfromyour formulas. 8. Resize andmove the gallerycontrol tofitthe available space onthe screen,leavingsome space at the bottomfor usinglater.
- 22. 9. Selectthe editiconfromthe gallery. 10. Adda label tothe gallery.
- 23. 11. Clickeditagainand add a Textinputbox tothe gallery. Resize andplace ittoline upwiththe label we’ve alreadyplaced. We will be updatinginventorycountsinthistextbox. 12. Rename the TextInputinventoryInput.Youare renamingthiscontrol soyoucan reference itfrom your formulas.
- 24. 13. Withfocus onthe Screen1inthe Tree view,clickinthe ribbononInsertandselectButton. 14. Drag and move the buttonto the bottomof the screen,double clickonitto editthe text.Rename it to Update.
- 25. 15. We will nowaddthe usermessage togive the userconfirmationtheirsubmissionwasaccepted;we will define thislogiclater. WithfocusonScreen1,inserta label,dragitto the bottomof the screen.
- 26. 16. We will nowaddlogicto the controlswe’ve placedonthe screen. Selectthe Galleryandreplace the Itemsformulawiththe following. 'Object Detection Products' 17. Selectthe label inyourgallery. Replace the Textformulawiththe following: ThisItem.Name 18. SelectinventoryInputandreplace the formulaforDefaultwiththe following: LookUp('Object Detection Products',Name = ThisItem.Name).'Inventory Total' 19. Selectthe otherlabel (the one thatshowsatthe bottom of the screen) andreplace itstextwiththe following: usermessage
- 27. 20. You’ll notice thatarea nowlooksblank. We will configure thatmessage inournextstep. 21. Selectthe buttoncontrol andreplace the OnSelectwiththe following: ForAll(productGallery.AllItems,Patch('Object Detection Products',LookUp('Object Detection Products',Name=DisplayName),{'Inventory
- 28. Total':Value(inventoryInput.Text)}));Set(usermessage,"Updated " & CountRows(productGallery.AllItems) & " items") 22. Playthe app again. 23. ClickDetect. 24. Selectanimage to evaluate. 25. Update the quantityforthe correct productand clickUpdate.
- 29. 26. The bottom shouldshowamessage now.