Introduction to AR with Unity3D
68 likes69,156 views
The document serves as an introduction to using Unity3D for Augmented Reality (AR), detailing various AR concepts such as detection methods, available libraries (like Vuforia and String), and popular applications in fields like gaming and education. It emphasizes the setup, performance comparison, and requirements for developing AR projects across platforms like iOS and Android. Additionally, it provides tips for optimization, troubleshooting, and resources for further learning.
Introduction to AR with Unity3D
- 1. AR with Unity3D Introduction to Augmented Reality with Unity3D Saturday, May 26, 12
- 2. Intro • Examples • Pantalla Global (CCCB) • Animac • Salón de Turisme AR with Unity3D 26/05/2012 - CatDroid Unity3D II @aquarioverde Saturday, May 26, 12
- 3. Koffa@Animac http://vimeo.com/37868229 AR with Unity3D 26/05/2012 - CatDroid Unity3D II @aquarioverde Saturday, May 26, 12
- 4. What is AR? • Virtual content overlayed onto the real word • 2D / 3D & audio • Realtime / Live camera view AR with Unity3D 26/05/2012 - CatDroid Unity3D II @aquarioverde Saturday, May 26, 12
- 5. Detection • Vision based • QR codes • Markers • Real life objects • Location based (GPS, Compass) AR with Unity3D 26/05/2012 - CatDroid Unity3D II @aquarioverde Saturday, May 26, 12
- 6. Vision Based AR • Tracking • Frame by frame analysis searching for “trackables” • Return position and orientation relative to device AR with Unity3D 26/05/2012 - CatDroid Unity3D II @aquarioverde Saturday, May 26, 12
- 7. AR Apps • Layar • http://www.layar.com/ • Junaio • http://www.junaio.com/ • Wikitude • http://www.wikitude.com/ • Appunta ;-) • http://appunta.com/ AR with Unity3D 26/05/2012 - CatDroid Unity3D II @aquarioverde Saturday, May 26, 12
- 8. Use • Arts & Gaming • Advertising • Navigation • Tourism • Education • ... and a lot more AR with Unity3D 26/05/2012 - CatDroid Unity3D II @aquarioverde Saturday, May 26, 12
- 9. Unity3D Libraries • String • www.poweredbystring.com • Vuforia (Qualcomm) • https://developer.qualcomm.com/mobile-development/mobile- technologies/augmented-reality AR with Unity3D 26/05/2012 - CatDroid Unity3D II @aquarioverde Saturday, May 26, 12
- 10. String - Tracking • Framed Markers • Multiple marker detection • <= 10 • Unflexible marker creation • black border surrounded by a white border necessary AR with Unity3D 26/05/2012 - CatDroid Unity3D II @aquarioverde Saturday, May 26, 12
- 11. String - Tracking • Any png • Full marker has to be in camera (>85%) • Markers not in scene • dificult scaling AR with Unity3D 26/05/2012 - CatDroid Unity3D II @aquarioverde Saturday, May 26, 12
- 12. Vuforia - Tracking • Image Target • Frame Markers • Simple 3D Objects (Boxes) • More flexible than String on marker creation AR with Unity3D 26/05/2012 - CatDroid Unity3D II @aquarioverde Saturday, May 26, 12
- 13. Vuforia - Tracking • Multiple Image Targets or Frame Markers • <= 5 • Works if only part of the marker is seen • Markers placed into 3D Scene • easy scaling • Virtual Buttons AR with Unity3D 26/05/2012 - CatDroid Unity3D II @aquarioverde Saturday, May 26, 12
- 14. Target Management • Online • ImageTargets • MultiTargets • jpg & png • RGB & B&W ! Rich in detail High in contrast • Scaling (Unity units) No repetitive patterns AR with Unity3D 26/05/2012 - CatDroid Unity3D II @aquarioverde Saturday, May 26, 12
- 15. Performance • String a little bit faster in detection and tracking • String is more stable AR with Unity3D 26/05/2012 - CatDroid Unity3D II @aquarioverde Saturday, May 26, 12
- 16. Platform support • Vuforia • Android & iOS • String • iOS AR with Unity3D 26/05/2012 - CatDroid Unity3D II @aquarioverde Saturday, May 26, 12
- 17. String - Setup • Download library • Unity3d • iOS • Add to Unity3D project • Add to XCode project AR with Unity3D 26/05/2012 - CatDroid Unity3D II @aquarioverde Saturday, May 26, 12
- 18. Vuforia - Setup • Download Vuforia library • available in Assets Store • Add to Unity3D Project AR with Unity3D 26/05/2012 - CatDroid Unity3D II @aquarioverde Saturday, May 26, 12
- 19. Testing • String: • In-Editor testing using webcam • Vuforia: ! Unity Remote • No image tracking nor life camera • Device Only Testing AR with Unity3D 26/05/2012 - CatDroid Unity3D II @aquarioverde Saturday, May 26, 12
- 20. Licensing • Unity / Unity iOS / Unity Android • https://store.unity3d.com/ • String • Expensive! • http://www.poweredbystring.com/licensing • Vuforia • Free!!! AR with Unity3D 26/05/2012 - CatDroid Unity3D II @aquarioverde Saturday, May 26, 12
- 21. Decision ??? AR with Unity3D 26/05/2012 - CatDroid Unity3D II @aquarioverde Saturday, May 26, 12
- 22. Vuforia Saturday, May 26, 12
- 23. Requirements iOS • Mac OS Snow Leopard & Lion • XCode • https://ar.qualcomm.at/qdevnet/sdk/ios AR with Unity3D 26/05/2012 - CatDroid Unity3D II @aquarioverde Saturday, May 26, 12
- 24. Requirements Android • MS Windows 7 & XP • Linux & Mac OS posssible but not documented • JDK + Eclipse + ADT • Android SDK + NDK • https://ar.qualcomm.at/qdevnet/sdk/android AR with Unity3D 26/05/2012 - CatDroid Unity3D II @aquarioverde Saturday, May 26, 12
- 25. Installation • Download pagackage vuforia-unity-OS- Version.zip and install • base extension & samples • Asset Store AR with Unity3D 26/05/2012 - CatDroid Unity3D II @aquarioverde Saturday, May 26, 12
- 26. Simple Project • New project • Add Vuforia • Add Dataset (markers) • Add ARCamera Prefab to scene • Add ImageTarget Prefab to scene • Select Dataset and ImageTarget • Add 3D objects + Light • Set Load Dataset • Deploy AR with Unity3D 26/05/2012 - CatDroid Unity3D II @aquarioverde Saturday, May 26, 12
- 27. Unity Folder Structure AR with Unity3D 26/05/2012 - CatDroid Unity3D II @aquarioverde Saturday, May 26, 12
- 28. Hands-On • DroidBalls • Floating spheres in space • Touch to dissappear AR with Unity3D 26/05/2012 - CatDroid Unity3D II @aquarioverde Saturday, May 26, 12
- 29. Raycasting • Detect 3D object touched on screen • Ray from touched screen point to camera • Detect object collisions AR with Unity3D 26/05/2012 - CatDroid Unity3D II @aquarioverde Saturday, May 26, 12
- 30. Tracking • ITrackableEventHandler • OnTrackableStateChanged • DefaultTrackableEventHandler • enable/disable rendering • ITrackerEventHandler • OnTrackablesUpdated AR with Unity3D 26/05/2012 - CatDroid Unity3D II @aquarioverde Saturday, May 26, 12
- 31. Splash Screen • Display when no marker is detected • Modify ITrackableEventHandler AR with Unity3D 26/05/2012 - CatDroid Unity3D II @aquarioverde Saturday, May 26, 12
- 32. Virtual Buttons • Hotspots on an ImageTarget • IVirtualButtonEventHandler • OnVirtualButtonPressed • OnVirtualButtonReleased • Can be created at runtime AR with Unity3D 26/05/2012 - CatDroid Unity3D II @aquarioverde Saturday, May 26, 12
- 33. Problems • Perfomance • Image Recognition • symmetry / similar elements • contrast / illumination • Real world sizing AR with Unity3D 26/05/2012 - CatDroid Unity3D II @aquarioverde Saturday, May 26, 12
- 34. Problems • Fixed camera vs fixed space • Physics!!! • Multiple markers • center??? AR with Unity3D 26/05/2012 - CatDroid Unity3D II @aquarioverde Saturday, May 26, 12
- 35. Tips & Advices • 3D Models • < 2K polygons overall • • < 200 polygons/object (draw calls) <= 3 materials/object ! Device dependent • Animations • < 30 bones • Use texture sequences AR with Unity3D 26/05/2012 - CatDroid Unity3D II @aquarioverde Saturday, May 26, 12
- 36. Tips & Advices • Attach markers to cardboard • Good lightning but avoid reflections • Disable objects that are not in view • Test using mouse instead of touch AR with Unity3D 26/05/2012 - CatDroid Unity3D II @aquarioverde Saturday, May 26, 12
- 37. Tips & Advices • Scale correctly from the beginning • Create simple reference objects to map to real world • e.g. cubes AR with Unity3D 26/05/2012 - CatDroid Unity3D II @aquarioverde Saturday, May 26, 12
- 38. Resources • Unity3D • http://forum.unity3d.com • http://answers.unity3d.com • Vuforia Forums • https://ar.qualcomm.at/qdevnet/forums • String ??? AR with Unity3D 26/05/2012 - CatDroid Unity3D II @aquarioverde Saturday, May 26, 12
- 39. Thank you! Andreas Blick @aquarioverde Saturday, May 26, 12