David Beard (PTC/Vuforia) Using Vuforia to Build Breakthrough Mixed Reality Experiences

Editor's Notes

• #2: Hello everyone. My name is David Beard, I’m a senior product manager for the Vuforia platform and our developer evangelist. My role is to help developers learn how to create AR/VR app using Vuforia, and on the best practices for designing mixed reality experiences. I’d started with Vuforia over 5 years ago as an engineer and am currently on the product management team, responsible for tools workflows.
• #3: Today I’m going to be discussing how Vuforia can be used to create mixed reality apps in Unity I’ll explain what mixed reality is and present some examples of mixed reality use cases Then I’ll describe the features of Vuforia 5.5 that support mixed reality But first, let me give you a brief introduction to Vuforia
• #4: Vuforia is PTC’s Mobile AR platform. It is an SDK that enables your apps to see things in the real world and then interact with them. We’ve done this by taking very sophisticated computer vision technologies, building them ground-up for mobile platforms, and making these technologies accessible through an easy to use API and developer workflows. …so that ANY app developer can quickly and very easily enable their apps with vision functionality.
• #5: Vuforia provides a very simple API. It tells developers what it sees and where it sees these objects precisely in the camera’s FoV…allowing you as developers to then render content accurately on or relative to the object Today with Vuforia you can see a number of things Images – these are every day images like magazine covers, product packaging, brochures Objects – rigid like toys and appliances – objects with little or no articulating or moving parts Text – we have text recognition for roman characters, such as English language words And finally with our Smart Terrain feature ….. meshes of an unknown environment…Vuforia can look at a table and will return the table surface and the props on the table in a mesh on top of which , you can overlay content…so you can drive a virtual car on a table that avoids a vase or a stack of books…or falls of the table. With Vuforia smart Terrain, we provide apps the awareness and intelligence to interact with their immediate environment SDKs We offer the flexibility to program with Java, C++/Objective C or with Unity About 85% of our developers use Unity because it provides an easy to use visual authoring workflow
• #6: Platform Review There are 3 main components to the platform The client side Engine Android and iOS devices Smart phones and tablets Digital Eyewear Tools to create and manage targets The Target manager is an online tool for creating, evaluating and managing targets You can create both device databases, which are embedded in your app, or cloud databases that are accessed using a web service The Object scanner tool is used to create Object Targets Eyewear calibration tool is for generating custom eye calibration profiles for users of optical see-through eyewear Cloud services enable you to deploy targets to your apps from a web service online. This service can support up to 1 million target images and delivers the same kind of trackable data as a device database. Cloud Recognition is on online image recognition service – you can query this service from your app Cloud Recognition also supports the Vuforia Web Service REST API which enables you to interface with the service from your own Content Management System
• #7: We’d originally developed Vuforia for hand held devices – smart phones and tablets Then as wearable displays became more readily available, we adapted our SDK to support optical and video see through eyewear Optical see through eyewear has a transparent lens, where digital content is rendered into the field of view of the wearer Examples include the Epson BT-200 and ODG R-6 and R-7 Video see-through eyewear relies on a device camera to see the world, which is presented to the user on the device screen Examples include Gear VR and Google Cardboard This evolution has enabled the development of a new class of AR/VR applications that support Mixed Reality experiences
• #8: Let me talk about Vuforia’s vision for mixed reality. People often describe AR and VR as distinct technologies, and there are some key differences between pure AR and VR But there are also significant similarities that make these complementary technologies They share a similar technology stack for computing, rendering, applications and sensor inputs They differ in their levels of immersion, between viewing the real world and experiencing a fully virtual world And these difference are actually complementary – they enable a new range of use cases, and more creative freedom
• #9: This is the new Mattel View Master You may remember this toy – it was originally a stereoscope. You would insert a disk of images and it enabled you to view them in 3D Mattel has updated this classic toy as an AR/VR viewer. Now the disks are AR targets that launch panoramic VR experiences View Master demonstrates how you can use AR to intuitively select and transition into VR experiences
• #11: Germbusters is a game developed by Realiteer, an independent developer He’s developed a set of VR controllers using AR targets – in this game the target represents a bubble gun This game shows that you can use an AR target as an interactive device in AR Notice that this is a fully immersive world – AR is simply used to recognize and track the game target
• #13: Cordy Orly is a app that enables you to view a proposed commercial development near the Cordy Orly airport in France You can view the plans in AR and then select a point of interest to view in VR This is a good example of a dual modalilty experience – it starts as handheld AR and transitions of VR in a Viewer
• #15: We introduced several new features to better support VR apps with Vuforia 5.5 – which were released a few weeks ago Stereo Rendering with Distortion Correction Rotational device tracking Mixed Reality Controller API Let me give you some details on each of these
• #16: Stereo scene rendering for digital eyewear apps – automatically corrects for pin cushion distortion High fidelity rendering with viewer and device specific lens distortion correction Custom viewers can be defined by developers
• #17: Provides a device independent, stand alone, solution for head tracking. You can use device tracking to track both the users head or their hand when they are holding the device The device tracker returns the rotation of the device around a central pivot point The pivot point is defined by a rotational model – we provide head and hand models that match typical human anthropometry ( body measurements ) We also enable you to change these models to define a custom rotational offset
• #18: The mixed reality controller is a utility API that handles AR / VR app lifecycle automatically It enables you to transition from AR to VR and also from handheld to viewer based experiences with a single method call This enables the AR/VR use cases I’ve shown and dual modality apps The nice thing is that you can develop a standard AR app for handheld mobile devices and easily enabled it for VR by simply implementing this API and a single method call.