Matteo Valoriani - How Augment your Reality: different perspective on the Reality-Virtuality Continuum - Codemotion Milan 2017
0 likes481 views
The document presents an overview of augmented reality (AR) and virtual reality (VR), discussing the technological evolution and differences in methodologies such as visual inertial odometry and the capabilities of devices like Hololens. Key concepts include motion tracking, depth sensing, and the importance of hardware and rendering optimizations in enhancing AR experiences. The text also references various applications and frameworks related to AR and VR technology development.
Matteo Valoriani - How Augment your Reality: different perspective on the Reality-Virtuality Continuum - Codemotion Milan 2017
- 1. How Augment your Reality: Different perspective on the Reality / Virtuality Continuum Matteo Valoriani CODEMOTION MILAN - SPECIAL EDITION 10 – 11 NOVEMBER 2017
- 2. How Augment your Reality: Different perspective on the Reality / Virtuality Continuum
- 3. Nice to Meet You matteo.valoriani@fifthingenium.com Matteo Valoriani, PhD CEO of FifthIngenium mvaloriani at gmail.com @MatteoValoriani Slideshare: www.slideshare.net/MatteoValoriani Linkedin: https://it.linkedin.com/in/matteovaloriani Blog: http://fifthingenium.com/blog GitHub: https://github.com/mvaloriani
- 5. How we are arrived here Virtual Continuum VR • Outside-in vs Inside-out AR • Visual Inertial Odometry (VIO) • ARKit/ARCore • HoloLens Agenda
- 6. 2010 - 2017
- 8. 1968, Apollo Guidance Computer 2000 Transistor 4k Memory
- 9. CPU evolution (Transistors) 5M 160M 1B 7B / 2B on mobile 10B / 3.3B on mobile
- 10. 230 4.400 32.000 90.000 (1996) (2003) (2008) (2016)
- 13. RECAP
- 24. Virtual Reality
- 25. CAVE vs HMD (Head Mounted Display)
- 26. Mobile VR
- 27. Console VR
- 29. Pros • Accuracy (you can add more trackers around the room to increase accuracy) • Latency very low (reduce VR sickness). Cons • Setup • Occlusion • Limitation of the cameras (field of view)
- 31. • . Pros • Simplicity / No Setup • Less hardware needed (no external camera) • No limit of space (potentially) Cons • Reduced accuracy and latency • Only the object in front to the user can be tracked • Powerful hardware
- 32. VR addons
- 33. VR Scenarios and Limitations
- 37. Optical vs Video see - through
- 38. Visual Inertial Odometry (VIO)
- 39. Visual Inertial Odometry (VIO) 6dof in real time : • 3D of xyz motion (translation) • 3D of pitch/yaw/roll (rotation) Pose is tracked: • via the Camera system, by matching a point in the real world to a pixel on the camera sensor each frame. • by the Inertial system (accelerometer & gyroscope — Inertial Measurement Unit or IMU) The output of both of those systems are then combined via a Kalman Filter which determines which of the two systems is providing the best estimate of your “real” position.
- 40. b 𝑑 = 𝑥𝑙 − 𝑥 𝑟 𝑏+𝑥 𝑙 − 𝑥 𝑟 𝑍−𝑓 = 𝑏 𝑍 Z = 𝑏∗𝑓 𝑑 Depth Sensing (Multi camera)
- 41. Single Camera?
- 42. The secret here is to have *really* good IMU error removal: Dead-reckoning
- 44. • . TrueDepth Camera Face tracking. Scene Understanding Detection of horizontal planes (tables, floors, …) and tracking of object. Lighting Estimation Estimation of light available in a scene in order to apply the correct amount of lighting to virtual objects. Hardware and Rendering Optimizations Apple A9, A10, and A11 processors.
- 47. • . Motion tracking Estimates a phone’s relative location based on internal sensors and video footage Scene Understanding Uses the camera to detect flat surfaces Lighting Estimation Helps virtual props cast accurate shadows and otherwise fit in with their surroundings Anchoring objects
- 50. • . ARCore vs Tango Tango’s dedicated infrared depth- sensing camera to create detailed 3D meshes. ARCore has to estimate scale based on the camera feed, while Tango directly measures distance. Nikhil Chandhok (AR director of product): “ARCore is for all the apps that we think that users want”
- 51. HoloLens
- 53. What is HoloLens? HoloLens is the first, fully wireless holographic computer that redefine personal computing and empowers people in new ways.
- 54. How to use
- 55. DEMO
- 57. Holographic Processing Unit v 1.0 TSMC-fabricated 28 nm co-processor. 24 Tensilica DSP cores (12 clusters) 65 million logic gates (used 50%) 8 MB of SRAM 1GB DRAM 1 Trillion Operation per second Sensor aggregator with gesture and environment processing 200x over software implementation Low Power (<10 Watts) 12 mm 12mm
- 58. AR Devices
- 60. • . I've seen things you people wouldn't believe. Attack ships on fire off the shoulder of Orion. I watched C-beams glitter in the dark near the Tannhäuser Gate. All those moments will be lost in time, like tears in rain. Time to die.
- 61. Developer Resources Preview Pricing Documentation Client SDKs Example Code Join Our Community Thank you!
- 68. Developer Resources Preview Pricing Documentation Client SDKs Example Code Join Our Community Thank you! How Augment your Reality Matteo Valoriani