Mobile Day - WebVR
2 likes265 views
This document provides an overview of VR, AR and MR technologies and the A-Frame framework for building VR experiences. It discusses that VR immerses the user in a simulated environment, AR overlays digital content on the real world, and MR combines both. Key factors that have enabled growth in these areas are improved smartphone cameras and processing power. The document outlines important input sensors, processing techniques like SLAM, graphics APIs, and frameworks like A-Frame that can be used to build VR/AR/MR applications. It encourages experimenting with new opportunities in this emerging field.
Mobile Day - WebVR
- 2. VR/AR/MR in a Nutshell Rabimba Karanjai, Mozilla Research Mixed Reality Mozilla Tech Speaker @rabimba
- 3. What the Heck Doodle? ARKit ARCore MR VR XR AR Oculus Sony Vive Meta HoloLens
- 4. What You’ll Learn Today ● What is VR / AR / MR? ● Why is this all happening now? ● What do I really need to know?
- 5. What is VR/AR/MR? VR: Virtual Reality AR: Augmented Reality MR: Mixed Reality XR: We don’t know Reality
- 6. Virtual Reality ● Completely immersive ● Wide field of view ● Fast refresh rate (90+fps) ● Fast sensor refresh
- 7. Augmented Reality ● Overlaid on real world ● Anchored to real world ● Non-immersive
- 9. Partial AR
- 13. Why Now?
- 14. The Long History of VR/AR/MR Ivan Sutherland, MIT Sword of Damocles (1968) 3D wireframe rooms
- 16. 1980s: VPL ● Data Glove ● Eye Phone ● Data Suit
- 17. Mattel’s Nintendo Power Glove
- 20. Virtual Reality Markup Language ● Scene based ● In the browser ● Slow to download ● Slow to render
- 22. Smartphone Era
- 23. Smartphone+ Era ● Dual cameras ● Depth sensing cameras ● Machine Learning in hardware ● Even more powerful GPU & CPU
- 24. What do I need to know?
- 25. Tech Stack for VR & AR Input: Sensors & Camera Processing: Image & Sensor Fusion Drawing: 3D Graphics API Application: Framework
- 26. Input: Sensors & Camera ● Accelerometer: relative to gravity ● Magnetometer: relative to earth’s magnetic field ● Gyroscope: relative angle movement ○ 9DOF sensor breakout board for 15$ @ Adafruit! ● Optical Camera: standard phone camera ● Stereo Camera: dual camera setup ● Depth Sensing Camera: Kinect, Face ID
- 27. Processing the 3D Environment ● Markers: Symbols or Beacons ● SLAM: Simultaneous Localization and Mapping ● MonoSLAM: SLAM in real-time with just one camera
- 28. HTC Vive Sensor Beacon two beacons + headset sensors = highly accurate
- 29. Processing: VR/AR APIs ● Vuforia ● ARToolkit ● WebVR ● ARKit ● ARCore ● WebXR
- 30. Vuforia
- 31. ARToolkit
- 32. WebVR ● Mobile: used by web apps for Google Cardboard on any phone. ● Desktop: works with major HMDs ● Shipping: Firefox for Windows ● Beta: Chrome, FF for other platforms
- 33. ARKit
- 34. Tango ARCore
- 35. WebXR: Work In Progress! ● Proposed Future Web Standard ● Extend WebVR with new features ● Work with ARKit & ARCore ● Polyfill for old browsers using WebRTC + sensors https://github.com/mozilla/webxr-api https://github.com/mozilla/webxr-polyfill
- 36. Drawing: Graphics APIs ● Medium Level ○ OpenGL (ES) ○ DirectX ○ WebGL ● Low Level ○ Metal ○ Vulkan ○ Direct X 12 ● High Level ○ Unity ○ Unreal ○ Three.js ○ Babylon.js
- 37. Application: Frameworks ● Scene Kit ● Cardboard & Daydream ● Hololens Academy ● React VR ● ArgonJS ● A-Frame
- 38. What do you *really* need to know? ● Start with web stack ● Follow Mozilla and Google and Microsoft ● Play with A-Frame ● Get a Cardboard viewer.
- 39. What can I do? ● Experiment! We are still in the Model T days! ● No one knows what the killer app will be. ● First new medium in decades ● Go play!
- 40. State of WebVR and Aframe The web framework for building VR experiences @rabimba | Mozilla VR | aframe.io
- 41. Friction of VR Ecosystems Gatekeepers Installs Closed
- 42. WebVR An open virtual reality platform with the advantages of the Web Open Connected Instant
- 43. Browser APIs that enable WebGL rendering to headsets and access to VR sensors
- 46. A-Frame A declarative framework for building virtual reality experiences on the Web
- 47. Hello World <a-scene> <a-box color="#4CC3D9" position="-1 0.5 -3" rotation="0 45 0"></a-box> <a-cylinder color="#FFC65D" position="1 0.75 -3" radius="0.5" height="1.5 <a-sphere color="#EF2D5E" position="0 1.25 -5" radius="1.25"></a-sphere> <a-plane color="#7BC8A4" position="0 0 -4" rotation="-90 0 0" width="4" h <a-sky color="#ECECEC"></a-sky> </a-scene>
- 48. Hello World
- 49. <a-animation> <a-box> <a-camera> <a-circle> <a-collada-model> <a-cone> <a-cursor> <a-curvedimage> <a-cylinder> <a-dodecahedron> <a-isocahedron> <a-image> <a-light> <a-obj-model> <a-plane> <a-ring> <a-tetrahedron> <a-torus> <a-torus-knot> <a-sky> <a-sound> <a-sphere> <a-videosphere>
- 50. Registry Curated collection of A-Frame components
- 52. Registry Curated collection of A-Frame components
- 53. Motion Capture Record then develop room scale VR on the go Remix Glitch
- 58. Journalism - Fear of the Sky Amnesty International UK
- 59. Journalism - Journey to Mars The Washington Post
- 60. Guri VR Build your own world in VR
- 63. Data Visualization - Adit @datatitian
- 65. AR - AR.js + A-Frame @jerome_etienne
- 66. Come build WebVR with us! aframe.io/community
- 67. Thank you @rabimba | rabimba@mozilla.com| aframe.io