Build Your Own VR Display Course - SIGGRAPH 2017: Part 1
- 1. Build Your Own VR Display An Introduction to VR Display Systems for Hobbyists and Educators Gordon Wetzstein Robert Konrad Nitish Padmanaban Hayato Ikoma
- 2. vir·tu·al re·al·i·ty vərCH(əw)əl rē‘alədē the computer-generated simulation of a three-dimensional image or environment that can be interacted with in a seemingly real or physical way by a person using special electronic equipment, such as a helmet with a screen inside or gloves fitted with sensors.
- 3. VPL Research
- 5. Other Applications of VR • architecture walkthroughs • remote operation of vehicles (e.g., drone racing!) • education • communication • virtual travel to expensive or risky environments • training surgeons, pilots,... • create empathy • … VR is a new medium ...
- 7. National Academy of Engineering “Enhance Virtual Reality” is 1 of 14 NAE grand challenges for engineering in the 21st century image from NAE
- 8. Exciting Engineering Aspects of VR/AR • CPU, GPU • IPU, DPU? • sensors & imaging • computer vision • scene understanding • human perception • displays: visual, auditory, vestibular, haptic, …• VR cameras • cloud computing • shared experiences • HCI • compression, streaming images from facebook, Microsoft
- 9. Personal Computer e.g. Commodore PET 1983 Laptop e.g. Apple MacBook Smartphone e.g. Google Pixel AR/VR e.g. Microsoft Hololens ???
- 10. A Brief History of Virtual Reality 1838 1968 2012-2017 Stereoscopes Wheatstone, Brewster, … VR & AR Ivan Sutherland VR explosion Oculus, Sony, HTC, MS, … Nintendo Virtual Boy 1995 ???
- 12. Ivan Sutherland’s HMD • optical see-through AR, including: • displays (2x 1” CRTs) • rendering • head tracking • interaction • model generation I. Sutherland “A head-mounted three-dimensional display”, Fall Joint Computer Conference 1968
- 13. Nintendo Virtual Boy • 770,000 units sold, commercial failure – judge for yourself Game: Red Alarm
- 14. = ?
- 15. 1968 1980s 2000s electronic / digital HCI / haptics low cost, high-res, low-latency!
- 16. • Samsung 5.7” AMOLED: 1920x1080px, 75Hz • 2 sets of lenses (for different prescriptions) • InvenSense 6-axis IMU • ARM Cortex-M3 MCU • … https://www.ifixit.com/Teardown/Oculus+Rift+Development+Kit+2+Teardown/27613 OculusDK2Teardown key factors: low latency & wide fov!
- 17. Field of View!
- 18. Simple Magnifier-type VR Displays
- 20. Google Glass
- 21. See-through AR Displays - Pepper’s Ghost 1862
- 22. See-through AR Displays: Waveguides Microsoft Hololens
- 23. Gordon Wetzstein Robert Konrad Nitish Padmanaban Hayato Ikoma Who We Are Assistant Professor of EE/CS Research Assistants and TAs of Stanford VR Course
- 24. Some Projects We’ve Worked On “Layered 3D” SIGGRAPH 2011, “Polarization Fields” SIGGRAPH Asia 2011, “Tensor Displays” SIGGRAPH 2012, “Focus 3D” ToG 2013, “Eyeglasses-free Displays” SIGGRAPH 2014, “Light Field Projection” SIGGRAPH 2015, …
- 25. Some Projects We’ve Worked On F. Huang, K. Chen, G. Wetzstein “The Light Field Stereoscope”, SIGGRAPH 2015 R. Konrad, E. Cooper, G. Wetzstein “Focus-tunable and Monovision Near-eye Displays”, SIGCHI 2016
- 26. Some Projects We’ve Worked On R. Konrad, N. Padmanaban, K. Molner, E. Cooper, G. Wetzstein “Accommodation- invariant Computational Near-eye Displays”, SIGGRAPH 2017 SIGGRAPH 2017 Paper, Wed 9-10:30am, Room 150/151
- 27. Some Projects We’ve Worked On N. Padmanaban, R. Konrad, T. Stramer, E. Cooper, G. Wetzstein “Optimizing VR for All Users Through Gaze-contingent Focus Displays”, PNAS 2017 SIGGRAPH 2017 Talk, Thu 3:45-5:15pm, Room 152
- 28. Stanford EE 267
- 29. About EE 267 • experimental hands-on class at Stanford, taught twice • lectures + assignments = one big project – build your own VR HMD • all hardware provided, but must return at the end • enrollment limited, because it’s a lab-based class and we only have limited hardware kits
- 30. About EE 267 • learning goals: • fundamental concepts of VR and Computer Graphics • implement software + hardware of a head mounted display • prerequisites: programming (C/C++, JavaScript), linear algebra • students: 80 undergrad + grad, teams of 2 get a hardware kit
- 31. About EE 267 Lecture slides, course notes, assignments, detailed grading breakdown, open source hardware, … available on http://stanford.edu/class/ee267/
- 32. SIGGRAPH Course Overview • Introduction and Overview, Gordon Wetzstein, 5 min • Overview of DIY Hardware and Software, Gordon Wetzstein, 10 min • Graphics Pipeline, Stereo Rendering, Lens Distortion, Robert Konrad, 60 min • IMUs, Orientation & Position Tracking, Sensor Fusion, Gordon Wetzstein, 60 min • Spatial Sound, Nitish Padmanaban, 30 min • Cinematic VR – Build your own VR camera, Robert Konrad, 25 min • Discussion and Q&A, All presenters, 5 min • Live Demos throughout the course: Hayato Ikoma More details & course notes: http://stanford.edu/class/ee267/
- 33. Build Your Own VR Display Overview of DIY Hardware and Software Gordon Wetzstein Stanford University stanford.edu/class/ee267/
- 34. HMD Housing & Lenses 6” LCD & HDMI Driver Board VRduino Flex Sensors Vibration Motors IMU & Teensy HDMI Cable 2x USB Cable
- 35. HMD Housing and Lenses • View-Master VR Starter Kit ($15- 20) or Deluxe VR Viewer ($23) • implements Google Cardboard 1.0/2.0 • very durable – protect flimsy LCDs • may need to drill additional holes
- 36. Display • Topfoison LCDs: • 1080p - $90 • 1440p (2K) - $100 • HDMI driver boards included • no audio jack! doesn’t fit in housing • super easy to use as external monitor on desktop or laptop
- 37. Display • same LCD you’ll find in many smartphones • reasonably robust (only broke a few) • ideally use OLEDs, but haven’t found any low-cost ones similar to these LCDs
- 38. Strobing the Display Backlight • for low-persistence applications, can strobe LED backlight at 500Hz • custom PCB with Arduino – email us if you’re interested
- 39. VRduino • Arduino-based open source platform for: • orientation tracking • positional tracking • interfacing with other IO devices • custom-design for EE 267 by Keenan Molner • all HW-related files on course website
- 40. VRduino • Teensy 3.2 microcontroller (48 MHz, $20) for all processing & IO • InvenSense 9250 IMU (9- DOF, $6) for orientation tracking • Triad photodiodes & precondition circuit ($1) for position tracking with HTC Lighthouse
- 41. VRduino
- 44. VRduino Photodiode 0 Photodiode 1 Photodiode 2 Photodiode 3
- 45. About EE 267 – VRduino IMU Teensy 3.2 4 photodiodes other GPIO pins
- 46. VRduino x=42mm, y=25mm x y x=42mm, y=-25mm x=-42mm, y=25mm x=-42mm, y=-25mm
- 47. VRduino pins 20,21 pins 22,23 pins 9,10 pins 5,6
- 48. VRduino SCL: pin 19 SDA: pin 18
- 50. Student-built Input Devices • data gloves with flex sensors • different types of controllers with tactile feedback via vibration motors • all connected to VRduino GPIO pins images from Adafruit.com
- 51. Live Demo