Developing VR in Unity
5 likes1,351 views
Adityo Pratomo discusses the development of virtual reality (VR) in Unity, covering various VR technologies, platforms, and interaction methods. He emphasizes the importance of experimenting with VR due to its newness and lack of established best practices, while also addressing the challenges such as simulation sickness and hardware constraints. Pratomo encourages developers to engage in VR projects to gain experience and contribute to the evolving field.
Developing VR in Unity
- 1. DEVELOPING VR IN UNITY ADITYO PRATOMO - LABTEK INDIE
- 2. DEVELOPING VR IN UNITY ABOUT ME ▸ Chief Technology Officer at Labtek Indie ▸ Chief Academic Officer at Froyo Framework ▸ Certified Unity Developer #20167UCD724 ▸ Loves to tinker with tech
- 3. DEVELOPING VR IN UNITY VR IN LABTEK INDIE It’s Strange (2015) Oculus Rift & Leap Motion, developed in Unity https://irunsiregar.itch.io/its-strange https://www.youtube.com/watch?v=oTii7SSfyLM
- 4. DEVELOPING VR IN UNITY VR IN LABTEK INDIE Combat League (2016) Oculus Rift, developed in Unreal Engine https://www.youtube.com/watch?v=1sHrSOe8JM8
- 5. DEVELOPING VR IN UNITY VR IN LABTEK INDIE 99 Names (2015) Web VR, developed in JS with ThreeJS 99names.adityo.net
- 6. DEVELOPING VR IN UNITY VR IN LABTEK INDIE Mixed Reality Experiment Google Cardboard devloped in Unity with Vuforia SDK http://www.adityo.net/?p=1146
- 7. DEVELOPING VR IN UNITY WHAT I’LL TALK ▸ VR Tech Overview ▸ Desktop VR vs Mobile VR ▸ VR in Unity ▸ Interaction in VR ▸ Towards Better VR Experience ▸ Future Prediction
- 8. NOW IS THE TIME TO START MAKING SOMETHING IN VR, EVEN IF YOU DON'T PLAN ON LAUNCHING IT. IT IS A REALLY GOOD IDEA TO START EXPERIMENTING AND SEE WHAT YOU CAN LEARN. EVEN IF YOU'RE NOT GOING TO LAUNCH SOMETHING, IT IS GOOD TO HAVE EXPERIENCE IN VIRTUAL REALITY SO THAT IF IT MAKES SENSE FOR YOU TO GET INTO IT, YOU'RE ABLE TO Palmer Luckey (Co-founder of Oculus) DEVELOPING VR IN UNITY
- 9. DEVELOPING VR IN UNITY VR TECH OVERVIEW ▸ VR is a tech that creates new perception of reality in the mind of its users ▸ Achieved through the use of HMD (Head Mounted Display) that tricks its user into thinking he/she is in a different place ▸ Closely related to AR, but not similar nor has hierarchical relation
- 10. DEVELOPING VR IN UNITY VR TECH OVERVIEW
- 11. DEVELOPING VR IN UNITY VR TECH OVERVIEW 1. Lens ‣ Emulating depth via focal point adjustment 2. Display ‣ Stereo rendering ‣ FOV emulates real life view 3. Tracking Technology ‣ Tracking head movement ‣ Uses gyro and accelerometer in smartphone 4. 3D Audio ‣ Immersive 3D audio
- 12. DEVELOPING VR IN UNITY CURRENT VR PLATFORMS PC ANDROID CONSOLE OCULUS RIFT HTC VIVE GOOGLE VR (CARDBOARD & DAYDREAM) SAMSUNG GEAR VR PS VR OSVR
- 13. DEVELOPING VR IN UNITY DESKTOP VR VS MOBILE VR: FROM THE TECH SIDE ▸ Desktop VR requires more horse power, but has potential for many kind of interaction and contents ▸ Mobile VR experience is hampered by the device’s spec ▸ Desktop VR can do positional tracking ▸ Mobile VR can only do head positional tracking ▸ Batteries vs Cable ▸ Both do VR content rendering in roughly the same way
- 14. DEVELOPING VR IN UNITY DESKTOP VR VS MOBILE VR: FROM THE BUSINESS SIDE ▸ Mobile VR has lower entry barrier, mid-high end Android phone is sufficient ▸ Desktop VR requires high end gaming rig ▸ Mobile VR has way larger potential market than Desktop ▸ Desktop VR will interests hardcore gamer ▸ Mobile VR will interests budget-oriented gadget enthusiasts
- 15. DEVELOPING VR IN UNITY DESKTOP VR VS MOBILE VR ▸ Mobile VR will help promoting VR to the masses ▸ Desktop VR will help pushing the boundaries, tech wise ▸ Both is ready for mass development, NOW!
- 16. DEVELOPING VR IN UNITY WHY DEVELOP VR NOW? ▸ Relatively new tech ▸ Still no best practice sets in stone ▸ Experiments are highly appreciated ▸ Big or small developers, everybody are still finding its feet ▸ YOU can make the difference
- 17. DEVELOPING VR IN UNITY VR IN UNITY PC OCULUS RIFT HTC VIVE OSVR OCULUS RIFT DK2 / CV1 WIN 7+ UNITY 5.X
- 18. DEVELOPING VR IN UNITY VR IN UNITY PC OCULUS RIFT HTC VIVE OSVR HTC VIVE WIN 7+ UNITY 5.X
- 19. DEVELOPING VR IN UNITY VR IN UNITY PC OCULUS RIFT HTC VIVE OSVR OSVR HMD WIN 7+ UNITY 5.X
- 20. DEVELOPING VR IN UNITY VR IN UNITY ANDROID GOOGLE VR (CARDBOARD & DAYDREAM) SAMSUNG GEAR VR ANDROID VR HMD ANDROID 5.1 + UNITY 5.X GOOGLE CARDBOARD SDK NEXUS 6 (DAYDREAM)
- 21. DEVELOPING VR IN UNITY VR IN UNITY ANDROID GOOGLE VR (CARDBOARD & DAYDREAM) SAMSUNG GEAR VR ANDROID VR HMD ANDROID 5.1 + UNITY 5.X VUFORIA SDK
- 22. DEVELOPING VR IN UNITY VR IN UNITY ANDROID GOOGLE VR (CARDBOARD & DAYDREAM) SAMSUNG GEAR VR ANDROID 5.1 + UNITY 5.X SAMSUNG GALAXY S7 SAMSUNG GEAR VR
- 23. DEVELOPING VR IN UNITY VR IN UNITY: GENERAL ARCHITECTURE ▸ Default camera is replaced by vendor-specific VR camera (usually comes as prefab) ▸ VR Camera will render the game scene to be VR-ready ▸ Other game objects are used as usual GAME SCENE VR CAMERA OTHER GAME OBJECTS
- 24. DEVELOPING VR IN UNITY VR IN UNITY: TIPS FOR THE FIRST TIMER ▸ Most of the time, you’ll wrestle with setting up specific environment ▸ Oculus: Get the latest SDK and fulfil the requirements, use Rift DK2 at least, don’t use laptop ▸ Android: setup Android SDK properly, Google and Vuforia SDKs are generally easy to set up
- 25. DEVELOPING VR IN UNITY VR IN UNITY ▸ After replacing the Main Camera object with the vendor- specific, AR Camera prefab, you’re done. ▸ EASY! ▸ Demo…
- 26. WHAT ELSE TO DO? if getting VR to up and run is easy, then
- 28. DEVELOPING VR IN UNITY INTERACTION IN VR ▸ With headset strapped on, it’s hard to look around ▸ Mouse + keyboard, won’t be the best way to interact
- 29. DEVELOPING VR IN UNITY INTERACTION IN VR LOOK AROUND GAMEPAD INTERACTION GAZE INTERACTION NATURAL INTERACTION (AVAILABLE NOW) OCULUS TOUCH VIVE CONTROLLER GOOGLE DAYDREAM LEAP MOTION EXPERIMENTAL INTERACTION (DIY) VOICE CONTROLLER CUSTOM ELECTRONIC (ARDUINO / RPI / ETC) ? easy to develop, minimal interaction harder to develop, rich interaction
- 30. DEVELOPING VR IN UNITY INTERACTION IN VR: LOOK AROUND ▸ The most basic thing you can do inside VR ▸ Replacing traditional FPS camera, controlled by mouse, with your head ▸ Results in a natural, immersive experience ▸ Used in many VR videos ▸ Minimum interaction, but can still produces excellent experience
- 31. DEVELOPING VR IN UNITY 360 PHOTOS / VIDEOS IN UNITY ▸ Generally speaking, here are the steps: ▸ Obtain 360 photo/video ▸ Make a skybox using sphere/cube ▸ Use the photo/video as the texture for the skybox
- 32. DEVELOPING VR IN UNITY 360 PHOTOS / VIDEOS IN UNITY
- 33. DEVELOPING VR IN UNITY 360 PHOTOS / VIDEOS IN UNITY ▸ Plugin: ▸ Easy Movie Texture (https://www.assetstore.unity3d.com/en/#!/ content/10032) ▸ Discussions: ▸ http://bernieroehl.com/360stereoinunity/ ▸ http://forum.unity3d.com/threads/projecting-a-360-degree-vr- video-on-a-sphere.376522/ ▸ http://talesfromtherift.com/use-a-360-degree-panorama-as-a- skybox/
- 34. DEVELOPING VR IN UNITY GAZE INTERACTION ▸ Using eye as an input ▸ Activating an object by looking at it for a certain period ▸ (Optional) use Gaze Reticle to help user find it’s view’s crosshair
- 35. DEVELOPING VR IN UNITY GAZE INTERACTION
- 36. DEVELOPING VR IN UNITY GAZE INTERACTION ▸ Google Cardboard SDK: ▸ Use GvrGaze script ▸ Drag the script from Assets/GoogleVR/Scripts/UI/ GvrGaze onto to the camera ▸ Implement IGvrGazePointer and IGvrGazeResponder to the interactive object ▸ Vuforia SDK uses Google’s Gaze
- 37. DEVELOPING VR IN UNITY NATURAL INTERACTION WITH LEAP MOTION ▸ Leap Motion offers the ability to track hand gestures ▸ It can be sticked in front of Oculus/Vive to enable user to see his/her hands ▸ Thus, enables a natural interaction using hands, while inside VR
- 38. DEVELOPING VR IN UNITY NATURAL INTERACTION WITH LEAP MOTION
- 39. DEVELOPING VR WITH UNITY NATURAL INTERACTION WITH LEAP MOTION ▸ Download and Import the Leap Motion Core Assets Unity Package ▸ Use the LeapOVRPlayerController Prefab from the Leap Motion + OVR directory as the main VR Camera ▸ The hand will be added to the screen and acts as a 3D objects, just like the others (with colliders too!)
- 40. DEVELOPING VR IN UNITY EXPERIMENTAL INTERACTION: VOICE CONTROL ▸ Use voice to give input to the VR world ▸ Closely tied to the deployment platform ▸ iOS, Android, Windows, Mac, has its own voice recognition system ▸ Unity 5.4 can natively do voice recognition in Windows (https://developer.microsoft.com/en-us/windows/ holographic/voice_input_in_unity)
- 41. DEVELOPING VR IN UNITY EXPERIMENTAL INTERACTION: VOICE CONTROL ▸ Unity-only solution: ▸ Ispikit: https://github.com/ispikit/ispikit-unity ▸ Word Detection: https://www.assetstore.unity3d.com/en/#!/ content/4518 ▸ Unity + other: ▸ Read Serial in Unity ▸ Do voice recognition in other using Python/NodeJS/etc and send serial from there
- 42. DEVELOPING VR IN UNITY EXPERIMENTAL INTERACTION: CUSTOM ELECTRONIC ▸ If you have something different, why not make it yourself ▸ Use Arduino / Raspberry Pi / etc. to read from certain sensor and send its data to Unity ▸ Vital to create haptic feedback and experimenting with different kinds of interaction ▸ Make one of a kind immersive experience
- 43. DEVELOPING VR IN UNITY EXPERIMENTAL INTERACTION: CUSTOM ELECTRONIC
- 44. DEVELOPING VR IN UNITY EXPERIMENTAL INTERACTION: CUSTOM ELECTRONIC ▸ Use Unity to receive Serial data ▸ Android & Microcontroller Bluetooth: https:// www.assetstore.unity3d.com/en/#!/content/16467 ▸ SimpleSerialPort: https://www.assetstore.unity3d.com/en/#!/ content/53594 ▸ Attach sensors to Arduino/Raspberry Pi/other ▸ Send data via Serial/Bluetooth ▸ Watch out for latency!
- 45. DEVELOPING VR IN UNITY TOWARDS BETTER VR EXPERIENCE: REMOVING SIMULATION SICKNESS ▸ Simulation sickness is VR’s biggest enemy ▸ Cause: brain is tricked to feel movement, while static ▸ When that trick is broken, nausea attacks, users opt out from the experience (and probably won’t come back)
- 46. DEVELOPING VR IN UNITY TOWARDS BETTER VR EXPERIENCE: REMOVING SIMULATION SICKNESS ▸ Technical causes: ▸ Latency (maximum should be 20ms) ▸ Many research has gone into how to solve this issue ▸ Many solutions, but none has quite hits the mark
- 47. DEVELOPING VR IN UNITY TOWARDS BETTER VR EXPERIENCE: REMOVING SIMULATION SICKNESS ▸ Hardware design solution: ▸ More comfortable lens ▸ More comfortable HMD when strapped on
- 48. DEVELOPING VR IN UNITY TOWARDS BETTER VR EXPERIENCE: REMOVING SIMULATION SICKNESS ▸ Content design: ▸ Adding nose and feet ▸ Shows a fixed reference point in a frame
- 49. DEVELOPING VR IN UNITY TOWARDS BETTER VR EXPERIENCE: REMOVING SIMULATION SICKNESS ▸ Game design - movement: ▸ In-game movement have more latency potential than standing still ▸ objects are continuously and dynamically rendered in real- time ▸ more processing power ▸ To move around, use teleport ▸ Teleport can be done with gaze interaction, no controller is required
- 50. DEVELOPING VR IN UNITY TOWARDS BETTER VR EXPERIENCE: REMOVING SIMULATION SICKNESS Keep Talking and Nobody Explodes www.keeptalkinggame.com
- 51. DEVELOPING VR IN UNITY TOWARDS BETTER VR EXPERIENCE: REMOVING SIMULATION SICKNESS ▸ Game design - movement: ▸ To move around: ▸ use teleport ▸ use pre-determined route ▸ Teleport can be done with gaze interaction, no controller is required ▸ Pre-determined route will ease us in optimizing assets ▸ Don’t add unnecessary vertical movement
- 52. DEVELOPING VR IN UNITY TOWARDS BETTER VR EXPERIENCE: REMOVING SIMULATION SICKNESS Teleport: https://developer.oculus.com/blog/introducing-the- oculus-sample-framework-for-unity-5/ INMIND VR: https://developer.oculus.com/blog/introducing-the- oculus-sample-framework-for-unity-5/
- 53. DEVELOPING VR IN UNITY TOWARDS BETTER VR EXPERIENCE: REMOVING SIMULATION SICKNESS ▸ Game design - camera view: ▸ It’s natural to have first person view, but it’s easier to result in simulation sickness ▸ To reduce that, use third person view, it will still have good experience
- 54. DEVELOPING VR IN UNITY TOWARDS BETTER VR EXPERIENCE: REMOVING SIMULATION SICKNESS Chronos: www.chronosvr.com Colosse: colossevr.tumblr.com
- 55. DEVELOPING VR IN UNITY TOWARDS BETTER VR EXPERIENCE: REMOVING SIMULATION SICKNESS ▸ Tech tips: ▸ Use low poly objects, with high-res textures to compensate ▸ Toon-shading is okay, photorealistic can probably causes latency
- 56. DEVELOPING VR IN UNITY TOWARDS BETTER VR EXPERIENCE: REMOVING SIMULATION SICKNESS Job Simulator: www.jobsimulatorgame.com Land’s End: www.landsendgame.com
- 57. DEVELOPING VR IN UNITY TOWARDS BETTER VR EXPERIENCE: REMOVING SIMULATION SICKNESS ▸ Tech tips: ▸ Use as many static object as possible (uses least RAM) ▸ Use LOD, culling and batching to prevent rendering too many object at the same time ▸ Use as many baked lighting as possible ▸ https://unity3d.com/learn/tutorials/topics/virtual-reality/ optimisation-vr-unity
- 58. DEVELOPING VR IN UNITY TOWARDS BETTER VR EXPERIENCE: REMOVING SIMULATION SICKNESS
- 59. DEVELOPING VR IN UNITY TOWARDS BETTER VR EXPERIENCE ▸ Experience design: ▸ Design experience from the ground up, don’t just adapt already available 3D game to VR ▸ VR is now best experienced in short but meaningful time (5-20 minutes) ▸ Design focal point, because the user can look into any direction
- 60. DEVELOPING VR IN UNITY TOWARDS BETTER VR EXPERIENCE ▸ UI design: ▸ Make sure the user can see the UI at any time ▸ Go for “no-UI” style, embed the required information in the objects, just like in normal life ▸ Enable the player to look around, even when in pause mode
- 61. DEVELOPING VR IN UNITY TOWARDS BETTER VR EXPERIENCE
- 62. DEVELOPING VR IN UNITY FUTURE PREDICTION ▸ Lower cost of desktop VR ▸ WebVR ▸ VR beyond entertainment
- 63. AT THE SAME TIME, I'D SAY DEVELOPERS SHOULD BE REALISTIC. DON'T BELIEVE THE MOST OPTIMISTIC THINGS THE ANALYSTS SAY. ESPECIALLY WHEN THOSE ANALYSTS DON'T ACTUALLY KNOW ANYTHING ABOUT VR. IT'S GOING TO TAKE TIME FOR VIRTUAL REALITY TO BECOME TRULY MASS MARKET AND SUCCESSFUL, SO PEOPLE SHOULD SCALE THEIR EFFORTS APPROPRIATELY. Palmer Luckey (Co-founder of Oculus) DEVELOPING VR IN UNITY