Hacking Reality: Browser-Based VR with HTML5

HACKING REALITY:
HTML5 AND
JAVASCRIPT
FOR CROSS-PLATFORM
VR
TONY PARISI
OCTOBER, 2014

ABOUT ME
CREDS
Co-creator, VRML and X3D
http://www.tonyparisi.com 10/21/2014
CONTACT
tony@vizi.gl
skype: auradeluxe
http://twitter.com/auradeluxe
http://www.tonyparisi.com/
http://www.learningwebgl.com/
GET VIZI
https://github.com/tparisi/Vizi
GET THE BOOKS!
WebGL: Up and Running
http://www.amazon.com/dp/144932357X
Programming 3D Applications with HTML and WebGL
http://www.amazon.com/Programming-Applications-HTML5-WebGL-Visualization/
dp/1449362966
MEETUPS
http://www.meetup.com/WebGL-Developers-Meetup/
http://www.meetup.com/Web-VR/
BOOK CODE
https://github.com/tparisi/WebGLBook
https://github.com/tparisi/Programming3DApplications
GET GLAM
http://www.glamjs.org/
https://github.com/tparisi/glam/
WORK
http://www.vizi.gl/

THE PROMISED LAND!
CONSUMER VR

VR TODAY
GIANT DOWNLOADS
SILO EXPERIENCES
CUMBERSOME NATIVE APPS AND INSTALLS
PROPRIETARY PLATFORMS AND
DEVELOPMENT STACKS
VR DOTH BE
HARD!

WHY I LOVE THE WEB
 INSTANT PUBLISHING
 INSTANT ACCESS TO INFORMATION
 NO TOLLS
 NOBODY CONTROLS IT
 CULTURE OF COLLABORATION
 VIEW SOURCE
…THE WEB WILL NEVER CLOSE UP SHOP.
image: Mark Surman
http://commonspace.wordpress.com/2014/03/12/happybirthday/

THE THREE D’S OF THE
WEB
 DEVELOPMENT
 CROSS-PLATFORM
 VENDOR-NEUTRAL
 OPEN SOURCE
 DEPLOYMENT
 CLOUD
 PUSH-BUTTON UPDATE AND PUBLISH
 DISTRIBUTION AND DISCOVERY
 EMBED IN OTHER PAGES
 SHARE WITH A HYPERLINK
 NO APP TO DOWNLOAD

VR AND THE WEB:
TWO GREAT TASTES… ?

WEB VR
FAST, CHEAP, AND TOTALLY
DEMOCRATIZED.
 BROWSER-BASED VIRTUAL
REALITY
 WEBGL
 CSS3
 VR SUPPORT NOW IN
BROWSER DEV BUILDS!!!
 NO BIG APP DOWNLOADS
AND INSTALLS!!!
 JUST ADD SMART PHONE
 “SMARTVR” USING
GOOGLE CARDBOARD
$25 CHEAP!

AN EXAMPLE
INFORMATION DIVING SHOWCASE
http://mozvr.github.io/infodive/
POWERED BY FIREFOX BUILT WITH VIZI

THE WEBVR API
1. QUERY FOR VR DEVICE(S) TO RENDER
// polyfill
var getVRDevices = navigator.mozGetVRDevices /* FF */ ||
navigator.getVRDevices; /* webkit */
var self = this;
getVRDevices().then( gotVRDevices );
function gotVRDevices( devices ) {
var vrHMD;
var error;
for ( var i = 0; i < devices.length; ++i ) {
if ( devices[i] instanceof HMDVRDevice ) {
vrHMD = devices[i];
self._vrHMD = vrHMD;
self.leftEyeTranslation = vrHMD.getEyeTranslation( "left" );
self.rightEyeTranslation = vrHMD.getEyeTranslation( "right" );
self.leftEyeFOV = vrHMD.getRecommendedEyeFieldOfView( "left" );
self.rightEyeFOV = vrHMD.getRecommendedEyeFieldOfView( "right" );
break; // We keep the first we encounter
}
}
}
get left/right eye
(camera) positions
get per-eye camera field of view; use
WebGL to render scene from two cameras

THE WEBVR API
2. GO FULLSCREEN (VR MODE)
fullscreen mode requires a DOM
element
var self = this;
var renderer = this._renderer;
var vrHMD = this._vrHMD;
var canvas = renderer.domElement;
var fullScreenChange =
canvas.mozRequestFullScreen? 'mozfullscreenchange' : 'webkitfullscreenchange';
document.addEventListener( fullScreenChange, onFullScreenChanged, false );
function onFullScreenChanged() {
if ( !document.mozFullScreenElement
&& !document.webkitFullScreenElement ) {
self.setFullScreen( false );
}
}
if ( canvas.mozRequestFullScreen ) {
canvas.mozRequestFullScreen( { vrDisplay: vrHMD } );
} else {
canvas.webkitRequestFullscreen( { vrDisplay: vrHMD } );
}
handle exiting fullscreen
mode
request fullscreen mode
for this VR device

THE WEBVR API
3. HEAD TRACKING
query HMD device state
// polyfill
var getVRDevices = navigator.mozGetVRDevices /* FF */ ||
navigator.getVRDevices; /* webkit */
var self = this;
getVRDevices().then( gotVRDevices );
function gotVRDevices( devices ) {
var vrInput;
var error;
for ( var i = 0; i < devices.length; ++i ) {
if ( devices[i] instanceof PositionSensorVRDevice ) {
vrInput = devices[i]
self._vrInput = vrInput;
break; // We keep the first we encounter
}
}
}
…
// called once per tick from requestAnimationFrame()
var update = function() {
var vrState = this.getVRState();
if ( !vrState ) {
return;
}
// vrState.hmd.rotation contains four floats, quaternion x,y,z,w
setCameraRotation(vrState.hmd.rotation);
};
get head-tracking VR device
update camera to match HMD
device orientation

WEBVR AND CARDBOARD
 GOOGLE CARDBOARD SHOWCASE
 Mobile Chrome http://g.co/chromevr
 Desktop Chrome http://vr.chromeexperiments.com/
 EVEN EASIER
 RENDER WEBGL SIDE-BY-SIDE STEREO (NO NEED TO QUERY
DEVICES)
 USE BROWSER DEVICE ORIENTATION API FOR HEAD TRACKING

WEBVR AND THREE.JS
 THE MOST POPULAR WEBGL LIBRARY
 http://threejs.org/
 LATEST STABLE VERSION (r68) INCLUDES STEREO
RENDERING AND HEAD TRACKING
 RENDERING
examples/js/effects/StereoEffect.js (Cardboard)
examples/js/effects/VREffect.js (Rift)
 HEAD TRACKING
examples/js/controls/DeviceOrientationControls.js (Cardboard)
examples/js/controls/VRControls.js (Rift)

VIZI: A FRAMEWORK FOR
WEBVR APPLICATIONS
 GOAL: MAKE IT EASY TO QUICKLY BUILD INTERESTING 3D
APPLICATIONS
 ARCHITECTURE INSPIRED INSPIRED BY MODERN GAME ENGINE
DESIGN
 COMPONENT-BASED DESIGN – EASILY PLUG NEW FEATURES INTO OBJECTS
 APPLICATION OBJECT – HANDLES EVENT LOOP, THREE.JS CREATION, PAGE RESIZE,
ROUTING EVENTS TO OBJECTS
 INTERACTIONS – MAKE IT EASY TO PUT MOUSE AND TOUCH INTERACTION ON OBJECTS
 BEHAVIORS – PREBUILT BEHAVIORS AUTOMATICALLY ROTATE, MOVE ETC.
 PRE-BUILT OBJECTS – FOR COMMONLY USED DESIGN PATTERNS
 SIMILAR IN DESIGN TO UNITY3D
 USES THREE.JS FOR ALL GRAPHICS
 ENHANCES THREE.JS VR RENDERING AND CONTROLLERS

OPEN TOOLS
FOR CROSS-PLATFORM VR
game engines/IDEs
Goo Enginehttp://www.gootechnologies.com/
Verold http://verold.com/
Turbulenz https://turbulenz.com/
PlayCanvas http://www.playcanvas.com/
Artillery Engine
https://artillery.com/
Sketchfab https://sketchfab.com/
Unreal https://www.unrealengine.com/
Unity http://unity3d.com/#unity-5
scene graph libraries/page frameworks
Three.js
http://threejs.org/
SceneJS
http://scenejs.org/
BabylonJS
http://www.babylonjs.com/
Vizi
Voodoo.js
http://www.voodoojs.com/
PhiloGL
http://www.senchalabs.org/philogl/
tQuery
http://jeromeetienne.github.io/tquery/

PRO TOOLS FOR WEB VR
Unreal 4 in WebGL
https://www.youtube.com/watch?v=c2uNDlP4RiE#t=42
60FPS
ported in 5 days
Unreal native C++ engine -> JavaScript
Emscripten + asm.js
EMSCRIPTEN -
THE COOLEST HACK
EVER!
https://github.com/kripken/ems
cripten
 CROSS-COMPILE C++
NATIVE CODE TO
JAVASCRIPT
 asm.js- LOW-LEVEL
OPTIMIZED JAVASCRIPT
 UNITY, UNREAL ENGINES
USE THIS TO DEPLOY ON
THE WEB
 WATCH OUT: HUGE
DOWNLOADS AND HARD TO
DEBUG…. !

WEBVR AND CSS
MOZILLA VR CSS SHOWCASE
http://mozvr.github.io/vr-web-examples/domvr-birds/

WEBVR AND CSS
<script type="text/javascript" src="js/vrutils.js"></script>
<script>
/* VR Headset and its associated orientation/position sensor */
var vrHMD, vrSensor;
/* Element that will serve as our camera, moving the rest of the scene around */
var cssCamera = document.getElementById("camera");
/* the camera's position, as a css transform string. For right now, we want it just in the middle. */
var cssCameraPositionTransform = "translate3d(0, 0, 0)";
/* Request animation frame loop. */
function animate() {
/* Acquire positional data from VR headset and convert into a transformation that can be applied to CSS. */
if (vrSensor !== undefined) {
var state = vrSensor.getState();
var cssOrientationMatrix = cssMatrixFromOrientation(state.orientation, true);
}
/* Apply positional data to camera element */
cssCamera.style.transform = cssOrientationMatrix + " " + cssCameraPositionTransform;
window.requestAnimationFrame(animate);
}
query HMD device state
calculate “camera” orientation
update “camera’s” CSS 3D transform

VR + ML
A MARKUP LANGUAGE FOR THE
METAVERSE?
 GLAM (GL AND MARKUP) - A
DECLARATIVE LANGUAGE FOR 3D
WEB CONTENT
 DEFINE 3D SCENE CONTENT IN
MARKUP; STYLE IT IN CSS
THE MARKUP
<glam>
<renderer type="rift"></renderer>
<scene>
<controller type="rift"></controller>
<background id="sb1" class="skybox">
</background>
<group y ='1' z='-3'>
<sphere class="bubble skybox”>
</sphere>
<sphere class="bubble skybox”>
</sphere>
</group>
…
THE CSS
<style>
.skybox {
envmap-right:url(../images/Park2/posx.jpg);
…
}
.bubble {
radius:.5;
shader-vertex:url(../shaders/fresnel.vs);
shader-fragment:url(../shaders/fresnel.fs);
shader-uniforms:mRefractionRatio f 1.02
mFresnelBias f 0.1 mFresnelPower f 2.0 mFresnelScale
f 1.0 tCube t null;
}
#sb1 {
background-type:box;
}
</style>

CHALLENGES
 WEBVR ON OCULUS IS NOT READY FOR PRIME TIME
 (THAT’S OK NEITHER IS OCULUS!)
 LATENCY IS THE BIGGEST ISSUE – BROWSER NEEDS TO UN-THROTTLE
AT 60FPS
 BUT WE’RE GOOD TO GO ON CARDBOARD!
 60FPS WORKS GREAT
 NEARLY 2 BILLION VR DEVICES ALREADY DEPLOYED!
 FRAMEWORKS AND TOOLS ARE TYPICALLY WEB-ISH: UNDER
DEVELOPMENT, ALWAYS IN FLUX, PRETTY MUCH OUT OF
CONTROL
 BUT OPEN SOURCE SO WE CAN LIVE WITH IT

LINKS
 BROWSER DEV BUILDS
Firefox http://blog.bitops.com/blog/2014/08/20/updated-firefox-vr-builds/
Chrome https://drive.google.com/folderview?id=0BzudLt22BqGRbW9WTHMtOWMzNjQ
 MOZILLA NEXT STEPS FOR VR (THURSDAY 23 OCT 2014)
https://air.mozilla.org/virtual-reality-the-web-next-steps/
 SAN FRANCISCO WEBVR MEETUP
 WEBVR MAILING LIST
web-vr-discuss@mozilla.org
 CARDBOARD VR SHOWCASE
http://vr.chromeexperiments.com/

KEEP IN TOUCH
CREDS
Co-creator, VRML and X3D
CONTACT
tony@vizi.gl
skype: auradeluxe
http://twitter.com/auradeluxe
http://www.tonyparisi.com/
http://www.learningwebgl.com/
GET VIZI
GET THE BOOKS!
WebGL: Up and Running
http://www.amazon.com/dp/144932357X
Programming 3D Applications with HTML and WebGL
http://www.amazon.com/Programming-Applications-HTML5-WebGL-Visualization/
dp/1449362966
MEETUPS
http://www.meetup.com/WebGL-Developers-Meetup/
BOOK CODE
https://github.com/tparisi/WebGLBook
https://github.com/tparisi/Programming3DApplications
GET GLAM
http://www.glamjs.org/
WORK
http://www.vizi.gl/

Hacking Reality: Browser-Based VR with HTML5

More Related Content

What's hot (20)

Viewers also liked (19)

Similar to Hacking Reality: Browser-Based VR with HTML5 (20)

More from Tony Parisi (13)

Recently uploaded (20)

Hacking Reality: Browser-Based VR with HTML5