Immersive Virtual Reality

Immersive Virtual Reality
Sébastien Kuntz
http://cb.nowan.net/blog
Sebastien.Kuntz@nowan.net

Last update
November 2012
Friday, November 16, 12

License
”Immersive Virtual Reality Introduction” by Sebastien Kuntz is
licensed under a Creative Commons Attribution-ShareAlike 3.0
Unported License. http://creativecommons.org/licenses/by/3.0/
• With the understanding that:
You are free: • Waiver — Any of the above conditions can be
• to Share — to copy, distribute and transmit the waived if you get permission from the copyright
work holder.
• Public Domain — Where the work or any of its
• to Remix — to adapt the work
elements is in the public domain under applicable
• to make commercial use of the work law, that status is in no way affected by the license.
Under the following conditions: • Other Rights — In no way are any of the following
• Attribution — You must attribute the work in the rights affected by the license:
manner speciﬁed by the author or licensor (but not in ◦ Your fair dealing or fair use rights, or other
any way that suggests that they endorse you or your use applicable copyright exceptions and limitations;
of the work). ◦ The author's moral rights;
• You must attribute “Immersive VR introduction” to ◦ Rights other persons may have either in the
(with this link): work itself or in how the work is used, such as
• Sébastien Kuntz - http://cb.nowan.net/blog publicity or privacy rights.
• Notice — For any reuse or distribution, you must
make clear to others the license terms of this work.
The best way to do this is with a link to this web
page.


Plan

Deﬁnitions
VR Systems
Applications
Techniques


WHO AM I ?
• Founder editor of

• http://www.imin-vr.com

• AFRV - French VR association: Board member - http://www.af-rv.fr

• VR Geeks association: President - http://www.vrgeeks.org

• Blog: http://cb.nowan.net/blog

• Twitter: @Cb_VRGeek

• 10 years in VR

• Virtools / Dassault Systèmes

• SNCF - French railways

What is VR for you ?


Immersive Virtual Reality
Presence in a virtual world

(c) Barco
(c) eMagin


Immersive VR = iVR

iVR is the science and technology required for a user to
feel present, via perceptive, cognitive
and functional immersion and interaction, in a
(computer) generated environment.


Presence
Cognitive presence
Perceptive presence

Results
Natural interactions
Natural reactions


What is VR ?
Realtime 3D != VR
Stereoscopy != VR
3D Tracking != VR But everything
Haptics != VR combined can be VR
3D Sound != VR
3D Interactions != VR


Ridiculous
If it’s real VR,
you probably
look ridiculous


Why now ?
Already changing world in industry and research
Democratization
Are we in the future ? No ! We’re still in prehistory !
WE can make a difference, invent systems, applications,
businesses, interactions

Measure presence ?


Measure presence ?

Can’t objectively measure presence (qualia)
What you can measure is if people respond to
this Virtual Environment as if it is real, which is
a result of presence
Compare real-life behavior vs virtual
If avatar: brain zone activates when user
accepts the avatar as real


Immersion parameters
Graphics framerate
Extent of tracking
Tracking latency (measure?)
Quality of the images
Field of view
Plausibility of behaviors
Range of sensory modalities
...

Paul Milgram


VR Systems


Projector based VR Systems


CAVE

http://www.jamstec.go.jp/esc/research/Perception/vr.en.html

Cave

(c) Sébastien Kuntz


http://www.youtube.com/watch?v=Hb29bLHEjUg
Clarte - SAS Cube

Immersia 3


Immersia 3 - IRISA Rennes

http://www.youtube.com/watch?v=cTd5DLe8KD0

Calit2 - StarCave

http://www.scivee.tv/node/3648
http://www.calit2.net/newsroom/release.php?id=1383

Projector based
Reality center - HoloStage

(c) Arnaud Muthelet

Projector based -
Workbench

http://liris.cnrs.fr/~sbrandel/en/research/VR/
LSIIT-IGG

Projector based

Workbench © PSA Peugeot Citroën Mediatheque.


Clarte R-Screen

http://www.youtube.com/watch?v=AcV5ma5nEVg

Panoscope 360°

(c) Matsushita

La Géode

jDome

http://www.youtube.com/watch?v=rNsFSVhZi9w

Toob
Immersive dome

http://youtu.be/N6LiLaNOHHU


Barco - RP-360

• 13 projectors
• Diamètre: 3.4m

Barco - RP-360

http://www.youtube.com/watch?v=3EbrQwpOx24

6 users: stereo + tracking

Weimar University (DE)
http://www.uni-weimar.de/cms/medien/vr/research/display-systems/c1x6-a-stereoscopic-six-user-display-for-co-located-collaboration-in-shared-virtual-environments.html

6 users
stereoscopy + tracking

• video

Weimar University (DE)
http://www.uni-weimar.de/cms/medien/vr/research/display-systems/c1x6-a-stereoscopic-six-user-display-for-co-located-collaboration-in-shared-virtual-environments.html

Screen based

Full-scale NexCAVE developed and deployed by Calit2 for Saudi
Arabia's King Abdullah University of Science and Technology -
http://www.calit2.net/newsroom/release.php?id=1743


Ultra-Short throw
projectors

• Sanyo PDG-DWL2500
• 1280x800
• 2500 lumens
• Canon LV-8235 UST


Ultra-Short throw projectors

http://www.youtube.com/watch?v=VSblSrTRJh0


TV-3D - 154”



Projector based VR Systems
(-)
(Very) expensive !
Lots of space ! ( front and behind, AC... )
Not 360°
Cluster hard to program !
(+)
+/- Collaborative
Not much to wear
High resolution, big FOV

HMD
Head mounted displays


Vuzix
•640x480
•32° FOV

•852x480
•32° FOV


eMagin Z800

800x600
40° FOV


Sony

HMZ-T1
1280x1024
45°


Sensics

piSight
Multiple screens
zSight
1280x1024
70° FOV


NVIS
nVisor SX60
1280x1024
60° FOV
nVisor SX111
1280x1024
111° FOV


Fakespace Labs

Wide5
150° horizontal FOV
1600x1200
Progressive


OCULUS RIFT
• Developer kit
• 90° horizontal
FOV
• 1280x800
• 640x800 per
eye


HMD
(-)
Low resolution, low FOV
Must wear, adjust
No peripheral vision
(+)
Lightweight
Compact
Easier to program
360° tracking


Summary
Projector/screen based
Cave, holostage, workbench
Collaborative, high FOV
Expensive, Complex
HMD
360°
“Cheap”
Low Fov/resolution


Depth perception


Depth perception

How do you perceive depth ?


Stereoscopy ?

http://www.youtube.com/watch?v=Jd3-eiid-Uw
Johnny Lee - Head Tracking with a wiimote

Stereoscopy ?

http://www.youtube.com/watch?v=nRYybo2WNwU
Esiea Virtual Showcase

Stereoscopy

You perceived depth without stereoscopy !


Depth perception
Monoscopic cues : Stereoscopic cues :
perspective
eye accomodation
atmospheric artifacts (fog)
relative sizes
eye convergence
angle
occlusion
parallax disparity between left
shadows and right images

Parallax can be more important for depth perception!
Stereoscopy is important for size and distance
evaluation

Stereoscopy


Stereoscopy
• Useful for
• Relative distance perception
• Detection of objects
• Discrimination between two objects
• Localisation
• But...
• Fusing problems
• Headaches
• Visual fatigue
• Accommodation / Vergence dissociation

Different stereoscopic
separation schemes
Active shutter glasses
Polarized glasses
Anaglyph
Inﬁtec
Autostereoscopic
Holograms


Active stereoscopy


Passive Stereoscopy

Linear
Circular Polarization
Polarization


Anaglyph


Inﬁtec

- Eye separation by color spectrum -


Light Efﬁciency

http://www.barco.com/projection_systems/downloads/Stereoscopic_technologies_overview.pdf

Autostereoscopic


Holograms

Stores ALL light interactions
through an entire “window”


Holograms


Stereoscopy for multiple
users ?


Stereoscopy for multiple
users ?

Active + polarized
Sequential


Multi-display


What system can we create
with only one graphics card?


What can we do with one
graphics card?


Nvidia SLI - ATI Crossﬁre
Use two graphics card for output on only one
graphics cards

Alternate frame rendering

Split frame rendering


Scalable graphics
Image decomposition ( ~= SLI )
Useful if application is fill-rate limited
frame rate decreases with increasing number of pixels (i.e. when the image resolution increases)


Scalable graphics
Database decomposition
Useful if application is CPU limited
frame rate decreases with increasing size of 3D scenes


Scalable graphics
Stereoscopy


How many displays with one
computer ?


Why multi-display ?
Multiple display surfaces
Cave
Increase display size
Increase brightness
CPU/GPU Bottlenecks
Cluster


How ?
Matrox DualHead2Go
Matrox TripleHead2Go
Multiple graphics card in a single PC
Often max 2 graphics cards
Cyviz


Multiple graphics cards in
one PC
Multi-pipe ~= Multi-GPU
One rendering context per GPU
Problems of resources sharing, threading
Need speciﬁc graphics engine adaptations
But best performances


NVidia Quadro Plex
•Up to :
•4 GPU
•4 outputs
•16gb memory
•Can combine two Quadro Plex


Quadro Mosaic Mode
Mosaic mode hides
multi-gpu
Only shows one gpu
to system
Can have 4 outputs
with 1 plex without
needing multi-pipe
adaptation
Good performances


ATI EyeFinity


Increase size or brightness
Mersive autocalibration


What if need more displays ?

96 projectors ?


Cluster !


Cluster Synchronisation

Scene synchronisation (Scenelock)
Swap-Buffer synchronisation (Swaplock)
Eyes synchronisation (Genlock)


Cluster

+
Scalable
Each node can be cheap
Easy node replacement
-
Complex to manage/program


Tracking


Tracking
Mechanical

Fakespace’s Boom
+
Precise
Fast
Help Lift weight
-
Big, discontinued


Tracking
Magnetic

Polhemus
Ascension
+
Precise, fast
-
Wires
Sensitive to metal

Tracking
Inertial

SpacePoint Fusion
XSens Mocap
Accelerometers,
Gyroscopes, magnetometers
+
Very fast
Light
-
Drift
Orientation only

Tracking
Optical

ART
ioTracker
OptiTrack
TrackIR
+
Wireless
No electronics
-
Install cameras
Line of sight

Leap motion

https://www.youtube.com/watch?v=N6hCwjwzUHg

Tracking
Mixing Optical

Intersense IS-900
Inertial + Ultrasound
IS-1200
Inertial + AR Patterns
Wiimote
Playstation Move


Optical Body Tracking

4D Views

http://www.4dviews.com/


Kinect fusion

http://www.youtube.com/watch?v=quGhaggn3cQ


Gloves


Haptics


Haptics
Force feedback


TN Games - Gaming Vest


Locomotion


Locomotion

CirculaFloor
• http://www.youtube.com/watch?v=rYsvB2y2Ero

Virtusphere
• http://www.youtube.com/watch?v=qTnnJR-hS7k

OmniDirectionnal Treadmill
• http://www.youtube.com/watch?v=BQw1tsgrJOs

Kuka robot

http://www.cyberneum.de/f1/

Force Dynamics

http://www.force-dynamics.com/

Wizdish

http://www.wizdish.com

Applications


IVR Applications
Design, prototyping
Marketing
Training
Medical
Research
Geoscience, ﬁnance, archeologie, urbanism, sport, art...


iVR Applications
Design, prototyping

Car, aircraft, boat, tractors,
house appliances
Collaborative design review (c) Peugeot PSA
Experience from user’s
point of view
Easy and fast to prototype
More iterations, better
products
(c) Miele

PSA Peugeot-Citroen - http://www.youtube.com/watch?v=WhMh191RltQ


FORD

http://cb.nowan.net/blog/2010/10/12/more-about-fords-vr-facilities/

VR4D

http://vr4d.ingenierium.com/

Ergo-Wide

http://www.clarte.asso.fr/realite-virtuelle.php/en/ERGOWIDE%C2%B3/
https://www.youtube.com/watch?v=aczLiNtVyDI

Lockheed Martin

http://www.youtube.com/watch?v=itOxIpj68Mw

Mockup-builder

http://www.youtube.com/watch?v=ZxJD9DXDB1E

iVR Applications
Marketing

Product experience (c) University of Gronigen

Shopping experience
Procter&Gamble
Kimberley Clarke

(c) Virtools


iVR Applications
Training
Repeatability
Controlled environment
(c) SNCF
Can practice rare / dangerous situations
Can analyze / quantify performance
No health risk for patients
Easy accessibility
Reduced costs
Adaptable difﬁculty and assistance
Multiple users trained at same time
(c) University of Florida
Attractive

SNCF - Fiacre


SNCF - Simurat


Saint-Gobain & Clarte

Use less material
Environment friendly
No water, nothing to clean
Faster to setup

http://www.youtube.com/watch?v=FEnalzHVmdI

RE-lion

http://vimeo.com/25032061

ESCIN - Sivic - Crime scene investigation

http://www.youtube.com/watch?v=FC0gHv-Snmw

iVR Applications
Research

Heal
Exposure therapy
PTSD, Phobias
Rehabilitation :
motor, addictions
Create Empathy
http://www.youtube.com/watch?v=SinlgK5Q4v4
Pain Distraction Crescent - Acrophobia


Burn pain - Snow World

http://www.youtube.com/watch?v=jNIqyyypojg

Fear of ﬂight

Arte - “A chacun sa réalité”


Summary

IVR strengths
Controlled / adaptable / safe environment
Natural interactions / reactions
Analyze


iVR Games ?

Cave Quake 3 (c) Paul Rajlich

Cave Unreal Tournament
(c) Jeffrey Jacobson

https://www.youtube.com/watch?v=r580mzSxSp8


Immersive rail shooter

http://vimeo.com/9549613

VR Angry-bot

http://www.youtube.com/watch?v=QSXxjr1SnqA

Battleﬁeld 3 simulator

www.youtube.com/watch?v=nQR49JGySTM


3D User Interactions


VR Techniques
3D User Interactions

Bowman classiﬁcation for generic interactions
Selection
Specifying one or more objects from a set
Manipulation
Modifying object properties (position, orientation,
scale, shape, color, texture, behavior, etc.)
Navigation
Travel
Wayﬁnding
System Control
Menus
Issuing command
Changing mode
Choosing tool
Symbolic Input
Other interactions
Measuring
…?
http://www.3dui.org

Navidget

http://www.youtube.com/watch?v=wpaKjLZll80

http://cb.nowan.net/blog/2010/03/25/3dui-grand-prize-2010-the-winners/


World builder

http://www.youtube.com/watch?v=VzFpg271sm8

Perceptive illusions


Body representation in VR

Proteus effect : virtual appearance can transform our
behaviors towards other people ( Jeremy Bailenson )
It seems possible to transfer our body in virtual bodies


Rubber hand illusion

http://www.youtube.com/watch?v=TCQbygjG0RU

Body transfer

http://www.youtube.com/watch?v=8oF8sQvnTlM

Barbie doll illusion

http://www.youtube.com/watch?v=_z8pSTMfGSo

First person VR illusion

http://www.youtube.com/watch?v=DlbSVnVRLTU

Perceptive
Ilusions
Pseudo haptics

http://kaji-lab.jp/ja/index.php?research

http://www.irisa.fr/tactiles

Perceptive Ilusions
Pseudo haptics


Perceptive Illusions
Play with perception’s limits
Our perception is dominated by visual sense
Balance
Redirected Walking :

(c) Franck Steinicke

Change blindness :

(c) Evan Suma - USC/ICT

Institute of creative technologies - change blindess

https://www.youtube.com/watch?v=FTWVjJ4um4A

Perceptive Illusions
Passive haptics

(c) Luv Kohli

Passive haptics

Ford PVM



Learn about human perception through its limitations
Improve VR experience by taking advantage of those
limitations


Conclusion

It is possible to transfer your body in VR
Reality is not what you think
VR is just another reality


http://www.af-rv.fr

AFRV

http://www.youtube.com/watch?v=lHZltjDZBJo&feature=plcp


www.VRGeeks.org

http://www.vrgeeks.org/vr-experiences/vrescape

http://www.vrgeeks.org/vr-experiences/vrsadistic

http://www.vrgeeks.org/vr-experiences/insnaketion

VR Challenges
What we must solve
VR is mainly a SOFTWARE problem
“3DUI is a hard and unexplored software engineering (and ergonomic) problem”
Markerless tracking
Minimum latency
HMDs
Lightweight
Wide FOV
Wireless
Full body haptics
Multi-sensorial experience
Locomotion
Avatars

VR For All

VR “Experiences”

Your goal :
Create presence
And don’t break it !!!

Verdun 1916-Time machine
http://www.youtube.com/watch?v=ZBfYIefenp0

Conclusion


Thank you !
Questions ?

Sebastien.Kuntz@nowan.net


Immersive Virtual Reality

More Related Content

What's hot (20)

Similar to Immersive Virtual Reality (20)

More from Sebastien Kuntz (9)

Immersive Virtual Reality