Everywhere but Here: how eXtended Reality is Changing The Ways We Communicate Online and Offline
- 1. Everywhere but Here: How eXtended Reality is Changing the Ways We Communicate Online and Offline Bektur Ryskeldiev, PhD Digital Nature Group, University of Tsukuba Tokyo, December 2019
- 2. Here’s my country Here’s Japan
- 3. I have been living in Japan for almost 7 years 5 of which I spent here And now I live here
- 4. Living in a remote location makes you very interested in remote communication technologies
- 6. How can we bridge the gap between hardware/software and interfaces we use to access them?
- 7. There are many ways to make your communication and interaction experience more immersive, you can involve: ● Stationary and mobile displays for mixed or virtual environments (e.g., holograms, AR/VR spaces) ● Objects around you (“smart” speakers, lights, projectors, drones and robots) ● Objects ON you, aka “wearables” (exoskeletons, haptic bracelets/vests, electric muscle stimulation devices)
- 8. Microsoft Research, “Holoportation” Microsoft Research, “IllumiRoom” P. Lopes et al, using EMS for MR interactions J. Rekimoto et al, ChameleonMask (aka “Human Uber”)
- 9. Terminology overview ● AR/VR/XR – Mixed Reality & eXtended Reality ● Telepresence & telexistence ● Wearables and haptics
- 10. eXtended Reality (XR) or MR + VR Source:https://en.wikipedia.org/wiki/Projection_augmented_model
- 11. How much system knows about the world around it (e.g. nothing vs full 3D mapping) Display type / resolution Whether the system creates a sense of presence (being somewhere) P. Milgram and F. Kishino, A taxonomy of mixed reality visual displays
- 12. Telepresence & Telexistence ● Telepresence – creating a sense of being somewhere remote (introduced by Marvin Minsky in 1980) ● Telexistence – creating a sense of existing somewhere remote (introduced by Susumu Tachi in 1980) ● Both terms are often being used interchangeably Images:SusumuTachi/VICEnews
- 13. In MR applications, wearable displays are usually employed as haptic displays that can create a sense of touch for an illusionary object via: ● vibration, ● changes in temperature, ● electric muscle stimulation (EMS), ● changes in weight/size, etc. TouchVRP.Lopesetal.,EMSforMR
- 14. So how exactly do these technologies change our online and offline communication?
- 15. Expanding visual and interactive space Problem: Single-viewpoint video streaming provides limited visible and interaction space Solution: Use 360˚ photo/video spheres to create a shared mixed reality space -> ->
- 16. Introducing immersive 360 and/or 3D content into remote communication makes collaborative experience more efficient B. Ryskeldiev, et al., StreamSpace: Pervasive mixed reality telepresence for remote collaboration on mobile devices -> ->
- 17. Creating a shared context and a more natural way of interaction Facebook SpacesS. Kim, et al., The Nomad and Couch Potato
- 18. Creating social mixed reality spaces with shared context for a better sense of being together (co-presence) Sharing mixed reality spaces anonymously via blockchain B. Ryskeldiev et al., ReactSpace & B. Ryskeldiev, et al, Distributed Metaverse
- 19. World mapping / AR Cloud How much system knows about the world around it (e.g. nothing vs full 3D mapping) Magic Leap Facebook
- 20. World mapping and view sharing evolution Mark Billinghurst, “Augmented Reality and Artificial Intelligence” http://empathiccomputing.org/
- 21. Privacy / AI + AR Group Anonymous, Wearable Face Projector Microsoft Research, Cardiolens Tencent, Native in-video advertising
- 22. Jun Rekimoto, Homo Cyberneticus S. Kasahara, J. Nishida, and P. Lopes, Preemptive Action Y. Saraiji et al., MetaLimbs Stelarc, Ping Body
- 23. Life imitating art imitating life Bladerunner 2049 Altered Carbon Serial Experiments Lain I. Sutherland, Sword of Damocles C. Ellingham, Memories of Tsukiji A Touring Machine, S. Feiner & B. McIntyre
- 24. Thank you for attention! You can find me at: Twitter: @parhelion318 | @rbektour | @ysrgbt Web: apolotary.com ysrgbt.work digitalnature.slis.tsukuba.ac.jp