Research Directions in Transitional Interfaces
1 like1,435 views
The document discusses research directions in transitional interfaces between real and digital worlds, particularly in mixed and cross-reality environments. It highlights the importance of seamless transitions along the virtuality continuum and collaboration techniques for augmented and virtual realities. Key research questions focus on user presence, awareness, interaction techniques, usability, and potential applications in collaborative settings.
Research Directions in Transitional Interfaces
- 1. RESEARCH DIRECTIONS IN TRANSITIONAL INTERFACES Mark Billinghurst mark.billinghurst@unisa.edu.au ACM ISS 2021 Workshop on Transitional Interfaces in Mixed and Cross-Reality: A new frontier? November 14th 2021
- 5. Computer Interfaces • Separation between real and digital worlds • WIMP (Windows, Icons, Menus, Pointer) metaphor
- 6. Rekimoto, J. and Nagao, K. 1995. The world through the computer: computer augmented interaction with real world environments. Making Interfaces Invisible (c) Internet of Things
- 7. Internet of Things (IoT).. • Embed computing and sensing in real world • Smart objects, sensors, etc.. (c) Internet of Things
- 8. Virtual Reality (VR) • Users immersed in Computer Generated environment • HMD, gloves, 3D graphics, body tracking
- 9. Augmented Reality (AR) • Virtual Images blended with the real world • See-through HMD, handheld display, viewpoint tracking, etc..
- 10. From Reality to Virtual Reality Internet of Things Augmented Reality Virtual Reality Real World Virtual World
- 11. Milgram’s Mixed Reality (MR) Continuum Augmented Reality Virtual Reality Real World Virtual World Mixed Reality "...anywhere between the extrema of the virtuality continuum." P. Milgram and A. F. Kishino, (1994) A Taxonomy of Mixed Reality Visual Displays Internet of Things
- 12. Milgram’s Mixed Reality (MR) Continuum Augmented Reality Virtual Reality Real World Virtual World Mixed Reality Internet of Things
- 13. Transitions (2001) • Interfaces need to support transitions along the RV continuum • Augmented Reality is preferred for: • augmenting real objects • co-located collaboration • Immersive Virtual Reality is preferred for: • experiencing world immersively • sharing views, remote collaboration
- 14. The MagicBook (2000) Reality Virtuality Augmented Reality (AR) Augmented Virtuality (AV)
- 16. Technology • Reality • No technology • Augmented Reality • Camera – tracking • Switch – fly in • Virtual Reality • Compass – tracking • Pressure pad – move • Switch – fly out
- 17. Features • Seamless transition from Reality to Virtuality • Reliance on real decreases as virtual increases • Supports egocentric and exocentric views • User can pick appropriate view • Independent Views • Privacy, role division, scalability • Collaboration on multiple levels: • Physical Object, AR Object, Immersive Virtual Space • Egocentric + exocentric collaboration • multiple multi-scale users
- 18. The MagicBook • Billinghurst, M., Kato, H., & Poupyrev, I. (2001). The MagicBook: a transitional AR interface. Computers & Graphics, 25(5), 745-753.
- 19. Mobile AR Transitional Interfaces • Using transition from Ego-centric to Exo-centric to increase awareness Mulloni, A., Dünser, A., & Schmalstieg, D. (2010, September). Zooming interfaces for augmented reality browsers. In Proceedings of the 12th international conference on Human computer interaction with mobile devices and services (pp. 161-170).
- 21. A Conceptual Model • Theoretical framework for understanding transitional interfaces Grasset, R., Mulloni, A., Billinghurst, M., & Schmalstieg, D. (2011). Navigation techniques in augmented and mixed reality: Crossing the virtuality continuum. In Handbook of Augmented Reality (pp. 379-407). Springer, New York, NY.
- 22. Applying Conceptual Model to MagicBook
- 23. Important Research Questions • Presence • How does user’s sense of Presence change? • Awareness • How does the user perceive the switch between contexts? • Interaction Techniques • What are the best interaction techniques for transitional interfaces? • Usability • How can we measure the usability of transitional interfaces? • Collaboration • How can transitional interfaces be used to improve collaboration?
- 24. Example: AR Lens for Transitional Interfaces Looser, J. C. A. (2007). AR magic lenses: Addressing the challenge of focus and context in augmented reality. • AR Lens • Create illusion that user is holding real lens • Support lens-based viewing/selection
- 25. AR Lens Demo
- 26. Example: Multi-Scale Collaboration Changing the user’s virtual body scale Piumsomboon, T., Lee, G. A., Irlitti, A., Ens, B., Thomas, B. H., & Billinghurst, M. (2019, May). On the shoulder of the giant: A multi-scale mixed reality collaboration with 360 video sharing and tangible interaction. In Proceedings of the 2019 CHI conference on human factors in computing systems (pp. 1-17).
- 28. Sharing: Separating Cues from Body • What happens when you can’t see your colleague/agent? Piumsomboon, T., Lee, G. A., Hart, J. D., Ens, B., Lindeman, R. W., Thomas, B. H., & Billinghurst, M. (2018, April). Mini-me: An adaptive avatar for mixed reality remote collaboration. In Proceedings of the 2018 CHI conference on human factors in computing systems (pp. 1-13). Collaborating Collaborator out of View
- 29. Mini-Me Communication Cues in MR • When lose sight of collaborator a Mini-Me avatar appears • Miniature avatar in real world • Mini-Me points to shared objects, show communication cues • Redirected gaze, gestures
- 31. User Study (16 participants) • Collaboration between user in AR, expert in VR • Hololens, HTC Vive • Two tasks: • (1) asymmetric, (2) symmetric • Key findings • Mini-Me significantly improved performance time (task1) (20% faster) • Mini-Me significantly improved Social Presence scores • 63% (task 2) – 75% (task 1) of users preferred Mini-Me “I feel like I am talking to my partner”
- 32. Jun Rekimoto, Sony CSL Invisible Interfaces
- 33. Invisible Interfaces Jun Rekimoto, Sony CSL
- 34. Invisible Interfaces Jun Rekimoto, Sony CSL
- 35. Example: Visualizing Sensor Networks • Network of Humidity Sensors • ZigBee wireless communication • Use Mobile AR to Visualize Humidity Rauhala, M., Gunnarsson, A. S., & Henrysson, A. (2006, September). A novel interface to sensor networks using handheld augmented reality. In Proceedings of the 8th conference on Human- computer interaction with mobile devices and services (pp. 145-148).
- 36. Invisible Interfaces Jun Rekimoto, Sony CSL
- 37. UbiVR – CAMAR CAMAR Companion CAMAR Viewer CAMAR Controller GIST - Korea
- 38. The Metaverse • Neal Stephenson’s “SnowCrash” • The Metaverse is the convergence of: • 1) virtually enhanced physical reality • 2) physically persistent virtual space • Metaverse Roadmap • http://metaverseroadmap.org/
- 39. Metaverse Dimensions • Augmentation technologies that layer information onto our perception of the physical environment. • Simulation refers to technologies that model reality • Intimate technologies are focused inwardly, on the identity and actions of the individual or object; • External technologies are focused outwardly, towards the world at large;
- 40. Metaverse Components • Four Key Components • Virtual Worlds • Augmented Reality • Mirror Worlds • Lifelogging
- 41. M ixed R eality
- 43. Possible Research Directions • Lifelogging to VR • Bringing real world actions into VR, VR to experience lifelogging data • AR to Lifelogging • Using AR to view lifelogging data in everyday life, Sharing physiological data • Mirror Worlds to VR • VR copy of the real world, Mirroring real world collaboration in VR • AR to Mirror Worlds • Visualizing the past in place, Asymmetric collaboration • And more..
- 44. Conclusions • Transitional interfaces • Research crossing boundaries • AR/VR Transitions along the MR continuum • Seamless transitions • Ego-centric and exo-centric views • Collaboration on multiple levels • Many directions for future research • Crossing boundaries, Metaverse