Extended Reality for Virtual Social Learning
Download as PPTX, PDF3 likes273 views
The document outlines the research and presentations related to Extended Reality (XR) applications in education, user experience, and social learning, authored by Zi Siang See and collaborators. It covers various educational theories like VARK learning styles and experiential learning, alongside practical applications of virtual reality (VR) and augmented reality (AR) in different fields including medicine, collaboration, and inclusivity. The study emphasizes the importance of user experience, empathic computing, and the potential of massive open online courses (MOOCs) for AR/VR training while identifying obstacles and challenges in implementation.
Extended Reality for Virtual Social Learning
- 1. Extended Reality for Virtual Social Learning Zi Siang See SMARTlab, School of Creative Industries, University of Newcastle Email: zisiang.see@newcastle.edu.au Research: https://www.newcastle.edu.au/profile/zisiang-see FB: www.facebook.com/zisiang Linkedin: https://au.linkedin.com/in/zisiangsee Demo: www.zisiangsee/demo Keynote Workshop: Zi Siang See, Xia Sheng Lee (Bruce) OpenLearning 2018 November 2018 Presentation: Zi Siang See, Xia Sheng Lee The transition of Emerging Technologies for Education Research Xia Sheng Lee (Bruce) Yalong Intelligent Equipment Group Co.,LTD. Email: leexiasheng@gmail.com Linkedin: https://www.linkedin.com/in/brucexslee FB: https://www.facebook.com/bruce.l.sheng Research: https://scholar.google.com/citations?user=5OdbSv4AAAAJ&hl=en Research & collaboration
- 2. User Experience: Quiver Print, Color, Play, try it at http://www.quivervision.com Presentation: Zi Siang See et al.
- 3. Is User Experience Important to You? Presentation: Zi Siang See et al.
- 4. Pedagogy and Learning Approach Presentation: Zi Siang See et al.
- 5. Neil D. Fleming David A. Kolb Maria Montessori Albert Bandura Pedagogy and Learning Approach Presentation: Zi Siang See et al.
- 6. Pedagogy and Learning Approach Neil D. Fleming VARK Learning Style • Visual • Auditory • Reading • Kinesthetic David A. Kolb Experiential Learning • Experience • Observation • Conceptualization • Experimentation Maria Montessori Montessori Learning • Freedom • Mixed-age classrooms • Uninterrupted choice • Exploration Albert Bandura Social Learning • Observation • Imitation • Modeling Presentation: Zi Siang See et al.
- 7. Research Presentation: Zi Siang See, Lizbeth Goodman A UON Project led by: Donovan Jones, Shanna Fealy (University of Newcastle) School of Nursing & Midwifery, Faculty of Health & Medicine
- 9. Pedagogy and Learning Approach Neil D. Fleming VARK Learning Style • Visual • Auditory • Reading • Kinesthetic David A. Kolb Experiential Learning • Experience • Observation • Conceptualization • Experimentation Maria Montessori Montessori Learning • Freedom • Mixed-age classrooms • Uninterrupted choice • Exploration Albert Bandura Social Learning • Observation • Imitation • Modeling Presentation: Zi Siang See et al.
- 10. Is social learning = you are who you help? Presentation: Zi Siang See et al.
- 11. Researcher: Zi Siang See Academic Researcher (Interaction Design), University of Newcastle Zi Siang See: Industry and Academic Research Presentation: Zi Siang See
- 12. Research and Specialization Subject Matter Expertise • Creative Multimedia • Interaction Design • Virtual Reality (VR) • Augmented Reality (AR) • Inclusive Design Presentation: Zi Siang See
- 13. Digital Technology Management Specialization Formerly at University of Reading Xia Sheng Lee (Bruce): Industry and Academic Research Presentation: Zi Siang See et al.
- 14. Research and Specialization Subject Matter Expertise • Digital Technology Management • Virtual Design in Construction • Building Information Modeling • 3D Terrestrial Laser Scanning • Virtual Learning Presentation: Zi Siang See et al.
- 15. Preview Research Presentation: Zi Siang See, Lizbeth Goodman
- 16. Preview Research Presentation: Zi Siang See, Lizbeth Goodman
- 17. Introduction Multimedia (Combination of Elements) • Text • Image • Sound • Video • Animation Industries • Creative and advertising • Corporate communications visuals • Architectural imaging • Cultural heritage preservation • Scientific achieve Presentation: Zi Siang See et al.
- 18. Introduction Multimedia (Combination of Elements) • Text • Image • Sound • Video • Animation • Touch • Scent • Taste • Sight Presentation: Zi Siang See Industries • Creative and advertising • Corporate communications visuals • Architectural imaging • Cultural heritage preservation • Scientific achieve Development /Trend • Augmented and Virtual Reality • Human Computer Interaction
- 19. Presentation: Zi Siang See et al.
- 20. Figure: a vision about 2000 imagined in 1910, through 24 illustrations by Villemard. Presentation: Zi Siang See et al.
- 21. Figure: a vision about 2000 imagined in 1910, through 24 illustrations by Villemard. Presentation: Zi Siang See et al.
- 22. Presentation: Zi Siang See et al.
- 23. Desk /Table Pocket /Palm /Hand Head /Eye Ubiquitous Major changes in computing platforms Presentation: Zi Siang See et al.
- 24. Presentation: Zi Siang See
- 25. Virtual Reality Figure: NASA, 1988 Virtual Environment System Presentation: Zi Siang See et al.
- 26. Figure: Super cockpit project, Tom Furness. Presentation: Zi Siang See et al.
- 27. Near-eye-display and fully augmented interactive display experience How far can we go? Presentation: Zi Siang See et al.
- 28. Augmented Reality Presentation: Zi Siang See et al.
- 29. Augmented Reality: Glass Exploration in modern wearable interface • Fashionable, daily life, UX practicality? Presentation: Zi Siang See et al.
- 30. Google Glass • Google Glass, developer version, 2013 • Can be configured on user’s customized or personal spectacles. Presentation: Zi Siang See et al.
- 31. Recon Jet • Sporty design • Cost “friendly” • Smartphone compatible • External Connectivity Presentation: Zi Siang See et al.https://www.youtube.com/watch?v=BhBtuNe_8is
- 32. Presentation: Zi Siang See et al.
- 33. • Displays, tracers, and AR systems in general need to become more accurate, lighter, cheaper, and less power consuming (Azuma, 2001). Figure: wearable computing, Steve Mann. Presentation: Zi Siang See et al.
- 34. Ideas worth spreading: How can we develop extended reality (XR) content or systems for good purposes and inclusive? Presentation: Zi Siang See et al.
- 35. Mixed Reality brings effective collaborative outcome? • Effects of Sharing Physiological States of Players in a Collaborative Virtual Reality Gameplay. (Dey et al., 2017) • Digital Taste Interface project, Electronic taste stimulation (Ranasinghe et al., 2011) • Billinghurst, M. (2011). The Future of Augmented Reality in Our Everyday Life. Presentation: Zi Siang See et al.
- 36. Google Cardboard Presentation: Zi Siang See et al.
- 37. Google Cardboard Programmable NFC Tag Low cost resin /plastic lens Mobile device as computing module Endless configurable size and shapes (and materials) Magnetometer Presentation: Zi Siang See et al.
- 38. Google Cardboard v2 Presses the screen Presentation: Zi Siang See et al.
- 39. Presentation: Zi Siang See et al. User Experience = full senses, realism, contextualization https://www.youtube.com/watch?v=sfowhrpS-6M
- 40. User Experience Presentation: Zi Siang See et al.
- 41. What do I see? http://www.zisiangsee.com/vtour/vtour_hdr_sphericalpanorama/index.html Presentation: Zi Siang See et al.https://www.youtube.com/watch?v=hjKd24UCPYY
- 42. Well, in a Social Learning Way Do you (do we): • Do what you can’t? • Do what you can? • Lead others by example? • Follow others? • No idea… Presentation: Zi Siang See et al.
- 43. Oculus Rift Figure: Oculus, by Facebook Presentation: Zi Siang See et al.
- 44. Android VR Figure: Android VR collaborating with other developers Google has announced eight hardware partners that will make Daydream-ready phones, including Samsung, HTC, LG, Xiaomi, Huawei, ZTE, Asus, and Alcatel. Presentation: Zi Siang See et al.
- 45. Apple + Metaio? Metaio was one of the most technologically-ambitious AR organization, acquired by Apple in 2015 Presentation: Zi Siang See et al.
- 46. Apple’s Initiatives SteamVR Coming To Mac (unveiled in WWDC 2017), AR Kit will be available too. Presentation: Zi Siang See et al.
- 47. Steam VR Presentation: Zi Siang SeeVinay Narayan - HTC Vive (https://visionsummit2017.com)
- 48. Steam VR Presentation: Zi Siang SeeVinay Narayan - HTC Vive (https://visionsummit2017.com)
- 49. Definitions Virtual Reality • Virtual Reality (VR) which can be referred to as immersive multimedia or computer- simulated life, replicates an environment that simulates physical presence in places in the real world or imagined worlds and lets the user interact in that world. Virtual reality artificially creates sensory experiences, which can even include sight, hearing, touch, and smell. Augmented Reality • Augmented Reality (AR) mixes a live real-world view with virtual interactive content on a mobile or wearable device. One of the key enablers for this is tracking technology, such as computer vision techniques for tracking off pre-defined markers or markerless images. Presentation: Zi Siang See et al.
- 50. Milgram’s Reality-Virtuality continuum Mixed Reality Reality - Virtuality (RV) Continuum Real Environment Augmented Reality (AR) Augmented Virtuality (AV) Virtual Environment "...anywhere between the extrema of the virtuality continuum." P. Milgram and A. F. Kishino, Taxonomy of Mixed Reality Visual Displays, IEICE Transactions on Information and Systems, E77-D(12), pp. 1321-1329, 1994. Presentation: Zi Siang See et al.
- 51. AR + VR + MR = XR? Presentation: Zi Siang See et al.
- 52. Gartner’s Hype Cycle Presentation: Zi Siang See et al.
- 53. Gartner’s Hype Cycle Presentation: Zi Siang See et al.
- 54. Gartner’s Hype Cycle (Emerging Technologies) Presentation: Zi Siang See et al.
- 55. Gartner’s Hype Cycle (Emerging Technologies) Presentation: Zi Siang See et al.
- 56. Platforms VR AR Sourced and reinterpreted from http://www.augment.com/blog/which-headset-is-right-for-you/ Presentation: Zi Siang See et al.
- 57. Case study: How can we combine inclusive and interaction design? Presentation: Zi Siang See et al.
- 58. Hair Raising Message Presentation: Zi Siang See et al.https://www.youtube.com/watch?v=064ipuBiWDg
- 59. Practical consideration • Ubiquitous computing – the installation of hardware are invisible to users • Focus on user experience • Has strong storyline • Brings social good and awareness Presentation: Zi Siang See et al.
- 60. Penguin Navi Presentation: Zi Siang See et al.https://www.youtube.com/watch?v=IK4-zPD_25U
- 61. Practical consideration • Mix computing and entertainment • High level of usability design • Focus on bringing the intended outcome – going to the aquarium • Make use of existing mobile devices and capability • Practical, practical, practical! Presentation: Zi Siang See et al.
- 62. How One Dad Is Using VR To Help His Daughter https://uploadvr.com/vr-good-one-dad-williams-syndrome Figure: Craig and Imogen Chaytor's Battle With Williams Syndrome Presentation: Zi Siang See et al.https://www.youtube.com/watch?v=LOHvriN7Xk4
- 63. Empathy Glasses Katsutoshi Masai, Kai Kunze, Maki sugimoto, Mark Billinghurst (2016) Empathy Glasses. Proceedings of the 2016 CHI Conference Extended Abstracts on Human Factors in Computing Systems Presentation: Zi Siang See et al.https://www.youtube.com/watch?v=CdgWVDbMwp4
- 64. Empathy Glasses Katsutoshi Masai, Kai Kunze, Maki sugimoto, Mark Billinghurst (2016) Empathy Glasses. Proceedings of the 2016 CHI Conference Extended Abstracts on Human Factors in Computing Systems Presentation: Zi Siang See et al.https://www.youtube.com/watch?v=CdgWVDbMwp4
- 65. Empathy Glasses Katsutoshi Masai, Kai Kunze, Maki sugimoto, Mark Billinghurst (2016) Empathy Glasses. Proceedings of the 2016 CHI Conference Extended Abstracts on Human Factors in Computing Systems Presentation: Zi Siang See et al.https://www.youtube.com/watch?v=CdgWVDbMwp4
- 66. Practical consideration • Emphatic computing • Shared experience, emotion, same viewpoint • Real-time • “Put myself in their shoes” • Achieve and complete tasks together Presentation: Zi Siang See et al.
- 67. Mobile Audible AR Zi Siang See, Mark Billinghurst, Vimalan Rengganaten, Shelly Soo (2016) Medical Learning Murmurs Simulation with Mobile Audible Augmented Reality. Siggraph Asia 2016 Mobile Graphics and Interactive Application Presentation: Zi Siang See et al.https://www.youtube.com/watch?v=VGsaTDKC2qY
- 68. Virtual Reality in Medicine Leadning University of Basel, Switzerland https://www.unibas.ch/en/News-Events/News/Uni-Research/Virtual-Reality-in-Medicine.html Presentation: Zi Siang See et al.https://youtu.be/AttXbcLUyR0
- 69. Journey to Birth Presentation: Zi Siang See et al.https://www.youtube.com/watch?v=7s3dP8QIyjc
- 70. Practical consideration • A key initiative by University of Newcastle • Midwifery and health sciences education • Can be used for educating mothers • Reduce confusion and provides clear understanding • Fit to the idea of portability in mixed reality context • Designed by midwifes for midwifes • Multi-disciplinary research Presentation: Zi Siang See et al.
- 71. Work Smarter (Collaboration and Outcome-orientated) Figure: Elevator technicians are using AR to improve safety and workflow https://uploadvr.com/microsoft-hololens-elevator-technicians/ Presentation: Zi Siang See et al.
- 72. Presentation: Zi Siang See et al.
- 73. Presentation: Zi Siang See et al.
- 74. VRPhysio Presentation: Zi Siang See et al.https://www.youtube.com/watch?v=WmSyWb6djDU
- 75. VRPhysio • Customizable • Clinic can track and help patients remotely • Analytics Presentation: Zi Siang See et al.https://www.youtube.com/watch?v=WmSyWb6djDU
- 76. VRPhysio Presentation: Zi Siang See et al.https://www.youtube.com/watch?v=WmSyWb6djDU
- 77. 54% Want To Use VR/AR In Physical Stores • 85% -- Travel experiences • 84% -- Museums and galleries • 84% -- Education, training • 83% -- For designing something • 79% -- Live entertainment • 77% -- Exercise or sport • 75% -- Navigation • 74% -- Retail and shopping • 70% -- Business • 69% -- Meditation • 63% -- Fully immersive gaming • 53% -- Gaming through mobile • Comprised interviews of 16,000 consumers. • Consumer's perspective: the main reason for not having tried VR or AR is the lack of opportunity, based on a new study. Presentation: Zi Siang See et al.https://www.mediapost.com/publications/article/301863/54-want-to-use-vrar-in-physical-stores.html
- 78. Presentation: Zi Siang See et al.
- 79. Presentation: Zi Siang See et al.
- 80. Implication Further reading (by Prof Mark Billinghurst): https://www.slideshare.net/marknb00/comp-4026-lecture1-introduction • Designing for “Everyone” pleases no one (Mark and Gun’s Advanced HCI slide, 2018) • Is managing user experience important to your project? Presentation: Zi Siang See et al.
- 81. Research Possibilities Exploration • Method and apparatus for augmented reality • Method and apparatus for virtual reality • Ubiquitous computing • Obstacles and issues • Technology management • Education technology • Engineering and concept design • User interface (UI) design • User experience (UX) Presentation: Zi Siang See et al.
- 82. Can we use VR/AR technologies to help more people? Effort • Empathic computing • Inclusive AR and VR • Experience synchronization • Improve well-being • Solving conflicts? • Medical-based and healthcare benefits • Archeology, cultural heritage: see the whole story in 360 (digital preservation) • Sharing community and social stories • Combine interest of different professional disciplines Presentation: Zi Siang See et al.
- 83. MOOC? Zi Siang See SMARTlab, School of Creative Industries, University of Newcastle Email: zisiang.see@newcastle.edu.au Research: https://www.newcastle.edu.au/profile/zisiang-see FB: www.facebook.com/zisiang Linkedin: https://au.linkedin.com/in/zisiangsee Demo: www.zisiangsee/demo Xia Sheng Lee (Bruce) Yalong Intelligent Equipment Group Co.,LTD. Email: leexiasheng@gmail.com Linkedin: https://www.linkedin.com/in/brucexslee FB: https://www.facebook.com/bruce.l.sheng Research: https://scholar.google.com/citations?user=5OdbSv4AAAAJ&hl=en Research: Zi Siang See, Xia Sheng Lee, Adam Brimo, Harold Thwaites, Lizbeth Goodman
- 84. Focus of this work Focus of this Study • Case study of massive open online course (MOOC) for AR VR. • Obstacles, challenges and usability issues (based on the MOOCs we developed). • Investigation of how MOOC addresses best practices and the fundamental requirements of AR and VR training, and direction of future AR/VR-enabled MOOC. MOOC for AR VR Training: Obstacles, Challenges and Usability Zi Siang See, Xia Sheng Lee, Adam Brimo, Harold Thwaites, Lizbeth Goodman (2018) MOOC for AR VR Training: Obstacles, Challenges and Usability. IEEE Games, Entertainment & Media Conference, Galway, Ireland 15th-17th August 2018 Research & collaboration Research: Zi Siang See, Xia Sheng Lee, Adam Brimo, Harold Thwaites, Lizbeth Goodman
- 85. Our MOOCS AR VR The Promise of Sci-Fi • http://www.openlearning.com/sunway-university BIM101 • https://www.openlearning.com/courses/revit101basicbim BIM101 • https://www.openlearning.com/courses/revit101basicbim Research: Zi Siang See, Xia Sheng Lee, Adam Brimo, Harold Thwaites, Lizbeth Goodman
- 86. Lizbeth Goodman Professor of Inclusive Design SMARTlab, University College Dublin Zi Siang See MOOC: AR VR The Promise of Sci-fi SMARTlab, University of Newcastle Xia Sheng Lee MOOC: BIM101, BIM202 University of Reading Adam Brimo Founder & CEO OpenLearning.com Harold Thwaites Professor, Digital Media CRCDM, Sunway University Research: Zi Siang See, Xia Sheng Lee, Adam Brimo, Harold Thwaites, Lizbeth Goodman
- 87. • MOOC brings a number of benefits (Chauhan et al., 2015). With right design/workflow of MOOC, instructors can support the learning process rather than take control of it. Background Contribution & intention - to provide an investigation on how some of fundamental criteria of AR and VR can be catered in MOOC setting. Research: Zi Siang See, Xia Sheng Lee, Adam Brimo, Harold Thwaites, Lizbeth Goodman
- 88. Augmented Reality • Handheld AR display • AR Browser (HTML5 + WebVR) • Unity Vuforia SDK • Visual Programming • Virtual Reality • Mobile VR display • Samsung Gear VR + phone • Unity SteamVR SDK • Visual Programming • Building Information Modelling • Authoring Development • Revit • May link to AR + VR Technologies involved in MOOC(s) Research: Zi Siang See, Xia Sheng Lee, Adam Brimo, Harold Thwaites, Lizbeth Goodman
- 89. Case Study and Approach • Classroom is vital for hands-on scenarios, however it may not be convenient for everyone • MOOC is accessible 24/7, but it requires effort (students & instructor) to solve non- standardized technical questions. (e.g. AR/VR, BIM, other tech-based development pace) • Unique characteristics of MOOC includes the option of paid or free certification, on- demand customized content dissimilar to conventional university syllabus. • Question: formal education vs social way of learning? Research: Zi Siang See, Xia Sheng Lee, Adam Brimo, Harold Thwaites, Lizbeth Goodman
- 90. Basic Goals • Image-based AR tracking • Way Finding Experience • Authoring Conventional Multimedia • Interaction and GUI High-end Goals • 3D object tracking • Markerless Tracking (e.g. SLAM) • Authoring AR-optimized multimedia • Customizing mechanics in AR Augmented Reality: Recommended Aspects to Cover in MOOC Research: Zi Siang See, Xia Sheng Lee, Adam Brimo, Harold Thwaites, Lizbeth Goodman
- 91. Basic Goals • Image-based AR tracking • Way Finding Experience • Authoring Conventional Multimedia • Interaction and GUI Augmented Reality: Recommended Aspects to Cover in MOOC Image-based tracking allows AR content to be overlayed on top of a 2D surface. This can be a magazine cover, book or a billboard. Some portal-based AR service providers provide features of overlaying text, image, video etc. Research: Zi Siang See, Xia Sheng Lee, Adam Brimo, Harold Thwaites, Lizbeth Goodman
- 92. Basic Goals • Image-based AR tracking • Way Finding Experience • Authoring Conventional Multimedia • Interaction and GUI Augmented Reality: Recommended Aspects to Cover in MOOC Way-finding AR uses sensors (GPS, compass, gyro, wifi, mixed-mode) tracking approaches for including enhanced location-based experience. Such scenarios can be seen in the popular Pokemon Go, Shopping AR Alert, Museum AR. Research: Zi Siang See, Xia Sheng Lee, Adam Brimo, Harold Thwaites, Lizbeth Goodman
- 93. Basic Goals • Image-based AR tracking • Way Finding Experience • Authoring Conventional Multimedia • Interaction and GUI Augmented Reality: Recommended Aspects to Cover in MOOC Production processes: Image and video editing, creative design process, sound montage, color management, file compressions, web-based content development, server hosting. Research: Zi Siang See, Xia Sheng Lee, Adam Brimo, Harold Thwaites, Lizbeth Goodman
- 94. Basic Goals • Image-based AR tracking • Way Finding Experience • Authoring Conventional Multimedia • Interaction and GUI Augmented Reality: Recommended Aspects to Cover in MOOC Hand gesture input is essential in mobile-based AR experience. It allows simple basic button to be pushed (for linking social media, Youtube) or scaling or rotating an object. Research: Zi Siang See, Xia Sheng Lee, Adam Brimo, Harold Thwaites, Lizbeth Goodman
- 95. Basic Goals • Image-based AR tracking • Way Finding Experience • Authoring Conventional Multimedia • Interaction and GUI High-end Goals • 3D object tracking • Markerless Tracking (e.g. SLAM) • Authoring AR-optimized multimedia • Customizing mechanics in AR Augmented Reality: Recommended Aspects to Cover in MOOC So far, we have no major problems introducing basic goals in MOOC Research: Zi Siang See, Xia Sheng Lee, Adam Brimo, Harold Thwaites, Lizbeth Goodman
- 96. Basic Goals • Image-based AR tracking • Way Finding Experience • Authoring Conventional Multimedia • Interaction and GUI High-end Goals • 3D object tracking • Markerless Tracking (e.g. SLAM) • Authoring AR-optimized multimedia • Customizing mechanics in AR Augmented Reality: Recommended Aspects to Cover in MOOC What about these high-end goals which may have special requirements? Research: Zi Siang See, Xia Sheng Lee, Adam Brimo, Harold Thwaites, Lizbeth Goodman
- 97. Augmented Reality: Recommended Aspects to Cover in MOOC Ideally, tracking real-world object is an essential elements for AR experience. However, it has been difficult to implement in actual project. High-end Goals • 3D object tracking • Markerless Tracking (e.g. SLAM) • Authoring AR-optimized multimedia • Customizing mechanics in AR Research: Zi Siang See, Xia Sheng Lee, Adam Brimo, Harold Thwaites, Lizbeth Goodman
- 98. Augmented Reality: Recommended Aspects to Cover in MOOC Markerless tracking is vital in the current or next stage or AR experience. However, it is only functionable in high-end mobile devices. Lower range devices may suffer from lack of computing power for rendering such content and environment tracking (Android ARCore vs Apple ARKIT) High-end Goals • 3D object tracking • Markerless Tracking (e.g. SLAM) • Authoring AR-optimized multimedia • Customizing mechanics in AR Research: Zi Siang See, Xia Sheng Lee, Adam Brimo, Harold Thwaites, Lizbeth Goodman
- 99. Augmented Reality: Recommended Aspects to Cover in MOOC FACT – currently there are no standardized ways to develop AR-enabled content. Versions of authoring platforms keep evolving (Unity 5 vs Unity 2018), and Android and Apple has very different development SDK requirements. High-end Goals • 3D object tracking • Markerless Tracking (e.g. SLAM) • Authoring AR-optimized multimedia • Customizing mechanics in AR Research: Zi Siang See, Xia Sheng Lee, Adam Brimo, Harold Thwaites, Lizbeth Goodman
- 100. Augmented Reality: Recommended Aspects to Cover in MOOC Some AR features require customized hardware and software aspects. These emerging media experience may only be suitable for resourceful organizations or creative projects. High-end Goals • 3D object tracking • Markerless Tracking (e.g. SLAM) • Authoring AR-optimized multimedia • Customizing mechanics in AR Research: Zi Siang See, Xia Sheng Lee, Adam Brimo, Harold Thwaites, Lizbeth Goodman
- 101. Q: can we still introduce high-end goals of AR authoring in a MOOC? Research: Zi Siang See, Xia Sheng Lee, Adam Brimo, Harold Thwaites, Lizbeth Goodman
- 102. Basic Goals • Environment simulation • VR360 (Spherical Panorama) • Locomotion, interactions • Authoring Conventional Multimedia High-end Goals • Presence • Hands or Avatar (self realisation) • Ambisonic audio • Authoring hyper-realistic VR scenes Virtual Reality: Recommended Aspects to Cover in MOOC Research: Zi Siang See, Xia Sheng Lee, Adam Brimo, Harold Thwaites, Lizbeth Goodman
- 103. Basic Goals • Environment simulation • VR360 (Spherical Panorama) • Locomotion, interactions • Authoring Conventional Multimedia Virtual Reality: Recommended Aspects to Cover in MOOC It is a visual programming creation process. Sometimes, also known as level design from the game development perspective, it involves a number of skills and technical knowledge during the creation process (3D modeling, photogrammetry, shader, texture, BIM) Research: Zi Siang See, Xia Sheng Lee, Adam Brimo, Harold Thwaites, Lizbeth Goodman
- 104. Basic Goals • Environment simulation • VR360 (Spherical Panorama) • Locomotion, interactions • Authoring Conventional Multimedia Virtual Reality: Recommended Aspects to Cover in MOOC The idea being expressed during the early 19 centuries of a panorama painter’s goal was to reproduce the real world so realistically that the onlookers could believe that what they were seeing was real (Benosman and Kang, 2001). Creating spherical panorama and HDR are core areas in computational photography. Research: Zi Siang See, Xia Sheng Lee, Adam Brimo, Harold Thwaites, Lizbeth Goodman
- 105. Basic Goals • Environment simulation • VR360 (Spherical Panorama) • Locomotion, interactions • Authoring Conventional Multimedia Virtual Reality: Recommended Aspects to Cover in MOOC Lomotion, means how users are allowed to “walked around” in virtual reality (3dof or 6dof). SteamVR allows users to have room-scale locomotion experience, this is an evolving aspect in VR tech development. Interactions such as working with objects in virtual scenes are essential. Research: Zi Siang See, Xia Sheng Lee, Adam Brimo, Harold Thwaites, Lizbeth Goodman
- 106. Basic Goals • Environment simulation • VR360 (Spherical Panorama) • Locomotion, interactions • Authoring Conventional Multimedia Virtual Reality: Recommended Aspects to Cover in MOOC Work with text, image, animation, video and sound. These process may be alright for multimedia-ready students or professionals, however it involves a range of know-how technical understanding for creating creative multimedia content. Research: Zi Siang See, Xia Sheng Lee, Adam Brimo, Harold Thwaites, Lizbeth Goodman
- 107. Basic Goals • Environment simulation • VR360 (Spherical Panorama) • Locomotion, interactions • Authoring Conventional Multimedia High-end Goals • Presence • Hands or Avatar (self realisation) • Ambisonic audio • Authoring hyper-realistic VR scenes Virtual Reality: Recommended Aspects to Cover in MOOC Some students find it difficult to learn these in MOOC vs classroom Research: Zi Siang See, Xia Sheng Lee, Adam Brimo, Harold Thwaites, Lizbeth Goodman
- 108. Basic Goals • Environment simulation • VR360 (Spherical Panorama) • Locomotion, interactions • Authoring Conventional Multimedia High-end Goals • Presence • Hands or Avatar (self realisation) • Ambisonic audio • Authoring hyper-realistic VR scenes Virtual Reality: Recommended Aspects to Cover in MOOC It may require specialized hardware and software, and non-standardized knowledge Research: Zi Siang See, Xia Sheng Lee, Adam Brimo, Harold Thwaites, Lizbeth Goodman
- 109. High-end Goals • Presence • Hands or Avatar (self realisation) • Ambisonic audio • Authoring hyper-realistic VR scenes Virtual Reality: Recommended Aspects to Cover in MOOC This is top priority in VR experience development. However, it can be extremely challenging to port in various real-world elements into VR world. One of the current and important topics in HCI is “extended reality (XR)”. Research: Zi Siang See, Xia Sheng Lee, Adam Brimo, Harold Thwaites, Lizbeth Goodman
- 110. High-end Goals • Presence • Hands or Avatar (self realisation) • Ambisonic audio • Authoring hyper-realistic VR scenes Virtual Reality: Recommended Aspects to Cover in MOOC This is top priority in VR (and AR) experience development. However, it can be extremely challenging to port in various real-world elements into VR world. One of the current and important topics in HCI is “extended reality (XR)”. Research: Zi Siang See, Xia Sheng Lee, Adam Brimo, Harold Thwaites, Lizbeth Goodman
- 111. High-end Goals • Presence • Hands or Avatar (self realisation) • Ambisonic audio • Authoring hyper-realistic VR scenes Virtual Reality: Recommended Aspects to Cover in MOOC Improving sense of presence involves a number of elements and this is a key area of research discussion, Hands VR and Avatar for self-realisation are some of the features developers could introduce in VR experience. However, some of these are challenging elements to be included in MOOC for VR training, in the situation of limited hardware access by students. Q: introduce shared learning facility? Research: Zi Siang See, Xia Sheng Lee, Adam Brimo, Harold Thwaites, Lizbeth Goodman
- 112. High-end Goals • Presence • Hands or Avatar (self realisation) • Ambisonic audio • Authoring hyper-realistic VR scenes Virtual Reality: Recommended Aspects to Cover in MOOC Ambisonic audio, is a specialized area in VR360 video or VR content development. Unity 2018 supports ambisonic audio decoder, but may only work with specialized settings. Note: we need Matthew Sansom and team to discuss about this technical area. Research: Zi Siang See, Xia Sheng Lee, Adam Brimo, Harold Thwaites, Lizbeth Goodman
- 113. High-end Goals • Presence • Hands or Avatar (self realisation) • Ambisonic audio • Authoring hyper-realistic VR scenes Virtual Reality: Recommended Aspects to Cover in MOOC One of the key areas in VR. This implies bringing in almost any real-world elements we could find, and attempt to implement it into virtual environment. This includes managing lighting, shadow, plants, terrain, water, wind, texture, dynamic, physics. Research: Zi Siang See, Xia Sheng Lee, Adam Brimo, Harold Thwaites, Lizbeth Goodman
- 114. Q: How can we mix AR and VR in a creative project? Research: Zi Siang See, Xia Sheng Lee, Adam Brimo, Harold Thwaites, Lizbeth Goodman
- 115. • Sustainable learning group. • Resembles the behaviors of learners similar to casual interaction within social media platforms. • For now, it was found that a number of users are unable to make positive progress in AR VR MOOC due to a number of reasons. • Can the problem of lack of hardware/software be solved? • Can learners and instructor easily solve technical problems in MOOC? Interactions of learners in 3 months Discussion Research: Zi Siang See, Xia Sheng Lee, Adam Brimo, Harold Thwaites, Lizbeth Goodman
- 116. • Collaborative learning • A near-linear and stable learning community if the subject is still valid. • New and old learners. • Can MOOC platform evolve to support native AR/VR functions? • Can MOOC of AR, VR, MR, VDC, BIM subjects being introduced as “non- conventional special content? • Will MOOC platforms support and produce “Netflix original” content? Interactions of learners in 5 years, VDC/BIM subject, involved about 5000 learners. Discussion Research: Zi Siang See, Xia Sheng Lee, Adam Brimo, Harold Thwaites, Lizbeth Goodman
- 117. • Livelihood of supporting learning community that is sustainable and scalable in MOOC platform. • Example: this figure demonstrates graphically a MOOC’s global diversity of learner profiles from 152 countries, involving 4903 learners. • In the future, the case study will also continue to facilitate and experiment with hybrid facilities such as 3D scanning and VR avatar. • We are contactable and reachable for INCLUSIVE collaborations. Diversity of students, VDC/BIM subject, involved about 5000 learners. Implication Research: Zi Siang See, Xia Sheng Lee, Adam Brimo, Harold Thwaites, Lizbeth Goodman
- 118. Questions in Mind for Future Studies • How can we make MOOC with AR and VR inclusive (for everyone)? • How can we introduce future learning center or facility for aiding AR/VR-enabled MOOC? Research: Zi Siang See, Xia Sheng Lee, Adam Brimo, Harold Thwaites, Lizbeth Goodman
- 119. AR VR Demo Zi Siang See SMARTlab, School of Creative Industries, University of Newcastle Email: zisiang.see@newcastle.edu.au Research: https://www.newcastle.edu.au/profile/zisiang-see FB: www.facebook.com/zisiang Linkedin: https://au.linkedin.com/in/zisiangsee Demo: www.zisiangsee/demo Xia Sheng Lee (Bruce) Yalong Intelligent Equipment Group Co.,LTD. Email: leexiasheng@gmail.com Linkedin: https://www.linkedin.com/in/brucexslee FB: https://www.facebook.com/bruce.l.sheng Research: https://scholar.google.com/citations?user=5OdbSv4AAAAJ&hl=en Research: Zi Siang See, Xia Sheng Lee, Adam Brimo, Harold Thwaites, Lizbeth Goodman
- 120. Demo (work-in-progress): Inclusive VR Museum VR • Electronic Cultural Atlas Initiative (ECAI) • Towards inclusive user experience and digital preservation of cultural artifacts Presentation: Zi Siang See et al.
- 121. Demo (work-in-progress): Inclusive VR Museum VR • Inclusive contributors (artists, creators, more) • Customization Presentation: Zi Siang See et al.
- 122. Preview Research Presentation: Zi Siang See, Lizbeth Goodman
- 123. Demo (work-in-progress): Inclusive VR Museum VR • Co-design with local contributors (Newcastle Library, Hunter Water) • Brings value to community, engagement, long-term involvement Research Presentation: Zi Siang See, Lizbeth Goodman
- 124. Preview Research Presentation: Zi Siang See, Lizbeth Goodman
- 125. Demo: Museum VR: The Tomb of Sultan • 3D Scanned /Photogrammetry of artifacts • User experience –revisiting an ancient site where the king rested. Research Presentation: Zi Siang See, Lizbeth Goodman
- 126. Demo: Museum VR: The Tomb of Sultan • Interaction: Can we see the hands? And can we gain some control? Physics? Presentation: Zi Siang See et al.
- 127. Presentation: Zi Siang SeePresentation: Zi Siang See et al.
- 128. Preview Research Presentation: Zi Siang See, Lizbeth Goodman
- 129. Presentation: Zi Siang See Creative Possibilities?
- 130. Example Setting • Mobile Devices • HMD compatibility • Cost-effective • Transmedia • Creative Branding • Public access Zi Siang See, Mohd Shahrizal Sunar, Mark Billinghurst, Arindam Dey, Delas Santano, Human Esmaeili, Harold Thwaites (2017) An Augmented Reality and Virtual Reality Pillar for Exhibitions: A Subjective Exploration. International Conference on Artificial Reality and Telexistence & Eurographics Symposium on Virtual Environments. Research Presentation: Zi Siang See
- 131. Will OpenLearning be inclusive and XR-enabled? Researchers: Zi Siang See, Xia Sheng Lee
- 132. Ideas for AR and VR? Start building yours :) Researchers: Zi Siang See, Xia Sheng Lee
- 133. Thank you / Discussion Researchers: Zi Siang See, Xia Sheng Lee Zi Siang See SMARTlab, School of Creative Industries, University of Newcastle Email: zisiang.see@newcastle.edu.au Research: https://www.newcastle.edu.au/profile/zisiang-see FB: www.facebook.com/zisiang Linkedin: https://au.linkedin.com/in/zisiangsee Demo: www.zisiangsee/demo Xia Sheng Lee (Bruce) Yalong Intelligent Equipment Group Co.,LTD. Email: leexiasheng@gmail.com Linkedin: https://www.linkedin.com/in/brucexslee FB: https://www.facebook.com/bruce.l.sheng Research: https://scholar.google.com/citations?user=5OdbSv4AAAAJ&hl=en