Multimodal Machines #JTELSS17 workshop
Download as PPTX, PDF1 like2,855 views
The workshop held in Aveiro, Portugal, focused on multimodality and its application in learning, particularly for robots and computers. Participants engaged in group assignments to create projects that illustrate multimodal concepts, such as sensory inputs and emotion sensors in educational environments. Notable experiments and research on multimodal learning analytics were discussed, showcasing the importance of this approach in enhancing learning outcomes.
Multimodal Machines #JTELSS17 workshop
- 1. Workshop @ JTELSS17 12th October 2017 – Aveiro, Portugal Multimodal Machines Daniele DI MITRI & Jan SCHNEIDER
- 2. What is multimodality? Pagina 2 Group assignment: divide into groups and find a picture which best describe the concepts of Multimodality.
- 3. What is multimodality? Pagina 3 https://www.youtube.com/watch?v=rn9V0cN4NWs https://www.youtube.com/watch?v=3kEHg7cYlP0
- 4. Pagina 4 How does multimodality work in learning?
- 5. Multimodality for robots Pagina 5 Group assignment: equip your robot with multimodal capabilities
- 6. They encode messages Pagina 6 Multimodality for computers e.g. 3D models in AR They decode sensor inputs e.g modern AR headsets like Hololenes
- 7. Observability Line INPUT SPACE OUTPUT SPACE Observable dimensions, can be tracked with sensors Unobservable dimensions, require human interpretation assessment Di Mitri, D., Drachsler, H., Specht, M. (2017) From signals to knowledge. A conceptual model for multimodal learning analytics. In press.
- 10. Put-that-there Pagina 10 Bolt, R. A. (1980). “Put-that-there”: Voice and gesture at the graphics interface (Vol. 14, No. 3, pp. 262-270). ACM.ISO 690 https://www.youtube.com/watch?v=RyBEUyEtxQo
- 11. Emotion sensors go to school Pagina 11 Arroyo, I., Cooper, D. G., Burleson, W., Woolf, B. P., Muldner, K., & Christopherson, R. (2009, July). Emotion Sensors Go To School. In AIED (Vol. 200, pp. 17-24)
- 12. Multimodal action based assessment Pagina 12 Worsley, M., & Blikstein, P. (2013, April). Towards the development of multimodal action based assessment. In Proceedings of the third international conference on learning analytics and knowledge (pp. 94-101). ACM.ISO 690
- 13. Analytics meet Patient Manikins Pagina 13 Martinez-Maldonado, R., Power, T., Hayes, C., Abdipranoto, A., Vo, T., Axisa, C., and Buckingham-Shum, S. (2017) Analytics Meet Patient Manikins: Challenges in an Authentic Small-Group Healthcare Simulation Classroom. International Conference on Learning Analytics and Knowledge, LAK 2017, 90-94.
- 15. Learning Pulse – are you in the Flow? Pagina 15 (Csikszentmihalyi, 1972) Di Mitri, D., Scheffel, M., Drachsler, H., Börner, D., Ternier, S., & Specht, M. (2017). Learning Pulse: a machine learning approach for predicting performance in self-regulated learning using multimodal data.ISO 690
- 16. Presentation Trainer Pagina 16 Schneider, J., Börner, D., Van Rosmalen, P., & Specht, M. (2015, November). Presentation trainer, your public speaking multimodal coach. In Proceedings of the 2015 ACM on International Conference on Multimodal Interaction (pp. 539-546). acm.ISO 690 https://www.youtube.com/watch?v=ElB6OvbL8fA
- 19. Q&A Thanks for listening! Daniele Di Mitri & Jan Schneider
Editor's Notes
- #3: 5 minutes