![Methodology.
• design-based research [Obrenović 2011]: design a tool, give
students & teachers and evaluate
• iterative process of design & evaluation
• paper mockups
• usability studies
• surveys on
usefulness
http://funnytogo.com/pictures/evolution/timeline.htm](https://image.slidesharecdn.com/learning-analytics-dashboards-for-learning-analytics-day-at-kuleuven-140522093640-phpapp02/85/Learning-Analytics-Dashboards-49-320.jpg)
Learning Analytics Dashboards
- 1. Learning Analytics dashboards Sten Govaerts EPFL, Zwitserland slides partially based on “Information Dashboard” by Katrien Verbert
- 2. about me
- 3. KULeuven team: Human-Computer Interaction ! •technology enhanced learning •music •research EPFL team: REACT - Real-time Coordination & Distributed Interaction Systems Group ! •technology enhanced learning •social media •transportation http://hci.cs.kuleuven.be/ http://react.epfl.ch/
- 4. •2008: context-aware music information retrieval •2009 •2010 •2011 •2012 •2013 •2014
- 5. •2008: context-aware music information retrieval •2009 •2010 •2011 •2012 •2013 •2014 information visualisation}
- 6. •2008: context-aware music information retrieval •2009 •2010 •2011 •2012 •2013 •2014 social recommendations } information visualisation}
- 7. •2008: context-aware music information retrieval •2009 •2010 •2011 •2012 •2013 •2014 social recommendations } information visualisation} learning analytics }
- 8. •2008: context-aware music information retrieval •2009 •2010 •2011 •2012 •2013 •2014 social recommendations } information visualisation} learning analytics }inquiry learning with online labs }
- 9. PhDstudent! @KULeuven •2008: context-aware music information retrieval •2009 •2010 •2011 •2012 •2013 •2014 social recommendations } information visualisation} learning analytics }inquiry learning with online labs }
- 10. PhDstudent! @KULeuven Postdoc @EPFL •2008: context-aware music information retrieval •2009 •2010 •2011 •2012 •2013 •2014 social recommendations } information visualisation} learning analytics }inquiry learning with online labs }
- 11. about you…
- 12. why are you interested in dashboards?
- 13. what I planned…
- 14. what I planned… what are dashboards?
- 15. what I planned… what are dashboards? dashboard examples
- 16. what I planned… what are dashboards? what is wrong with most dashboards? dashboard examples
- 17. what I planned… what are dashboards? what is wrong with most dashboards? how can you design better dashboards? dashboard examples
- 18. what I planned… what are dashboards? what is wrong with most dashboards? how can you design better dashboards? dashboard examples learning analytics dashboard examples
- 19. what I planned… what are dashboards? what is wrong with most dashboards? how can you design better dashboards? dashboard examples how can you quickly get started making your own dashboards? learning analytics dashboard examples
- 26. What is an information dashboard? A dashboard is a visual display of the most important information needed to achieve one or more objectives; consolidated and arranged on a single screen so the information can be monitored at a glance. - Stephen Few Stephen Few, "Dashboard Confusion," Intelligent Enterprise, March 20, 2004.
- 27. Learning Analytics! dashboards (& visualisations ;-)
- 29. GISMO http://www.noe-kaleidoscope.org/pub/researcher/resources/video/aied/mazza.html
- 31. Backstage Pohl A, Bry F, Schwarz J, Gottstein M (2012) Sensing the classroom: improving awareness and self-awareness of students with backstage. In: Proceedings of the international conference on interactive and collaborative learning (ICL), Villach, Austria, 26–28 September 2012.
- 32. Dollár A, Steif PS (2012) Web-based statics course with learning dashboard for instructors. In: Uskov V (ed) Proceedings of computers and advanced technology in education (CATE 2012), June 25–27, 2012, Napoli, Italy OLI Dashboard
- 33. LARA Learning Analytics Reflection & Awareness environment
- 36. also on tabletop & large displays
- 37. J. L. Santos, K.Verbert, S. Govaerts, and E. Duval. Visualizing PLE Usage. Proceedings of EFEPLE11: 1st Workshop on Exploring the Fitness and Evolvability of Personal Learning Environments, CEUR Laclusaz, France, 30-31 March 2011
- 38. Student Activity Meter Govaerts, Sten;Verbert, Katrien, Duval, Erik; Pardo,Abelardo, The Student Activity Meter for Awareness and Self-reflection. http://ariadne.cs.kuleuven.be/monitorwidget-chi
- 41. Objectives • self-monitoring for learners http://free-extras.com/images/obama_hope_progress-2848.htm
- 42. Objectives • self-monitoring for learners • awareness for teachers http://free-extras.com/images/obama_hope_progress-2848.htm
- 43. Objectives • self-monitoring for learners • awareness for teachers • time tracking http://free-extras.com/images/obama_hope_progress-2848.htm
- 44. Objectives • self-monitoring for learners • awareness for teachers • time tracking • learning resource use and recommendations http://free-extras.com/images/obama_hope_progress-2848.htm
- 49. Methodology. • design-based research [Obrenović 2011]: design a tool, give students & teachers and evaluate • iterative process of design & evaluation • paper mockups • usability studies • surveys on usefulness http://funnytogo.com/pictures/evolution/timeline.htm
- 50. iter. evaluation goal methodology demo-graphics data I. usability, satisfaction, preliminary usefulness task-based interview with think-aloud, SUS & MSDT 12 CS students at KULeuven timetracking viaTwitter II. assessing teacher needs, use & usefulness online survey with Likert items, yes/no and open questions 19 teachers & TA’s with CGIAR courses Moodle logs III. assessing teacher needs, expert feedback, use & usefulness online survey with Likert items, yes/no and open questions 12 participants from LAK MOOC Moodle logs IV. use, usefulness & satisfaction structured face-to-face interview with tasks & open questions 11 teachers & TA’s at Carlos III Uni. Madrid VM with extensive tool tracking
- 51. Conclusion • SAM is perceived useful and contributes to better awareness.
- 52. Conclusion • SAM is perceived useful and contributes to better awareness. • SAM enables:
- 53. Conclusion • SAM is perceived useful and contributes to better awareness. • SAM enables: • to detect potential students doing well and at risk
- 54. Conclusion • SAM is perceived useful and contributes to better awareness. • SAM enables: • to detect potential students doing well and at risk • a better course overview
- 55. Conclusion • SAM is perceived useful and contributes to better awareness. • SAM enables: • to detect potential students doing well and at risk • a better course overview • understanding student time spending
- 56. Conclusion • SAM is perceived useful and contributes to better awareness. • SAM enables: • to detect potential students doing well and at risk • a better course overview • understanding student time spending • to find a wide variety of new insights
- 57. Conclusion • SAM is perceived useful and contributes to better awareness. • SAM enables: • to detect potential students doing well and at risk • a better course overview • understanding student time spending • to find a wide variety of new insights • almost all want to continue using SAM
- 67. STRUCTURE • explain main idea • science edu -> to labs -> lab fire -> need something safer • objectives golab • partner list • react’s job • portal • architecture
- 69. An overview of where students are Currently there are 4 students in the orientation phase, 8 in the conceptualization phase, 3 in the investigation phase and one discussing. Student currently in phase 4 8 3 0 1
- 70. A detailed view of where each student is Tommy is in the discussion phase, Becky in the investigation phase, etc. Student current phase Tommy Becky Jessi Alex
- 71. An overview of the time spent in each space for all students There are currently students in the conclusion and the discussion phases, the average time spent in the discussion phase so far is 20 seconds and the average time spent in the investigation phase is 15:30 minutes Average time per phase 2:50 3:01 15:30 4:20 0:20
- 72. The number of time students change phases Alex has been changing 12 times between phases so far, Jessi only 5 times. Number of phase changes Tommy Becky Jessi Alex 4 8 5 12
- 73. An overview of who handed in their assignment Tommy, Jessi and Alex have already handed in their assignments. Student reports Tommy Becky Jessi Alex report.pdf Jessi.pdf Pat pat.pdf
- 74. Student reports Tommy Becky Jessi Alex report.pdf Jessi.pdf Pat pat.pdf Number of phase changes Tommy Becky Jessi Alex 4 8 5 12 Connected Students 11 Connected Students 4 Connected Stud 6 Student current phase Tommy Becky Jessi Alex
- 79. inadequate context for the data.
- 80. inadequate context for the data.
- 93. bad encoding of data
- 94. not highlighting important data.
- 96. too much colour.
- 97. dashboards require thoughtful design…
- 98. how to design better dashboards?
- 99. how to design better dashboards? 1. reduce the non-data pixels
- 100. how to design better dashboards? 1. reduce the non-data pixels 2. enhance the data pixels
- 101. how to design better dashboards? 1. reduce the non-data pixels 2. enhance the data pixels 3. design for usability
- 102. how to design better dashboards? 1. reduce the non-data pixels 2. enhance the data pixels 3. design for usability
- 108. Enhance data pixels
- 109. Enhance data pixels
- 111. Design for usability. 1. organize information to support its meaning & use
- 112. Design for usability. 1. organize information to support its meaning & use 2. make it aesthetically pleasing!
- 113. Design for usability. 1. organize information to support its meaning & use 2. make it aesthetically pleasing! 3. design for a launch pad
- 114. Design for usability. 1. organize information to support its meaning & use 2. make it aesthetically pleasing! 3. design for a launch pad
- 115. Design for usability. 1. organize information to support its meaning & use 2. make it aesthetically pleasing! 3. design for a launch pad
- 116. How to evaluate? usability System Usability Scale (SUS)
- 117. How to evaluate? usability System Usability Scale (SUS)
- 119. How to evaluate? usability interviews: task list think aloud+
- 120. How to evaluate? usability interviews: task list think aloud+ observe+
- 121. How to evaluate? usability interviews: task list think aloud+ observe+ understand what the user does understand what the issues are +
- 123. How to evaluate? usefulness&impact much HARDER…
- 124. How to evaluate? usefulness&impact interviews much HARDER…
- 125. How to evaluate? usefulness&impact interviews much HARDER… perceived usefulness surveys
- 126. How to evaluate? usefulness&impact interviews much HARDER… perceived usefulness surveys longitudinal studies
- 127. How to quickly build your own dashboard?
- 128. How to quickly build your own dashboard? get data
- 129. How to quickly build your own dashboard? get data (clean data)
- 130. How to quickly build your own dashboard? get data (clean data) visualise!
- 131. get data import.io
- 133. Organic Chemistry Organic chemistry Organical Chemestry OrganicChemistry Organ Chemistry (clean data)
- 134. Organic Chemistry Organic chemistry Organical Chemestry OrganicChemistry Organ Chemistry (clean data) Google Refine/OpenRefine http://openrefine.org/
- 135. Organic Chemistry Organic chemistry Organical Chemestry OrganicChemistry Organ Chemistry (clean data) Google Refine/OpenRefine http://openrefine.org/ DataWrangler http://vis.stanford.edu/wrangler/
- 143. there are also all-in- one solutions…
- 144. How many kids are repeating a year in elementary school?
- 145. • http://www.wolframalpha.com/input/?i=how+many+kids +are+repeating+primary+school+in+Belgium • http://www.wolframalpha.com/input/?i=university+with+the +largest+enrollment&lk=3 • http://www.wolframalpha.com/input/?i=university+with +the+largest+enrollment+in+Belgium • http://www.wolframalpha.com/input/?i=Belgium+vs+USA +education+expenditures+per+high+school+student
- 146. further reading… http://pdf.th7.cn/down/files/1312/information_dashboard_design.pdf
- 147. Edward Tufte.