Visualizing Student Feedback
0 likes761 views
This presentation discusses using a mixed quantitative and qualitative approach to visualizing student feedback data. It argues that while quantitative feedback via ratings can identify strong and weak areas, simply reporting average ratings is insufficient. The presentation proposes using a heatmap visualization of course ratings that shows the distribution of scores and how each course compares to the average. This type of visual overview is easier for staff to understand and use for curriculum planning compared to tables of numbers. The approach was inspired by techniques used in bioinformatics and initial student research found the visual format took less time to analyze.
Visualizing Student Feedback
- 1. Visualizing Student Feedback: a case for mixed quant-qual approach Margus Niitsoo, Kaspar Kruup University of Tartu
- 2. This talk ● Quantitative feedback – Why? ● Visualization? What for? ● Feedback for curriculum level?
- 3. My perspective ● as a student – graduated just 2 years ago ● as a lecturer – 5 years teaching experience ● as the curriculum manager – past 2 years ● as an engineer – MSc and PhD in Computer Science
- 4. Qualitative feedback ● “Stricter deadlines would have disciplined us to study more!”
- 5. Qualitative feedback ● “Stricter deadlines would have disciplined us to study more!” ● “It was hard to hear the lecturer”
- 6. Qualitative feedback ● “Stricter deadlines would have disciplined us to study more!” ● “It was hard to hear the lecturer” ● “The course sucked!”
- 7. Qualitative feedback “The lecturer was one of the best I have seen in my studies” vs “The lecturer was hard to follow, monotonous, inept and boring” Which is (more) true?
- 8. Quantitative feedback The averaged student rating for the course was: 4.2
- 9. Quantitative feedback The averaged student rating for the course was: 4.2 / 5
- 10. Quantitative feedback The averaged student rating for the course was: 4.2 / 5 1 2 3 4 5
- 11. Quantitative feedback The averaged student rating for the course was: 4.2 / 5 1 2 3 4 5 Respondents: 14/100
- 13. University of Amsterdam All in all, I give this class the following grade: (1=very bad, 5=excellent)
- 14. An often unanswerable question Does the score of 4.2 mean that I am above average?
- 15. Add the averages (UT): Institute avg | Faculty avg | Uni. avg
- 16. Next problem Does the score of 4.2 mean that I am above average? By how much?
- 17. Easy! The institute average was 4.0 with SD 0.3
- 18. Easy? The average for courses was 4.0 with SD 0.3 What (the hell) does SD mean?
- 19. Standard deviation Course avg: 4.2 Avg. for courses: 4.0 SD for courses: 0.3
- 20. Standard deviation Course avg: 4.2 Avg. for courses: 4.0 SD for courses: 0.3 Quick calculation: (4.2-4.0)/0.3 = 0.66
- 21. Standard deviation Course avg: 4.2 Avg. for courses: 4.0 SD for courses: 0.3 Quick calculation: (4.2-4.0)/0.3 = 0.66 Translation: Better than ~70%
- 22. We are bad with numbers! ● Analysis of numbers is slow ● It is hard (requires effort) ● We are prone to mistakes!
- 23. We are bad with numbers! ● Analysis of numbers is slow ● It is hard (requires effort) ● We are prone to mistakes! Also: a picture is worth a thousand words! (We can represent more with less)
- 24. The UT solution
- 25. The UT solution (University of Amsterdam)
- 26. The UT solution
- 27. The UT solution
- 28. The UT solution
- 29. The UT solution
- 30. Pros ● Good quick overview – With details also available, if needed ● Better information – Ranking and distribution implicit ● Emphasizes important – Comparison with mean and other courses ● Increased interest in feedback – More visually attractive and accessible than numbers
- 31. Some research: ● On-line survey among students – 107 respondents – 4 tasks of feedback analysis + opinion survey – 3 groups for each task ● Tables with numbers, Simplified graph, Full graph ● Main findings – Graphs took less time to process in analysis tasks – Most preferred graphs over numbers (but some had a strong preference for tables with numbers so both needed) – Workload question result often misinterpreted (since the “best” response for that question is 0 instead of 2) (this is joint work with Kaspar)
- 32. Curriculum management ● Course planning – What to teach when – Who teaches what ● Quality assurance – Identify bright and problem spots – Spread good practices – Aid lecturers with problem spots
- 33. Good overview essential! ● Course planning – What to teach when – Who teaches what ● Quality assurance – Identify bright and problem spots – Spread good practices – Aid lecturers with problem spots
- 35. Problems (as before) ● Absolute values – Is 3.2 good or bad? ● Just one value – Leads to oversimplification ● Mostly numbers – Hard to get an overview
- 36. Proposed solution (Inspired by heatmaps used in Bioinformatics)
- 37. Summary ● Quantitative feedback has a use – Finding the bright spots and problem areas ● Presentation of data is important – Although sadly this is often neglected
- 38. Further work ● Qualitative research into the perceptions of the academic staff of the new system – Currently in progress with a larger team ● Good review of systems currently used throughout the world – Does your uni. have something better?
- 39. Thank you! Any questions and comments would be welcome! (If you have any later, try Margus.Niitsoo@ut.ee)