Bridging the Gap from Knowledge to Action: Putting Analytics in the Hands of Academic Advisors
2 likes846 views
The document discusses a project aimed at utilizing data from campus learning management systems to enhance the effectiveness of academic mentoring for historically underrepresented engineering students at the University of Michigan. It highlights the development of metrics and visualization tools to provide relevant performance data to mentors, ultimately shortening the intervention time for students in need. The findings emphasize the role of early warning systems in improving student outcomes through timely communication and support strategies by academic mentors.
Bridging the Gap from Knowledge to Action: Putting Analytics in the Hands of Academic Advisors
- 1. Bridging the Gap from Knowledge to Action: Putting Analytics in the Hands of Academic Advisors Steven Lonn Andrew Krumm R. Joseph Waddington Stephanie Teasley USE Lab University of Michigan Digital Media Commons www.umich.edu/~uselab
- 2. Research Setting: M-STEM Academy • Undergraduate engineering mentoring program • Historically underrepresented students • 200 Engineering students in 4 cohorts USE Lab Digital Media Commons 2
- 3. Goals of the project • Utilize data stored in campus learning management system to: • Provide timely and targeted data on student performance to M-STEM mentors • Shorten the timespan from problem identification to intervention USE Lab Digital Media Commons 3
- 4. Supporting M-STEM mentors • Iteratively develop • Metrics for comparing students using LMS data • Classification schemes • Visualizations of student performances • Send mentors weekly updates Photo%Credit:%h,p://teacherrogers.wordpress.com USE Lab Digital Media Commons 4
- 5. How does mentor’s use of EWS affect student outcomes? USE Lab Digital Media Commons 5
- 6. Improved Performance Action academic resources, Face-to-Face / Email Communication study strategies Mentor Audience Student EWS Product Classification Analysis Data 6
- 7. Improved Performance Action academic resources, Face-to-Face / Email Communication study strategies Mentor Audience Student EWS Product Classification Analysis Data 6
- 8. Assignments Gradebook Weekly%query%of%LMS%for%courses% that%include%an%MASTEM%student%and% use%the%Gradebook%or%Assignments% “Presence” tool 8
- 9. Measures from LMS data • Gradebook and Assignments tools allow up-to-date tracking of student performances • Report student-level information for M-STEM students • Percent of available points earned • Course averages (all students) USE Lab Digital Media Commons 9
- 10. Measures from LMS data • “Presence” events serve as a proxy for effort and are events common to all courses • Cumulative and week-to-week “Presence” USE Lab Digital Media Commons 10
- 11. 75% mean 25% USE Lab Digital Media Commons 11
- 12. Cumulative “Presence” events can be highly predictive for students’ final 75% course grade performance mean 25% USE Lab Digital Media Commons 11
- 13. USE Lab Digital Media Commons 12
- 14. Classification scheme • Absolute grade thresholds • Difference from course average • Presence cutoff USE Lab Digital Media Commons 12
- 15. Classification scheme • Absolute grade thresholds • Difference from course average • Presence cutoff Comparisons • Grades (course average) • Percentile Ranks (presence) USE Lab Digital Media Commons 12
- 16. Classification scheme Student % Relative Distance Presence Percentile Rank E3 >=0.85 . . Encourage 0.75<=X<0.85 <-0.15 . Explore 0.75<=X<0.85 >=-0.15 <0.25 Explore 0.75<=X<0.85 >=-0.15 >=0.25 Encourage 0.65<=X<0.75 <-0.15 <0.25 Engage 0.65<=X<0.75 <-0.15 >=0.25 Explore 0.65<=X<0.75 >=-0.15 . Explore 0.55<=X<0.65 >=-0.10 . Explore 0.55<=X<0.65 <-0.10 . Engage <0.55 . . Engage USE Lab Digital Media Commons 13
- 17. Mentor summary USE Lab Digital Media Commons 14
- 18. Student Detail Report USE Lab Digital Media Commons 15
- 19. Benefits of EWS use • Contacting students • Shortening time to intervention • Viewing longitudinal trends • By individual course • Across all courses • Contextualizing M-STEM student performance USE Lab Digital Media Commons 16
- 20. How mentors use the EWS USE Lab Digital Media Commons 17
- 21. Next Steps • New infrastructure • New versions • Instructor • Students • Messaging system • Recommendations (from person, from system) USE Lab Digital Media Commons 18
- 22. Improved Performance Action academic resources, Face-to-Face / Email Communication study strategies Mentor Audience Student EWS Product Classification Analysis USE Lab Digital Media Commons Data 19
- 23. Conclusion • Closing the gap between problem identification and intervention • Organizational capacity and the success of learning analytics • “Analytics” is but a small part • Information is always subject to interpretation • How can we scaffold interpretation and effective action-taking? USE Lab Digital Media Commons 20
- 24. Collaborators M-STEM ITS • Cinda-Sue Davis • Bryan Hartman • Guy Meadows • Jeff Jenkins • James Holloway • Dan Kiskis • Daryl Koch • Mark Jones USE Lab • Debbie Taylor • Gierad Laput USE Lab Digital Media Commons 21
- 25. Questions Steve slonn@umich.edu @stevelonn Stephanie steasley@umich.edu @stephteasley www.umich.edu/~uselab slides: www.slideshare.net/stevelonn USE Lab Digital Media Commons 22