Deciphering the MOOC ecosystem: what makes successful MOOCs
1 like601 views
(1) The document analyzes data from MOOCs offered by the University of British Columbia and the University of Edinburgh to identify factors that contribute to MOOC success. (2) It finds that course location influences enrollment, with more local students participating, and that different courses have unique characteristics in terms of video length and structure that impact student engagement and retention. (3) Overall longer total video lengths per week are negatively correlated with student retention, but the length of individual videos does not show the same relationship.
Deciphering the MOOC ecosystem: what makes successful MOOCs
- 1. DECIPHERING THE MASSIVE OPEN ONLINE COURSES (MOOCS) ECOSYSTEM: WHAT MAKES SUCCESSFUL MOOCS? Ido Roll, Will Engle, Simon Bates! @hummus_monster @infology @simonpbates @CTLT_UBC! Centre for Teaching, Learning, and Technology! University of British Columbia 1
- 2. @hummus_monster @simonpbates @UBC_CTLT #STLHE2014 Please raise your hand if you have pre-teen child(ren) at home.! Please keep it up if you know where the following sentence is taken from:! “The snow glows white on the mountain tonight” 2
- 3. @hummus_monster @simonpbates @UBC_CTLT #STLHE2014 LEARNING FROM VIDEO ! IS POSSIBLE 3 If someone wants to learn, you cannot stop them.! @eric_mazur #STLHE2014
- 4. @hummus_monster @simonpbates @UBC_CTLT #STLHE2014 WE KNOW A LOT ABOUT UNIVERSITY TEACHING What works?! Small classrooms! Interactivity! With material! With peers! With instructors! Feedback! … 4 MOOCs
- 5. @hummus_monster @simonpbates @UBC_CTLT #STLHE2014 THREE KEY QUESTIONS 5 Who are ! our learners? What defines ! success? What course design supports that?
- 6. @hummus_monster @simonpbates @UBC_CTLT #STLHE2014 CONTEXT Four UBC courses (Spring/Summer 2013)! Climate Literacy; Useful Genetics; Program Design; Game Theory! 330,000 learners registered. ! Five Edinburgh courses (Winter/Spring 2013)! Introduction to Philosophy; Artificial Intelligence; Astrobiology; Nutrition; Critical Thinking.! 309,000 learners registered.! https://www.era.lib.ed.ac.uk/handle/1842/6683! 6
- 7. @hummus_monster @simonpbates @UBC_CTLT #STLHE2014 VIDEO PRODUCTION 7 Averagelengthpervideo(min) 0 6 12 18 24 Average # of videos per week 0 4 8 12 16 UBC 1 UBC 2 UBC 3 UBC 4
- 8. @hummus_monster @simonpbates @UBC_CTLT #STLHE2014 THREE KEY QUESTIONS 8 Who are our learners? What defines ! success? What course design supports that?
- 9. @hummus_monster @simonpbates @UBC_CTLT #STLHE2014 WHO ARE OUR LEARNERS? General demographics consistent across courses and with Coursera. 9 Education level %responses 0 25 50 75 100 UBC 1 UBC 2 UBC 3 Coursera High School Diploma Professional School Some College Bachelor’s Degree Master’s Degree Doctorate Degree
- 10. @hummus_monster @simonpbates @UBC_CTLT #STLHE2014 WHO ARE OUR LEARNERS? General demographics consistent across courses and with Coursera. 10 Ethnicity %responses 0 25 50 75 100 UBC 1 UBC 2 UBC 3 Coursera White or Caucasian South Asian East Asian Other Asian Black or African American Decline to State
- 11. @hummus_monster @simonpbates @UBC_CTLT #STLHE2014 ROLE OF GEOGRAPHY Does location matter? Are MOOCs “local”? 11 Country of Origin %responses 0 15 30 45 60 UBC overall Coursera Edinburgh 11 3.6 3.9 United States India Canada United Kingdom Spain Brazil Germany
- 12. @hummus_monster @simonpbates @UBC_CTLT #STLHE2014 ROLE OF GEOGRAPHY Does location matter? Are MOOCs “local”? 12 Country of Origin %responses 0 15 30 45 60 UBC overall Coursera Edinburgh 43.9 5.2 United States India Canada United Kingdom Spain Brazil Germany
- 13. @hummus_monster @simonpbates @UBC_CTLT #STLHE2014 ROLE OF GEOGRAPHY Break down by course. weighting by overall registration, population of country:! How many more students are local, per 10,000 registrants, per capita?! In Canada, ! ~150 additional local students! ~500 Canadians overall 13 Extralocallearnersper10,000 registrants,1,000,000residents 0 2.5 5 7.5 10 UBC Edinburgh 2.0 1.0 55.7 3.3 5.5 4.8
- 14. @hummus_monster @simonpbates @UBC_CTLT #STLHE2014 THREE KEY QUESTIONS 14 Who are ! our learners? What defines success? What course design support that?
- 15. @hummus_monster @simonpbates @UBC_CTLT #STLHE2014 WHAT DEFINES SUCCESS? Definitions:! Visitors: log in at least once. Our denominator. 15 Activities Videos Website Week 0 1 2 3 4 5 6 7 8
- 16. @hummus_monster @simonpbates @UBC_CTLT #STLHE2014 METRICS OF SUCCESS Definitions:! Visitors: log in at least once. Our denominator.! Tasters: watch a video on Week 2. 16 Activities Videos Website Week 0 1 2 3 4 5 6 7 8
- 17. @hummus_monster @simonpbates @UBC_CTLT #STLHE2014 METRICS OF SUCCESS Definitions:! Visitors: log in at least once. Our denominator.! Tasters: watch a video on Week 2.! Learners: watch videos all the way through. 17 Activities Videos Website Week 0 1 2 3 4 5 6 7 8
- 18. @hummus_monster @simonpbates @UBC_CTLT #STLHE2014 METRICS OF SUCCESS Definitions:! Visitors: log in at least once. Our denominator.! Tasters: watch a video on Week 2.! Learners: watch videos all the way through. ! Active learners: problem sets, discussion forums. 18 Activities Videos Website Week 0 1 2 3 4 5 6 7 8
- 19. @hummus_monster @simonpbates @UBC_CTLT #STLHE2014 METRICS OF SUCCESS Definitions:! Visitors: log in at least once. Our denominator.! Tasters: watch a video on Week 2.! Learners: watch videos all the way through. ! Active learners: problem sets, discussion forums. 19 Activities Videos Website Week 0 1 2 3 4 5 6 7 8
- 20. @hummus_monster @simonpbates @UBC_CTLT #STLHE2014 METRICS OF SUCCESS Metrics:! Attractiveness: convertVisitors to Tasters.! Retention: convert Tasters to Learners! Interactivity: convert Learners to Active Learners. 20 Activities Videos Website Week 0 1 2 3 4 5 6 7 8 Visitors Tasters Learners Active Learners
- 21. @hummus_monster @simonpbates @UBC_CTLT #STLHE2014 METRICS OF SUCCESS 21 0 0.15 0.3 0.45 0.6 Attractiveness Retention Interactivity 48% 54% 35% 50% 31%29% 52% 58% 36% 35% 57% 24% Activities Videos Website Week 0 1 2 3 4 5 6 7 8 Visitors Tasters Learners Active Learners 17k 13k 49k 85k 2k 3k 4k 16k
- 22. @hummus_monster @simonpbates @UBC_CTLT #STLHE2014 THREE KEY QUESTIONS 22 Who are ! our learners? What defines ! success? What course design support that?
- 23. @hummus_monster @simonpbates @UBC_CTLT #STLHE2014 WHAT AFFECTS CONVERSION RATES? Correlate with:! Average length per video! Average length per week 23 Activities Videos Website Week 0 1 2 3 4 5 6 7 8 Visitors Tasters Learners Active Learners
- 24. @hummus_monster @simonpbates @UBC_CTLT #STLHE2014 ATTRACTIVENESS Visitors -> Tasters: How many of those who visited the website watched Week 2 videos?! By length per week: r = -0.72*, r² = 0.52! 24 %ofVisitorswho arealsoTasters 0% 20% 40% 60% Overall legth of videos per week (min) 0 45 90 135 180 UBC Edinburgh
- 25. @hummus_monster @simonpbates @UBC_CTLT #STLHE2014 ATTRACTIVENESS Visitors -> Tasters: How many of those who visited the website watched Week 2 videos?! By length per video: r = -0.33! 25 %ofVisitorswho arealsoTasters 0% 20% 40% 60% Average length per video 5 8 11 14 17 UBC Edinburgh
- 26. @hummus_monster @simonpbates @UBC_CTLT #STLHE2014 RETENTION Tasters -> Learners: How many of those who watched @ Week 2 also watched @ Week 5?! By length per week: r = -0.29! 26 %ofTasterswho arealsoLearners 0% 26.667% 53.333% 80% Overall legth of videos per week (min) 0 45 90 135 180 UBC Edinburgh
- 27. @hummus_monster @simonpbates @UBC_CTLT #STLHE2014 RETENTION Tasters -> Learners: How many of those who watched @ Week 2 also watched @ Week 5?! By length per video: r = 0.55, r² = 0.30 27 %ofTasterswho arealsoLearners 0% 20% 40% 60% 80% Average length per video 5 8 11 14 17 UBC Edinburgh
- 28. @hummus_monster @simonpbates @UBC_CTLT #STLHE2014 INTERACTIVITY How many of the students who watch videos also submit problem sets? 28 %ofvideowatchers whosubmitted problemsets 20% 30% 40% 50% 60% 70% Week 0 2 4 6 8 UBC 1 UBC 2 UBC 3 UBC 4
- 29. @hummus_monster @simonpbates @UBC_CTLT #STLHE2014 SUMMARY Local matters! Courses have unique “signatures”! Structure, e.g., # and length of videos! Engagement, e.g., % active learning.! Overall length of videos is negatively correlated with retention! But length of individual videos does not show such relationship. ! True for both universities 29
- 30. @hummus_monster @simonpbates @UBC_CTLT #STLHE2014 DESIGN RECOMMENDATIONS Long videos may be okay! Especially later in the course! Keep overall load reasonable.! Active learning is not affected by week-by-week variations! Two-stage courses: attractiveness and retention. 30