ocwc2014

Editor's Notes

• #3: This presentation will serve these three purposes and also propose that the OCW Consortium take a leadership role in serving as a clearing house and advocate for the sharing of data and experimental results across institutions, in order to advance the use of open material to fuel education innovation.
• #4: All our investigations ultimately come down to an examination of the imperative of universal higher education—the idea that ultimately what we will experience is the fact that everyone will be able to learn anything, anytime, any place, for free. The ubiquitous educational product will merge learning and life in such a way that learning projects will become less and less distinguishable from other like activities, and the public will adopt learning projects of varying length and complexity and seek the best means of achieving their learning goals. We examine MOOCs here against that broader framework.
• #5: UC Irvine is among the leaders in offering MOOCs. UCI has offered 15 MOOCs (one was repeated) mostly on Coursera and enrolled over 640,000 people giving us a solid base for investigation.
• #6: One example of the other open material UCI offers is the video lectures posted on YouTube. For the first three months of 2014 we are averaging over 100,000 views and almost 1,000,000 minutes watched per month.
• #7: Most of the MOOC research has centered around the MOOC phenomenon itself—what is the appeal of MOOCS and how do people use them, what effect will MOOCs have on education, how disruptive will they be, how can credit be grated for MOOCs, and what business models are emerging.A major question/issue has arisen because of the relatively low completion rates of those who enroll in MOOCs. So, much of the attention has centered on how to measure participation rates short of full completion of the course. A second category of research questions little on the largely untapped question over the role that MOOCs play or might play in actually helping people learn.
• #8: These results have been replicated in a number of simple studies across a large number of “n’s”. But this measures only the current dominant form of MOOC offerings. As MOOCs evolve and begin treating subjects other than those represented by undergraduate courses from top universities, the utilization of MOOCs will also evolve.
• #9: This indicates that of the 444,178 participants who enrolled in UCI’s Coursera’s courses, 222,862 (50%) did not engage at all in the learning activity.
• #10: Coursera offers the above scheme to categorize participation levels among MOOCs. The establishment of such a scheme helps us structure queries of the data that can be compared across intuitions and MOOC instances. However, this categorization scheme can be usefully expanded to include those who persist through the final assignment (exam) and pass the course.
• #11: This represents the data from 11 MOOC courses offered by UCI. Note that about 11% completed final exams but only 7.4% passed them. Participationin the course declines as the course progresses. For instance, while 74% watched the first video, only 25% watched at the midpoint. But the rate of videos watched by the end of the course was about 18%, which indicates higher rates of persistence on the relatively passive measure of video watching. For the more active measure, like taking an exam, the numbers were 40.6%, 15% and 11%, a similar pattern.
• #12: This is an example of the second category of MOOC research questions—those having to do with the use of MOOCs in actually producing meaningful learning in individuals. A number of MOOC related events have cast doubt on the ability of a free course such as a MOOC to help students prepare for the rigors of university-level study. An example is the pre-calculus course offered through Udacity and San Jose State University in which very few students persisted and passed the preliminary course. UCI is adding data to the answer to the question by doing a pre-bio course for freshman entering UCI in the Fall 2013 term. These students (about 500) were given the chance to take a pre-bio course on Coursera during the Summer 2013. The results of their learning activities is now being compared with those not taking the MOOC to determine if they do better in the Fall Bio-1 course.
• #13: As will all education research, the ability to conduct rigorous scientific research is hampered by the fact that it is usually very difficult to control all the variables needed for the full application of statistics and the scientific method. Comparing two groups of learners who take different learning treatments inevitably involves the introduction of variables in the control and experimental treatment that makes cause and effect relationships verifiable. On the other hand, with very large groups of learners it is possible to query the data for valid relationships (rather than cause and effect). The large “ns” of most MOOCs make valid conclusions about relationships possible.
• #14: A number of laws and common sense require that personally identifiable information such as student names, social security numbers and other references that can be traced back to an individual should be protected and not revealed in any way, including in the results of research studies. Coursera has gone further in categorizing such data and its related states. Unanonymizable data is data such as PII that is revealed by, say, another student in a class in relation to the class including peer assessments and forum comments. Public forum and course data are data revealing PII which has been voluntarily placed in public view by the student involved such as posts to a public forum with the student author name and other PII attached. Because the assessment of learning is such an important issue for measurement, the ability to examine individual responses to learning necessary and this is difficult without using PII. This generally means that human subject protocols or IRBs must be filed by researchers on their campuses.
• #15: Since most MOOC data is now generated by other sources than universities themselves (Coursera, EdX, Udacity, etc.), the data is usually owned by these entities who must release it to university researchers even when that data relates to the students taking the MOOC offered by the university. Because the use of data including PII is so sensitive, the passing of responsibility from these entities to university researchers presents some issues.
• #16: Despite these problems, MOOC related research has a number of advantages. The large numbers involved, and the ease of collecting data about those large numbers is a big advantage and mediates the problems created by the number of variables introduced. There is a strong impulse by the owners of data to share that data in the service of openness and the usefulness and utility that sharing serves. Both Coursera and EdX have recently announced efforts to promote the collecting and sharing of data. Further, because MOOC providers are interested the relative effectiveness of their own MOOCs there is an impulse for the replication of data and again, the sharing of it.