Big Data for Student Learning
3 likes1,266 views
The document discusses how learning analytics and educational data mining can leverage big data to enhance student learning and improve educational practices. It outlines the developments in technology and data collection methods, discusses the potential benefits and applications, and highlights challenges such as privacy and the need for data-driven decision-making. The role of educators, administrators, and students is evolving as data becomes integral to the educational experience.
Big Data for Student Learning
- 1. SRI International Center for Technology in Learning Learning Analytics: Can Big Data Help Improve Student Learning? Marie Bienkowski October 2, 2012 Virginia Military Institute, Lexington, VA
- 2. Overview • Intersecting developments – An Instrumented World – The Quantified Self – Advances in Computing – Interconnected and Real-Time Feedback Loops • Defining Educational Data Mining, Learning Analytics, and Visual Data Analytics • Examples of How Data Gets Used • Challenges and Final Thoughts
- 3. An (Increasingly) Online World • Learning Online – Learning and Content Management Systems (Blackboard, Sakai, Moodle) – OER Delivery & Social Platforms (Edmodo, Gooru) – Self-Help, Homework Sites, Flipped Classroom (Khan Academy, CrackOChem) – Games, Simulations, Digital Textbooks – Social Learning (Discussion forums, twitter, Facebook) • Collecting interaction data: We’ve come a long way from clicker systems & online gradebooks
- 4. The (Increasingly) Quantified Self • Mobile devices capture activities • Improved biometric sensors – FitBit Activity Monitor, Zeo Sleep Manager • Social networks for data aggregation, crowd sourcing data – myfitnesspal.com, mapmyrun • Self-help sites for time management, exercise, weight loss, etc.
- 5. Advances in Computing • Mobile computing devices • Capturing click stream data • Cheap data storage • Improvements in computer methods – Processing big data – Machine learning – Open source software • Social networking web sites
- 6. Interconnected Feedback Loops National Educational Technology Plan “The goal of creating an interconnected feedback system would be to ensure that key decisions about learning are informed by data and that data are aggregated and made accessible at all levels of the education system for continuous improvement.” (U.S. Department of Education 2010, p. 35)
- 7. Inspiration National Educational Technology Plan “Twenty-first century learning powered by technology”
- 8. New Forms of Assessment National Educational Technology Plan “When students are learning online, there are multiple opportunities to exploit the power of technology for formative assessment. The same technology that supports learning activities gathers data in the course of learning that can be used for assessment…. An online system can collect much more and much more detailed information about how students are learning than manual methods. As students work, the system can capture their inputs and collect evidence of their problem- solving sequences, knowledge, and strategy use, as reflected by the information each student selects or inputs, the number of attempts the student makes, the number of hints and feedback given, and the time allocation across parts of the problem.” (U.S. Department of Education 2010, p. 30)
- 9. Research in Data Mining/Analytics • Educational Data Mining: develops new techniques, tests learning theories and informs educational practice. Looks for patterns in unstructured data. Generally automates responses to learners. • Learning analytics: Applies techniques and takes a “system-level” view of teaching and learning, including at the institutional level. Generally supports human decision making vs. automating responses. • Visual Data Analytics: taps the ability of humans to discern patterns in visually represented complex datasets.
- 10. Examples • Predicting student success: CourseSignals from Purdue. • Detecting useful help-seeking behavior and adapting accordingly. • Presenting actionable data to various stakeholders. • Research: combine observation data with click stream data to detect boredom, frustration, gaming the system.
- 11. Research Example • Evidence of off-task behavior – Learner completes module in less time than expected and performs poorly on exam, suggesting the learner is not engaged in attempting to use the software to learn – Very fast actions immediately before or after very slow actions – Extremely fast actions or extremely slow actions
- 12. Research Example • Evidence of gaming – Systematic and rapid incorrect answers – Quickly and repeatedly asking for help – Sustained and/or systematic guessing
- 13. Model of Adaptive Learning Systems
- 14. Application Areas • User Modeling: Model a learner’s knowledge, behavior, motivation, experience, and satisfaction. • User Profiling: Cluster users into similar groups. • Domain Modeling: Decompose content to be learned into components and sequences. • Effectiveness: Test learning principles, curricula, etc. • Trend Analysis: Track changes over time. • Recommendations and Improvements: Suggest resources and actions to learners; adapt system to learners.
- 18. Challenges • Technical: Handling big data; interoperability of data systems; asking the right questions • Institutional: Requires a culture of data-driven decision making and transparency in models that analyze data • Privacy and Ethics: Maintain student and teacher privacy while allowing data aggregation to drive powerful models; who owns the data?
- 19. Final Thoughts • Changing role of teachers: Do sense-making with patterns that the data mining algorithms find. Challenge the models and override when needed. • Changing role of administrators: Use data as pointers to areas to investigate. Exemplify savvy data use. Be data-curious. • Changing role of students: Learn how to learn online. Become interested in reflecting on own efforts and managing learning.
- 20. Enhancing Teaching and Learning Through Educational Data Mining and Learning Analytics An Issue Brief Issue Brief forthcoming (Fall, 2012) from US Department of Education Office of Educational Technology. Contact marie.bienkowski@sri.com U.S. Department of Education
Editor's Notes
- #2: Good afternoon. Here from SRI International. Silicon Valley. Founded by Stanford University in 1946; separated in 1970. Facilities throughout the US, including biosciences in Harrisonburg, Virginia
- #3: I can give you the answer to the Q “Can Big Data Help Improve Student Learning?” now—”Yes, but it’s complicated” I have been in the technology field for 30 years as a computer scientist and while I, as all of us--, have seen technology change our lives, it’s always through an interplay of the technical and social systems. And data mining and analytics applied to education is no exception.Leveraging big data is an area in that many stakeholders in the education enterprise have an interest in and it truly is used K-20. And not just by teachers and learners and education administrators. Education researchers and software developers and commercial enterprises also have a great interest in what they can learn from using big data.In my experience, that’s unique across any technology.So this talk will be part background, and part examples of this growing area. You should leave with an understanding of what technology is being applied and what areas seem promising, as well as the challenges faced.
- #4: Early on, companies who do business online realized the value of leveraging the traces or “data exhaust” or activity streams left by users. The examples now are so common: Amazon’s recommender system; NetFlix’s ratings, Pandora’s music genome. These examples show the power of collective intelligence: pooling data from many people provides value (sometimes) to one. Search companies use different kinds of big data—consider how google predicts search terms and how Facebook games keep their users engaged.We might stop and consider how learning today is “instrumented”: teachers observations, homework, in-class assessments, and standardized tests. As we progress, it might be instructive to keep in mind a simple example: showing how you got to a solution in your math homework.
- #5: Another thread, perhaps less prominent in the arena of teaching and learning with big data, but one which I think will have a big impact on the uptake and social acceptability of using big data, is the rise of the quantified self. This skirts the issue of who owns the data. Example of weight loss site. People enter common items and their nutritional information. Others vote on the accuracy of the entries. Imagine using that data for two purposes: 1. directing advertisements to people or deciding how to redo a product for weight loss vs. finding patterns in the data: people like you (e.g., exercise amount, weight, gender) who ate more of X lost weight. One of these seems more socially acceptable, right?
- #6: Advanced in computing are of course behind all of this, and that’s often not obvious to people. Processors are getting smaller—hand-held—as well as having networked capabilities. Internet client technology—browsers—have more sophisticated methods of both presenting and capturing data from users. This data can be stored cheaply and without a lot of processing at the time of capture. Then, advances in how to spread computing over multiple machines and then search for patterns in the data have moved the field forward. Finally, the open-source software movement is not to be underestimated, because it has led to sharing of methods that operate on big data.A final advance is the increase in people’s willingness to share data. Social networking sites are the most obvious examples, but there are others as well: ratings, comments, tagging, etc. These are all ways that data gets into the data stream for processing.
- #7: In 2010, the US Department of Education released the National Educational Technology Plan. That plan reflected goals for improving the educational system—how we make decisions—by envisioning feedback loops for all levels. This diagram, adapted from one by Candace Thille of the Open Learning Initiative at CMU, shows the players who are involved in this enterprise. I would add Researchers onto Developers.
- #8: So we’ve seen the factors and drivers that make it possible now harness big data for teaching and learning. What does the plan say about using data.
- #9: In 2010, the focus of using big data was on assessment of student learning. And that’s probably the most common way people think of using big data—to discern whether students have learned or not. At the time of the writing of this plan, that was about the most obvious application of using big data. But since that time—just 2 years later, we’ve learned about many more possible applications. But before we go there, let’s define a few terms.
- #10: Here are some definitions that are useful as background. You might have heard already of data mining—it’s been in the popular press as a method for snooping on people’s behavior. But of course it’s just a computer science technique for finding patterns in data—generally unstructured data—data that’s not already formatted and stored neatly in a database. An obvious example of data mining it building models that predict credit card fraud. It’s not only a big data area, it’s a big money area! The machine is given examples of normal buying and abnormal buying. From these, it learns a predictive model. This model is then let loose on real transactions and it catches fraud. And educational data mining finds patterns in education data. Learning analytics generally takes those models and applies them in system-level situations: mainly at higher education. These distinctions are still evolving. Finally, visual data analytics are added on for completeness—here again I am blending research and application.
- #11: So how are data mining and analytics used in teaching and learning? Here are some examples that go beyond the assessment focus we talked about earlier
- #14: We like models too. I like to show this model of a general way
- #15: Think beyond assessment!
- #16: Dreambox Learning
- #17: Khan Academy
- #18: In Thille’s OLI Statistics Course:More detailed information:Class’s learning of sub-objectivesLearning of individual studentsCommon misconceptions