Development and Adoption of an Adaptive Learning System: Reflections and Lessons Learned
- 1. Development and Adoption of an Adaptive Learning System Development and Adoption of an Adaptive Learning System: Reflections and Lessons Learned Dr Hassan Khosravi The University of Queensland Brisbane, QLD, Australia h.khosravi@uq.edu.au Professor Shazia Sadiq The University of Queensland Brisbane, QLD, Australia shazia@itee.uq.edu.au Professor Dragan Gasevic Maonash University Clayton, Vic, Australia Dragan.Gasevic@monash.edu
- 2. Development and Adoption of an Adaptive Learning System 2 Overview Challenges in delivering learner-centered learning at scale Adaptive learning systems provide a potential solution. But they are expensive to purchase or develop. RiPPLE: A discipline-agnostic cost-effective crowdsourced adaptive learning system Reflections and lessons learned from development and adoption of RiPPLE.
- 3. Development and Adoption of an Adaptive Learning SystemDevelopment and Adoption of an Adaptive Learning System 3 Developing Cost-Effective ALSs Adaptive Learning Systems (ALSs) Reflections and Lessons Learned Conclusion The RiPPLE Platform
- 4. Development and Adoption of an Adaptive Learning System 4 Adaptive Learning Systems Adaptive Learning Systems make use of data about students, learning processes, and learning products to provide an efficient, effective and customised learning experience for students by dynamically adapting learning content to suit their individual abilities or preferences. Domain Model Learner Model Content Repository Adaptation Engine
- 5. Development and Adoption of an Adaptive Learning System 5 Effectiveness of Adaptive Learning Systems This review found that the effect size of human tutoring is d = 0.79 and the effect size of intelligent tutoring systems was 0.76, so they are nearly as effective as human tutoring A meta-analysis of 107 studies on ITSs involving 14,321 participants found that: ITS were associated with higher achievement relative to teacher-led large-group instruction, non-ITS computer based- instruction, and texbooks or workbooks. This paper conducted a study on 3422 students from 198 offerings that have used ALEKS reporting significantly higher pass rates amongst students using ALEKS. Yilmaz The meta-analysis indicated that Intelligent Tutoring Systems produced a large effect size on reading comprehension when compared to traditional instruction (0.86)
- 6. Development and Adoption of an Adaptive Learning System 6 Developing Adaptive Educational Systems Publisher Model: designed with pre-existing content. • Examples: Pearsons MyLabs, McGraw-Hills LearnSmart and ALEKS • Successful in K12 where content is standardized. • Expensive to use Platform Model: provides a content-agnostic system infrastructure that enables instructors to develop content. • Examples: Smart Sparrow, Desire2Learn and edX incorporate adaptive • Introduce significant overhead for instructors • 25 hours of an expert's time for each hour of adaptive instruction (Aleven et al., 2006). • Both types are very expensive to develop /purchase and challenging to scale across different domains.
- 7. Development and Adoption of an Adaptive Learning SystemDevelopment and Adoption of an Adaptive Learning System 7 Adaptive Learning Systems (ALSs) Reflections and Lessons Learned Conclusion The RiPPLE Platform Developing Cost-Effective ALSs
- 8. Development and Adoption of an Adaptive Learning System 8 Developing Cost-Effective Adaptive Learning Systems
- 9. Development and Adoption of an Adaptive Learning System 9 Successful Crowdsourcing Stories Outside of Education Crowdsourcing Knowledge Crowdsourcing information Crowdsourcing Service Crowdsourcing micro tasks Crowdsourcing answers Crowdfunding
- 10. Development and Adoption of an Adaptive Learning System 10 Successful Crowdsourcing Stories in Education PeerWise: Crowdsourcing learning resources Peer grading and evaluation system Crowdy: Interactive, Collaborative, Crowd- powered Video Learning Piazza: crowdsources answers in discussion forums
- 11. Development and Adoption of an Adaptive Learning SystemDevelopment and Adoption of an Adaptive Learning System 13 Adaptive Learning Systems (ALSs) Reflections and Lessons Learned Conclusion Developing Cost-Effective ALSs The RiPPLE Platform
- 12. Development and Adoption of an Adaptive Learning System 12 The RiPPLE Platform
- 13. Development and Adoption of an Adaptive Learning System 13 The RiPPLE Platform Match students for peer study sessions Provide clicker-based in-class activity Free and open access to use See ripplelearning.org
- 14. Development and Adoption of an Adaptive Learning SystemDevelopment and Adoption of an Adaptive Learning System 13 Adaptive Learning Systems (ALSs) The RiPPLE Platform Conclusion Developing Cost-Effective ALSs Reflections and Lessons Learned
- 15. Development and Adoption of an Adaptive Learning System 15 Reflections and Lessons Learned
- 16. Development and Adoption of an Adaptive Learning System 16 1. Harness the Creativity Power of Your Students Question writing frequency correlated most strongly with summative performance (Spearman's rank: 0.24, p=<0.001). Only two questions of the 300 'most-answered' questions analysed had an unacceptable discriminatory value (<0.2)
- 17. Development and Adoption of an Adaptive Learning System 17 1. Harness the Creativity Power of Your Students “60% of all explanations classified as being of high or outstanding quality. Overall, 75% of questions met combined quality criteria” “People with greater expertise tend to make assumptions about student learning that turn out to be in conflict with students’ actual performance and developmental propensities.” “Crowdsourcing can efficiently yield high-quality assessment items that meet rigorous judgmental and statistical criteria. ” Overall, 86% of the examples classified in this multiple cohort investigation were found to be a ‘High quality question’ by being coherent, correct, requiring more than just simple factual recall, possessing a valid solution and reasonable distractors.
- 18. Development and Adoption of an Adaptive Learning System 18 • Incentivizing students to “eat their broccoli” is challenging! 1. Harness the Creativity Power of Your Students Frequent reminder of the benefits Inclusion of the activity as part of their assessment and bonus marks Availability of gamification aspects Co-development of sample solutions to questions from past exams
- 19. Development and Adoption of an Adaptive Learning System 19 2. Provide Rich and Transparent Models of the Learners (1) Interpretation: easier understand and interpret recommendations (2) Rationale: easier to understand the rationale behind recommendations (3) Trust: increases trust in the recommendations (4) Motivation: increases motivation to study or further use the platform
- 20. Development and Adoption of an Adaptive Learning System 20 3. Put Instructors in Charge Provide flexibility in use Adopting co-creation strategies to develop the platform and features. Providing small workshops and online help center. Comprehensive set of reports to provides rich analytics on students
- 21. Development and Adoption of an Adaptive Learning System 21 4. Support Ethical Low Cost Empirical Educational research Develop guidelines referring to consent, transparency and benevolence Develop mechanisms for instructors to run controlled experiments. Provide access to rich analytics to support observational studies. Develop mechanisms for instructors to run quasi- experiments.
- 22. Development and Adoption of an Adaptive Learning System 22 5. Provide Mechanisms to Motivate Students to be Actively Engaged Leaderboards Awards Rating Systems
- 23. Development and Adoption of an Adaptive Learning System 23 6. Usability, Flexibility and Scalability Usabiity Flexibility Scalability 1. Observing new users interacting with the system. 2. Holding focus groups with students and instructors. 3. Addition of feedback and survey 4. Adding Google Analytics to capture users’ interactions and behavior. 1. Started by hosting locally 2. Running stress test 3. Be cloud ready
- 24. Development and Adoption of an Adaptive Learning SystemDevelopment and Adoption of an Adaptive Learning System 13 Adaptive Learning Systems (ALSs) The RiPPLE Platform Reflections and Lessons Learned Developing Cost-Effective ALSs Conclusion
- 25. Development and Adoption of an Adaptive Learning System 25 Conclusion Educational technologies can help with delivering learner-centered learning at scale RiPPLE: A discipline-agnostic cost-effective crowdsourced adaptive learning system Reflections and lessons learned from Development and adoption of RiPPLE. Dr Hassan Khosravi The University of Queensland Brisbane, QLD, Australia h.khosravi@uq.edu.au