An Overview of Recent Developments in Intelligent e-Textbooks and Reading Analytics
- 1. An Overview of Recent Developments in Intelligent e-Textbooks and Reading Analytics BY DAVID BOULANGER AND VIVEKANANDAN KUMAR Athabasca University | Canada
- 2. 2 LITERATURE 13 9 1 Conference Journal Workshop 9 conferences 9 journals 1 workshop 23 References 3 8 5 6 1 2015 2016 2017 2018 2019
- 3. DATASETS 3 # of events Events / reader # of readers Reading episode Domain Software Type of reading 2,812,727 939 2993 Semester (43 courses) BookLooper Lecture 567,193 7988 71 Semester Human Computer Interaction, Information Retrieval Reading Circle Textbook, research publication, etc. 129,451 555 233 Semester (11 courses) CourseSmart Textbook 75,748 276 274 Semester Social Sciences, Business, Education - Textbook 65,755 608 108 Semester (2 courses) BookRoll Lecture 10,994 - - ~1 year (110 different magazine issues) Viewerplus + APP- BI Magazine 10,188 35 289 Semester Interactive Systems Design AnnotatED+, Reading Circle Textbook ~7200 109 66 Semester Research Methods - Textbook 1370 80 17 1.5 hours Educational Technology DITeL Journal article 65 7 9 30 minutes Introductory Biology - Textbook In-house readers except for CourseSmart 4 types of reading The semester is the typical period during which reading behavior is evaluated.
- 4. CONCEPT MAP OF READING ANALYTICS & INTELLIGENT E-TEXTBOOKS 4 E-Reader Document Document Format Hardware Reading Behavior Software Metrics Engagement Model Gaming BehaviorTechniques Accuracy Feature SelectionSmart Features Interactive Components Adaptive Engine Instructional Modeling Student Modeling Content Modeling Student DOCUMENT • Page • Section • Chapter Raw Data Interaction Data Self-Reported Data DOCUMENT FORMAT • PDF • EPUB • Word documents • Web documents • Any reading resource
- 5. CONCEPT MAP OF READING ANALYTICS & INTELLIGENT E-TEXTBOOKS 5 E-Reader Document Document Format Hardware Reading Behavior Software Metrics Engagement Model Gaming BehaviorTechniques Accuracy Feature SelectionSmart Features Interactive Components Adaptive Engine Instructional Modeling Student Modeling Content Modeling Student Raw Data Interaction Data Self-Reported Data HARDWARE • Augmented reality • Eye tracker • Mobile • Remote • Smartphones • Tablets • E-readers • Computers SOFTWARE • BookLooper • Reading Circle • CourseSmart • BookRoll • Viewerplus • AnnotatED+ • DITeL • Adobe Reader • Kindle
- 6. CONCEPT MAP OF READING ANALYTICS & INTELLIGENT E-TEXTBOOKS 6 E-Reader Document Document Format Hardware Reading Behavior Software Metrics Engagement Model Gaming BehaviorTechniques Accuracy Smart Features Interactive Components Adaptive Engine Instructional Modeling Student Modeling Content Modeling Student INTERACTION DATA • Next page • Previous page • Jump to page • Annotations • Marking of unknown words • Comments • Highlighting • Underlining • Changing marker color • Zooming in/out • Searching • Search jump • Screen orientation • Portrait • Landscape • Scrolling • Bookmarks • Opening, exiting, minimizing e-textbooks SELF-REPORTED DATA • Survey • Interview • Questionnaires Raw Data Interaction Data Self-Reported Data • Online • Offline Feature Selection
- 7. CONCEPT MAP OF READING ANALYTICS & INTELLIGENT E-TEXTBOOKS 7 E-Reader Document Document Format Hardware Reading Behavior Software Metrics Engagement Model Gaming BehaviorTechniques Accuracy Smart Features Interactive Components Adaptive Engine Instructional Modeling Student Modeling Content Modeling Student METRICS • # of blinks • Distances of eye movements • Coordinates of eye gazes • Fixations • Saccades • Time of adoption • Reading speed • Engagement level • Reading session • Attention • Reading/visit time • Last pages read • Pages previewedRaw Data Interaction Data Self-Reported Data Feature Selection
- 8. CONCEPT MAP OF READING ANALYTICS & INTELLIGENT E-TEXTBOOKS 8 E-Reader Document Document Format Hardware Reading Behavior Software Metrics Engagement Model Gaming BehaviorTechniques Accuracy Smart Features Interactive Components Adaptive Engine Instructional Modeling Student Modeling Content Modeling Student READING BEHAVIOR • Progressive sequential analysis • Self-reported behavior • Perception gap Raw Data Interaction Data Self-Reported Data ENGAGEMENT MODEL • Blocked linear regression • Clustering • None • Low • Medium • High GAMING BEHAVIOR • Classifier (gaming, normal) • Logistic regression • KNN • Naïve Bayes • Decision tree • SVM Feature Selection
- 9. CONCEPT MAP OF READING ANALYTICS & INTELLIGENT E-TEXTBOOKS 9 E-Reader Document Document Format Hardware Reading Behavior Software Metrics Engagement Model Gaming BehaviorTechniques Accuracy Smart Features Interactive Components Adaptive Engine Instructional Modeling Student Modeling Content Modeling Student TECHNIQUES • Gradient tree boosting • Survival analysis • PCA • Regression • Simple linear • Non-linear • Mixed-model linear • Deep learning • Stochastic block model • Descriptive statistics • Classification • Random forest • KNN • Support vector machine Raw Data Interaction Data Self-Reported Data FEATURE SELECTION • Random forest regressor • F-regression • K-means transformations • t-test ACCURACY • Training/testing set • Cross-validation • AUC • RMSE/MSE Feature Selection
- 10. CONCEPT MAP OF READING ANALYTICS & INTELLIGENT E-TEXTBOOKS 10 E-Reader Document Document Format Hardware Reading Behavior Software Metrics Engagement Model Gaming BehaviorTechniques Accuracy Smart Features Interactive Components Adaptive Engine Instructional Modeling Student Modeling Content Modeling Student SMART FEATURES • Prediction of student performance • Identification of at-risk students/drop-outs • Real-time improvement of learning materials • Provision of teacher annotations • Measurement of reader’s interest and competence • Assessment of a concept’s difficulty level • Automatic correction of students’ answers • Provision of formative feedback • Recommendation of next topics to be learned • Recommendation of effective learning strategies • Real-time lecture supporting system • Many more … Raw Data Interaction Data Self-Reported Data Feature Selection
- 11. CONCEPT MAP OF READING ANALYTICS & INTELLIGENT E-TEXTBOOKS 11 E-Reader Document Document Format Hardware Reading Behavior Software Metrics Engagement Model Gaming BehaviorTechniques Accuracy Smart Features Interactive Components Adaptive Engine Instructional Modeling Student Modeling Content Modeling Student INTERACTIVE COMPONENTS • Multimedia/visuals • Video • Sketch • Animation • Diagram • Problems to solve • Collaborative • Q&A • Quizzes (multiple- choice/open-ended questions) • Explanations of key concepts • Examples with different parameters Raw Data Interaction Data Self-Reported Data Feature Selection
- 12. CONCEPT MAP OF READING ANALYTICS & INTELLIGENT E-TEXTBOOKS 12 E-Reader Document Document Format Hardware Reading Behavior Software Metrics Engagement Model Gaming BehaviorTechniques Accuracy Smart Features Interactive Components Adaptive Engine Instructional Modeling Student Modeling Content Modeling Student ADAPTIVE ENGINE • Student model • Learning style • Instructional model • Knowledge level • Content model • Curriculum Raw Data Interaction Data Self-Reported Data STUDENT MODELING • Transfer GPA • Demographics CONTENT MODELING • Concept-based hyperspace organization • Automatic • Bag-of-words • LDA • Manual INSTRUCTIONAL MODELING • Knowledge Tracing • Hidden Markov model • Behavior model • Performance model • Behavior-performance model Feature Selection
- 13. FUTURE WORK 13 ▪ Benchmark datasets on both reading activities and heterogeneous learning activities ▪ Further experimentation of deep learning techniques (e.g., multimodal RNNs) ▪ Analysis of the reading process in the broader frame of the learning process (e.g., how reading is used as a learning strategy) ▪ Impact of e-textbook’s smart features on reading and learning performance ▪ Discovery of optimal reading behaviors ▪ E-textbook’s interactive components implemented in virtual/augmented reality environments ▪ Intelligent printed textbooks/documents through augmented reality ▪ Intelligent adaptivity to nurture self-regulatory traits ▪ Adaptivity (computer-driven) vs. adaptability (human-driven)
- 14. http://www.bbc.com/capital/story/20190127-humanics-a-way-to-robot-proof-your-career LEARNING ANALYTICS, COGNIFICATION & HUMANICS 14 Measuring the impact of learning analytics on learning performance • Accelerating learning time • Increasing knowledge retention • Formation of soft skills • Greater transfer of learning Humanics-oriented life-long learning Ensuring humans remain relevant in the world of AI. • Mastery of learning content: • Traditional literacies • Tech literacy • Data literacy • Human literacy • Nurture and assessment of cognitive capacities: • Creativity • Mental flexibility • Critical thinking • Systems thinking • Leadership Cognification of learning The process of making learning objects and environments increasingly and ethically smarter. • Intelligent textbook • Competence assessment • Automated scoring • Formative feedback • Remedial interventions “A generation ago, the half-life of a skill was about 26 years. Today, it’s 4 and half years and dropping.” – Indranil Roy
- 15. Learner Instructor Organization Governance Stakeholders IoT Conversations Data Artisanship Dashboard Insights Descriptive Diagnostic Predictive Prescriptive Dimensions Causality Observational Randomized Longitudinal Meta-analysis Meta-analysis Meta-analysis Data Mining & Research Methods Knowledge Base “An ethics-bound, semi-autonomous, and trust-enabled human-AI fusion that measures and advances knowledge boundaries in human learning.” A DEFINITION OF LEARNING ANALYTICS 15 Kumar, V., Boulanger, D., Fraser, S.: Inferring Causal Effects from Learning Analytics: Discovering the Nature of Bias (2019).
- 16. Instructional Model Topic E Topic D Topic G Topic H Topic J Assessment of Decision Making Content Model Expert Machine Learning Topics ready to be learned Topic D Topic E Learner Model Adaptive Engine Learning Object Repository Trust Model Talent SRL Learning Style Grit Interest Engage- ment Demo- graphics Learning Strategy Moti- vation Prior Know- ledge Learner/Teacher/AI Decision Making Pedagogical Strategy 2x E 1x D 3x G 5x H 2x J ADAPTIVITY 16