Sam Spaulding - Emotion AI Developer Day 2016
4 likes544 views
The document discusses socially assistive robots (SARs) and their role in education, particularly focusing on their ability to provide emotional and social support to enhance learning outcomes. It highlights the importance of affective computing in adaptive tutoring systems, describing the development of intelligent robotic tutors that personalize educational experiences based on emotional interactions with students. The study on 'Tega,' a social robot, explores its long-term engagement with preschool children, indicating that emotional personalization can improve learning and emotional expressiveness during robot interactions.
Sam Spaulding - Emotion AI Developer Day 2016
- 1. Socially Assistive Robots, Educational Tutoring, and Affective Computing Sam Spaulding MIT Media Lab
- 2. Social Robots Inventing our Future while Learning about Ourselves Robotic Engineering Artificial Intelligence Studies of Human Behavior Human-Robot Interaction Educational Companion Aging-in-Place HelperAssembly-mate
- 3. Socially Assistive Robots (SAR) Socially Assistive Robots are designed to leverage their social and affective attributes to provide social support to people in order to sustain engagement, motivate, coach, monitor, educate, or facilitate communication & teamwork for improved outcomes.
- 5. Embodiment matters! Robots produce higher learning gains Robots are more able to form long-term bonds Robots produce greater compliance Leyzberg et al. (2012) Bainbridge et al. (2012) Kidd & Breazeal (2008) Compared to screen-based representations...
- 6. Intelligent Tutoring Systems Use domain-general inference and modeling algorithms Have been extensively tested in real-world environments over long periods of time
- 7. Intelligent Tutoring SystemsRobotic Tutors The best of both... Social Presence Onboard Sensing Agent-based Interaction Adaptive Personalization Data-driven Student Models Physically embodied robot tutors that: sense and understand emotions build models of students based on affective data act intelligently as a result of the model info
- 8. Affect-aware Student Models for Robot Tutors - Child and Robot interact through shared “Storymaker” game context - Robot framed as peer, periodically asks child to demonstrate reading ability (to be presented at AAMAS ’16)
- 9. Knowledge State Estimation - A key challenge for adaptive, computational tutors is - “how to personalize experience?” - In order to provide personalized curriculum, the tutor must first determine students’ initial knowledge state Task Difficulty Student Ability Boredom Overwhelmed Flow: Optimal Challenge “Flow”
- 10. Bayesian Knowledge Tracing (BKT) Bayesian Knowledge Tracing (BKT) P(Li t) Correctt P(Li t+1) P(Li t+2) ... ... Correct t+1 Correct t+2 Background Model Domain Evaluation Contributions Sparse channel for knowledge, but widely studied Each ‘traced’ skill modeled by an HMM
- 11. Affective Bayesian Knowledge Tracing Affective data drawn from 5s before question asked to 5s after question answered
- 12. Affect-aware Student Models for Robot Tutors - 25 children came and played with Dragonbot - 13 children did same interaction with Tablet only - Experiment conducted in Summer 2014, affective analysis completed 6mo. later (to be presented at AAMAS ’16)
- 13. Are Children More Emotionally Expressive When Interacting with A Robot? ... ... ... ... } { Avg Smile: 18 Avg BrowFurrow: 7 Avg BrowRaise: 62 Avg LipDepress: 4 AvgValence: -24 Avg Engage: 67 } {Median Filter } {Mean Metric Value Session Footage Raw Affdex Measurements Median-smoothed Affdex Data Subject Bag Average MetricValue over Interaction
- 14. Are Children More Emotionally Expressive When Interacting with A Robot? * * * * p < .05 n=25 “Robot” condition n=13 “Tablet condition
- 15. Training and Evaluating Skill Models BKT and Aff-BKT models trained for 4 Skills via Expectation Maximization Model classes evaluated via log-likelihood comparison, with Leave-one-out cross-validation
- 16. Does Including Affective Data in Training Yield Better Models? ... }Affective + Right/Wrong Training Data Subject Bag Expectation Maximization { P(Li t) CorrecttSmilet P(Li t+1) ... Engagedt Correctt+1Smilet+1 Engagedt+1 ... P(Li t) CorrecttSmilet P(Li t+1) ... Engagedt Correctt+1Smilet+1 Engagedt+1 ... } Training Data Trained Model Trained Model Subset {Model Subset P(Li t) CorrecttSmilet P(Li t+1) ... Engagedt Correctt+1Smilet+1 Engagedt+1 ... P(Li t) Correctt P(Li t+1) ... Correctt+1 ... Held-out Right/Wrong Test Data, D' Likelihood of Model, Given Test Data D’ } D’ P(D’| θaff) θaff = maxθ P(Daff|θ) θaff, a subset of θaff, containing only BKT model parameters θaff ^ ^ ^ Daff
- 17. Does Including Affective Data in Training Yield Better Models? [ [ [ [ Exact-Correct BKT Aff-BKT BKT Aff-BKT BKT Aff-BKT BKT Aff-BKT First-Letter Length Last-Letter
- 18. Does Including Affective Data in Training Yield Better Models? p < 1.0 x 10-4 * * ** * * ** **p < 1.0 x 10-5 p < 1.0 x 10-6 * ***
- 19. Are children more emotionally expressive when interacting with robots? Can we leverage emotional expression data to create better student models? Two main questions:
- 20. Are children more emotionally expressive when interacting with robots? Can we leverage emotional expression data to create better student models? Two main questions:
- 21. Intelligent Tutoring SystemsRobotic Tutors The best of both... Social Presence Onboard Sensing Agent-based Interaction Adaptive Personalization Data-driven Student Models Physically embodied robot tutors that: sense and understand emotions build models of students based on affective data act intelligently as a result of the model info
- 22. Meet Tega!
- 23. Tega: a “real-world ready” social robot! Student models allow us to personalize curricular content. How do we personalize affective support? Affective Personalization
- 24. Affective Personalization of a Social Robot Tutor for SSL (presented at AAAI ’16) •What’s it all about? •Children learning Spanish as a second language with a robot companion •An Integrated System •Tega Robot •Custom Educational Game • Affdex affective sensor •All synchronized and coordinated through a ROS- based cognitive architecture •The Study •Long-term, in-the-wild, fully autonomous interaction •Personalization of affective response
- 25. Integrated System: Software Game: - Unity-based sprite game - 8 sessions of content + review - Fully autonomous play - Virtual “instructor” character, robot as peer Affdex phone: - Real-time detection of facial expressions - Valence / Engagement used as reward to RL algorithm
- 26. Educational Context Robot again framed as peer, with ‘bilingual’ Toucan as teacher As the student plays through the game, the robot provides affective support through verbal + nonverbal actions
- 27. Reinforcement Learning on Affective Data SARSA Algorithm Reward = .4(Engagement) +.6( (Valence+100) ) 2 State Space = 3 x 2 x 2 x 2 = 24 states total Neg./Neut./Pos. Valence Hi/Lo Engagement On/Off Task Right/Wrong Last Question Action Space = 3 x 2 + No-action = 7 action classes total ɛ-greedy algorithm, with ɛ decreasing across sessions
- 28. Sample Actions
- 29. Timeline • 8 Weeks of In-Class deployment • 1 Pre-test Session • 6 Study Sessions (part review, part new content) • 1 Post-test Session
- 31. Affective Personalization * Appropriate affective responses are critical to avoiding “novelty effect”
- 32. Did they actually learn? *
- 33. Contributions • Novelty of current study: • Long-term interaction (8 sessions) • Fully autonomous social robot (Tega) • In-the-wild experiment (inside a classroom) • Affective personalization (Affdex) • Age of participants (3-6 yrs) Social robot personalized its affective response, thus increasing children’s valence during long-term interaction.
- 34. Looking ahead… Old challenges are becoming tractable: sensing, deployment, robust systems. New challenges are conceptual and computational - i.e. how to fully integrate emotions into an agent’s cognition
- 35. Collaborators and Supporters This research was supported by the National Science Foundation (NSF) under Grant CCF-1138986 and Graduate Research Fellowship Grant No 1122374. Luke Plummer JinJoo Lee Goren Gordon Jacqueline KoryProf. Cynthia Breazeal