User Control in AIED (Artificial Intelligence in Education)
0 likes246 views
This document summarizes research on improving user control and personalization in artificial intelligence for education (AIED) systems. It discusses several AIED systems that provide adaptive navigation support and annotation based on user models while allowing user control over sequencing and navigation. Evaluation of these systems found they can reduce effort, encourage exploration, and increase learning outcomes when users are able to follow or override advice. The document also presents approaches that improve transparency and control through open learner models, controllable ranking, visualization of recommendation models, and balancing adaptation with user exploration.
User Control in AIED (Artificial Intelligence in Education)
- 1. Two Brains are Better than One: User Control in AIED Peter Brusilovsky with Denis Parra, Chun-Hua Tsai, Jordan Barria-Pineda, Kamil Akhuseyinoglu, Behnam Rahdari PAWS Lab School of Computing and Information University of Pittsburgh
- 2. AI or Humans + AI? 2
- 3. Two Brains are Better than One! 3 Image credit: https://towardsdatascience.com
- 4. Adaptive Hypermedia: Human-AI Collaboration Adding AI to user-controlled information access environment 4
- 5. Navigation vs. Adaptive Sequencing 5 Human makes navigation decision AI makes navigation decision
- 6. ELM-ART: Adaptive Annotation (1996) Weber, G. and Brusilovsky, P. (2001) ELM-ART: An adaptive versatile system for Web-based instruction. International Journal of Artificial Intelligence in Education 12 (4), 351-384.
- 7. ELM-ART: Evaluation • No formal classroom study • Users provided their experience • Drop-out evaluation technology • 33 subjects – visited more than 5 pages – have no experience with Lisp – did not finish lesson 3 – 14/19 with/without programming experience
- 8. ELM-ART: Value of ANS Mean number of pages which the users with experience in at least one programming language completed with ELM-ART
- 9. ELM-ART: Value of ANS Mean number of pages which the users with no experience in programming languages completed with ELM-ART
- 11. Adaptive Annotation Can: • Reduce navigation efforts • Reduce repetitive visits to learning content pages • Encourage non-sequential navigation • Increase learning outcome • For those who is ready to follow and advice • Make system more attractive for students • Students stay much longer without any reward
- 12. More Control! Open Learner Model 12 Weber, G. and Brusilovsky, P. (2001) ELM-ART: An adaptive versatile system for Web-based instruction. International Journal of Artificial Intelligence in Education 12 (4), 351-384.
- 13. Simple Ranking Control in Search and Recommendation Allow the user to control how the ranking list is produced to adapt personalization for the current context as well as better explore recommendation results 13
- 14. TaskSieve: Controllable Personalized Search Ahn, Jae-wook, Peter Brusilovsky, Daqing He, Jonathan Grady, and Qi Li. 2008. "Personalized Web Exploration with Task Models." In the 17th international conference on World Wide Web, WWW '08, 1-10. Beijing, China: ACM.
- 15. TaskSieve Controllable Ranking • Post-filtering • Combine query relevance and task relevance – Alpha * Task_Model_Score + (1-alpha) * Search Score – Alpha : user control (0.0, 0.5, or 1.0) • Results – Better than regular adaptive search – Better then non adaptive baseline even in cases when profile was excluded – Users were really good in deciding when to engage the profile and how 15
- 16. O'Donovan, John, Barry Smyth, Brynjar Gretarsson, Svetlin Bostandjiev, and Tobias Höllerer. 2008. "PeerChooser: visual interactive recommendation." In Proceedings of the twenty-sixth annual SIGCHI conference on Human factors in computing systems, 1085-88. Florence, Italy: ACM. PeerChooser: Controllable CF 16
- 17. Complex Control + Transparency Make the ranking process transparent and explorable. Allow users to play with presentation parameters to understand aspects of relevance and find best items in the given context 17
- 18. Control and Transparency: Two Sides of the Same Coin Explain Visualize Explore Control 18 Transparency Controllability No full transparency without controllability Control is challenging without transparency
- 19. RelevanceTuner: Control+Visualization in a Hybrid Social Recommender Tsai, Chun-Hua and Peter Brusilovsky (2018) Beyond the Ranked List: User-Driven Exploration and Diversification of Social Recommendation. In 23rd International Conference on Intelligent User Interfaces, 239--50. Tokyo, Japan: ACM.
- 20. Mastery Grids: Personalized Practice System with OSLM Loboda, T. D., Guerra, J., Hosseini, R., & Brusilovsky, P. (2014, September). Mastery grids: An open source social educational progress visualization. In European conference on technology enhanced learning (pp. 235-248). Springer, Cham. Learning content OSLM Features
- 21. Student-Controllable Social Comparison Default Class Average Change to Lower progress Change to Higher progress Akhuseyinoglu, K., Barria-Pineda, J., Sosnovsky, S., Lamprecht, A.-L., Guerra, J., and Brusilovsky, P. (2020) Exploring Student-Controlled Social Comparison. In: Proceedings of European Conference on Technology Enhanced Learning EC-TEL 2020, Cham, 14-18 September, 2020, Springer International Publishing, pp. 244-258.
- 24. Total 706 group changes (n=40, M=17.65, SD=19.09) – 91% (n=40) 1+ changes – 77% (n=32) 5+ changes – 34% (n=15) 10+changes Did Students Use Control Features? 24
- 25. 67% exploratory group changes Stability of Group Selections Change distribution: 21% 47% 32% Dominant group: n=3 n=30 n=7 Avg. practice time: 22 mins 180 mins 75 mins 25
- 26. More Controllable SC 26 Sosnovsky, S., Fang, Q., de Vries, B., Luehof, S., & Wiegant, F. (2020). Towards Adaptive Social Comparison for Education. In C. Alario-Hoyos, M.J. Rodríguez-Triana, M. Scheffel, I. Arnedillo-Sánchez, & S.M. Dennerlein (Eds.) Proceedings of EC-TEL’2020: 15th European Conference on Technology Enhanced Learning (pp. 421-426). Berlin/Heidelberg, Germany: Springer
- 27. Jordan Barria-Pineda and Peter Brusilovsky. 2019. Making Educational Recommendations Transparent through a Fine-Grained Open Learner Model. In IUI’19 Workshops. APCSE Workshop Mastery Grids with transparent recommenda- tions Transparency for Navigation Support
- 28. Grapevine: Personalized Exploration with Open UM 28 Rahdari, B., Brusilovsky, P., and Babichenko, D. (2020) Personalizing Information Exploration with an Open User Model. In: Proceedings of 31st ACM Conference on Hypertext and Social Media, July 13-15, 2020, ACM, pp. 167-176.
- 29. Personalization – Exploration Loop Adapt-Discover-Control-Adapt-Discover- 29
- 30. Questions? 30
- 31. Readings • Ahn, Jae-wook, Peter Brusilovsky, Jonathan Grady, Daqing He, and Sue Yeon Syn (2007) Open user profiles for adaptive news systems: help or harm? In the 16th international conference on World Wide Web, WWW '07, 11-20. • Ahn, Jae-wook, Peter Brusilovsky, Daqing He, Jonathan Grady, and Qi Li.( 2008.) Personalized Web Exploration with Task Models."In the 17th international conference on World Wide Web, WWW '08, 1-10. Beijing, China:. • Ahn, J. and Brusilovsky, P. (2013) Adaptive visualization for exploratory information retrieval. Information Processing and Management 49 (5), 1139–1164. • Ahn, J., Brusilovsky, P., and Han, S. (2015) Personalized Search: Reconsidering the Value of Open User Models. In: Proceedings of Proceedings of the 20th International Conference on Intelligent User Interfaces, Atlanta, Georgia, USA, March 29-April 1, 2015, ACM, pp. 202-212 • Verbert, K., Parra-Santander, D., and Brusilovsky, P. (2016) Agents Vs. Users: Visual Recommendation of Research Talks with Multiple Dimension of Relevance. ACM Transactions on Interactive Intelligent Systems 6 (2), Article No. 11 • Parra, D. and Brusilovsky, P. (2015) User-controllable personalization: A case study with SetFusion. International Journal of Human-Computer Studies 78, 43–67. • Cardoso, Bruno, Gayane Sedrakyan, Francisco Gutiérrez, Denis Parra, Peter Brusilovsky, and Katrien Verbert (2019). IntersectionExplorer, a multi-perspective approach for exploring recommendations, International Journal of Human-Computer Studies, 121: 73-92. • Verbert, K., Parra-Santander, D., Brusilovsky, P., Cardoso, B., and Wongchokprasitti, C. (2017) Supporting Conference Attendees with Visual Decision Making Interfaces. In: Companion of the 22nd International Conference on Intelligent User Interfaces (IUI '17), Limassol, Cyprus, ACM. • Tsai, Chun-Hua and Peter Brusilovsky (2018) Beyond the Ranked List: User-Driven Exploration and Diversification of Social Recommendation. In 23rd International Conference on Intelligent User Interfaces, 239--50. Tokyo, Japan: ACM. • Rahdari, B., Brusilovsky, P., and Babichenko, D. (2020) Personalizing Information Exploration with an Open User Model. In: Proceedings of 31st ACM Conference on Hypertext and Social Media, July 13-15, 2020, ACM, pp. 167-176. 31