![Most recent related work
"Rational explanations are only
effective on users that report being
very unfamiliar with a task" [1]
18
[1] Schaffer, J., Playa Vista, C. A., O’Donovan, J., Michaelis, J., Raglin, A., & Höllerer, T. (2019, March). I can
do better than your AI: expertise and explanations. In Proceedings of the 24th International Conference
on Intelligent User Interfaces (pp. 240-251). ACM.
[2] Dodge, J., Liao, Q. V., Zhang, Y., Bellamy, R. K., & Dugan, C. (2019). Explaining Models: An Empirical
Study of How Explanations Impact Fairness Judgment. arXiv preprint arXiv:1901.07694.
"We highlight that there is no one-size-fits-all
solution for effective explanations-it depends on
the kind of fairness issues and user profiles" [2]](https://image.slidesharecdn.com/iui2019-190905120830/85/To-explain-or-not-to-explain-18-320.jpg)
To explain or not to explain
- 1. To explain or not to explain
- 2. To explain or not to explain The effects of personal characteristics when explaining music recommendations. 2
- 3. Martijn Millecamp* Nyi Nyi Htun* Katrien Verbert*Cristina Conati** 3 * ** Augment
- 4. Why? 4
- 5. 5
- 6. 6
- 7. Problem? 7
- 8. 8 ?
- 9. Solution 9
- 10. Scrutability Tintarev, N., & Masthoff, J. (2007, April). A survey of explanations in recommender systems. In 2007 IEEE 23rd international conference on data engineering workshop (pp. 801-810). IEEE. 10
- 11. PeerChooser O'Donovan, J., Smyth, B., Gretarsson, B., Bostandjiev, S., & Höllerer, T. (2008, April). PeerChooser: visual interactive recommendation. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (pp. 1085-1088). ACM. 11
- 12. SmallWorlds 12 Gretarsson, B., O'Donovan, J., Bostandjiev, S., Hall, C., & Höllerer, T. (2010, June). Smallworlds: visualizing social recommendations. In Computer Graphics Forum (Vol. 29, No. 3, pp. 833-842). Oxford, UK: Blackwell Publishing Ltd.
- 13. Beyond the ranked list Tsai, C. H., & Brusilovsky, P. (2018, March). Beyond the ranked list: User-driven exploration and diversification of social recommendation. In 23rd International Conference on Intelligent User Interfaces (pp. 239-250). ACM. 13
- 14. SetFusion 14 Parra, D., Brusilovsky, P., 2015. User-controllable personalization: A case study with setfusion.
- 15. Intersection Explorer 15 Verbert, K., Parra, D., Brusilovsky, P., & Duval, E. (2013, March). Visualizing recommendations to support exploration, transparency and controllability. In Proceedings of the 2013 international conference on Intelligent user interfaces (pp. 351-362). ACM.
- 16. TasteWeights 16 Bostandjiev, S., O'Donovan, J., & Höllerer, T. (2012, September). Tasteweights: a visual interactive hybrid recommender system
- 17. 17 Solution?
- 18. Most recent related work "Rational explanations are only effective on users that report being very unfamiliar with a task" [1] 18 [1] Schaffer, J., Playa Vista, C. A., O’Donovan, J., Michaelis, J., Raglin, A., & Höllerer, T. (2019, March). I can do better than your AI: expertise and explanations. In Proceedings of the 24th International Conference on Intelligent User Interfaces (pp. 240-251). ACM. [2] Dodge, J., Liao, Q. V., Zhang, Y., Bellamy, R. K., & Dugan, C. (2019). Explaining Models: An Empirical Study of How Explanations Impact Fairness Judgment. arXiv preprint arXiv:1901.07694. "We highlight that there is no one-size-fits-all solution for effective explanations-it depends on the kind of fairness issues and user profiles" [2]
- 19. User-centered evaluation framework 19 Knijnenburg, B. P., Willemsen, M. C., Gantner, Z., Soncu, H., & Newell, C. (2012). Explaining the user experience of recommender systems. User Modeling and User-Adapted Interaction, 22(4-5), 441-504.
- 20. ValueChart 20 Conati, C., Carenini, G., Hoque, E., Steichen, B., & Toker, D. (2014, June). Evaluating the impact of user characteristics and different layouts on an interactive visualization for decision making. In Computer Graphics Forum (Vol. 33, No. 3, pp. 371-380).
- 21. ScaViz 21 Jin, Y., Tintarev, N., & Verbert, K. (2018, September). Effects of personal characteristics on music recommender systems with different levels of controllability. In Proceedings of the 12th ACM Conference on Recommender Systems (pp. 13-21). ACM.
- 22. Argumentation-Based Explanations 22 Naveed, S., Donkers, T., & Ziegler, J. (2018, July). Argumentation-Based Explanations in Recommender Systems: Conceptual Framework and Empirical Results. In Adjunct Publication of the 26th Conference on User Modeling, Adaptation and Personalization (pp. 293-298). ACM.
- 23. Combination? 23
- 25. 1. How do personal characteristics impact user perception of the system when recommendations are explained? 2. How do personal characteristics impact user interaction with the system when recommendations are explained? 25
- 26. Interface 26
- 29. Personal Characteristics ◦ Locus of control ▫ "The extent to which people believe the have power over events in their lives" Fourier ◦ Need for cognition ▫ "Tendency for an individual to engage in, and enjoy effortful cognitive activities" Cacioppo ◦ Visualisation Literacy ▫ "The ability to interpret and to make meaning from information presented in the form of images and graphs" Boy ◦ Visual Working memory ▫ "The part of our cognitive system that is responsible for short-term holding of information for further processing" Corsi ◦ Musical Experience ▫ "The construct that can refer to musical skills, expertise, achievements, and related behaviours across a range of facets" Gold-MSI 29
- 31. Participants ◦ N = 71) ◦ Amazon Mechanical Turk ◦ $2 31 PC Mean SD Loc 12.099 4.58 NFC 62.845 14.96 VisLit -0.073 0.977 VWM 5.93 1.477 MS 67.673 20.051
- 32. Procedure ◦ Pilot study ◦ N = 20 ◦ Playlist of 15 songs ◦ Recommend 10 songs 32
- 33. Procedure 33
- 34. Measurements perception ◦ Recommender effectiveness ▫ The songs recommended to me match my interest ▫ The recommender helped to find good songs for my playlist ◦ Good understanding ▫ I understood why the songs were recommended to me. ▫ The information provided for the recommended songs is sufficient to make a decision ▫ The songs recommended to me had similar attributes to my preference ◦ Trust ▫ I trust the system to suggest good songs ◦ Novelty ▫ The recommender system helped me discover new songs ◦ Use intention ▫ I will use the system again ◦ Satisfaction ▫ Overall, I am satisfied with the recommended system ◦ Confidence ▫ I am confident about the playlist I have created 34
- 35. Measurements interaction ◦ Nb_slider ▫ The number of times the participant used a slider ◦ precision ▫ Songs liked / songs recommended ◦ Nb_play ▫ Total number of songs played ◦ Nb_why ▫ Number of explanations clicked open ◦ Duration_why ▫ The amount of time the explanation is open 35
- 36. 36 So what? To explain or not to explain?
- 37. Impact on user perception 37 X2 (1) =8.73, p=0.003 Cai, C. J., Jongejan, J., & Holbrook, J. (2019, March). The effects of example-based explanations in a machine learning interface. In Proceedings of the 24th International Conference on Intelligent User Interfaces (pp. 258-262). ACM.
- 38. Impact on user interaction 38 t = -2.975, p=0.00592
- 39. Impact on user interaction 39 t = 2.052, p=0.04408
- 40. Impact on user interaction 40 t = -2.795, p=0.00592
- 41. Impact on user perception 41 t = -2.755, p=0.00757
- 42. Thematic analysis 42 Explanations Musical attributes Novelty Quality of recommendations
- 43. Thematic analysis 43 Explanations Low NFC High NFC
- 44. Thematic analysis 44 Musical attributes Low NFC High NFC
- 45. Thematic analysis 45 Novelty Low MS High MS
- 46. Thematic analysis 46 Quality of recommendations
- 47. 47
- 48. Guidelines 48
- 49. Guidelines 49