Agents vs Users: Visual Recommendation of Research Talks with Multiple Dimension of Relevance
1 like387 views
The document discusses the challenges and prospects of personalized social tagging systems in providing multiple relevance dimensions for users. It presents a new approach that visualizes recommendations as agents alongside user interactions, enhancing exploration and control. Evaluation studies indicate that considering multiple entities significantly improves the effectiveness of recommendation systems.
Agents vs Users: Visual Recommendation of Research Talks with Multiple Dimension of Relevance
- 1. Agents vs Users: Visual Recommendation of Research Talks with Multiple Dimension of Relevance Katrien Verbert – KU Leuven – @katrien_v Denis Parra – PUC Chile - @denisparra Peter Brusilovsky – University of Pittsburgh - @peterpaws Presented at IUI 2017
- 2. Motivation • multiple relevance prospects in personalized social tagging systems o community relevance prospects o social relevance prospect o content relevance prospect • existing personalized social systems o do not allow to explore and combine multiple relevance prospects o only one prospect can be explored at a given time 2
- 3. Also recommendations personalized relevance prospect 3
- 4. Shortcomings recommender systems • cold-start issues • difficult to explain the rationale behind recommendations • user control has positive effect o several ways to control elements o which are most effective for the user experience? 4
- 5. 5 He, C., Parra, D., & Verbert, K. (2016). Interactive recommender systems: A survey of the state of the art and future research challenges and opportunities. Expert Systems with Applications, 56, 9-27.
- 6. John O'Donovan, Barry Smyth, Brynjar Gretarsson, Svetlin Bostandjiev, and Tobias Höllerer. 2008. PeerChooser: visual interactive recommendation. CHI '08 Related work: PeerChooser 6
- 7. Related work: Smallworlds Gretarsson, B., O'Donovan, J., Bostandjiev, S., Hall, C. and Höllerer, T. SmallWorlds: Visualizing Social Recommendations. Comput. Graph. Forum, 29, 3 (2010), 833-842. 7
- 9. Contributions • new approach to support exploration, transparency and controllability o recommender systems are shown as agents o in parallel to real users and tags o users can interrelate entities to find items • evaluation study that assesses o effectiveness o probability of item selection 9
- 10. Conference Navigator The studies were conducted using Conference Navigator 3 http://halley.exp.sis.pitt.edu/cn3/10
- 11. TalkExplorer - I 11 Entities Tags, Recommender Agents, Users
- 12. TalkExplorer - II Recommender Recommender Cluster with intersect ion of entities Cluster (of talks) associated to only one entity Canvas Area: Intersections of Different Entities User
- 13. TalkExplorer - III 13 Items Talks explored by the user
- 14. Our Assumptions • Items which are relevant in more that one aspect could be more valuable to the users • Displaying multiple aspects of relevance visually is important for the users in the process of item’s exploration 14
- 15. User study 1 • Setup o supervised user study o 21 participants at UMAP 2012 and ACM Hypertext 2012 conferences • Procedure o Tasks • interact with users and their bookmarks • interact with agents • interact with tags o Post-questionnaire 15
- 16. Evaluation • Data collection o recordings of voice and screen using camtasia studio o system logs • Measurements o effectiveness: # bookmarked items / #explorations o yield: : # bookmarked items / sum of items in selection set 16
- 17. Effectiveness results Summary explored actions and their effectiveness Effectiveness = # bookmarked items / #explorations 17
- 18. Summary results Sign. effect. Sign. yield multiple versus one entity 0.003 <0.001 user versus (user + entity) 0.593 <0.001 agent versus (agent + entity) 0.341 <0.001
- 21. User study 2 • Setup o Unsupervised user study o Conducted at LAK 2012 and ECTEL 2013 (18 users) o Subjects familiar with visualizations, but not much with RecSys • Procedure o Users were left free to explore the interface. o Interactions were logged o Post-questionnaire 21
- 22. Summary results 22 Sign. effect. multiple versus one entity <0.001 user versus (user + entity) 0.3682 agent versus (agent + entity) 0.4426
- 23. Summary results Sign. effectiveness ranked list versus agents 0.7008 ranked list versus (agent + entity) <0.001 ranked list versus multiple entities 0.0012 23
- 24. Results of Studies 1 & 2 • Effectiveness increases with intersections of more entities • Effectiveness wasn’t affected in the field study (study 2) • … but exploration distribution was affected Average effectiveness Total number of explorations 24
- 25. Drawback: visualizing intersections Clustermap Venn diagram Venn diagram: more natural way to visualize intersections 25 Verbert, K., Parra, D., Brusilovksy, P. (2014). The effect of different set-based visualizations on user exploration of recommendations. In : IntRS@RecSys, 2014 (pp. 37-44).
- 26. IntersectionExplorer (IEx) 26 Verbert, K., Seipp, K., He, C., Parra, D., Wongchokprasitti, C., Brusilovsky, P. (2016). Scalable exploration of relevance prospects to support decision making. In : IIntRS@RecSys. 2016 (pp. 28-35). CEUR-WS.
- 28. Feedback very welcome! Please participate in a short user study: http://halley.exp.sis.pitt.edu/cn3/iestudy3.php?conferenceID= 148 28
Editor's Notes
- #3: Our work has been motivated by the presence of multiple relevance prospects in modern social tagging systems. An important feature pioneered by social tagging systems and later used in other kinds of social systems is the ability to explore different community relevance prospects by examining items bookmarked by a specific user or items associated by various users with a specific tag. Items bookmarked by a specific user offer a social relevance prospect: if this user is known and trustable or appears to be like-minded (bookmarked a number of items known as interesting) a collection of his or her bookmarks is perceived as an interesting and relevant set that is worth to explore for more useful items. Similarly, items marked by a specific tag offer a content relevance prospect. Items related to a tag of interest or a tag that was used to mark many known interesting items are also perceived as potentially relevant and worth to explore. In a social tagging system extended with a personalized recommender engine
- #7: Related work: peerchooser John O&apos;Donovan, Barry Smyth, Brynjar Gretarsson, Svetlin Bostandjiev, and Tobias Höllerer. 2008. PeerChooser: visual interactive recommendation. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI &apos;08).
- #25: - In field study, users explore mainly clusters connected to one entity Users don’t even get to explore interactions between 4 items enabling users to explore interrelationships between prospects increases probability of finding a relevant item interrelating tags with other entities increase their effectiveness significantly