Measuring Diversity in algorithmic recommendation sets
Download as PPTX, PDF1 like426 views
The document discusses measuring diversity in news recommender systems. It examines which systems produce the largest and smallest amounts of pluralism across different dimensions of diversity. The researchers analyze 1000 simulated recommendation sets from different algorithmic settings based on data from a Dutch newspaper. They measure diversity using entropy in features and distances between features in multidimensional spaces. The computational approach involves constructing feature and document distance matrices to assess diversity in recommendations. Future work will apply the methods to more complex datasets and include feedback loops.
Measuring Diversity in algorithmic recommendation sets
- 1. Measuring diversity Judith Möller, Damian Trilling, Natali Helberger, Bram van Es University of Amsterdam
- 2. Does algorithmic recommendation lead to less diversity? 2
- 3. Research question Which news recommender system produces the largest and smallest amount of pluralism on the different dimensions of diversity? 3
- 4. Methods Data: 1000 Simulated recommendation sets in different algorithmic settings based of data on Volkskrant.nl published between 19.9.2016 and 26.9 2016, N=21,973 articles • Benchmark: Recommendation by the human editor 4
- 5. 5
- 6. Measuring diversity? Possible: Entropy in features (topic, named entities….) Much better: Distance of features in a space
- 7. 7
- 10. Computational approach • Feature distance matrix: Compute Euclidian distances between the occurring feature values (topics, polarity, subjectivity, etc.) • Document matrices: Apply document weights to the feature values • Document distance matrix: Collect the Frobenius norms of the differences between the document matrices • Apply document distance matrix to recommendations
- 11. Work in progress…. Feature space
- 12. Next steps Method assumes spaces of topics Apply to more complex and diverse data sets Future work needs to also include feedback loops Development of DART (Diversity Assessment and Recommendation toolkit)
- 13. Results
- 14. Thanks Want to know more? j.moller@uva.nl; @judith_moeller Personalised-communication.net
Editor's Notes
- #8: This is the result of a topic model, more in particular an LDA (Latent Dirichlet Allocation). We then used multi-dimensional scaling to place the resulting topics in a two-dimensional space (HIER DUS AANGEVEN DAT ALLEMAAL NIET ZO ZEER UITMAAKT, ANDERE METHODE KAN OOK, BELANGRIJKSTE PUNTEN: JE MOET IN STAAT ZIJN OM ___AFSTANDEN____ TE BEPALEN, IN ONS GEVAL Euclidean distance
- #12: Red circle is what was recommended to one specific user, How to optimize the recommender, mover users towards norm (0), and you want to make the circle larger