SlideShare a Scribd company logo

Natural Language Justifications for Recommender Systems Exploiting Text Summarization and Sentiment Analysis

Cataldo Musto, profile picture
Cataldo Musto

The document describes a method for generating natural language justifications for recommender systems using text summarization and sentiment analysis techniques. It discusses prior approaches using descriptive properties or review-based features, and proposes a new approach that exploits automatic text summarization of user reviews. The proposed method involves extracting aspects from reviews, ranking aspects based on frequency, sentiment, and importance, and generating a summary justification using a centroid-based text summarization algorithm on filtered sentences from reviews. The goal is to provide a higher-quality justification by summarizing relevant information from multiple reviews.

Technology
Related topics:
Intelligenza Artificiale • Sentiment Analysis Methods
1 of 49
Downloaded 14 times
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
@cataldomusto cataldo.musto@uniba.it Natural Language Justifications for Recommender Systems Exploiting Text Summarization and Sentiment Analysis CATALDO MUSTO, GAETANO ROSSIELLO, MARCO DE GEMMIS, PASQUALE LOPS AND GIOVANNI SEMERARO UNIVERSITÀ DEGLI STUDI DI BARI ALDO MORO - ITALY
Recommender Systems Cataldo Musto, Gaetano Rossiello, Marco de Gemmis, Pasquale Lops, Giovanni Semeraro. Natural Language Justifications for Recommender Systems Exploiting Text Summarization and Sentiment Analysis. AI*IA 2019, Rende, November 22, 2019 2
The Explanation Problem Recommendation I suggest you… 3Cataldo Musto, Gaetano Rossiello, Marco de Gemmis, Pasquale Lops, Giovanni Semeraro. Natural Language Justifications for Recommender Systems Exploiting Text Summarization and Sentiment Analysis. AI*IA 2019, Rende, November 22, 2019
The Explanation Problem Recommendation 4Cataldo Musto, Gaetano Rossiello, Marco de Gemmis, Pasquale Lops, Giovanni Semeraro. Natural Language Justifications for Recommender Systems Exploiting Text Summarization and Sentiment Analysis. AI*IA 2019, Rende, November 22, 2019
Early approaches[*]: descriptive properties Recommendation I suggest you The Ring because you often like movies with Naomi Watts as 21 grams and Mulholland Drive. Furthermore, you like films about ghosts such as The Sixth Sense. 5Cataldo Musto, Gaetano Rossiello, Marco de Gemmis, Pasquale Lops, Giovanni Semeraro. Natural Language Justifications for Recommender Systems Exploiting Text Summarization and Sentiment Analysis. AI*IA 2019, Rende, November 22, 2019 [*] Cataldo Musto, Fedelucio Narducci, Pasquale Lops, Marco de Gemmis, Giovanni Semeraro: ExpLOD: A Framework for Explaining Recommendations based on the Linked Open Data Cloud. RecSys 2016: 151-154
More recently[*]: review-based features I recommend you The Ring because people who liked the movie think that it delivers some bone-chilling terror. Moreover, people liked The Ring since the casting is pretty good. 6Cataldo Musto, Gaetano Rossiello, Marco de Gemmis, Pasquale Lops, Giovanni Semeraro. Natural Language Justifications for Recommender Systems Exploiting Text Summarization and Sentiment Analysis. AI*IA 2019, Rende, November 22, 2019 [*] Cataldo Musto, Pasquale Lops, Marco de Gemmis, Giovanni Semeraro: Justifying Recommendations through Aspect-based Sentiment Analysis of Users Reviews. UMAP 2019: 4-12
More recently: review-based features To identify relevant and distinguishing characteristics of the recommended item by mining users’ reviews 7Cataldo Musto, Gaetano Rossiello, Marco de Gemmis, Pasquale Lops, Giovanni Semeraro. Natural Language Justifications for Recommender Systems Exploiting Text Summarization and Sentiment Analysis. AI*IA 2019, Rende, November 22, 2019
Why should we use reviews? Intense thriller Pretty good casting Well-plotted investigation Impressive horror ...... To identify relevant and distinguishing characteristics of the recommended item by mining users’ reviews 8Cataldo Musto, Gaetano Rossiello, Marco de Gemmis, Pasquale Lops, Giovanni Semeraro. Natural Language Justifications for Recommender Systems Exploiting Text Summarization and Sentiment Analysis. AI*IA 2019, Rende, November 22, 2019
Why should we use reviews? To identify relevant and distinguishing characteristics of the recommended item by mining users’ reviews 9Cataldo Musto, Gaetano Rossiello, Marco de Gemmis, Pasquale Lops, Giovanni Semeraro. Natural Language Justifications for Recommender Systems Exploiting Text Summarization and Sentiment Analysis. AI*IA 2019, Rende, November 22, 2019 I recommend you The Ring because people who liked the movie think that it delivers some bone-chilling terror. Moreover, people liked The Ring since the casting is pretty good.
Cataldo Musto. Advances in Content-based Recommender Systems – Explanation Strategies. ACM Summer School on Recommender Systems 2019, Gothenburg (Sweden), September 13 2019 Our first methodology has two main weaknesses • Very naïve strategy for detecting more relevant aspects • Very static template-based to generate natural language explanations 10 Why do we need another approach?
Review-based Explanations exploiting Automatic Text Summarization In this talk Cataldo Musto, Gaetano Rossiello, Marco de Gemmis, Pasquale Lops, Giovanni Semeraro: Combining Text Summarization and Aspect-based Sentiment Analysis of Users’ Reviews to Justify Recommendations. ACM RecSys 2019, pp. 383-387 11Cataldo Musto, Gaetano Rossiello, Marco de Gemmis, Pasquale Lops, Giovanni Semeraro. Natural Language Justifications for Recommender Systems Exploiting Text Summarization and Sentiment Analysis. AI*IA 2019, Rende, November 22, 2019
To exploit automatic text summarization techniques to build an higher-quality justifications. Intuition 12Cataldo Musto, Gaetano Rossiello, Marco de Gemmis, Pasquale Lops, Giovanni Semeraro. Natural Language Justifications for Recommender Systems Exploiting Text Summarization and Sentiment Analysis. AI*IA 2019, Rende, November 22, 2019
To exploit automatic text summarization techniques to build an higher-quality justifications. We conceive our justification as a summary of the information conveyed by all the available reviews. Intuition 13Cataldo Musto, Gaetano Rossiello, Marco de Gemmis, Pasquale Lops, Giovanni Semeraro. Natural Language Justifications for Recommender Systems Exploiting Text Summarization and Sentiment Analysis. AI*IA 2019, Rende, November 22, 2019
Workflow 14Cataldo Musto, Gaetano Rossiello, Marco de Gemmis, Pasquale Lops, Giovanni Semeraro. Natural Language Justifications for Recommender Systems Exploiting Text Summarization and Sentiment Analysis. AI*IA 2019, Rende, November 22, 2019
Workflow 15Cataldo Musto, Gaetano Rossiello, Marco de Gemmis, Pasquale Lops, Giovanni Semeraro. Natural Language Justifications for Recommender Systems Exploiting Text Summarization and Sentiment Analysis. AI*IA 2019, Rende, November 22, 2019
Aspect Extraction Goal: to identify the aspects that are discussed when people talk about the item 16 reviews aspects Input: reviews of the item i R = {ri1, ri2 … rin} Output: aspects A = {ai1, ai2 … aik} Cataldo Musto, Gaetano Rossiello, Marco de Gemmis, Pasquale Lops, Giovanni Semeraro. Natural Language Justifications for Recommender Systems Exploiting Text Summarization and Sentiment Analysis. AI*IA 2019, Rende, November 22, 2019
Aspect Extraction Statistical approach based on the Kullback-Leibler (KL) Divergence Measures the difference between the distribution of a term in a generic corpus (e.g. BNC) and its distribution in a domain corpus (e.g. movie reviews) 17Cataldo Musto, Gaetano Rossiello, Marco de Gemmis, Pasquale Lops, Giovanni Semeraro. Natural Language Justifications for Recommender Systems Exploiting Text Summarization and Sentiment Analysis. AI*IA 2019, Rende, November 22, 2019
Aspect Extraction Measures the difference between the distribution of a term in a generic corpus (e.g. BNC) and its distribution in a domain corpus (e.g. movie reviews) Insight: the higher the divergence, the higher the importance of the term in the domain t = term ca = corpus A cb = corpus B Statistical approach based on the Kullback-Leibler (KL) Divergence 18Cataldo Musto, Gaetano Rossiello, Marco de Gemmis, Pasquale Lops, Giovanni Semeraro. Natural Language Justifications for Recommender Systems Exploiting Text Summarization and Sentiment Analysis. AI*IA 2019, Rende, November 22, 2019
Aspect Extraction Measures the difference between the distribution of a term in a generic corpus (e.g. BNC) and its distribution in a domain corpus (e.g. movie reviews) Insight: the higher the divergence, the higher the importance of the term in the domain KL(cast, BNC, movie-reviews) >> 0 KL(actor, BNC, movie-reviews) > 0 KL(city, BNC, movie-reviews) ~ 0 KL(woman, BNC, movie-reviews) ~ 0 We label as ‘aspects’ the nouns whose KL-divergence is higher than zero Statistical approach based on the Kullback-Leibler (KL) Divergence 19Cataldo Musto, Gaetano Rossiello, Marco de Gemmis, Pasquale Lops, Giovanni Semeraro. Natural Language Justifications for Recommender Systems Exploiting Text Summarization and Sentiment Analysis. AI*IA 2019, Rende, November 22, 2019
Aspect Extraction Measures the difference between the distribution of a term in a generic corpus (e.g. BNC) and its distribution in a domain corpus (e.g. movie reviews) Insight: the higher the divergence, the higher the importance of the term in the domain KL(cast, BNC, movie-reviews) >> 0 YES KL(actor, BNC, movie-reviews) > 0 YES KL(city, BNC, movie-reviews) ~ 0 NO KL(woman, BNC, movie-reviews) ~ 0 NO Statistical approach based on the Kullback-Leibler (KL) Divergence 20Cataldo Musto, Gaetano Rossiello, Marco de Gemmis, Pasquale Lops, Giovanni Semeraro. Natural Language Justifications for Recommender Systems Exploiting Text Summarization and Sentiment Analysis. AI*IA 2019, Rende, November 22, 2019
Aspect Ranking 21Cataldo Musto, Gaetano Rossiello, Marco de Gemmis, Pasquale Lops, Giovanni Semeraro. Natural Language Justifications for Recommender Systems Exploiting Text Summarization and Sentiment Analysis. AI*IA 2019, Rende, November 22, 2019
Aspect Ranking aspects top-k aspects Input: aspects A = {ai1, ai2 … aim} Output: top-k aspects A = {ai1, ai2 … aik} 22Cataldo Musto, Gaetano Rossiello, Marco de Gemmis, Pasquale Lops, Giovanni Semeraro. Natural Language Justifications for Recommender Systems Exploiting Text Summarization and Sentiment Analysis. AI*IA 2019, Rende, November 22, 2019
Aspect Ranking Goal: to identify distinguishing aspects that are discussed with a positive sentiment when people talk about the item 23Cataldo Musto, Gaetano Rossiello, Marco de Gemmis, Pasquale Lops, Giovanni Semeraro. Natural Language Justifications for Recommender Systems Exploiting Text Summarization and Sentiment Analysis. AI*IA 2019, Rende, November 22, 2019
Aspect Ranking Goal: to identify distinguishing aspects that are discussed with a positive sentiment when people talk about the item 24 How many times aspect ‘a’ appears in the reviews of item ‘i’ (frequency of the aspect) Cataldo Musto, Gaetano Rossiello, Marco de Gemmis, Pasquale Lops, Giovanni Semeraro. Natural Language Justifications for Recommender Systems Exploiting Text Summarization and Sentiment Analysis. AI*IA 2019, Rende, November 22, 2019
Aspect Ranking Goal: to identify distinguishing aspects that are discussed with a positive sentiment when people talk about the item 25 How positive is the opinion of the users when they talk about aspect ‘a’ (opinion towards the aspect) Cataldo Musto, Gaetano Rossiello, Marco de Gemmis, Pasquale Lops, Giovanni Semeraro. Natural Language Justifications for Recommender Systems Exploiting Text Summarization and Sentiment Analysis. AI*IA 2019, Rende, November 22, 2019
Aspect Ranking Goal: to identify distinguishing aspects that are discussed with a positive sentiment when people talk about the item 26 How distinguishing is the aspect ‘a’ (KL-divergence score) Cataldo Musto, Gaetano Rossiello, Marco de Gemmis, Pasquale Lops, Giovanni Semeraro. Natural Language Justifications for Recommender Systems Exploiting Text Summarization and Sentiment Analysis. AI*IA 2019, Rende, November 22, 2019
Aspect Ranking Goal: to identify distinguishing aspects that are discussed with a positive sentiment when people talk about the item 27 Intuition: our formula gives an higher score to the aspects that are frequently mentioned in the reviews with a positive sentiment. Moreover, it also rewards less popular aspects (higher KL-DIV). Cataldo Musto, Gaetano Rossiello, Marco de Gemmis, Pasquale Lops, Giovanni Semeraro. Natural Language Justifications for Recommender Systems Exploiting Text Summarization and Sentiment Analysis. AI*IA 2019, Rende, November 22, 2019
Generation 28Cataldo Musto, Gaetano Rossiello, Marco de Gemmis, Pasquale Lops, Giovanni Semeraro. Natural Language Justifications for Recommender Systems Exploiting Text Summarization and Sentiment Analysis. AI*IA 2019, Rende, November 22, 2019
Generation Intuition: we conceive our justification as a summary of the information conveyed by all the available reviews 29Cataldo Musto, Gaetano Rossiello, Marco de Gemmis, Pasquale Lops, Giovanni Semeraro. Natural Language Justifications for Recommender Systems Exploiting Text Summarization and Sentiment Analysis. AI*IA 2019, Rende, November 22, 2019
Generation Intuition: we conceive our justification as a summary of the information conveyed by all the available reviews Approach: we exploited a centroid-based method for automatic text summarization. Very good performance in multi-document summarization scenarios. Assumption: each review can be considered as ‘document’ thus the corpus of the reviews can be used to feed the algorithm 30Cataldo Musto, Gaetano Rossiello, Marco de Gemmis, Pasquale Lops, Giovanni Semeraro. Natural Language Justifications for Recommender Systems Exploiting Text Summarization and Sentiment Analysis. AI*IA 2019, Rende, November 22, 2019
Generation Generation process is in turn split into two steps • Sentence Filtering • Text Summarization Sentence Filtering is used to feed the summarization algorithm with compliant sentences. We selected sentences that matched the following criterions: 31Cataldo Musto, Gaetano Rossiello, Marco de Gemmis, Pasquale Lops, Giovanni Semeraro. Natural Language Justifications for Recommender Systems Exploiting Text Summarization and Sentiment Analysis. AI*IA 2019, Rende, November 22, 2019
Generation Generation process is in turn split into two steps • Sentence Filtering • Text Summarization Sentence Filtering is used to feed the summarization algorithm with compliant sentences. We selected sentences that matched the following criterions: • The sentence contains a main aspect • The sentence is longer than 5 tokens • The sentence expresses a positive sentiment • The sentence does not contain first-person personal or possessive pronouns 32Cataldo Musto, Gaetano Rossiello, Marco de Gemmis, Pasquale Lops, Giovanni Semeraro. Natural Language Justifications for Recommender Systems Exploiting Text Summarization and Sentiment Analysis. AI*IA 2019, Rende, November 22, 2019
Generation Text Summarization Algorithm Input: item i, sentences s1…sn, word limit k Output: summary for item i consisting of k words 1. Build a vector space representation for each sentence 2. Merge all the sentences in a pseudo-document that represents the item (centroid vector) 3. Until the word limit is not reached 3.1 Go through the sentences and calculate cosine similarity 3.2 Pick the one with the highest cosine similarity to the centroid (and not that similar to the sentences previously picked) 3.3 Add it to the summary 33Cataldo Musto, Gaetano Rossiello, Marco de Gemmis, Pasquale Lops, Giovanni Semeraro. Natural Language Justifications for Recommender Systems Exploiting Text Summarization and Sentiment Analysis. AI*IA 2019, Rende, November 22, 2019
Generation Text Summarization Algorithm Input: item i, sentences s1…sn, word limit k Output: summary for item i consisting of k words 1. Build a vector space representation for each sentence 2. Merge all the sentences in a pseudo-document that represents the item (centroid vector) 3. Until the word limit is not reached 3.1 Go through the sentences and calculate cosine similarity 3.2 Pick the one with the highest cosine similarity to the centroid (and not that similar to the sentences previously picked) 3.3 Add it to the summary 34Cataldo Musto, Gaetano Rossiello, Marco de Gemmis, Pasquale Lops, Giovanni Semeraro. Natural Language Justifications for Recommender Systems Exploiting Text Summarization and Sentiment Analysis. AI*IA 2019, Rende, November 22, 2019
Generation Text Summarization Algorithm Input: item i, sentences s1…sn, word limit k Output: summary for item i consisting of k words 1. Build a vector space representation for each sentence 2. Merge all the sentences in a pseudo-document that represents the item (centroid vector) 3. Until the word limit is not reached 3.1 Go through the sentences and calculate cosine similarity 3.2 Pick the one with the highest cosine similarity to the centroid (and not that similar to the sentences previously picked) 3.3 Add it to the summary 35Cataldo Musto, Gaetano Rossiello, Marco de Gemmis, Pasquale Lops, Giovanni Semeraro. Natural Language Justifications for Recommender Systems Exploiting Text Summarization and Sentiment Analysis. AI*IA 2019, Rende, November 22, 2019
Generation Text Summarization Algorithm Input: item i, sentences s1…sn, word limit k Output: summary for item i consisting of k words 1. Build a vector space representation for each sentence 2. Merge all the sentences in a pseudo-document that represents the item (centroid vector) 3. Until the word limit is not reached 3.1 Go through the sentences and calculate cosine similarity 3.2 Pick the one with the highest cosine similarity to the centroid (and not that similar to the sentences previously picked) 3.3 Add it to the summary 36Cataldo Musto, Gaetano Rossiello, Marco de Gemmis, Pasquale Lops, Giovanni Semeraro. Natural Language Justifications for Recommender Systems Exploiting Text Summarization and Sentiment Analysis. AI*IA 2019, Rende, November 22, 2019
Generation Text Summarization Algorithm Input: item i, sentences s1…sn, word limit k Output: summary for item i consisting of k words 1. Build a vector space representation for each sentence 2. Merge all the sentences in a pseudo-document that represents the item (centroid vector) 3. Until the word limit is not reached 3.1 Go through the sentences and calculate cosine similarity 3.2 Pick the one with the highest cosine similarity to the centroid (and not that similar to the sentences previously picked) 3.3 Add it to the summary 37Cataldo Musto, Gaetano Rossiello, Marco de Gemmis, Pasquale Lops, Giovanni Semeraro. Natural Language Justifications for Recommender Systems Exploiting Text Summarization and Sentiment Analysis. AI*IA 2019, Rende, November 22, 2019
Generation – Final Output “If you like or love the blood and gore kinds of films, this movie will certainly disappoint you as the focus is on character, story, mood and unique special effects. The Ring is a story about supernatural evil therefore, it is a horror film, done very much in the style of the psychological thriller.” Legenda Red: aspects (k=4) Black: compliant excerpts 38Cataldo Musto, Gaetano Rossiello, Marco de Gemmis, Pasquale Lops, Giovanni Semeraro. Natural Language Justifications for Recommender Systems Exploiting Text Summarization and Sentiment Analysis. AI*IA 2019, Rende, November 22, 2019
Experimental Evaluation Research Question 1 How effective are the justifications generated through the pipeline, on varying of different combinations of the parameters? Research Question 2 How does our justifications perform with respect to a simple review-based explanation? Experimental Design User Study with a Web Application 141 subjects Movie Domain. 300 movies. ~150k reviews. Metrics: Transparency, Engagement, Persuasion, Trust, Effectiveness [^] Parameters: Justification Length (Short=50 words, Long=100) and #Aspects (10 and 30). Between-subjects for Research Question 1, Within-subjects for Research Question 2 [^] Tintarev, N., & Masthoff, J. Designing and evaluating explanations for recommender systems. In Recommender systems handbook. pp. 479-510. Springer, Boston, MA. 2011 39Cataldo Musto, Gaetano Rossiello, Marco de Gemmis, Pasquale Lops, Giovanni Semeraro. Natural Language Justifications for Recommender Systems Exploiting Text Summarization and Sentiment Analysis. AI*IA 2019, Rende, November 22, 2019
Experimental Protocol (Research Question 1) 1. Gathering movie preferences Users rated their favourite movies 40Cataldo Musto, Gaetano Rossiello, Marco de Gemmis, Pasquale Lops, Giovanni Semeraro. Natural Language Justifications for Recommender Systems Exploiting Text Summarization and Sentiment Analysis. AI*IA 2019, Rende, November 22, 2019
Experimental Protocol (Research Question 1) 2. Recommendation is obtained Personalized PageRank as algorithm 1. Gathering movie preferences Users rated their favourite movies 41Cataldo Musto, Gaetano Rossiello, Marco de Gemmis, Pasquale Lops, Giovanni Semeraro. Natural Language Justifications for Recommender Systems Exploiting Text Summarization and Sentiment Analysis. AI*IA 2019, Rende, November 22, 2019
Experimental Protocol (Research Question 1) 2. Recommendation is obtained Personalized PageRank as algorithm 3. Explanation is generated Random Configuration (users not aware) 1. Gathering movie preferences Users rated their favourite movies 42Cataldo Musto, Gaetano Rossiello, Marco de Gemmis, Pasquale Lops, Giovanni Semeraro. Natural Language Justifications for Recommender Systems Exploiting Text Summarization and Sentiment Analysis. AI*IA 2019, Rende, November 22, 2019
Experimental Protocol (Research Question 1) 1. Gathering movie preferences Users rated their favourite movies 2. Recommendation is obtained Personalized PageRank as algorithm 3. Explanation is generated Random Configuration (users not aware) 4. Metrics are calculated Through Questionnaires 43Cataldo Musto, Gaetano Rossiello, Marco de Gemmis, Pasquale Lops, Giovanni Semeraro. Natural Language Justifications for Recommender Systems Exploiting Text Summarization and Sentiment Analysis. AI*IA 2019, Rende, November 22, 2019
Results (Research Question 1) MOVIES Transparency Persuasion Engagement Trust Effectiveness Top-10 Short 2.83 3.06 3.06 2.83 0.89 Top-30 Long 3.16 3.06 2.69 3.19 0.94 Top-10 Short 3.95 3.64 3.37 3.55 0.55 Top-30 Long 3.24 3.18 3.12 3.22 0.38 Finding 1 Long justifications better than short justifications, on average Finding 2 Top-10 aspect provide better explanations than Top-30 aspects Finding 3 Long explanations based on Top-10 aspects lead to the best results 44Cataldo Musto, Gaetano Rossiello, Marco de Gemmis, Pasquale Lops, Giovanni Semeraro. Natural Language Justifications for Recommender Systems Exploiting Text Summarization and Sentiment Analysis. AI*IA 2019, Rende, November 22, 2019
Experimental Protocol (Research Question 2) Review-based Explanation + Summarization Review-based Explanation 45Cataldo Musto, Gaetano Rossiello, Marco de Gemmis, Pasquale Lops, Giovanni Semeraro. Natural Language Justifications for Recommender Systems Exploiting Text Summarization and Sentiment Analysis. AI*IA 2019, Rende, November 22, 2019
Results (Research Question 2) MOVIES Review+ Summary Review- based Indiffer. Transparency 54.5% 40.9% 4.6% Persuasion 77.3% 13.6% 9.1% Engagement 63.6% 27.3% 9.1% Trust 68.2% 4.5% 27.3% Outcome: automatic Text Summarization provides users with the best explanation. Confirmed for all the metrics and both the domains 46Cataldo Musto, Gaetano Rossiello, Marco de Gemmis, Pasquale Lops, Giovanni Semeraro. Natural Language Justifications for Recommender Systems Exploiting Text Summarization and Sentiment Analysis. AI*IA 2019, Rende, November 22, 2019
Recap “If you like or love the blood and gore kinds of films, this movie will certainly disappoint you as the focus is on character, story, mood and unique special effects. The Ring is a story about supernatural evil therefore, it is a horror film, done very much in the style of the psychological thriller.” Review-based explanation using automatic text summarization 47Cataldo Musto, Gaetano Rossiello, Marco de Gemmis, Pasquale Lops, Giovanni Semeraro. Natural Language Justifications for Recommender Systems Exploiting Text Summarization and Sentiment Analysis. AI*IA 2019, Rende, November 22, 2019
Take-home Messages 48 1. 2. Textual features extracted from reviews can be used to generate a natural language justifications supporting a recommendation Cataldo Musto, Gaetano Rossiello, Marco de Gemmis, Pasquale Lops, Giovanni Semeraro. Natural Language Justifications for Recommender Systems Exploiting Text Summarization and Sentiment Analysis. AI*IA 2019, Rende, November 22, 2019 48 Text summarization algorithms are suitable to build a less ‘static’ justifications based on review excerpts 3. Shorter Justifications based on Top-10 aspects obtained the best results
49 Grazie! cataldo.musto@uniba.it @cataldomusto Contacts Cataldo Musto, Gaetano Rossiello, Marco de Gemmis, Pasquale Lops, Giovanni Semeraro. Natural Language Justifications for Recommender Systems Exploiting Text Summarization and Sentiment Analysis. AI*IA 2019, Rende, November 22, 2019

More Related Content

PDF
Hybrid Semantics aware Recommendations Exploiting Knowledge Graph Embeddings
Cataldo Musto
 
PDF
Justifying Recommendations through Aspect-based Sentiment Analysis of Users R...
Cataldo Musto
 
PDF
Explanation Strategies - Advances in Content-based Recommender System
Cataldo Musto
 
PPTX
Exploiting Distributional Semantics Models for Natural Language Context-aware...
GiuseppeSpillo
 
PDF
Exploiting Distributional Semantics Models for Natural Language Context-aware...
Cataldo Musto
 
PDF
Semantics-aware Techniques for Social Media Analysis, User Modeling and Recom...
Cataldo Musto
 
PDF
Semantics-aware Content-based Recommender Systems
Pasquale Lops
 
PPTX
Recsys 2017 -- Understanding How People Use Natural Language to Ask for Recom...
Max Harper
 
Hybrid Semantics aware Recommendations Exploiting Knowledge Graph Embeddings
Cataldo Musto
 
Justifying Recommendations through Aspect-based Sentiment Analysis of Users R...
Cataldo Musto
 
Explanation Strategies - Advances in Content-based Recommender System
Cataldo Musto
 
Exploiting Distributional Semantics Models for Natural Language Context-aware...
GiuseppeSpillo
 
Exploiting Distributional Semantics Models for Natural Language Context-aware...
Cataldo Musto
 
Semantics-aware Techniques for Social Media Analysis, User Modeling and Recom...
Cataldo Musto
 
Semantics-aware Content-based Recommender Systems
Pasquale Lops
 
Recsys 2017 -- Understanding How People Use Natural Language to Ask for Recom...
Max Harper
 

Similar to Natural Language Justifications for Recommender Systems Exploiting Text Summarization and Sentiment Analysis (8)

PDF
Veda Semantics - introduction document
rajatkr
 
PPTX
Presentation on the use of AI tools.pptx
AMAECHICHINEDUM1
 
PDF
Narrative-Driven Recommendation for Casual Leisure Needs
Marijn Koolen
 
PDF
Veda Semantics - introduction document
rajatkr
 
PDF
AUTOMATIC ANALYSIS OF DOCUMENT SENTIMENT
Stephen Faucher
 
PDF
Mining Opinion Features in Customer Reviews
IJCERT JOURNAL
 
PPTX
Abstractive Review Summarization
Cognizant Technology Solutions
 
PDF
Combining Distributional Semantics and Entity Linking for Context-aware Conte...
Cataldo Musto
 
Veda Semantics - introduction document
rajatkr
 
Presentation on the use of AI tools.pptx
AMAECHICHINEDUM1
 
Narrative-Driven Recommendation for Casual Leisure Needs
Marijn Koolen
 
Veda Semantics - introduction document
rajatkr
 
AUTOMATIC ANALYSIS OF DOCUMENT SENTIMENT
Stephen Faucher
 
Mining Opinion Features in Customer Reviews
IJCERT JOURNAL
 
Abstractive Review Summarization
Cognizant Technology Solutions
 
Combining Distributional Semantics and Entity Linking for Context-aware Conte...
Cataldo Musto
 
Ad

More from Cataldo Musto (20)

PDF
MyrrorBot: a Digital Assistant Based on Holistic User Models for Personalize...
Cataldo Musto
 
PDF
Fairness and Popularity Bias in Recommender Systems: an Empirical Evaluation
Cataldo Musto
 
PDF
Intelligenza Artificiale e Social Media - Monitoraggio della Farnesina e La M...
Cataldo Musto
 
PDF
Exploring the Effects of Natural Language Justifications in Food Recommender ...
Cataldo Musto
 
PDF
Towards a Knowledge-aware Food Recommender System Exploiting Holistic User Mo...
Cataldo Musto
 
PDF
Towards Queryable User Profiles: Introducing Conversational Agents in a Platf...
Cataldo Musto
 
PDF
L'IA per l'Empowerment del Cittadino: Hate Map, Myrror, PA Risponde
Cataldo Musto
 
PDF
ExpLOD: un framework per la generazione di spiegazioni per recommender system...
Cataldo Musto
 
PDF
Myrror: una piattaforma per Holistic User Modeling e Quantified Self
Cataldo Musto
 
PDF
Semantic Holistic User Modeling for Personalized Access to Digital Content an...
Cataldo Musto
 
PDF
Holistic User Modeling for Personalized Services in Smart Cities
Cataldo Musto
 
PDF
A Framework for Holistic User Modeling Merging Heterogeneous Digital Footprints
Cataldo Musto
 
PDF
eHealth, mHealth in Otorinolaringoiatria: innovazioni dirompenti o disastrose?
Cataldo Musto
 
PDF
Semantics-aware Recommender Systems Exploiting Linked Open Data and Graph-bas...
Cataldo Musto
 
PDF
Il Linguaggio dell'Odio sui Social Network
Cataldo Musto
 
PDF
Mappare l'Odio - Hate Speech & Social Media
Cataldo Musto
 
PDF
A Multi-Criteria Recommender System Exploiting Aspect-based Sentiment Analysi...
Cataldo Musto
 
PDF
Recommender Systems based on Linked Open Data
Cataldo Musto
 
PDF
A Deep Architecture for Content-based Recommendations Exploiting Recurrent Ne...
Cataldo Musto
 
PDF
Temporal and Holistic User Modeling Workshop - THUM@UMAP 2017
Cataldo Musto
 
MyrrorBot: a Digital Assistant Based on Holistic User Models for Personalize...
Cataldo Musto
 
Fairness and Popularity Bias in Recommender Systems: an Empirical Evaluation
Cataldo Musto
 
Intelligenza Artificiale e Social Media - Monitoraggio della Farnesina e La M...
Cataldo Musto
 
Exploring the Effects of Natural Language Justifications in Food Recommender ...
Cataldo Musto
 
Towards a Knowledge-aware Food Recommender System Exploiting Holistic User Mo...
Cataldo Musto
 
Towards Queryable User Profiles: Introducing Conversational Agents in a Platf...
Cataldo Musto
 
L'IA per l'Empowerment del Cittadino: Hate Map, Myrror, PA Risponde
Cataldo Musto
 
ExpLOD: un framework per la generazione di spiegazioni per recommender system...
Cataldo Musto
 
Myrror: una piattaforma per Holistic User Modeling e Quantified Self
Cataldo Musto
 
Semantic Holistic User Modeling for Personalized Access to Digital Content an...
Cataldo Musto
 
Holistic User Modeling for Personalized Services in Smart Cities
Cataldo Musto
 
A Framework for Holistic User Modeling Merging Heterogeneous Digital Footprints
Cataldo Musto
 
eHealth, mHealth in Otorinolaringoiatria: innovazioni dirompenti o disastrose?
Cataldo Musto
 
Semantics-aware Recommender Systems Exploiting Linked Open Data and Graph-bas...
Cataldo Musto
 
Il Linguaggio dell'Odio sui Social Network
Cataldo Musto
 
Mappare l'Odio - Hate Speech & Social Media
Cataldo Musto
 
A Multi-Criteria Recommender System Exploiting Aspect-based Sentiment Analysi...
Cataldo Musto
 
Recommender Systems based on Linked Open Data
Cataldo Musto
 
A Deep Architecture for Content-based Recommendations Exploiting Recurrent Ne...
Cataldo Musto
 
Temporal and Holistic User Modeling Workshop - THUM@UMAP 2017
Cataldo Musto
 
Ad

Recently uploaded (20)

PDF
Empathic Computing: Creating Shared Understanding
Mark Billinghurst
 
PPT
“AI and Expert System Decision Support & Business Intelligence Systems”
ZubinRadhakrishnan
 
PDF
Architecting across the Boundaries of two Complex Domains - Healthcare & Tech...
Peter Tan
 
PPTX
Effective Security Operations Center (SOC) A Modern, Strategic, and Threat-In...
ReZa AdineH
 
PDF
Network Security Unit 5.pdf for BCA BBA.
Serpent6
 
PDF
Advanced methodologies resolving dimensionality complications for autism neur...
IAESIJAI
 
PDF
Encapsulation theory and applications.pdf
gurumoop
 
PPTX
VMware vSphere Foundation How to Sell Presentation-Ver1.4-2-14-2024.pptx
blackmambaettijean
 
PDF
Dropbox Q2 2025 Financial Results & Investor Presentation
Dropbox
 
PDF
Per capita expenditure prediction using model stacking based on satellite ima...
IAESIJAI
 
PPTX
Programs and apps: productivity, graphics, security and other tools
4mqw9zch22
 
PDF
MIND Revenue Release Quarter 2 2025 Press Release
MIND CTI
 
PDF
The Rise and Fall of 3GPP – Time for a Sabbatical?
3G4G
 
PDF
Electronic commerce courselecture one. Pdf
OmerMohamed64
 
PDF
Encapsulation_ Review paper, used for researhc scholars
gurumoop
 
PDF
Unlocking AI with Model Context Protocol (MCP)
Brian McKeiver
 
PDF
TokAI - TikTok AI Agent : The First AI Application That Analyzes 10,000+ Vira...
SOFTTECHHUB
 
PDF
Building Integrated photovoltaic BIPV_UPV.pdf
SandeepSingh286037
 
PDF
Agricultural_Statistics_at_a_Glance_2022_0.pdf
SandeepSingh286037
 
PPTX
MYSQL Presentation for SQL database connectivity
Swati270511
 
Empathic Computing: Creating Shared Understanding
Mark Billinghurst
 
“AI and Expert System Decision Support & Business Intelligence Systems”
ZubinRadhakrishnan
 
Architecting across the Boundaries of two Complex Domains - Healthcare & Tech...
Peter Tan
 
Effective Security Operations Center (SOC) A Modern, Strategic, and Threat-In...
ReZa AdineH
 
Network Security Unit 5.pdf for BCA BBA.
Serpent6
 
Advanced methodologies resolving dimensionality complications for autism neur...
IAESIJAI
 
Encapsulation theory and applications.pdf
gurumoop
 
VMware vSphere Foundation How to Sell Presentation-Ver1.4-2-14-2024.pptx
blackmambaettijean
 
Dropbox Q2 2025 Financial Results & Investor Presentation
Dropbox
 
Per capita expenditure prediction using model stacking based on satellite ima...
IAESIJAI
 
Programs and apps: productivity, graphics, security and other tools
4mqw9zch22
 
MIND Revenue Release Quarter 2 2025 Press Release
MIND CTI
 
The Rise and Fall of 3GPP – Time for a Sabbatical?
3G4G
 
Electronic commerce courselecture one. Pdf
OmerMohamed64
 
Encapsulation_ Review paper, used for researhc scholars
gurumoop
 
Unlocking AI with Model Context Protocol (MCP)
Brian McKeiver
 
TokAI - TikTok AI Agent : The First AI Application That Analyzes 10,000+ Vira...
SOFTTECHHUB
 
Building Integrated photovoltaic BIPV_UPV.pdf
SandeepSingh286037
 
Agricultural_Statistics_at_a_Glance_2022_0.pdf
SandeepSingh286037
 
MYSQL Presentation for SQL database connectivity
Swati270511
 

Natural Language Justifications for Recommender Systems Exploiting Text Summarization and Sentiment Analysis

  • 1. @cataldomusto cataldo.musto@uniba.it Natural Language Justifications for Recommender Systems Exploiting Text Summarization and Sentiment Analysis CATALDO MUSTO, GAETANO ROSSIELLO, MARCO DE GEMMIS, PASQUALE LOPS AND GIOVANNI SEMERARO UNIVERSITÀ DEGLI STUDI DI BARI ALDO MORO - ITALY
  • 2. Recommender Systems Cataldo Musto, Gaetano Rossiello, Marco de Gemmis, Pasquale Lops, Giovanni Semeraro. Natural Language Justifications for Recommender Systems Exploiting Text Summarization and Sentiment Analysis. AI*IA 2019, Rende, November 22, 2019 2
  • 3. The Explanation Problem Recommendation I suggest you… 3Cataldo Musto, Gaetano Rossiello, Marco de Gemmis, Pasquale Lops, Giovanni Semeraro. Natural Language Justifications for Recommender Systems Exploiting Text Summarization and Sentiment Analysis. AI*IA 2019, Rende, November 22, 2019
  • 4. The Explanation Problem Recommendation 4Cataldo Musto, Gaetano Rossiello, Marco de Gemmis, Pasquale Lops, Giovanni Semeraro. Natural Language Justifications for Recommender Systems Exploiting Text Summarization and Sentiment Analysis. AI*IA 2019, Rende, November 22, 2019
  • 5. Early approaches[*]: descriptive properties Recommendation I suggest you The Ring because you often like movies with Naomi Watts as 21 grams and Mulholland Drive. Furthermore, you like films about ghosts such as The Sixth Sense. 5Cataldo Musto, Gaetano Rossiello, Marco de Gemmis, Pasquale Lops, Giovanni Semeraro. Natural Language Justifications for Recommender Systems Exploiting Text Summarization and Sentiment Analysis. AI*IA 2019, Rende, November 22, 2019 [*] Cataldo Musto, Fedelucio Narducci, Pasquale Lops, Marco de Gemmis, Giovanni Semeraro: ExpLOD: A Framework for Explaining Recommendations based on the Linked Open Data Cloud. RecSys 2016: 151-154
  • 6. More recently[*]: review-based features I recommend you The Ring because people who liked the movie think that it delivers some bone-chilling terror. Moreover, people liked The Ring since the casting is pretty good. 6Cataldo Musto, Gaetano Rossiello, Marco de Gemmis, Pasquale Lops, Giovanni Semeraro. Natural Language Justifications for Recommender Systems Exploiting Text Summarization and Sentiment Analysis. AI*IA 2019, Rende, November 22, 2019 [*] Cataldo Musto, Pasquale Lops, Marco de Gemmis, Giovanni Semeraro: Justifying Recommendations through Aspect-based Sentiment Analysis of Users Reviews. UMAP 2019: 4-12
  • 7. More recently: review-based features To identify relevant and distinguishing characteristics of the recommended item by mining users’ reviews 7Cataldo Musto, Gaetano Rossiello, Marco de Gemmis, Pasquale Lops, Giovanni Semeraro. Natural Language Justifications for Recommender Systems Exploiting Text Summarization and Sentiment Analysis. AI*IA 2019, Rende, November 22, 2019
  • 8. Why should we use reviews? Intense thriller Pretty good casting Well-plotted investigation Impressive horror ...... To identify relevant and distinguishing characteristics of the recommended item by mining users’ reviews 8Cataldo Musto, Gaetano Rossiello, Marco de Gemmis, Pasquale Lops, Giovanni Semeraro. Natural Language Justifications for Recommender Systems Exploiting Text Summarization and Sentiment Analysis. AI*IA 2019, Rende, November 22, 2019
  • 9. Why should we use reviews? To identify relevant and distinguishing characteristics of the recommended item by mining users’ reviews 9Cataldo Musto, Gaetano Rossiello, Marco de Gemmis, Pasquale Lops, Giovanni Semeraro. Natural Language Justifications for Recommender Systems Exploiting Text Summarization and Sentiment Analysis. AI*IA 2019, Rende, November 22, 2019 I recommend you The Ring because people who liked the movie think that it delivers some bone-chilling terror. Moreover, people liked The Ring since the casting is pretty good.
  • 10. Cataldo Musto. Advances in Content-based Recommender Systems – Explanation Strategies. ACM Summer School on Recommender Systems 2019, Gothenburg (Sweden), September 13 2019 Our first methodology has two main weaknesses • Very naïve strategy for detecting more relevant aspects • Very static template-based to generate natural language explanations 10 Why do we need another approach?
  • 11. Review-based Explanations exploiting Automatic Text Summarization In this talk Cataldo Musto, Gaetano Rossiello, Marco de Gemmis, Pasquale Lops, Giovanni Semeraro: Combining Text Summarization and Aspect-based Sentiment Analysis of Users’ Reviews to Justify Recommendations. ACM RecSys 2019, pp. 383-387 11Cataldo Musto, Gaetano Rossiello, Marco de Gemmis, Pasquale Lops, Giovanni Semeraro. Natural Language Justifications for Recommender Systems Exploiting Text Summarization and Sentiment Analysis. AI*IA 2019, Rende, November 22, 2019
  • 12. To exploit automatic text summarization techniques to build an higher-quality justifications. Intuition 12Cataldo Musto, Gaetano Rossiello, Marco de Gemmis, Pasquale Lops, Giovanni Semeraro. Natural Language Justifications for Recommender Systems Exploiting Text Summarization and Sentiment Analysis. AI*IA 2019, Rende, November 22, 2019
  • 13. To exploit automatic text summarization techniques to build an higher-quality justifications. We conceive our justification as a summary of the information conveyed by all the available reviews. Intuition 13Cataldo Musto, Gaetano Rossiello, Marco de Gemmis, Pasquale Lops, Giovanni Semeraro. Natural Language Justifications for Recommender Systems Exploiting Text Summarization and Sentiment Analysis. AI*IA 2019, Rende, November 22, 2019
  • 14. Workflow 14Cataldo Musto, Gaetano Rossiello, Marco de Gemmis, Pasquale Lops, Giovanni Semeraro. Natural Language Justifications for Recommender Systems Exploiting Text Summarization and Sentiment Analysis. AI*IA 2019, Rende, November 22, 2019
  • 15. Workflow 15Cataldo Musto, Gaetano Rossiello, Marco de Gemmis, Pasquale Lops, Giovanni Semeraro. Natural Language Justifications for Recommender Systems Exploiting Text Summarization and Sentiment Analysis. AI*IA 2019, Rende, November 22, 2019
  • 16. Aspect Extraction Goal: to identify the aspects that are discussed when people talk about the item 16 reviews aspects Input: reviews of the item i R = {ri1, ri2 … rin} Output: aspects A = {ai1, ai2 … aik} Cataldo Musto, Gaetano Rossiello, Marco de Gemmis, Pasquale Lops, Giovanni Semeraro. Natural Language Justifications for Recommender Systems Exploiting Text Summarization and Sentiment Analysis. AI*IA 2019, Rende, November 22, 2019
  • 17. Aspect Extraction Statistical approach based on the Kullback-Leibler (KL) Divergence Measures the difference between the distribution of a term in a generic corpus (e.g. BNC) and its distribution in a domain corpus (e.g. movie reviews) 17Cataldo Musto, Gaetano Rossiello, Marco de Gemmis, Pasquale Lops, Giovanni Semeraro. Natural Language Justifications for Recommender Systems Exploiting Text Summarization and Sentiment Analysis. AI*IA 2019, Rende, November 22, 2019
  • 18. Aspect Extraction Measures the difference between the distribution of a term in a generic corpus (e.g. BNC) and its distribution in a domain corpus (e.g. movie reviews) Insight: the higher the divergence, the higher the importance of the term in the domain t = term ca = corpus A cb = corpus B Statistical approach based on the Kullback-Leibler (KL) Divergence 18Cataldo Musto, Gaetano Rossiello, Marco de Gemmis, Pasquale Lops, Giovanni Semeraro. Natural Language Justifications for Recommender Systems Exploiting Text Summarization and Sentiment Analysis. AI*IA 2019, Rende, November 22, 2019
  • 19. Aspect Extraction Measures the difference between the distribution of a term in a generic corpus (e.g. BNC) and its distribution in a domain corpus (e.g. movie reviews) Insight: the higher the divergence, the higher the importance of the term in the domain KL(cast, BNC, movie-reviews) >> 0 KL(actor, BNC, movie-reviews) > 0 KL(city, BNC, movie-reviews) ~ 0 KL(woman, BNC, movie-reviews) ~ 0 We label as ‘aspects’ the nouns whose KL-divergence is higher than zero Statistical approach based on the Kullback-Leibler (KL) Divergence 19Cataldo Musto, Gaetano Rossiello, Marco de Gemmis, Pasquale Lops, Giovanni Semeraro. Natural Language Justifications for Recommender Systems Exploiting Text Summarization and Sentiment Analysis. AI*IA 2019, Rende, November 22, 2019
  • 20. Aspect Extraction Measures the difference between the distribution of a term in a generic corpus (e.g. BNC) and its distribution in a domain corpus (e.g. movie reviews) Insight: the higher the divergence, the higher the importance of the term in the domain KL(cast, BNC, movie-reviews) >> 0 YES KL(actor, BNC, movie-reviews) > 0 YES KL(city, BNC, movie-reviews) ~ 0 NO KL(woman, BNC, movie-reviews) ~ 0 NO Statistical approach based on the Kullback-Leibler (KL) Divergence 20Cataldo Musto, Gaetano Rossiello, Marco de Gemmis, Pasquale Lops, Giovanni Semeraro. Natural Language Justifications for Recommender Systems Exploiting Text Summarization and Sentiment Analysis. AI*IA 2019, Rende, November 22, 2019
  • 21. Aspect Ranking 21Cataldo Musto, Gaetano Rossiello, Marco de Gemmis, Pasquale Lops, Giovanni Semeraro. Natural Language Justifications for Recommender Systems Exploiting Text Summarization and Sentiment Analysis. AI*IA 2019, Rende, November 22, 2019
  • 22. Aspect Ranking aspects top-k aspects Input: aspects A = {ai1, ai2 … aim} Output: top-k aspects A = {ai1, ai2 … aik} 22Cataldo Musto, Gaetano Rossiello, Marco de Gemmis, Pasquale Lops, Giovanni Semeraro. Natural Language Justifications for Recommender Systems Exploiting Text Summarization and Sentiment Analysis. AI*IA 2019, Rende, November 22, 2019
  • 23. Aspect Ranking Goal: to identify distinguishing aspects that are discussed with a positive sentiment when people talk about the item 23Cataldo Musto, Gaetano Rossiello, Marco de Gemmis, Pasquale Lops, Giovanni Semeraro. Natural Language Justifications for Recommender Systems Exploiting Text Summarization and Sentiment Analysis. AI*IA 2019, Rende, November 22, 2019
  • 24. Aspect Ranking Goal: to identify distinguishing aspects that are discussed with a positive sentiment when people talk about the item 24 How many times aspect ‘a’ appears in the reviews of item ‘i’ (frequency of the aspect) Cataldo Musto, Gaetano Rossiello, Marco de Gemmis, Pasquale Lops, Giovanni Semeraro. Natural Language Justifications for Recommender Systems Exploiting Text Summarization and Sentiment Analysis. AI*IA 2019, Rende, November 22, 2019
  • 25. Aspect Ranking Goal: to identify distinguishing aspects that are discussed with a positive sentiment when people talk about the item 25 How positive is the opinion of the users when they talk about aspect ‘a’ (opinion towards the aspect) Cataldo Musto, Gaetano Rossiello, Marco de Gemmis, Pasquale Lops, Giovanni Semeraro. Natural Language Justifications for Recommender Systems Exploiting Text Summarization and Sentiment Analysis. AI*IA 2019, Rende, November 22, 2019
  • 26. Aspect Ranking Goal: to identify distinguishing aspects that are discussed with a positive sentiment when people talk about the item 26 How distinguishing is the aspect ‘a’ (KL-divergence score) Cataldo Musto, Gaetano Rossiello, Marco de Gemmis, Pasquale Lops, Giovanni Semeraro. Natural Language Justifications for Recommender Systems Exploiting Text Summarization and Sentiment Analysis. AI*IA 2019, Rende, November 22, 2019
  • 27. Aspect Ranking Goal: to identify distinguishing aspects that are discussed with a positive sentiment when people talk about the item 27 Intuition: our formula gives an higher score to the aspects that are frequently mentioned in the reviews with a positive sentiment. Moreover, it also rewards less popular aspects (higher KL-DIV). Cataldo Musto, Gaetano Rossiello, Marco de Gemmis, Pasquale Lops, Giovanni Semeraro. Natural Language Justifications for Recommender Systems Exploiting Text Summarization and Sentiment Analysis. AI*IA 2019, Rende, November 22, 2019
  • 28. Generation 28Cataldo Musto, Gaetano Rossiello, Marco de Gemmis, Pasquale Lops, Giovanni Semeraro. Natural Language Justifications for Recommender Systems Exploiting Text Summarization and Sentiment Analysis. AI*IA 2019, Rende, November 22, 2019
  • 29. Generation Intuition: we conceive our justification as a summary of the information conveyed by all the available reviews 29Cataldo Musto, Gaetano Rossiello, Marco de Gemmis, Pasquale Lops, Giovanni Semeraro. Natural Language Justifications for Recommender Systems Exploiting Text Summarization and Sentiment Analysis. AI*IA 2019, Rende, November 22, 2019
  • 30. Generation Intuition: we conceive our justification as a summary of the information conveyed by all the available reviews Approach: we exploited a centroid-based method for automatic text summarization. Very good performance in multi-document summarization scenarios. Assumption: each review can be considered as ‘document’ thus the corpus of the reviews can be used to feed the algorithm 30Cataldo Musto, Gaetano Rossiello, Marco de Gemmis, Pasquale Lops, Giovanni Semeraro. Natural Language Justifications for Recommender Systems Exploiting Text Summarization and Sentiment Analysis. AI*IA 2019, Rende, November 22, 2019
  • 31. Generation Generation process is in turn split into two steps • Sentence Filtering • Text Summarization Sentence Filtering is used to feed the summarization algorithm with compliant sentences. We selected sentences that matched the following criterions: 31Cataldo Musto, Gaetano Rossiello, Marco de Gemmis, Pasquale Lops, Giovanni Semeraro. Natural Language Justifications for Recommender Systems Exploiting Text Summarization and Sentiment Analysis. AI*IA 2019, Rende, November 22, 2019
  • 32. Generation Generation process is in turn split into two steps • Sentence Filtering • Text Summarization Sentence Filtering is used to feed the summarization algorithm with compliant sentences. We selected sentences that matched the following criterions: • The sentence contains a main aspect • The sentence is longer than 5 tokens • The sentence expresses a positive sentiment • The sentence does not contain first-person personal or possessive pronouns 32Cataldo Musto, Gaetano Rossiello, Marco de Gemmis, Pasquale Lops, Giovanni Semeraro. Natural Language Justifications for Recommender Systems Exploiting Text Summarization and Sentiment Analysis. AI*IA 2019, Rende, November 22, 2019
  • 33. Generation Text Summarization Algorithm Input: item i, sentences s1…sn, word limit k Output: summary for item i consisting of k words 1. Build a vector space representation for each sentence 2. Merge all the sentences in a pseudo-document that represents the item (centroid vector) 3. Until the word limit is not reached 3.1 Go through the sentences and calculate cosine similarity 3.2 Pick the one with the highest cosine similarity to the centroid (and not that similar to the sentences previously picked) 3.3 Add it to the summary 33Cataldo Musto, Gaetano Rossiello, Marco de Gemmis, Pasquale Lops, Giovanni Semeraro. Natural Language Justifications for Recommender Systems Exploiting Text Summarization and Sentiment Analysis. AI*IA 2019, Rende, November 22, 2019
  • 34. Generation Text Summarization Algorithm Input: item i, sentences s1…sn, word limit k Output: summary for item i consisting of k words 1. Build a vector space representation for each sentence 2. Merge all the sentences in a pseudo-document that represents the item (centroid vector) 3. Until the word limit is not reached 3.1 Go through the sentences and calculate cosine similarity 3.2 Pick the one with the highest cosine similarity to the centroid (and not that similar to the sentences previously picked) 3.3 Add it to the summary 34Cataldo Musto, Gaetano Rossiello, Marco de Gemmis, Pasquale Lops, Giovanni Semeraro. Natural Language Justifications for Recommender Systems Exploiting Text Summarization and Sentiment Analysis. AI*IA 2019, Rende, November 22, 2019
  • 35. Generation Text Summarization Algorithm Input: item i, sentences s1…sn, word limit k Output: summary for item i consisting of k words 1. Build a vector space representation for each sentence 2. Merge all the sentences in a pseudo-document that represents the item (centroid vector) 3. Until the word limit is not reached 3.1 Go through the sentences and calculate cosine similarity 3.2 Pick the one with the highest cosine similarity to the centroid (and not that similar to the sentences previously picked) 3.3 Add it to the summary 35Cataldo Musto, Gaetano Rossiello, Marco de Gemmis, Pasquale Lops, Giovanni Semeraro. Natural Language Justifications for Recommender Systems Exploiting Text Summarization and Sentiment Analysis. AI*IA 2019, Rende, November 22, 2019
  • 36. Generation Text Summarization Algorithm Input: item i, sentences s1…sn, word limit k Output: summary for item i consisting of k words 1. Build a vector space representation for each sentence 2. Merge all the sentences in a pseudo-document that represents the item (centroid vector) 3. Until the word limit is not reached 3.1 Go through the sentences and calculate cosine similarity 3.2 Pick the one with the highest cosine similarity to the centroid (and not that similar to the sentences previously picked) 3.3 Add it to the summary 36Cataldo Musto, Gaetano Rossiello, Marco de Gemmis, Pasquale Lops, Giovanni Semeraro. Natural Language Justifications for Recommender Systems Exploiting Text Summarization and Sentiment Analysis. AI*IA 2019, Rende, November 22, 2019
  • 37. Generation Text Summarization Algorithm Input: item i, sentences s1…sn, word limit k Output: summary for item i consisting of k words 1. Build a vector space representation for each sentence 2. Merge all the sentences in a pseudo-document that represents the item (centroid vector) 3. Until the word limit is not reached 3.1 Go through the sentences and calculate cosine similarity 3.2 Pick the one with the highest cosine similarity to the centroid (and not that similar to the sentences previously picked) 3.3 Add it to the summary 37Cataldo Musto, Gaetano Rossiello, Marco de Gemmis, Pasquale Lops, Giovanni Semeraro. Natural Language Justifications for Recommender Systems Exploiting Text Summarization and Sentiment Analysis. AI*IA 2019, Rende, November 22, 2019
  • 38. Generation – Final Output “If you like or love the blood and gore kinds of films, this movie will certainly disappoint you as the focus is on character, story, mood and unique special effects. The Ring is a story about supernatural evil therefore, it is a horror film, done very much in the style of the psychological thriller.” Legenda Red: aspects (k=4) Black: compliant excerpts 38Cataldo Musto, Gaetano Rossiello, Marco de Gemmis, Pasquale Lops, Giovanni Semeraro. Natural Language Justifications for Recommender Systems Exploiting Text Summarization and Sentiment Analysis. AI*IA 2019, Rende, November 22, 2019
  • 39. Experimental Evaluation Research Question 1 How effective are the justifications generated through the pipeline, on varying of different combinations of the parameters? Research Question 2 How does our justifications perform with respect to a simple review-based explanation? Experimental Design User Study with a Web Application 141 subjects Movie Domain. 300 movies. ~150k reviews. Metrics: Transparency, Engagement, Persuasion, Trust, Effectiveness [^] Parameters: Justification Length (Short=50 words, Long=100) and #Aspects (10 and 30). Between-subjects for Research Question 1, Within-subjects for Research Question 2 [^] Tintarev, N., & Masthoff, J. Designing and evaluating explanations for recommender systems. In Recommender systems handbook. pp. 479-510. Springer, Boston, MA. 2011 39Cataldo Musto, Gaetano Rossiello, Marco de Gemmis, Pasquale Lops, Giovanni Semeraro. Natural Language Justifications for Recommender Systems Exploiting Text Summarization and Sentiment Analysis. AI*IA 2019, Rende, November 22, 2019
  • 40. Experimental Protocol (Research Question 1) 1. Gathering movie preferences Users rated their favourite movies 40Cataldo Musto, Gaetano Rossiello, Marco de Gemmis, Pasquale Lops, Giovanni Semeraro. Natural Language Justifications for Recommender Systems Exploiting Text Summarization and Sentiment Analysis. AI*IA 2019, Rende, November 22, 2019
  • 41. Experimental Protocol (Research Question 1) 2. Recommendation is obtained Personalized PageRank as algorithm 1. Gathering movie preferences Users rated their favourite movies 41Cataldo Musto, Gaetano Rossiello, Marco de Gemmis, Pasquale Lops, Giovanni Semeraro. Natural Language Justifications for Recommender Systems Exploiting Text Summarization and Sentiment Analysis. AI*IA 2019, Rende, November 22, 2019
  • 42. Experimental Protocol (Research Question 1) 2. Recommendation is obtained Personalized PageRank as algorithm 3. Explanation is generated Random Configuration (users not aware) 1. Gathering movie preferences Users rated their favourite movies 42Cataldo Musto, Gaetano Rossiello, Marco de Gemmis, Pasquale Lops, Giovanni Semeraro. Natural Language Justifications for Recommender Systems Exploiting Text Summarization and Sentiment Analysis. AI*IA 2019, Rende, November 22, 2019
  • 43. Experimental Protocol (Research Question 1) 1. Gathering movie preferences Users rated their favourite movies 2. Recommendation is obtained Personalized PageRank as algorithm 3. Explanation is generated Random Configuration (users not aware) 4. Metrics are calculated Through Questionnaires 43Cataldo Musto, Gaetano Rossiello, Marco de Gemmis, Pasquale Lops, Giovanni Semeraro. Natural Language Justifications for Recommender Systems Exploiting Text Summarization and Sentiment Analysis. AI*IA 2019, Rende, November 22, 2019
  • 44. Results (Research Question 1) MOVIES Transparency Persuasion Engagement Trust Effectiveness Top-10 Short 2.83 3.06 3.06 2.83 0.89 Top-30 Long 3.16 3.06 2.69 3.19 0.94 Top-10 Short 3.95 3.64 3.37 3.55 0.55 Top-30 Long 3.24 3.18 3.12 3.22 0.38 Finding 1 Long justifications better than short justifications, on average Finding 2 Top-10 aspect provide better explanations than Top-30 aspects Finding 3 Long explanations based on Top-10 aspects lead to the best results 44Cataldo Musto, Gaetano Rossiello, Marco de Gemmis, Pasquale Lops, Giovanni Semeraro. Natural Language Justifications for Recommender Systems Exploiting Text Summarization and Sentiment Analysis. AI*IA 2019, Rende, November 22, 2019
  • 45. Experimental Protocol (Research Question 2) Review-based Explanation + Summarization Review-based Explanation 45Cataldo Musto, Gaetano Rossiello, Marco de Gemmis, Pasquale Lops, Giovanni Semeraro. Natural Language Justifications for Recommender Systems Exploiting Text Summarization and Sentiment Analysis. AI*IA 2019, Rende, November 22, 2019
  • 46. Results (Research Question 2) MOVIES Review+ Summary Review- based Indiffer. Transparency 54.5% 40.9% 4.6% Persuasion 77.3% 13.6% 9.1% Engagement 63.6% 27.3% 9.1% Trust 68.2% 4.5% 27.3% Outcome: automatic Text Summarization provides users with the best explanation. Confirmed for all the metrics and both the domains 46Cataldo Musto, Gaetano Rossiello, Marco de Gemmis, Pasquale Lops, Giovanni Semeraro. Natural Language Justifications for Recommender Systems Exploiting Text Summarization and Sentiment Analysis. AI*IA 2019, Rende, November 22, 2019
  • 47. Recap “If you like or love the blood and gore kinds of films, this movie will certainly disappoint you as the focus is on character, story, mood and unique special effects. The Ring is a story about supernatural evil therefore, it is a horror film, done very much in the style of the psychological thriller.” Review-based explanation using automatic text summarization 47Cataldo Musto, Gaetano Rossiello, Marco de Gemmis, Pasquale Lops, Giovanni Semeraro. Natural Language Justifications for Recommender Systems Exploiting Text Summarization and Sentiment Analysis. AI*IA 2019, Rende, November 22, 2019
  • 48. Take-home Messages 48 1. 2. Textual features extracted from reviews can be used to generate a natural language justifications supporting a recommendation Cataldo Musto, Gaetano Rossiello, Marco de Gemmis, Pasquale Lops, Giovanni Semeraro. Natural Language Justifications for Recommender Systems Exploiting Text Summarization and Sentiment Analysis. AI*IA 2019, Rende, November 22, 2019 48 Text summarization algorithms are suitable to build a less ‘static’ justifications based on review excerpts 3. Shorter Justifications based on Top-10 aspects obtained the best results
  • 49. 49 Grazie! cataldo.musto@uniba.it @cataldomusto Contacts Cataldo Musto, Gaetano Rossiello, Marco de Gemmis, Pasquale Lops, Giovanni Semeraro. Natural Language Justifications for Recommender Systems Exploiting Text Summarization and Sentiment Analysis. AI*IA 2019, Rende, November 22, 2019