SlideShare a Scribd company logo

A Multi-Criteria Recommender System Exploiting Aspect-based Sentiment Analysis of Users' Reviews

Cataldo Musto, profile picture
Cataldo Musto

The document describes a multi-criteria recommender system that exploits aspect-based sentiment analysis of user reviews. It involves a two-step methodology: 1) performing aspect extraction and sentiment analysis on user reviews using an algorithm based on SABRE to identify aspects, sub-aspects, and sentiment, and 2) creating and populating a multi-criteria data model with the extracted information and using it to generate recommendations. The system aims to develop a multi-criteria data model for recommendations without overwhelming users by automatically extracting product aspects and sentiments from reviews rather than requiring users to manually evaluate each aspect.

Education
Related topics:
Natural Language Processing • Sentiment Analysis Methods
1 of 42
Downloaded 49 times
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
@cataldomusto @pasqualelops @semeraro_g @SWAP_research A Multi-criteria Recommender System Exploiting Aspect-based Sentiment Analysis of Users’ Reviews CATALDO MUSTO, MARCO DE GEMMIS, GIOVANNI SEMERARO, PASQUALE LOPS UNIVERSITÀ DEGLI STUDI DI BARI ‘ALDO MORO’ - ITALY RecSys 2017 - 11th ACM Conference on Recommender Systems Como, Italy August 30, 2017 cataldo.musto@uniba.it
Multi-Criteria RecSys Cataldo Musto, Marco de Gemmis, Giovanni Semeraro. Pasquale Lops A Multi-criteria Recommender System Exploiting Aspect-based Sentiment Analysis of Users’ Reviews. RECSYS 2017. Como, Italy. August 30, 2017 Not a new concept (*) Each user evaluates each aspect of the item (*) Adomavicius, Gediminas, and YoungOk Kwon. "Multi-criteria recommender systems." Recommender Systems Handbook. Springer US, 2015. 847-880.
Multi-Criteria RecSys Cataldo Musto, Marco de Gemmis, Giovanni Semeraro. Pasquale Lops A Multi-criteria Recommender System Exploiting Aspect-based Sentiment Analysis of Users’ Reviews. RECSYS 2017. Como, Italy. August 30, 2017 Not a new concept (*) Each user evaluates each aspect of the item (*) Adomavicius, Gediminas, and YoungOk Kwon. "Multi-criteria recommender systems." Recommender Systems Handbook. Springer US, 2015. 847-880. Problem: Overwhelming!
Multi-Criteria RecSys Cataldo Musto, Marco de Gemmis, Giovanni Semeraro. Pasquale Lops A Multi-criteria Recommender System Exploiting Aspect-based Sentiment Analysis of Users’ Reviews. RECSYS 2017. Como, Italy. August 30, 2017 Not a new concept (*) Each user evaluates each aspect of the item (*) Adomavicius, Gediminas, and YoungOk Kwon. "Multi-criteria recommender systems." Recommender Systems Handbook. Springer US, 2015. 847-880. Problem: Aspects are not fixed!
Multi-Criteria RecSys Cataldo Musto, Marco de Gemmis, Giovanni Semeraro. Pasquale Lops A Multi-criteria Recommender System Exploiting Aspect-based Sentiment Analysis of Users’ Reviews. RECSYS 2017. Como, Italy. August 30, 2017 Not a new concept (*) Each user evaluates each aspect of the item (*) Adomavicius, Gediminas, and YoungOk Kwon. "Multi-criteria recommender systems." Recommender Systems Handbook. Springer US, 2015. 847-880. Problem: Aspects can be further modeled as a hierarchy
Research Question Cataldo Musto, Marco de Gemmis, Giovanni Semeraro. Pasquale Lops A Multi-criteria Recommender System Exploiting Aspect-based Sentiment Analysis of Users’ Reviews. RECSYS 2017. Como, Italy. August 30, 2017 How to develop a multi-criteria data model without overwhelming the user ?
Research Question Cataldo Musto, Marco de Gemmis, Giovanni Semeraro. Pasquale Lops A Multi-criteria Recommender System Exploiting Aspect-based Sentiment Analysis of Users’ Reviews. RECSYS 2017. Como, Italy. August 30, 2017 What is the performance of such a data model in a collaborative recommendation scenario?
Our contribution Cataldo Musto, Marco de Gemmis, Giovanni Semeraro. Pasquale Lops A Multi-criteria Recommender System Exploiting Aspect-based Sentiment Analysis of Users’ Reviews. RECSYS 2017. Como, Italy. August 30, 2017 A multi-criteria collaborative recommendation methodology exploiting aspect-based sentiment analysis of users’ reviews
Methodology Cataldo Musto, Marco de Gemmis, Giovanni Semeraro. Pasquale Lops A Multi-criteria Recommender System Exploiting Aspect-based Sentiment Analysis of Users’ Reviews. RECSYS 2017. Como, Italy. August 30, 2017 Input: textual reviews Step 1: aspect extraction and sentiment analysis Step 2: creating multi-criteria data model Output: recommendations
Methodology Cataldo Musto, Marco de Gemmis, Giovanni Semeraro. Pasquale Lops A Multi-criteria Recommender System Exploiting Aspect-based Sentiment Analysis of Users’ Reviews. RECSYS 2017. Como, Italy. August 30, 2017 Input: textual reviews Step 1: aspect extraction and sentiment analysis Step 2: creating multi-criteria data model Output: recommendations
Methodology Cataldo Musto, Marco de Gemmis, Giovanni Semeraro. Pasquale Lops A Multi-criteria Recommender System Exploiting Aspect-based Sentiment Analysis of Users’ Reviews. RECSYS 2017. Como, Italy. August 30, 2017 Input: textual reviews Step 1: aspect extraction and sentiment analysis Step 2: creating and filling our multi-criteria data model Output: recommendations
Methodology Cataldo Musto, Marco de Gemmis, Giovanni Semeraro. Pasquale Lops A Multi-criteria Recommender System Exploiting Aspect-based Sentiment Analysis of Users’ Reviews. RECSYS 2017. Como, Italy. August 30, 2017 Input: textual reviews Step 1: aspect extraction and sentiment analysis Step 2: creating and filling our multi-criteria data model Output: recommendations
Aspect Extraction and Sentiment Analysis Cataldo Musto, Marco de Gemmis, Giovanni Semeraro. Pasquale Lops A Multi-criteria Recommender System Exploiting Aspect-based Sentiment Analysis of Users’ Reviews. RECSYS 2017. Como, Italy. August 30, 2017 Algorithm based on SABRE(*) (Sentiment Aspect-based Retrieval Engine) (*) Caputo, A., Basile, P., de Gemmis, M., Lops, P., Semeraro, G., & Rossiello, G. (2017). SABRE: A Sentiment Aspect-Based Retrieval Engine. In Information Filtering and Retrieval (pp. 63-78). 𝑅 = {𝑟1, 𝑟2 … 𝑟 𝑛}Input: set of reviews Output: quintuples < 𝑟𝑖, 𝑎𝑖𝑗, 𝑎𝑖𝑗𝑘, 𝑟𝑒𝑙 𝑎𝑖𝑗𝑘, 𝑟𝑖 , 𝑠𝑒𝑛𝑡(𝑎𝑖𝑗𝑘, 𝑟𝑖) >
Aspect Extraction and Sentiment Analysis Cataldo Musto, Marco de Gemmis, Giovanni Semeraro. Pasquale Lops A Multi-criteria Recommender System Exploiting Aspect-based Sentiment Analysis of Users’ Reviews. RECSYS 2017. Como, Italy. August 30, 2017 Algorithm based on SABRE(*) (Sentiment Aspect-based Retrieval Engine) (*) Caputo, A., Basile, P., de Gemmis, M., Lops, P., Semeraro, G., & Rossiello, G. (2017). SABRE: A Sentiment Aspect-Based Retrieval Engine. In Information Filtering and Retrieval (pp. 63-78). 𝑅 = {𝑟1, 𝑟2 … 𝑟 𝑛}Input: set of reviews Output: quintuples < 𝑟𝑖, 𝑎𝑖𝑗, 𝑎𝑖𝑗𝑘, 𝑟𝑒𝑙 𝑎𝑖𝑗𝑘, 𝑟𝑖 , 𝑠𝑒𝑛𝑡(𝑎𝑖𝑗𝑘, 𝑟𝑖) > 𝑟𝑖 = 𝑎𝑖𝑗, 𝑎𝑖𝑗𝑘= 𝑟𝑒𝑙 𝑎𝑖𝑗𝑘, 𝑟𝑖 , 𝑠𝑒𝑛𝑡(𝑎𝑖𝑗𝑘, 𝑟𝑖) = i-th review j-th aspect and k-th sub-aspect in the i-th review relevance and sentiment
Aspect Extraction and Sentiment Analysis Cataldo Musto, Marco de Gemmis, Giovanni Semeraro. Pasquale Lops A Multi-criteria Recommender System Exploiting Aspect-based Sentiment Analysis of Users’ Reviews. RECSYS 2017. Como, Italy. August 30, 2017 Algorithm based on SABRE(*) (Sentiment Aspect-based Retrieval Engine) (*) Caputo, A., Basile, P., de Gemmis, M., Lops, P., Semeraro, G., & Rossiello, G. (2017). SABRE: A Sentiment Aspect-Based Retrieval Engine. In Information Filtering and Retrieval (pp. 63-78). 𝑅 = {𝑟1, 𝑟2 … 𝑟 𝑛}Input: set of reviews Output: quintuples < 𝑟𝑖, 𝑎𝑖𝑗, 𝑎𝑖𝑗𝑘, 𝑟𝑒𝑙 𝑎𝑖𝑗𝑘, 𝑟𝑖 , 𝑠𝑒𝑛𝑡(𝑎𝑖𝑗𝑘, 𝑟𝑖) > 𝑟𝑖 = 𝑎𝑖𝑗, 𝑎𝑖𝑗𝑘= 𝑟𝑒𝑙 𝑎𝑖𝑗𝑘, 𝑟𝑖 , 𝑠𝑒𝑛𝑡(𝑎𝑖𝑗𝑘, 𝑟𝑖) = i-th review j-th aspect and k-th sub-aspect in the i-th review relevance and sentiment How do we extract aspects, relevance and sentiment?
Aspect Extraction and Sentiment Analysis Cataldo Musto, Marco de Gemmis, Giovanni Semeraro. Pasquale Lops A Multi-criteria Recommender System Exploiting Aspect-based Sentiment Analysis of Users’ Reviews. RECSYS 2017. Como, Italy. August 30, 2017 Aspect Extraction (*) Caputo, A., Basile, P., de Gemmis, M., Lops, P., Semeraro, G., & Rossiello, G. (2017). SABRE: A Sentiment Aspect-Based Retrieval Engine. In Information Filtering and Retrieval (pp. 63-78). Statistical approach based on the Kullback-Leibler (KL) Divergence
Aspect Extraction and Sentiment Analysis Cataldo Musto, Marco de Gemmis, Giovanni Semeraro. Pasquale Lops A Multi-criteria Recommender System Exploiting Aspect-based Sentiment Analysis of Users’ Reviews. RECSYS 2017. Como, Italy. August 30, 2017 Aspect Extraction (*) Caputo, A., Basile, P., de Gemmis, M., Lops, P., Semeraro, G., & Rossiello, G. (2017). SABRE: A Sentiment Aspect-Based Retrieval Engine. In Information Filtering and Retrieval (pp. 63-78). Statistical approach based on the Kullback-Leibler (KL) Divergence Measures the difference between the distribution of a term in a generic corpus (e.g. BNC) and its distribution in a domain corpus (e.g. hotel reviews) Insight: the higher the divergence, the higher the importance of the term in the domain
Aspect Extraction and Sentiment Analysis Cataldo Musto, Marco de Gemmis, Giovanni Semeraro. Pasquale Lops A Multi-criteria Recommender System Exploiting Aspect-based Sentiment Analysis of Users’ Reviews. RECSYS 2017. Como, Italy. August 30, 2017 Aspect Extraction (*) Caputo, A., Basile, P., de Gemmis, M., Lops, P., Semeraro, G., & Rossiello, G. (2017). SABRE: A Sentiment Aspect-Based Retrieval Engine. In Information Filtering and Retrieval (pp. 63-78). Statistical approach based on the Kullback-Leibler (KL) Divergence Measures the difference between the distribution of a term in a generic corpus (e.g. BNC) and its distribution in a domain corpus (e.g. hotel reviews) Insight: the higher the divergence, the higher the importance of the term in the domain KL(room, BNC, hotel-reviews) >> 0 KL(food, BNC, hotel-reviews) > 0 KL(place, BNC, hotel-reviews) ~ 0 KL(politics, BNC, hotel-reviews) ~ 0 We label as ‘aspects’ the nouns whose KL-divergence is higher than zero
Aspect Extraction and Sentiment Analysis Cataldo Musto, Marco de Gemmis, Giovanni Semeraro. Pasquale Lops A Multi-criteria Recommender System Exploiting Aspect-based Sentiment Analysis of Users’ Reviews. RECSYS 2017. Como, Italy. August 30, 2017 Aspect Extraction (*) Caputo, A., Basile, P., de Gemmis, M., Lops, P., Semeraro, G., & Rossiello, G. (2017). SABRE: A Sentiment Aspect-Based Retrieval Engine. In Information Filtering and Retrieval (pp. 63-78). Statistical approach based on the Kullback-Leibler (KL) Divergence Measures the difference between the distribution of a term in a generic corpus (e.g. BNC) and its distribution in a domain corpus (e.g. hotel reviews) Insight: the higher the divergence, the higher the importance of the term in the domain KL(room, BNC, hotel-reviews) >> 0 YES KL(food, BNC, hotel-reviews) > 0 YES KL(place, BNC, hotel-reviews) ~ 0 NO KL(politics, BNC, hotel-reviews) ~ 0 NO
Aspect Extraction and Sentiment Analysis Cataldo Musto, Marco de Gemmis, Giovanni Semeraro. Pasquale Lops A Multi-criteria Recommender System Exploiting Aspect-based Sentiment Analysis of Users’ Reviews. RECSYS 2017. Como, Italy. August 30, 2017 Aspect Extraction (*) Caputo, A., Basile, P., de Gemmis, M., Lops, P., Semeraro, G., & Rossiello, G. (2017). SABRE: A Sentiment Aspect-Based Retrieval Engine. In Information Filtering and Retrieval (pp. 63-78). Statistical approach based on the Kullback-Leibler (KL) Divergence Measures the difference between the distribution of a term in a generic corpus (e.g. BNC) and its distribution in a domain corpus (e.g. hotel reviews) Insight: the higher the divergence, the higher the importance of the term in the domain KL(room, BNC, hotel-reviews) >> 0 YES KL(food, BNC, hotel-reviews) > 0 YES KL(place, BNC, hotel-reviews) ~ 0 NO KL(politics, BNC, hotel-reviews) ~ 0 NO Distinguishing aspect: the set is not fixed!
Aspect Extraction and Sentiment Analysis Cataldo Musto, Marco de Gemmis, Giovanni Semeraro. Pasquale Lops A Multi-criteria Recommender System Exploiting Aspect-based Sentiment Analysis of Users’ Reviews. RECSYS 2017. Como, Italy. August 30, 2017 Sub-aspect Extraction (*) Caputo, A., Basile, P., de Gemmis, M., Lops, P., Semeraro, G., & Rossiello, G. (2017). SABRE: A Sentiment Aspect-Based Retrieval Engine. In Information Filtering and Retrieval (pp. 63-78). Another distinguishing aspect: we can extract a hierarchy of terms Based on Phraseness and Informativeness: They measure the gain in information if two terms are modeled together Insight: if phraseness and informativeness are high, the terms have an high cohesion
Aspect Extraction and Sentiment Analysis Cataldo Musto, Marco de Gemmis, Giovanni Semeraro. Pasquale Lops A Multi-criteria Recommender System Exploiting Aspect-based Sentiment Analysis of Users’ Reviews. RECSYS 2017. Como, Italy. August 30, 2017 Sub-aspect Extraction (*) Caputo, A., Basile, P., de Gemmis, M., Lops, P., Semeraro, G., & Rossiello, G. (2017). SABRE: A Sentiment Aspect-Based Retrieval Engine. In Information Filtering and Retrieval (pp. 63-78). Another distinguishing aspect: we can extract a hierarchy of terms Based on Phraseness and Informativeness: They measure the gain in information if two terms are modeled together Insight: if phraseness and informativeness are high, the terms have an high cohesion SUB(room, food, hotel-reviews) ~ 0 NO SUB(room, shower, hotel-reviews) > 0 YES SUB(food, wine, hotel-reviews) > 0 YES SUB(food, service, hotel-reviews) ~ 0 NO
Aspect Extraction and Sentiment Analysis Cataldo Musto, Marco de Gemmis, Giovanni Semeraro. Pasquale Lops A Multi-criteria Recommender System Exploiting Aspect-based Sentiment Analysis of Users’ Reviews. RECSYS 2017. Como, Italy. August 30, 2017 Sabre@Work (*) Real review of the hotel we actually stay in Como :) (*)
Aspect Extraction and Sentiment Analysis Cataldo Musto, Marco de Gemmis, Giovanni Semeraro. Pasquale Lops A Multi-criteria Recommender System Exploiting Aspect-based Sentiment Analysis of Users’ Reviews. RECSYS 2017. Como, Italy. August 30, 2017 < 𝑟, 𝑏𝑟𝑒𝑎𝑘𝑓𝑎𝑠𝑡,∗, 1.5, 0.5 > Sabre@Work (*) Real review of the hotel we actually stay in Como :) (*)
Aspect Extraction and Sentiment Analysis Cataldo Musto, Marco de Gemmis, Giovanni Semeraro. Pasquale Lops A Multi-criteria Recommender System Exploiting Aspect-based Sentiment Analysis of Users’ Reviews. RECSYS 2017. Como, Italy. August 30, 2017 < 𝑟, 𝑏𝑟𝑒𝑎𝑘𝑓𝑎𝑠𝑡,∗, 1.5, 0.5 > No sub-aspects Relevance=KL-divergence score Sentiment = lexicon-based approach based on AFINN wordlist (*) or machine-learning based approach based on CoreNLP (^) (*) http://neuro.imm.dtu.dk/wiki/AFINN (^) https://nlp.stanford.edu/sentiment/code.html Sabre@Work (*) Real review of the hotel we actually stay in Como :) (*)
Aspect Extraction and Sentiment Analysis Cataldo Musto, Marco de Gemmis, Giovanni Semeraro. Pasquale Lops A Multi-criteria Recommender System Exploiting Aspect-based Sentiment Analysis of Users’ Reviews. RECSYS 2017. Como, Italy. August 30, 2017 < 𝑟, 𝑏𝑟𝑒𝑎𝑘𝑓𝑎𝑠𝑡,∗, 1.5, 0.5 > < 𝑟, 𝑟𝑜𝑜𝑚, 𝑠ℎ𝑜𝑤𝑒𝑟, 1.2, −0.5 > Sabre@Work (*) Real review of the hotel we actually stay in Como :) (*)
Aspect Extraction and Sentiment Analysis Cataldo Musto, Marco de Gemmis, Giovanni Semeraro. Pasquale Lops A Multi-criteria Recommender System Exploiting Aspect-based Sentiment Analysis of Users’ Reviews. RECSYS 2017. Como, Italy. August 30, 2017 < 𝑟, 𝑏𝑟𝑒𝑎𝑘𝑓𝑎𝑠𝑡,∗, 1.5, 0.5 > < 𝑟, 𝑟𝑜𝑜𝑚, 𝑠ℎ𝑜𝑤𝑒𝑟, 1.2, −0.5 > < 𝑟, 𝑟𝑜𝑜𝑚,∗, 1.3, 0.2 > Sabre@Work (*) Real review of the hotel we actually stay in Como :) (*)
Aspect Extraction and Sentiment Analysis Cataldo Musto, Marco de Gemmis, Giovanni Semeraro. Pasquale Lops A Multi-criteria Recommender System Exploiting Aspect-based Sentiment Analysis of Users’ Reviews. RECSYS 2017. Como, Italy. August 30, 2017 < 𝑟, 𝑏𝑟𝑒𝑎𝑘𝑓𝑎𝑠𝑡,∗, 1.5, 0.5 > < 𝑟, 𝑟𝑜𝑜𝑚, 𝑠ℎ𝑜𝑤𝑒𝑟, 1.2, −0.5 > < 𝑟, 𝑟𝑜𝑜𝑚,∗, 1.3, 0.2 > Sabre@Work … … . 𝑒𝑡𝑐. (*) Real review of the hotel we actually stay in Como :) (*)
Multi-Criteria Data Model Cataldo Musto, Marco de Gemmis, Giovanni Semeraro. Pasquale Lops A Multi-criteria Recommender System Exploiting Aspect-based Sentiment Analysis of Users’ Reviews. RECSYS 2017. Como, Italy. August 30, 2017 A multi-criteria data model is automatically filled by exploiting the aspects extracted from the review and their sentiment Finer-Grained Representation!
Providing Recommendations Cataldo Musto, Marco de Gemmis, Giovanni Semeraro. Pasquale Lops A Multi-criteria Recommender System Exploiting Aspect-based Sentiment Analysis of Users’ Reviews. RECSYS 2017. Como, Italy. August 30, 2017 Similarity is calculated through multi-criteria Euclidean distance Recommendations are provided by exploiting both User-to-User and Item-to-Item Collaborative Filtering Recommendation
Framework Recap Cataldo Musto, Marco de Gemmis, Giovanni Semeraro. Pasquale Lops A Multi-criteria Recommender System Exploiting Aspect-based Sentiment Analysis of Users’ Reviews. RECSYS 2017. Como, Italy. August 30, 2017
Experiments Which combination of the parameters led to the best predictive accuracy? How does our framework perform when compared to single-criteria recommendations and matrix factorization tecniques? Cataldo Musto, Marco de Gemmis, Giovanni Semeraro. Pasquale Lops A Multi-criteria Recommender System Exploiting Aspect-based Sentiment Analysis of Users’ Reviews. RECSYS 2017. Como, Italy. August 30, 2017
Datasets Yelp 45,981 users 11,537 items 229,606 ratings(*) 99.95% sparsity TripAdvisor 536,952 users 3,945 items 796,958 ratings(*) 99.96% sparsity Amazon 826,773 users 50,210 items 1,324,759 ratings(*) 99.99% sparsity (*) Ratings = ratings + reviews Cataldo Musto, Marco de Gemmis, Giovanni Semeraro. Pasquale Lops A Multi-criteria Recommender System Exploiting Aspect-based Sentiment Analysis of Users’ Reviews. RECSYS 2017. Como, Italy. August 30, 2017
Datasets Yelp 45,981 users 11,537 items 229,606 ratings(*) 99.95% sparsity TripAdvisor 536,952 users 3,945 items 796,958 ratings(*) 99.96% sparsity Amazon 826,773 users 50,210 items 1,324,759 ratings(*) 99.99% sparsity (*) Ratings = ratings + reviews Cataldo Musto, Marco de Gemmis, Giovanni Semeraro. Pasquale Lops A Multi-criteria Recommender System Exploiting Aspect-based Sentiment Analysis of Users’ Reviews. RECSYS 2017. Como, Italy. August 30, 2017
Experimental Settings Review Processing ◦ Stop-Word removed ◦ Entity and Collocations recognized SABRE parameters ◦ With/without subaspects ◦ #aspects/#subaspects = 10, 50 ◦ KL-divergence threshold = 0.1 ◦ Only nouns! Recommendations ◦ Multi-Criteria U2U and I2I Metric ◦ MAE (calculated with Rival framework) Cataldo Musto, Marco de Gemmis, Giovanni Semeraro. Pasquale Lops A Multi-criteria Recommender System Exploiting Aspect-based Sentiment Analysis of Users’ Reviews. RECSYS 2017. Como, Italy. August 30, 2017
Baselines Single-Criteria Recommendations techniques ◦ User-to-User Collaborative Filtering ◦ Item-to-Item Collaborative Filtering Static Multi-Criteria Recommendations ◦ Only on TripAdvisor data Matrix Factorization techniques ◦ SGD (Stochastic Gradient Descent) ◦ ParallelSGD ◦ ALSWR ◦ Implementations available in Mahout ◦ Tuning of parameters Alessandro Suglia, Claudio Greco, Cataldo Musto, Marco de Gemmis, Pasquale Lops, Giovanni Semeraro. A Deep Architecture for Content-based Recommendations Exploiting Recurrent Neural Networks. UMAP 2017. Bratislava, Slovakia. July 12, 2017
Top-10 aspects Place Food Service Restaurant Price Menu Staff Drink Lunch Table Hotel Room Staff Location Service Breakfast Restaurant Bathroom Price View Game Graphic Story Character Player Price Gameplay Controller Level Music Cataldo Musto, Marco de Gemmis, Giovanni Semeraro. Pasquale Lops A Multi-criteria Recommender System Exploiting Aspect-based Sentiment Analysis of Users’ Reviews. RECSYS 2017. Como, Italy. August 30, 2017
0,7111 0,7564 0,7269 0,8007 0,65 0,67 0,69 0,71 0,73 0,75 0,77 0,79 0,81 0,83 TripAdvisor 10 neigh. / 10 aspects / sub-aspects 10 neigh. / 10 aspects / no sub-aspects 10 neigh. / 50 aspects / sub-aspects 10 neigh. / 50 aspects / no sub-aspects Outcomes Best-results obtained with 10 aspects Best-results obtained by also introducing sub-aspects (Amazon had a different behavior) Lower MAE! Results – Multi-Criteria User-to-User CF Cataldo Musto, Marco de Gemmis, Giovanni Semeraro. Pasquale Lops A Multi-criteria Recommender System Exploiting Aspect-based Sentiment Analysis of Users’ Reviews. RECSYS 2017. Como, Italy. August 30, 2017
0,7111 0,798 0,8245 0,8429 0,6 0,65 0,7 0,75 0,8 0,85 TripAdvisor Multi-U2U Static-Multi-U2U Multi-I2I Static-Multi-I2I Outcomes TripAdvisors data included ratings about six static aspects (cleanliness, location, value, service, sleep quality, overall) Our approach based on unsupervised aspect extraction also improved these results Results – vs. Static Multi-Criteria RecSys Cataldo Musto, Marco de Gemmis, Giovanni Semeraro. Pasquale Lops A Multi-criteria Recommender System Exploiting Aspect-based Sentiment Analysis of Users’ Reviews. RECSYS 2017. Como, Italy. August 30, 2017
0,7111 0,8337 0,745 0,7449 0,9053 0,65 0,7 0,75 0,8 0,85 0,9 0,95 1 TripAdvisor Multi-U2U Single-U2U Ratings-SGD Parallel-SGD ALSWR Outcomes Our approach overcomes all the baselines. Our framework wins the comparisons to Single-U2U and Single-I2I Also matrix factorization techniques got an higher MAE Results – Baselines Cataldo Musto, Marco de Gemmis, Giovanni Semeraro. Pasquale Lops A Multi-criteria Recommender System Exploiting Aspect-based Sentiment Analysis of Users’ Reviews. RECSYS 2017. Como, Italy. August 30, 2017
Recap Results ☺ Our framework significantly improves all the baselines ☺ Unsupervised Aspect Extraction also overcomes static aspects Future Work: evaluate data model with more sophisticated algorithms Cataldo Musto, Marco de Gemmis, Giovanni Semeraro. Pasquale Lops A Multi-criteria Recommender System Exploiting Aspect-based Sentiment Analysis of Users’ Reviews. RECSYS 2017. Como, Italy. August 30, 2017
Thanks! cataldo.musto@uniba.it @cataldomusto, @semeraro_g @pasqualelops, @SWAP_research Cataldo Musto, Marco de Gemmis, Giovanni Semeraro. Pasquale Lops A Multi-criteria Recommender System Exploiting Aspect-based Sentiment Analysis of Users’ Reviews. RECSYS 2017. Como, Italy. August 30, 2017

More Related Content

PDF
Introduction to Genetic Algorithms and Evolutionary Computation
Aleksander Stensby
 
PDF
Tutorial on Deep Learning in Recommender System, Lars summer school 2019
Anoop Deoras
 
PPTX
Recommender Systems
Girish Khanzode
 
PDF
Deep Learning for Recommender Systems RecSys2017 Tutorial
Alexandros Karatzoglou
 
PDF
Calibrated Recommendations
Harald Steck
 
PPTX
Recommender systems: Content-based and collaborative filtering
Viet-Trung TRAN
 
PDF
Classification Based Machine Learning Algorithms
Md. Main Uddin Rony
 
PPTX
Recurrent Neural Network (RNN) | RNN LSTM Tutorial | Deep Learning Course | S...
Simplilearn
 
Introduction to Genetic Algorithms and Evolutionary Computation
Aleksander Stensby
 
Tutorial on Deep Learning in Recommender System, Lars summer school 2019
Anoop Deoras
 
Recommender Systems
Girish Khanzode
 
Deep Learning for Recommender Systems RecSys2017 Tutorial
Alexandros Karatzoglou
 
Calibrated Recommendations
Harald Steck
 
Recommender systems: Content-based and collaborative filtering
Viet-Trung TRAN
 
Classification Based Machine Learning Algorithms
Md. Main Uddin Rony
 
Recurrent Neural Network (RNN) | RNN LSTM Tutorial | Deep Learning Course | S...
Simplilearn
 

What's hot (20)

PDF
Performance Evaluation for Classifiers tutorial
Bilkent University
 
PDF
Personalizing the listening experience
Mounia Lalmas-Roelleke
 
PPTX
Depth estimation using deep learning
University of Oklahoma
 
PDF
Introduction to Generative Adversarial Networks (GANs)
Appsilon Data Science
 
PDF
Recent Trends in Personalization at Netflix
Förderverein Technische Fakultät
 
PPTX
Collaborative Filtering Recommendation System
Milind Gokhale
 
PDF
Sentiment Analysis
Data Science Society
 
PPTX
Linear models and multiclass classification
NdSv94
 
PPTX
Data Analysis: Evaluation Metrics for Supervised Learning Models of Machine L...
Md. Main Uddin Rony
 
PDF
Bayes Belief Networks
Sai Kumar Kodam
 
PDF
Multi-armed Bandits
Dongmin Lee
 
PPT
Amazon Item-to-Item Recommendations
Roger Chen
 
PDF
Hands on Explainable Recommender Systems with Knowledge Graphs @ RecSys22
GiacomoBalloccu
 
PDF
Visualization of Deep Learning Models (D1L6 2017 UPC Deep Learning for Comput...
Universitat Politècnica de Catalunya
 
PDF
Boston ML - Architecting Recommender Systems
James Kirk
 
PDF
XGBoost & LightGBM
Gabriel Cypriano Saca
 
PDF
Deep Learning for Recommender Systems
Yves Raimond
 
PDF
Graph Neural Networks for Recommendations
WQ Fan
 
PDF
Time, Context and Causality in Recommender Systems
Yves Raimond
 
PDF
Sequential Decision Making in Recommendations
Jaya Kawale
 
Performance Evaluation for Classifiers tutorial
Bilkent University
 
Personalizing the listening experience
Mounia Lalmas-Roelleke
 
Depth estimation using deep learning
University of Oklahoma
 
Introduction to Generative Adversarial Networks (GANs)
Appsilon Data Science
 
Recent Trends in Personalization at Netflix
Förderverein Technische Fakultät
 
Collaborative Filtering Recommendation System
Milind Gokhale
 
Sentiment Analysis
Data Science Society
 
Linear models and multiclass classification
NdSv94
 
Data Analysis: Evaluation Metrics for Supervised Learning Models of Machine L...
Md. Main Uddin Rony
 
Bayes Belief Networks
Sai Kumar Kodam
 
Multi-armed Bandits
Dongmin Lee
 
Amazon Item-to-Item Recommendations
Roger Chen
 
Hands on Explainable Recommender Systems with Knowledge Graphs @ RecSys22
GiacomoBalloccu
 
Visualization of Deep Learning Models (D1L6 2017 UPC Deep Learning for Comput...
Universitat Politècnica de Catalunya
 
Boston ML - Architecting Recommender Systems
James Kirk
 
XGBoost & LightGBM
Gabriel Cypriano Saca
 
Deep Learning for Recommender Systems
Yves Raimond
 
Graph Neural Networks for Recommendations
WQ Fan
 
Time, Context and Causality in Recommender Systems
Yves Raimond
 
Sequential Decision Making in Recommendations
Jaya Kawale
 
Ad

Similar to A Multi-Criteria Recommender System Exploiting Aspect-based Sentiment Analysis of Users' Reviews (20)

PPTX
IIR 2017, Lugano Switzerland
Marco Polignano
 
PDF
[MMIR@MM2023] On Popularity Bias of Multimodal-aware Recommender Systems: A M...
Daniele Malitesta
 
PDF
Correlation of feature score to to overall sentiment score for identifying th...
International Journal of Advance Research and Innovative Ideas in Education
 
PDF
The sarcasm detection with the method of logistic regression
EditorIJAERD
 
PDF
Affect- and Personality-based Recommender Systems Part I: Motivation, Models
Marko Tkalčič
 
PDF
Semantics-aware Recommender Systems Exploiting Linked Open Data and Graph-bas...
Cataldo Musto
 
PDF
QuESo: a Quality Model for Open Source Software Ecosystems
GESSI UPC
 
PDF
D018212428
IOSR Journals
 
PDF
Analysis, design and implementation of a Multi-Criteria Recommender System ba...
Davide Giannico
 
PDF
IRJET- Review on Different Recommendation Techniques for GRS in Online Social...
IRJET Journal
 
PDF
A Review on Sentimental Analysis of Application Reviews
IJMER
 
PDF
Ijmer 46067276
IJMER
 
PDF
Ijmer 46067276
IJMER
 
DOCX
httpowl.english.purdue.eduowlresource54401 The Pur
PazSilviapm
 
PDF
A REVIEW PAPER ON BFO AND PSO BASED MOVIE RECOMMENDATION SYSTEM | J4RV4I1015
Journal For Research
 
PDF
Streaming Analytics
IJARIIT
 
PDF
Music Recommendation System with User-based and Item-based Collaborative Filt...
ijeei-iaes
 
PDF
A Novel Latent Factor Model For Recommender System
Andrew Parish
 
PDF
A Framework for Holistic User Modeling Merging Heterogeneous Digital Footprints
Cataldo Musto
 
PDF
Modeling For Sustainability: Or How to Make Smart CPS Smarter?
Benoit Combemale
 
IIR 2017, Lugano Switzerland
Marco Polignano
 
[MMIR@MM2023] On Popularity Bias of Multimodal-aware Recommender Systems: A M...
Daniele Malitesta
 
Correlation of feature score to to overall sentiment score for identifying th...
International Journal of Advance Research and Innovative Ideas in Education
 
The sarcasm detection with the method of logistic regression
EditorIJAERD
 
Affect- and Personality-based Recommender Systems Part I: Motivation, Models
Marko Tkalčič
 
Semantics-aware Recommender Systems Exploiting Linked Open Data and Graph-bas...
Cataldo Musto
 
QuESo: a Quality Model for Open Source Software Ecosystems
GESSI UPC
 
D018212428
IOSR Journals
 
Analysis, design and implementation of a Multi-Criteria Recommender System ba...
Davide Giannico
 
IRJET- Review on Different Recommendation Techniques for GRS in Online Social...
IRJET Journal
 
A Review on Sentimental Analysis of Application Reviews
IJMER
 
Ijmer 46067276
IJMER
 
Ijmer 46067276
IJMER
 
httpowl.english.purdue.eduowlresource54401 The Pur
PazSilviapm
 
A REVIEW PAPER ON BFO AND PSO BASED MOVIE RECOMMENDATION SYSTEM | J4RV4I1015
Journal For Research
 
Streaming Analytics
IJARIIT
 
Music Recommendation System with User-based and Item-based Collaborative Filt...
ijeei-iaes
 
A Novel Latent Factor Model For Recommender System
Andrew Parish
 
A Framework for Holistic User Modeling Merging Heterogeneous Digital Footprints
Cataldo Musto
 
Modeling For Sustainability: Or How to Make Smart CPS Smarter?
Benoit Combemale
 
Ad

More from Cataldo Musto (20)

PDF
MyrrorBot: a Digital Assistant Based on Holistic User Models for Personalize...
Cataldo Musto
 
PDF
Fairness and Popularity Bias in Recommender Systems: an Empirical Evaluation
Cataldo Musto
 
PDF
Intelligenza Artificiale e Social Media - Monitoraggio della Farnesina e La M...
Cataldo Musto
 
PDF
Exploring the Effects of Natural Language Justifications in Food Recommender ...
Cataldo Musto
 
PDF
Exploiting Distributional Semantics Models for Natural Language Context-aware...
Cataldo Musto
 
PDF
Towards a Knowledge-aware Food Recommender System Exploiting Holistic User Mo...
Cataldo Musto
 
PDF
Towards Queryable User Profiles: Introducing Conversational Agents in a Platf...
Cataldo Musto
 
PDF
Hybrid Semantics aware Recommendations Exploiting Knowledge Graph Embeddings
Cataldo Musto
 
PDF
Natural Language Justifications for Recommender Systems Exploiting Text Summa...
Cataldo Musto
 
PDF
L'IA per l'Empowerment del Cittadino: Hate Map, Myrror, PA Risponde
Cataldo Musto
 
PDF
Explanation Strategies - Advances in Content-based Recommender System
Cataldo Musto
 
PDF
Justifying Recommendations through Aspect-based Sentiment Analysis of Users R...
Cataldo Musto
 
PDF
ExpLOD: un framework per la generazione di spiegazioni per recommender system...
Cataldo Musto
 
PDF
Myrror: una piattaforma per Holistic User Modeling e Quantified Self
Cataldo Musto
 
PDF
Semantic Holistic User Modeling for Personalized Access to Digital Content an...
Cataldo Musto
 
PDF
Holistic User Modeling for Personalized Services in Smart Cities
Cataldo Musto
 
PDF
eHealth, mHealth in Otorinolaringoiatria: innovazioni dirompenti o disastrose?
Cataldo Musto
 
PDF
Il Linguaggio dell'Odio sui Social Network
Cataldo Musto
 
PDF
Mappare l'Odio - Hate Speech & Social Media
Cataldo Musto
 
PDF
Recommender Systems based on Linked Open Data
Cataldo Musto
 
MyrrorBot: a Digital Assistant Based on Holistic User Models for Personalize...
Cataldo Musto
 
Fairness and Popularity Bias in Recommender Systems: an Empirical Evaluation
Cataldo Musto
 
Intelligenza Artificiale e Social Media - Monitoraggio della Farnesina e La M...
Cataldo Musto
 
Exploring the Effects of Natural Language Justifications in Food Recommender ...
Cataldo Musto
 
Exploiting Distributional Semantics Models for Natural Language Context-aware...
Cataldo Musto
 
Towards a Knowledge-aware Food Recommender System Exploiting Holistic User Mo...
Cataldo Musto
 
Towards Queryable User Profiles: Introducing Conversational Agents in a Platf...
Cataldo Musto
 
Hybrid Semantics aware Recommendations Exploiting Knowledge Graph Embeddings
Cataldo Musto
 
Natural Language Justifications for Recommender Systems Exploiting Text Summa...
Cataldo Musto
 
L'IA per l'Empowerment del Cittadino: Hate Map, Myrror, PA Risponde
Cataldo Musto
 
Explanation Strategies - Advances in Content-based Recommender System
Cataldo Musto
 
Justifying Recommendations through Aspect-based Sentiment Analysis of Users R...
Cataldo Musto
 
ExpLOD: un framework per la generazione di spiegazioni per recommender system...
Cataldo Musto
 
Myrror: una piattaforma per Holistic User Modeling e Quantified Self
Cataldo Musto
 
Semantic Holistic User Modeling for Personalized Access to Digital Content an...
Cataldo Musto
 
Holistic User Modeling for Personalized Services in Smart Cities
Cataldo Musto
 
eHealth, mHealth in Otorinolaringoiatria: innovazioni dirompenti o disastrose?
Cataldo Musto
 
Il Linguaggio dell'Odio sui Social Network
Cataldo Musto
 
Mappare l'Odio - Hate Speech & Social Media
Cataldo Musto
 
Recommender Systems based on Linked Open Data
Cataldo Musto
 

Recently uploaded (20)

PPTX
1st Inaugural Professorial Lecture held on 19th February 2020 (Governance and...
kawisojames10
 
PPTX
Institutional Correction lecture only . . .
limvtheghins
 
PDF
O5-L3 Freight Transport Ops (International) V1.pdf
labyankof
 
PDF
RMMM.pdf make it easy to upload and study
sajedul2
 
PDF
TR - Agricultural Crops Production NC III.pdf
avgilmegino3
 
PPTX
PPH.pptx obstetrics and gynecology in nursing
SrideviDevaraj5
 
PDF
Computing-Curriculum for Schools in Ghana
abigailanane203
 
PDF
Module 4: Burden of Disease Tutorial Slides S2 2025
Jonathan Hallett
 
PDF
grade 11-chemistry_fetena_net_5883.pdf teacher guide for all student
banteamlakmebratu738
 
PPTX
master seminar digital applications in india
ananyaray35
 
PPTX
Cell Types and Its function , kingdom of life
ClaireMangundayao1
 
PPTX
PPT- ENG7_QUARTER1_LESSON1_WEEK1. IMAGERY -DESCRIPTIONS pptx.pptx
KMDee
 
PDF
Chapter 2 Heredity, Prenatal Development, and Birth.pdf
MarjorieLopezTiu
 
PPTX
Renaissance Architecture: A Journey from Faith to Humanism
DrMonicaSharma
 
PPTX
GDM (1) (1).pptx small presentation for students
shrawanbpkihs
 
PDF
3rd Neelam Sanjeevareddy Memorial Lecture.pdf
Academy of Grassroots Studies and Research of India (AGRASRI)
 
PDF
Complications of Minimal Access Surgery at WLH
mohitsuren827
 
PDF
Pre independence Education in Inndia.pdf
goofy5603
 
PPTX
Final Presentation General Medicine 03-08-2024.pptx
ZaheerAhmad228692
 
PDF
Insiders guide to clinical Medicine.pdf
AbhishekPatil315128
 
1st Inaugural Professorial Lecture held on 19th February 2020 (Governance and...
kawisojames10
 
Institutional Correction lecture only . . .
limvtheghins
 
O5-L3 Freight Transport Ops (International) V1.pdf
labyankof
 
RMMM.pdf make it easy to upload and study
sajedul2
 
TR - Agricultural Crops Production NC III.pdf
avgilmegino3
 
PPH.pptx obstetrics and gynecology in nursing
SrideviDevaraj5
 
Computing-Curriculum for Schools in Ghana
abigailanane203
 
Module 4: Burden of Disease Tutorial Slides S2 2025
Jonathan Hallett
 
grade 11-chemistry_fetena_net_5883.pdf teacher guide for all student
banteamlakmebratu738
 
master seminar digital applications in india
ananyaray35
 
Cell Types and Its function , kingdom of life
ClaireMangundayao1
 
PPT- ENG7_QUARTER1_LESSON1_WEEK1. IMAGERY -DESCRIPTIONS pptx.pptx
KMDee
 
Chapter 2 Heredity, Prenatal Development, and Birth.pdf
MarjorieLopezTiu
 
Renaissance Architecture: A Journey from Faith to Humanism
DrMonicaSharma
 
GDM (1) (1).pptx small presentation for students
shrawanbpkihs
 
3rd Neelam Sanjeevareddy Memorial Lecture.pdf
Academy of Grassroots Studies and Research of India (AGRASRI)
 
Complications of Minimal Access Surgery at WLH
mohitsuren827
 
Pre independence Education in Inndia.pdf
goofy5603
 
Final Presentation General Medicine 03-08-2024.pptx
ZaheerAhmad228692
 
Insiders guide to clinical Medicine.pdf
AbhishekPatil315128
 

A Multi-Criteria Recommender System Exploiting Aspect-based Sentiment Analysis of Users' Reviews

  • 1. @cataldomusto @pasqualelops @semeraro_g @SWAP_research A Multi-criteria Recommender System Exploiting Aspect-based Sentiment Analysis of Users’ Reviews CATALDO MUSTO, MARCO DE GEMMIS, GIOVANNI SEMERARO, PASQUALE LOPS UNIVERSITÀ DEGLI STUDI DI BARI ‘ALDO MORO’ - ITALY RecSys 2017 - 11th ACM Conference on Recommender Systems Como, Italy August 30, 2017 cataldo.musto@uniba.it
  • 2. Multi-Criteria RecSys Cataldo Musto, Marco de Gemmis, Giovanni Semeraro. Pasquale Lops A Multi-criteria Recommender System Exploiting Aspect-based Sentiment Analysis of Users’ Reviews. RECSYS 2017. Como, Italy. August 30, 2017 Not a new concept (*) Each user evaluates each aspect of the item (*) Adomavicius, Gediminas, and YoungOk Kwon. "Multi-criteria recommender systems." Recommender Systems Handbook. Springer US, 2015. 847-880.
  • 3. Multi-Criteria RecSys Cataldo Musto, Marco de Gemmis, Giovanni Semeraro. Pasquale Lops A Multi-criteria Recommender System Exploiting Aspect-based Sentiment Analysis of Users’ Reviews. RECSYS 2017. Como, Italy. August 30, 2017 Not a new concept (*) Each user evaluates each aspect of the item (*) Adomavicius, Gediminas, and YoungOk Kwon. "Multi-criteria recommender systems." Recommender Systems Handbook. Springer US, 2015. 847-880. Problem: Overwhelming!
  • 4. Multi-Criteria RecSys Cataldo Musto, Marco de Gemmis, Giovanni Semeraro. Pasquale Lops A Multi-criteria Recommender System Exploiting Aspect-based Sentiment Analysis of Users’ Reviews. RECSYS 2017. Como, Italy. August 30, 2017 Not a new concept (*) Each user evaluates each aspect of the item (*) Adomavicius, Gediminas, and YoungOk Kwon. "Multi-criteria recommender systems." Recommender Systems Handbook. Springer US, 2015. 847-880. Problem: Aspects are not fixed!
  • 5. Multi-Criteria RecSys Cataldo Musto, Marco de Gemmis, Giovanni Semeraro. Pasquale Lops A Multi-criteria Recommender System Exploiting Aspect-based Sentiment Analysis of Users’ Reviews. RECSYS 2017. Como, Italy. August 30, 2017 Not a new concept (*) Each user evaluates each aspect of the item (*) Adomavicius, Gediminas, and YoungOk Kwon. "Multi-criteria recommender systems." Recommender Systems Handbook. Springer US, 2015. 847-880. Problem: Aspects can be further modeled as a hierarchy
  • 6. Research Question Cataldo Musto, Marco de Gemmis, Giovanni Semeraro. Pasquale Lops A Multi-criteria Recommender System Exploiting Aspect-based Sentiment Analysis of Users’ Reviews. RECSYS 2017. Como, Italy. August 30, 2017 How to develop a multi-criteria data model without overwhelming the user ?
  • 7. Research Question Cataldo Musto, Marco de Gemmis, Giovanni Semeraro. Pasquale Lops A Multi-criteria Recommender System Exploiting Aspect-based Sentiment Analysis of Users’ Reviews. RECSYS 2017. Como, Italy. August 30, 2017 What is the performance of such a data model in a collaborative recommendation scenario?
  • 8. Our contribution Cataldo Musto, Marco de Gemmis, Giovanni Semeraro. Pasquale Lops A Multi-criteria Recommender System Exploiting Aspect-based Sentiment Analysis of Users’ Reviews. RECSYS 2017. Como, Italy. August 30, 2017 A multi-criteria collaborative recommendation methodology exploiting aspect-based sentiment analysis of users’ reviews
  • 9. Methodology Cataldo Musto, Marco de Gemmis, Giovanni Semeraro. Pasquale Lops A Multi-criteria Recommender System Exploiting Aspect-based Sentiment Analysis of Users’ Reviews. RECSYS 2017. Como, Italy. August 30, 2017 Input: textual reviews Step 1: aspect extraction and sentiment analysis Step 2: creating multi-criteria data model Output: recommendations
  • 10. Methodology Cataldo Musto, Marco de Gemmis, Giovanni Semeraro. Pasquale Lops A Multi-criteria Recommender System Exploiting Aspect-based Sentiment Analysis of Users’ Reviews. RECSYS 2017. Como, Italy. August 30, 2017 Input: textual reviews Step 1: aspect extraction and sentiment analysis Step 2: creating multi-criteria data model Output: recommendations
  • 11. Methodology Cataldo Musto, Marco de Gemmis, Giovanni Semeraro. Pasquale Lops A Multi-criteria Recommender System Exploiting Aspect-based Sentiment Analysis of Users’ Reviews. RECSYS 2017. Como, Italy. August 30, 2017 Input: textual reviews Step 1: aspect extraction and sentiment analysis Step 2: creating and filling our multi-criteria data model Output: recommendations
  • 12. Methodology Cataldo Musto, Marco de Gemmis, Giovanni Semeraro. Pasquale Lops A Multi-criteria Recommender System Exploiting Aspect-based Sentiment Analysis of Users’ Reviews. RECSYS 2017. Como, Italy. August 30, 2017 Input: textual reviews Step 1: aspect extraction and sentiment analysis Step 2: creating and filling our multi-criteria data model Output: recommendations
  • 13. Aspect Extraction and Sentiment Analysis Cataldo Musto, Marco de Gemmis, Giovanni Semeraro. Pasquale Lops A Multi-criteria Recommender System Exploiting Aspect-based Sentiment Analysis of Users’ Reviews. RECSYS 2017. Como, Italy. August 30, 2017 Algorithm based on SABRE(*) (Sentiment Aspect-based Retrieval Engine) (*) Caputo, A., Basile, P., de Gemmis, M., Lops, P., Semeraro, G., & Rossiello, G. (2017). SABRE: A Sentiment Aspect-Based Retrieval Engine. In Information Filtering and Retrieval (pp. 63-78). 𝑅 = {𝑟1, 𝑟2 … 𝑟 𝑛}Input: set of reviews Output: quintuples < 𝑟𝑖, 𝑎𝑖𝑗, 𝑎𝑖𝑗𝑘, 𝑟𝑒𝑙 𝑎𝑖𝑗𝑘, 𝑟𝑖 , 𝑠𝑒𝑛𝑡(𝑎𝑖𝑗𝑘, 𝑟𝑖) >
  • 14. Aspect Extraction and Sentiment Analysis Cataldo Musto, Marco de Gemmis, Giovanni Semeraro. Pasquale Lops A Multi-criteria Recommender System Exploiting Aspect-based Sentiment Analysis of Users’ Reviews. RECSYS 2017. Como, Italy. August 30, 2017 Algorithm based on SABRE(*) (Sentiment Aspect-based Retrieval Engine) (*) Caputo, A., Basile, P., de Gemmis, M., Lops, P., Semeraro, G., & Rossiello, G. (2017). SABRE: A Sentiment Aspect-Based Retrieval Engine. In Information Filtering and Retrieval (pp. 63-78). 𝑅 = {𝑟1, 𝑟2 … 𝑟 𝑛}Input: set of reviews Output: quintuples < 𝑟𝑖, 𝑎𝑖𝑗, 𝑎𝑖𝑗𝑘, 𝑟𝑒𝑙 𝑎𝑖𝑗𝑘, 𝑟𝑖 , 𝑠𝑒𝑛𝑡(𝑎𝑖𝑗𝑘, 𝑟𝑖) > 𝑟𝑖 = 𝑎𝑖𝑗, 𝑎𝑖𝑗𝑘= 𝑟𝑒𝑙 𝑎𝑖𝑗𝑘, 𝑟𝑖 , 𝑠𝑒𝑛𝑡(𝑎𝑖𝑗𝑘, 𝑟𝑖) = i-th review j-th aspect and k-th sub-aspect in the i-th review relevance and sentiment
  • 15. Aspect Extraction and Sentiment Analysis Cataldo Musto, Marco de Gemmis, Giovanni Semeraro. Pasquale Lops A Multi-criteria Recommender System Exploiting Aspect-based Sentiment Analysis of Users’ Reviews. RECSYS 2017. Como, Italy. August 30, 2017 Algorithm based on SABRE(*) (Sentiment Aspect-based Retrieval Engine) (*) Caputo, A., Basile, P., de Gemmis, M., Lops, P., Semeraro, G., & Rossiello, G. (2017). SABRE: A Sentiment Aspect-Based Retrieval Engine. In Information Filtering and Retrieval (pp. 63-78). 𝑅 = {𝑟1, 𝑟2 … 𝑟 𝑛}Input: set of reviews Output: quintuples < 𝑟𝑖, 𝑎𝑖𝑗, 𝑎𝑖𝑗𝑘, 𝑟𝑒𝑙 𝑎𝑖𝑗𝑘, 𝑟𝑖 , 𝑠𝑒𝑛𝑡(𝑎𝑖𝑗𝑘, 𝑟𝑖) > 𝑟𝑖 = 𝑎𝑖𝑗, 𝑎𝑖𝑗𝑘= 𝑟𝑒𝑙 𝑎𝑖𝑗𝑘, 𝑟𝑖 , 𝑠𝑒𝑛𝑡(𝑎𝑖𝑗𝑘, 𝑟𝑖) = i-th review j-th aspect and k-th sub-aspect in the i-th review relevance and sentiment How do we extract aspects, relevance and sentiment?
  • 16. Aspect Extraction and Sentiment Analysis Cataldo Musto, Marco de Gemmis, Giovanni Semeraro. Pasquale Lops A Multi-criteria Recommender System Exploiting Aspect-based Sentiment Analysis of Users’ Reviews. RECSYS 2017. Como, Italy. August 30, 2017 Aspect Extraction (*) Caputo, A., Basile, P., de Gemmis, M., Lops, P., Semeraro, G., & Rossiello, G. (2017). SABRE: A Sentiment Aspect-Based Retrieval Engine. In Information Filtering and Retrieval (pp. 63-78). Statistical approach based on the Kullback-Leibler (KL) Divergence
  • 17. Aspect Extraction and Sentiment Analysis Cataldo Musto, Marco de Gemmis, Giovanni Semeraro. Pasquale Lops A Multi-criteria Recommender System Exploiting Aspect-based Sentiment Analysis of Users’ Reviews. RECSYS 2017. Como, Italy. August 30, 2017 Aspect Extraction (*) Caputo, A., Basile, P., de Gemmis, M., Lops, P., Semeraro, G., & Rossiello, G. (2017). SABRE: A Sentiment Aspect-Based Retrieval Engine. In Information Filtering and Retrieval (pp. 63-78). Statistical approach based on the Kullback-Leibler (KL) Divergence Measures the difference between the distribution of a term in a generic corpus (e.g. BNC) and its distribution in a domain corpus (e.g. hotel reviews) Insight: the higher the divergence, the higher the importance of the term in the domain
  • 18. Aspect Extraction and Sentiment Analysis Cataldo Musto, Marco de Gemmis, Giovanni Semeraro. Pasquale Lops A Multi-criteria Recommender System Exploiting Aspect-based Sentiment Analysis of Users’ Reviews. RECSYS 2017. Como, Italy. August 30, 2017 Aspect Extraction (*) Caputo, A., Basile, P., de Gemmis, M., Lops, P., Semeraro, G., & Rossiello, G. (2017). SABRE: A Sentiment Aspect-Based Retrieval Engine. In Information Filtering and Retrieval (pp. 63-78). Statistical approach based on the Kullback-Leibler (KL) Divergence Measures the difference between the distribution of a term in a generic corpus (e.g. BNC) and its distribution in a domain corpus (e.g. hotel reviews) Insight: the higher the divergence, the higher the importance of the term in the domain KL(room, BNC, hotel-reviews) >> 0 KL(food, BNC, hotel-reviews) > 0 KL(place, BNC, hotel-reviews) ~ 0 KL(politics, BNC, hotel-reviews) ~ 0 We label as ‘aspects’ the nouns whose KL-divergence is higher than zero
  • 19. Aspect Extraction and Sentiment Analysis Cataldo Musto, Marco de Gemmis, Giovanni Semeraro. Pasquale Lops A Multi-criteria Recommender System Exploiting Aspect-based Sentiment Analysis of Users’ Reviews. RECSYS 2017. Como, Italy. August 30, 2017 Aspect Extraction (*) Caputo, A., Basile, P., de Gemmis, M., Lops, P., Semeraro, G., & Rossiello, G. (2017). SABRE: A Sentiment Aspect-Based Retrieval Engine. In Information Filtering and Retrieval (pp. 63-78). Statistical approach based on the Kullback-Leibler (KL) Divergence Measures the difference between the distribution of a term in a generic corpus (e.g. BNC) and its distribution in a domain corpus (e.g. hotel reviews) Insight: the higher the divergence, the higher the importance of the term in the domain KL(room, BNC, hotel-reviews) >> 0 YES KL(food, BNC, hotel-reviews) > 0 YES KL(place, BNC, hotel-reviews) ~ 0 NO KL(politics, BNC, hotel-reviews) ~ 0 NO
  • 20. Aspect Extraction and Sentiment Analysis Cataldo Musto, Marco de Gemmis, Giovanni Semeraro. Pasquale Lops A Multi-criteria Recommender System Exploiting Aspect-based Sentiment Analysis of Users’ Reviews. RECSYS 2017. Como, Italy. August 30, 2017 Aspect Extraction (*) Caputo, A., Basile, P., de Gemmis, M., Lops, P., Semeraro, G., & Rossiello, G. (2017). SABRE: A Sentiment Aspect-Based Retrieval Engine. In Information Filtering and Retrieval (pp. 63-78). Statistical approach based on the Kullback-Leibler (KL) Divergence Measures the difference between the distribution of a term in a generic corpus (e.g. BNC) and its distribution in a domain corpus (e.g. hotel reviews) Insight: the higher the divergence, the higher the importance of the term in the domain KL(room, BNC, hotel-reviews) >> 0 YES KL(food, BNC, hotel-reviews) > 0 YES KL(place, BNC, hotel-reviews) ~ 0 NO KL(politics, BNC, hotel-reviews) ~ 0 NO Distinguishing aspect: the set is not fixed!
  • 21. Aspect Extraction and Sentiment Analysis Cataldo Musto, Marco de Gemmis, Giovanni Semeraro. Pasquale Lops A Multi-criteria Recommender System Exploiting Aspect-based Sentiment Analysis of Users’ Reviews. RECSYS 2017. Como, Italy. August 30, 2017 Sub-aspect Extraction (*) Caputo, A., Basile, P., de Gemmis, M., Lops, P., Semeraro, G., & Rossiello, G. (2017). SABRE: A Sentiment Aspect-Based Retrieval Engine. In Information Filtering and Retrieval (pp. 63-78). Another distinguishing aspect: we can extract a hierarchy of terms Based on Phraseness and Informativeness: They measure the gain in information if two terms are modeled together Insight: if phraseness and informativeness are high, the terms have an high cohesion
  • 22. Aspect Extraction and Sentiment Analysis Cataldo Musto, Marco de Gemmis, Giovanni Semeraro. Pasquale Lops A Multi-criteria Recommender System Exploiting Aspect-based Sentiment Analysis of Users’ Reviews. RECSYS 2017. Como, Italy. August 30, 2017 Sub-aspect Extraction (*) Caputo, A., Basile, P., de Gemmis, M., Lops, P., Semeraro, G., & Rossiello, G. (2017). SABRE: A Sentiment Aspect-Based Retrieval Engine. In Information Filtering and Retrieval (pp. 63-78). Another distinguishing aspect: we can extract a hierarchy of terms Based on Phraseness and Informativeness: They measure the gain in information if two terms are modeled together Insight: if phraseness and informativeness are high, the terms have an high cohesion SUB(room, food, hotel-reviews) ~ 0 NO SUB(room, shower, hotel-reviews) > 0 YES SUB(food, wine, hotel-reviews) > 0 YES SUB(food, service, hotel-reviews) ~ 0 NO
  • 23. Aspect Extraction and Sentiment Analysis Cataldo Musto, Marco de Gemmis, Giovanni Semeraro. Pasquale Lops A Multi-criteria Recommender System Exploiting Aspect-based Sentiment Analysis of Users’ Reviews. RECSYS 2017. Como, Italy. August 30, 2017 Sabre@Work (*) Real review of the hotel we actually stay in Como :) (*)
  • 24. Aspect Extraction and Sentiment Analysis Cataldo Musto, Marco de Gemmis, Giovanni Semeraro. Pasquale Lops A Multi-criteria Recommender System Exploiting Aspect-based Sentiment Analysis of Users’ Reviews. RECSYS 2017. Como, Italy. August 30, 2017 < 𝑟, 𝑏𝑟𝑒𝑎𝑘𝑓𝑎𝑠𝑡,∗, 1.5, 0.5 > Sabre@Work (*) Real review of the hotel we actually stay in Como :) (*)
  • 25. Aspect Extraction and Sentiment Analysis Cataldo Musto, Marco de Gemmis, Giovanni Semeraro. Pasquale Lops A Multi-criteria Recommender System Exploiting Aspect-based Sentiment Analysis of Users’ Reviews. RECSYS 2017. Como, Italy. August 30, 2017 < 𝑟, 𝑏𝑟𝑒𝑎𝑘𝑓𝑎𝑠𝑡,∗, 1.5, 0.5 > No sub-aspects Relevance=KL-divergence score Sentiment = lexicon-based approach based on AFINN wordlist (*) or machine-learning based approach based on CoreNLP (^) (*) http://neuro.imm.dtu.dk/wiki/AFINN (^) https://nlp.stanford.edu/sentiment/code.html Sabre@Work (*) Real review of the hotel we actually stay in Como :) (*)
  • 26. Aspect Extraction and Sentiment Analysis Cataldo Musto, Marco de Gemmis, Giovanni Semeraro. Pasquale Lops A Multi-criteria Recommender System Exploiting Aspect-based Sentiment Analysis of Users’ Reviews. RECSYS 2017. Como, Italy. August 30, 2017 < 𝑟, 𝑏𝑟𝑒𝑎𝑘𝑓𝑎𝑠𝑡,∗, 1.5, 0.5 > < 𝑟, 𝑟𝑜𝑜𝑚, 𝑠ℎ𝑜𝑤𝑒𝑟, 1.2, −0.5 > Sabre@Work (*) Real review of the hotel we actually stay in Como :) (*)
  • 27. Aspect Extraction and Sentiment Analysis Cataldo Musto, Marco de Gemmis, Giovanni Semeraro. Pasquale Lops A Multi-criteria Recommender System Exploiting Aspect-based Sentiment Analysis of Users’ Reviews. RECSYS 2017. Como, Italy. August 30, 2017 < 𝑟, 𝑏𝑟𝑒𝑎𝑘𝑓𝑎𝑠𝑡,∗, 1.5, 0.5 > < 𝑟, 𝑟𝑜𝑜𝑚, 𝑠ℎ𝑜𝑤𝑒𝑟, 1.2, −0.5 > < 𝑟, 𝑟𝑜𝑜𝑚,∗, 1.3, 0.2 > Sabre@Work (*) Real review of the hotel we actually stay in Como :) (*)
  • 28. Aspect Extraction and Sentiment Analysis Cataldo Musto, Marco de Gemmis, Giovanni Semeraro. Pasquale Lops A Multi-criteria Recommender System Exploiting Aspect-based Sentiment Analysis of Users’ Reviews. RECSYS 2017. Como, Italy. August 30, 2017 < 𝑟, 𝑏𝑟𝑒𝑎𝑘𝑓𝑎𝑠𝑡,∗, 1.5, 0.5 > < 𝑟, 𝑟𝑜𝑜𝑚, 𝑠ℎ𝑜𝑤𝑒𝑟, 1.2, −0.5 > < 𝑟, 𝑟𝑜𝑜𝑚,∗, 1.3, 0.2 > Sabre@Work … … . 𝑒𝑡𝑐. (*) Real review of the hotel we actually stay in Como :) (*)
  • 29. Multi-Criteria Data Model Cataldo Musto, Marco de Gemmis, Giovanni Semeraro. Pasquale Lops A Multi-criteria Recommender System Exploiting Aspect-based Sentiment Analysis of Users’ Reviews. RECSYS 2017. Como, Italy. August 30, 2017 A multi-criteria data model is automatically filled by exploiting the aspects extracted from the review and their sentiment Finer-Grained Representation!
  • 30. Providing Recommendations Cataldo Musto, Marco de Gemmis, Giovanni Semeraro. Pasquale Lops A Multi-criteria Recommender System Exploiting Aspect-based Sentiment Analysis of Users’ Reviews. RECSYS 2017. Como, Italy. August 30, 2017 Similarity is calculated through multi-criteria Euclidean distance Recommendations are provided by exploiting both User-to-User and Item-to-Item Collaborative Filtering Recommendation
  • 31. Framework Recap Cataldo Musto, Marco de Gemmis, Giovanni Semeraro. Pasquale Lops A Multi-criteria Recommender System Exploiting Aspect-based Sentiment Analysis of Users’ Reviews. RECSYS 2017. Como, Italy. August 30, 2017
  • 32. Experiments Which combination of the parameters led to the best predictive accuracy? How does our framework perform when compared to single-criteria recommendations and matrix factorization tecniques? Cataldo Musto, Marco de Gemmis, Giovanni Semeraro. Pasquale Lops A Multi-criteria Recommender System Exploiting Aspect-based Sentiment Analysis of Users’ Reviews. RECSYS 2017. Como, Italy. August 30, 2017
  • 33. Datasets Yelp 45,981 users 11,537 items 229,606 ratings(*) 99.95% sparsity TripAdvisor 536,952 users 3,945 items 796,958 ratings(*) 99.96% sparsity Amazon 826,773 users 50,210 items 1,324,759 ratings(*) 99.99% sparsity (*) Ratings = ratings + reviews Cataldo Musto, Marco de Gemmis, Giovanni Semeraro. Pasquale Lops A Multi-criteria Recommender System Exploiting Aspect-based Sentiment Analysis of Users’ Reviews. RECSYS 2017. Como, Italy. August 30, 2017
  • 34. Datasets Yelp 45,981 users 11,537 items 229,606 ratings(*) 99.95% sparsity TripAdvisor 536,952 users 3,945 items 796,958 ratings(*) 99.96% sparsity Amazon 826,773 users 50,210 items 1,324,759 ratings(*) 99.99% sparsity (*) Ratings = ratings + reviews Cataldo Musto, Marco de Gemmis, Giovanni Semeraro. Pasquale Lops A Multi-criteria Recommender System Exploiting Aspect-based Sentiment Analysis of Users’ Reviews. RECSYS 2017. Como, Italy. August 30, 2017
  • 35. Experimental Settings Review Processing ◦ Stop-Word removed ◦ Entity and Collocations recognized SABRE parameters ◦ With/without subaspects ◦ #aspects/#subaspects = 10, 50 ◦ KL-divergence threshold = 0.1 ◦ Only nouns! Recommendations ◦ Multi-Criteria U2U and I2I Metric ◦ MAE (calculated with Rival framework) Cataldo Musto, Marco de Gemmis, Giovanni Semeraro. Pasquale Lops A Multi-criteria Recommender System Exploiting Aspect-based Sentiment Analysis of Users’ Reviews. RECSYS 2017. Como, Italy. August 30, 2017
  • 36. Baselines Single-Criteria Recommendations techniques ◦ User-to-User Collaborative Filtering ◦ Item-to-Item Collaborative Filtering Static Multi-Criteria Recommendations ◦ Only on TripAdvisor data Matrix Factorization techniques ◦ SGD (Stochastic Gradient Descent) ◦ ParallelSGD ◦ ALSWR ◦ Implementations available in Mahout ◦ Tuning of parameters Alessandro Suglia, Claudio Greco, Cataldo Musto, Marco de Gemmis, Pasquale Lops, Giovanni Semeraro. A Deep Architecture for Content-based Recommendations Exploiting Recurrent Neural Networks. UMAP 2017. Bratislava, Slovakia. July 12, 2017
  • 37. Top-10 aspects Place Food Service Restaurant Price Menu Staff Drink Lunch Table Hotel Room Staff Location Service Breakfast Restaurant Bathroom Price View Game Graphic Story Character Player Price Gameplay Controller Level Music Cataldo Musto, Marco de Gemmis, Giovanni Semeraro. Pasquale Lops A Multi-criteria Recommender System Exploiting Aspect-based Sentiment Analysis of Users’ Reviews. RECSYS 2017. Como, Italy. August 30, 2017
  • 38. 0,7111 0,7564 0,7269 0,8007 0,65 0,67 0,69 0,71 0,73 0,75 0,77 0,79 0,81 0,83 TripAdvisor 10 neigh. / 10 aspects / sub-aspects 10 neigh. / 10 aspects / no sub-aspects 10 neigh. / 50 aspects / sub-aspects 10 neigh. / 50 aspects / no sub-aspects Outcomes Best-results obtained with 10 aspects Best-results obtained by also introducing sub-aspects (Amazon had a different behavior) Lower MAE! Results – Multi-Criteria User-to-User CF Cataldo Musto, Marco de Gemmis, Giovanni Semeraro. Pasquale Lops A Multi-criteria Recommender System Exploiting Aspect-based Sentiment Analysis of Users’ Reviews. RECSYS 2017. Como, Italy. August 30, 2017
  • 39. 0,7111 0,798 0,8245 0,8429 0,6 0,65 0,7 0,75 0,8 0,85 TripAdvisor Multi-U2U Static-Multi-U2U Multi-I2I Static-Multi-I2I Outcomes TripAdvisors data included ratings about six static aspects (cleanliness, location, value, service, sleep quality, overall) Our approach based on unsupervised aspect extraction also improved these results Results – vs. Static Multi-Criteria RecSys Cataldo Musto, Marco de Gemmis, Giovanni Semeraro. Pasquale Lops A Multi-criteria Recommender System Exploiting Aspect-based Sentiment Analysis of Users’ Reviews. RECSYS 2017. Como, Italy. August 30, 2017
  • 40. 0,7111 0,8337 0,745 0,7449 0,9053 0,65 0,7 0,75 0,8 0,85 0,9 0,95 1 TripAdvisor Multi-U2U Single-U2U Ratings-SGD Parallel-SGD ALSWR Outcomes Our approach overcomes all the baselines. Our framework wins the comparisons to Single-U2U and Single-I2I Also matrix factorization techniques got an higher MAE Results – Baselines Cataldo Musto, Marco de Gemmis, Giovanni Semeraro. Pasquale Lops A Multi-criteria Recommender System Exploiting Aspect-based Sentiment Analysis of Users’ Reviews. RECSYS 2017. Como, Italy. August 30, 2017
  • 41. Recap Results ☺ Our framework significantly improves all the baselines ☺ Unsupervised Aspect Extraction also overcomes static aspects Future Work: evaluate data model with more sophisticated algorithms Cataldo Musto, Marco de Gemmis, Giovanni Semeraro. Pasquale Lops A Multi-criteria Recommender System Exploiting Aspect-based Sentiment Analysis of Users’ Reviews. RECSYS 2017. Como, Italy. August 30, 2017
  • 42. Thanks! cataldo.musto@uniba.it @cataldomusto, @semeraro_g @pasqualelops, @SWAP_research Cataldo Musto, Marco de Gemmis, Giovanni Semeraro. Pasquale Lops A Multi-criteria Recommender System Exploiting Aspect-based Sentiment Analysis of Users’ Reviews. RECSYS 2017. Como, Italy. August 30, 2017