![Related work
▪ Online Travel Agencies reshaped the ecosystem [1]
▪ …and eWOM has a strong bias/influence on online decision making [2,4,5]
▪ …as well as to predict business performance [3]
▪ From a CS perspective eWOM is a main ingredient for algorithmic decision support
mechanism like RS
▪ Experiments in psychology literature[6] revealed users with different decision
making styles
[1] Xiang, Z. et al. Information technology and consumer behavior in travel and tourism: Insights from travel planning using the internet. JRCS 2015
[2] Ulrike Gretzel and Kyung Hyan Yoo. Use and impact of online travel reviews. Information and Communication Technologies in Tourism 2008
[3] Xie, K. et al. The business value of online consumer reviews and management response to hotel performance. IJHM 2014
[4] Xiang, Z. et al. A comparative analysis of major online review platforms: Implications for social media analytics inhospitality and tourism. TM 2017
[5] Xie, H. et al. Consumers responses to ambivalent online hotel reviews: The role of perceived source credibility and predecisional disposition. IJHM 2011
[6] Schwartz, B. et al. Maximizing versus satisficing: Happiness is a matter of choice. JPSP 2002
4](https://image.slidesharecdn.com/enter19-190203142244/85/Decision-Making-Based-on-Bimodal-Rating-Summary-Statistics-An-Eye-Tracking-Study-of-Hotels-4-320.jpg){kind=tracking}
![Decision Making on Multi-attribute Items
▪ Non-Compensatory Strategies [1]:
▫ Compare items based on one attribute
▫ Perform intra-dimensional comparisons
▫ Perform less comparisons
▪ Compensatory Strategies [1]:
▫ All attributes meet a minimum requirement
▫ Multiple inter-dimensional comparisons
▫ Spend more time on items
Eye movement is an indicator of the screening of the choices [2].
7[1] John W Payne. Task complexity and contingentprocessing in decision making: An information search and protocol analysis. Organizational Behavior and Human
Performance, 1976
[2] Jacob L. Orquin and Simone Mueller Loose. Attention and choice: A review on eye movements in decision making. Acta Psychologica 2013](https://image.slidesharecdn.com/enter19-190203142244/85/Decision-Making-Based-on-Bimodal-Rating-Summary-Statistics-An-Eye-Tracking-Study-of-Hotels-7-320.jpg){kind=tracking}
![Decision making strategies
▪ Interpersonal differences
▪ Satisficer / Maximizers [1]
▪ Three subdimensions[2]:
▫ Decision Difficulty
▫ Alternative Search
▫ High Standards
8
[1] Herbert A Simon. A behavioral model of rational choice. The quarterly journal of economics, 1955
[2] Schwartz et al., 2002, Maximizing versus satisficing: Happiness is a matter of choice. Journal of Personality and Social Psychology 1983
Herbert Simon](https://image.slidesharecdn.com/enter19-190203142244/85/Decision-Making-Based-on-Bimodal-Rating-Summary-Statistics-An-Eye-Tracking-Study-of-Hotels-8-320.jpg){kind=tracking}
![Earlier Work
▪ We run a set of 3 experiments to understand trade-off
mechanisms between decision strategies
▪ Decomposing rating summaries: Different types of explanations,
Number of ratings, Mean of the ratings, Variance, Skewness
▪ Respondents:
▫ Relied highly on the mean rating
▫ Non linear influence of overall Number of Ratings
▫ Variance and skewness remain largely unnoticed
▫ Maximizers vs. Satisficers display different preferences
9
[1] Coba L., Zanker M., Rook L., Symeonidis P.: Exploring Users' Perception of Rating Summary Statistics. UMAP ‘18
[2] Coba L., Zanker M., Rook L., Symeonidis P.: Exploring Users' Perceptionof Collaborative Explanation Styles. CBI ’18
[3] Coba L., Zanker M., Rook L., Symeonidis P.: Decision Making Strategies Differ in the Presenceof Collaborative Explanations: Two Conjoint Studies. IUI
’19](https://image.slidesharecdn.com/enter19-190203142244/85/Decision-Making-Based-on-Bimodal-Rating-Summary-Statistics-An-Eye-Tracking-Study-of-Hotels-9-320.jpg){kind=tracking}
![Data
▪ Data driven levels [1]
▪ J-shaped [2]
▪ Bimodality coefficient[3]:
12
[1] Markus Zanker and Martin Schoberegger. An empirical study on the persuasiveness
of fact-based explanations for recommender systems. RecSys 2014
[2] Hu N, Zhang J, Pavlou PA (2009) Overcomingthe J-shaped distribution of product reviews.
Commun ACM
[3] Pfister R, Schwarz KA, Janczyk M, Dale R, Freeman JB (2013) Good things peak in pairs: a note on
the bimodality coefficient. Front Psychol](https://image.slidesharecdn.com/enter19-190203142244/85/Decision-Making-Based-on-Bimodal-Rating-Summary-Statistics-An-Eye-Tracking-Study-of-Hotels-12-320.jpg){kind=tracking}
![Eye-tracking Metrics
▪ Area of Interest (AOI)[1]
▪ Fixation times
▫ Geometrical mean [2]
▪ Revisits [3]
15
[1]Kenneth Holmqvist, Marcus Nystroom, Richard Andersson, Richard Dewhurst,
Halszka Jarodzka, and Joost Van De Weijer. Eye tracking. A comprehensive guide to
methods and measures. Oxford University Press, 2011.
[2] Jeff Sauro and James R. Lewis. Average task times in usability tests. CHI’10
[3] John W Payne. Task complexity and contingent processing in decision making: An
information search and protocol analysis. Organizational Behavior and Human
Performance, 1976](https://image.slidesharecdn.com/enter19-190203142244/85/Decision-Making-Based-on-Bimodal-Rating-Summary-Statistics-An-Eye-Tracking-Study-of-Hotels-15-320.jpg){kind=tracking}
Decision Making Based on Bimodal Rating Summary Statistics - An Eye-Tracking Study of Hotels
- 1. Ludovik Coba lucoba@unibz.it Markus Zanker mzanker@unibz.it Laurens Rook l.rook@tudelft.nl DecisionMakingBasedon BimodalRatingSummary Statistics An Eye-TrackingStudyofHotels
- 2. Introduction Intro – Related work – Earlier Work
- 3. Examples: Electronic Word-of-Mouth (Ratings Summary Statistics) 3
- 4. Related work ▪ Online Travel Agencies reshaped the ecosystem [1] ▪ …and eWOM has a strong bias/influence on online decision making [2,4,5] ▪ …as well as to predict business performance [3] ▪ From a CS perspective eWOM is a main ingredient for algorithmic decision support mechanism like RS ▪ Experiments in psychology literature[6] revealed users with different decision making styles [1] Xiang, Z. et al. Information technology and consumer behavior in travel and tourism: Insights from travel planning using the internet. JRCS 2015 [2] Ulrike Gretzel and Kyung Hyan Yoo. Use and impact of online travel reviews. Information and Communication Technologies in Tourism 2008 [3] Xie, K. et al. The business value of online consumer reviews and management response to hotel performance. IJHM 2014 [4] Xiang, Z. et al. A comparative analysis of major online review platforms: Implications for social media analytics inhospitality and tourism. TM 2017 [5] Xie, H. et al. Consumers responses to ambivalent online hotel reviews: The role of perceived source credibility and predecisional disposition. IJHM 2011 [6] Schwartz, B. et al. Maximizing versus satisficing: Happiness is a matter of choice. JPSP 2002 4
- 5. 5 The goal of this research is determine how rating summary statistics are guiding users’ choices in the online scenario … in order to develop more efficient algorithms. Research Goals
- 6. Decomposing rating summaries We consider them to be multi-attribute objects: ▪ Number of ratings ▪ Mean of the ratings ▪ Bimodality ▪ Variance ▪ Skewness ▪ Origin of Ratings 6
- 7. Decision Making on Multi-attribute Items ▪ Non-Compensatory Strategies [1]: ▫ Compare items based on one attribute ▫ Perform intra-dimensional comparisons ▫ Perform less comparisons ▪ Compensatory Strategies [1]: ▫ All attributes meet a minimum requirement ▫ Multiple inter-dimensional comparisons ▫ Spend more time on items Eye movement is an indicator of the screening of the choices [2]. 7[1] John W Payne. Task complexity and contingentprocessing in decision making: An information search and protocol analysis. Organizational Behavior and Human Performance, 1976 [2] Jacob L. Orquin and Simone Mueller Loose. Attention and choice: A review on eye movements in decision making. Acta Psychologica 2013
- 8. Decision making strategies ▪ Interpersonal differences ▪ Satisficer / Maximizers [1] ▪ Three subdimensions[2]: ▫ Decision Difficulty ▫ Alternative Search ▫ High Standards 8 [1] Herbert A Simon. A behavioral model of rational choice. The quarterly journal of economics, 1955 [2] Schwartz et al., 2002, Maximizing versus satisficing: Happiness is a matter of choice. Journal of Personality and Social Psychology 1983 Herbert Simon
- 9. Earlier Work ▪ We run a set of 3 experiments to understand trade-off mechanisms between decision strategies ▪ Decomposing rating summaries: Different types of explanations, Number of ratings, Mean of the ratings, Variance, Skewness ▪ Respondents: ▫ Relied highly on the mean rating ▫ Non linear influence of overall Number of Ratings ▫ Variance and skewness remain largely unnoticed ▫ Maximizers vs. Satisficers display different preferences 9 [1] Coba L., Zanker M., Rook L., Symeonidis P.: Exploring Users' Perception of Rating Summary Statistics. UMAP ‘18 [2] Coba L., Zanker M., Rook L., Symeonidis P.: Exploring Users' Perceptionof Collaborative Explanation Styles. CBI ’18 [3] Coba L., Zanker M., Rook L., Symeonidis P.: Decision Making Strategies Differ in the Presenceof Collaborative Explanations: Two Conjoint Studies. IUI ’19
- 10. Approach Experimental design – Metrics – Setup 10
- 11. Conjoint experiment to quantify users’ preferences Ranking based Conjoint Methodology: ▪ Used in product design/development ▪ Items can be seen as a bundle of attributes ▪ Goal to identify the utility contribution of each attribute of the rating summary statistics separately 11
- 12. Data ▪ Data driven levels [1] ▪ J-shaped [2] ▪ Bimodality coefficient[3]: 12 [1] Markus Zanker and Martin Schoberegger. An empirical study on the persuasiveness of fact-based explanations for recommender systems. RecSys 2014 [2] Hu N, Zhang J, Pavlou PA (2009) Overcomingthe J-shaped distribution of product reviews. Commun ACM [3] Pfister R, Schwarz KA, Janczyk M, Dale R, Freeman JB (2013) Good things peak in pairs: a note on the bimodality coefficient. Front Psychol
- 13. Design ▪ Full-factorial design with: ▫ 2 levels of the Number of rating ▫ 3 levels of Mean ▫ 3 levels of Bimodality ▪ 3 screens with 6 items to rank 13
- 14. Additive utility model Different attributes contribute independently to the overall utility. The perceived utility of an item/profile is determined as: 𝑢 = 𝑥𝑖 𝛽 + 𝜀 𝑥𝑖 vector characterizing a profile i, 𝛽 vector with (unknown) preferences for each attribute level, 𝜀 is the residual error. Respondents are supposed to select the alternative with, in their eyes, maximal utility u. 14
- 15. Eye-tracking Metrics ▪ Area of Interest (AOI)[1] ▪ Fixation times ▫ Geometrical mean [2] ▪ Revisits [3] 15 [1]Kenneth Holmqvist, Marcus Nystroom, Richard Andersson, Richard Dewhurst, Halszka Jarodzka, and Joost Van De Weijer. Eye tracking. A comprehensive guide to methods and measures. Oxford University Press, 2011. [2] Jeff Sauro and James R. Lewis. Average task times in usability tests. CHI’10 [3] John W Payne. Task complexity and contingent processing in decision making: An information search and protocol analysis. Organizational Behavior and Human Performance, 1976
- 16. Results
- 17. Respondents’ demographics 0 10 20 30 40 Age 40+ 31-40 25-30 18-24 0 10 20 30 Gender No answer Female Male 0 5 10 15 20 Country Others Germany Albania Italy 42 Respondents 17 All respondents were acquainted to online booking scenarios
- 18. Split on Decision Difficulty: Parameter estimates 18
- 19. Non - compensatory strategy ▪ Compare items based on one attribute ▪ Perform intra-dimensional comparisons ▪ Perform less comparisons 19
- 20. Compensatory strategy ▪ All attributes meet a minimum requirement ▪ Multiple inter-dimensional comparison ▪ Spend more time on items 20
- 21. Max vs. Sat: Time spent on items 21 Geometrical mean of the time spent on item (confidence level of 95%), median split on decision difficulty sub-scale.
- 22. Max vs. Sat: Revisits 22 Mean number of revisits per item(confidence level of 95%), median split on decision difficulty sub-scale.
- 23. Gini-index 23
- 24. Conclusions
- 25. Conclusions ▪ Maximizers and satisficers expose different decision making behavior ▫ Choice is dominated by mean and number of ratings ▫ Bimodality showed no significant influence ▫ Compensatory vs. non compensatory ▪ Rating summaries influence/bias users’ choice ▫ Not considered when interpreting implicit user feedback ▪ Our results indicate that more aspects need to be considered to optimize recommendations based on explainability/persuasiveness 25
- 26. Thank you! Ludovik Coba lucoba@unibz.it Markus Zanker mzanker@unibz.it Laurens Rook l.rook@tudelft.nl