SlideShare a Scribd company logo
Affect- and Personality-based Recommender Systems
Part I: Motivation, Models
Marko Tkalčič, Free University of Bozen-Bolzano
ACM Summer School on Recommender Systems 2017
Marko Tkalčič, RecSys2017SummerSchool-Part1-AffectRecSys 1/64
Netflix
Netflix . . . one of the greatest players in recommender
systems!
Marko Tkalčič, RecSys2017SummerSchool-Part1-AffectRecSys 2/64
Netflix
Netflix . . . one of the greatest players in recommender
systems!
What is Netflix recommending us?
Marko Tkalčič, RecSys2017SummerSchool-Part1-AffectRecSys 2/64
Netflix
Netflix . . . one of the greatest players in recommender
systems!
What is Netflix recommending us?
Movies/films . . . really?
Marko Tkalčič, RecSys2017SummerSchool-Part1-AffectRecSys 2/64
Netflix
Netflix . . . one of the greatest players in recommender
systems!
What is Netflix recommending us?
Movies/films . . . really?
“I want to watch a funny movie tonight”
Marko Tkalčič, RecSys2017SummerSchool-Part1-AffectRecSys 2/64
Netflix
Netflix . . . one of the greatest players in recommender
systems!
What is Netflix recommending us?
Movies/films . . . really?
“I want to watch a funny movie tonight”
Funny is all you want?
Marko Tkalčič, RecSys2017SummerSchool-Part1-AffectRecSys 2/64
But there’s more!!
Marko Tkalčič, RecSys2017SummerSchool-Part1-AffectRecSys 3/64
But there’s more!!
Question:
Can (rating/genre/year/director)
summarize that rollercoaster?
Thanks to Shlomo Berkovski for the inspiring example from the EMPIRE 2015 keynote.
Image source: http://yhvh.name/?w=2646
Marko Tkalčič, RecSys2017SummerSchool-Part1-AffectRecSys 3/64
Table of Contents
Introduction
Why are theory-driven models important?
Models of Emotion and Personality
Marko Tkalčič, RecSys2017SummerSchool-Part1-AffectRecSys 4/64
Who am I?
Marko Tkalčič
• 2016 - now : assistant professor at Free University of
Bozen-Bolzano
• 2013 - 2015: postdoc at Johannes Kepler University,
Linz
• 2011 - 2012: postdoc at University of Ljubljana
• 2008 - 2010: PhD student at University of Ljubljana
My research explores ways in which psychologically-motivated user characteristics,
such as emotions and personality, can be used to improve recommender systems
(personalized systems in general). It employs methods such as user studies and
machine learning.
Marko Tkalčič, RecSys2017SummerSchool-Part1-AffectRecSys 5/64
Book, 2016
• Tkalčič, M., Carolis, B. De, Gemmis, M. de, Odić, A., &
Košir, A. (Eds.). (2016). Emotions and Personality in
Personalized Services. Springer International Publishing.
https://doi.org/10.1007/978-3-319-31413-6
• Authors from
• Stanford, Cambridge, Imperial College, UCL . . .
• topics:
• psychological models
• acquisition of emotions/personality
• personalization techniques
• http://www.springer.com/gp/book/9783319314112
Marko Tkalčič, RecSys2017SummerSchool-Part1-AffectRecSys 6/64
Goal of this Talk/Learning Outcomes
• complements other talks of the Summer School
• off the beaten track . . . meant to open new ideas
• part of the audience should say
• this is BS
• this is inspiring
Marko Tkalčič, RecSys2017SummerSchool-Part1-AffectRecSys 7/64
Goal of this Talk/Learning Outcomes
• complements other talks of the Summer School
• off the beaten track . . . meant to open new ideas
• part of the audience should say
• this is BS
• this is inspiring
We will learn
• Part I (Tuesday, 16:30 - 18:30)
• why models borrowed from psychology and social sciences are important
• models (emotions, personality)
Marko Tkalčič, RecSys2017SummerSchool-Part1-AffectRecSys 7/64
Goal of this Talk/Learning Outcomes
• complements other talks of the Summer School
• off the beaten track . . . meant to open new ideas
• part of the audience should say
• this is BS
• this is inspiring
We will learn
• Part I (Tuesday, 16:30 - 18:30)
• why models borrowed from psychology and social sciences are important
• models (emotions, personality)
• Part II (Thursday, 8:15 - 10:15)
• tools for acquiring E&P
• usage of E&P in recommender systems
Marko Tkalčič, RecSys2017SummerSchool-Part1-AffectRecSys 7/64
Table of Contents
Introduction
Why are theory-driven models important?
Models of Emotion and Personality
Marko Tkalčič, RecSys2017SummerSchool-Part1-AffectRecSys 8/64
What are we predicting in RS?
. . . for each user, we want to choose such item that maximizes the user’s utility/rating.
(Adomavicius, Tuzhilin, 2005)
Marko Tkalčič, RecSys2017SummerSchool-Part1-AffectRecSys 9/64
What are we predicting in RS?
. . . for each user, we want to choose such item that maximizes the user’s utility/rating.
(Adomavicius, Tuzhilin, 2005)
Recommender Systems (RSs) are software tools and techniques that provide suggestions
for items that are most likely of interest to a particular user (Ricci et al., 2015)
Marko Tkalčič, RecSys2017SummerSchool-Part1-AffectRecSys 9/64
What are we predicting in RS?
. . . for each user, we want to choose such item that maximizes the user’s utility/rating.
(Adomavicius, Tuzhilin, 2005)
Recommender Systems (RSs) are software tools and techniques that provide suggestions
for items that are most likely of interest to a particular user (Ricci et al., 2015)
Marko Tkalčič, RecSys2017SummerSchool-Part1-AffectRecSys 9/64
What are we predicting in RS?
. . . for each user, we want to choose such item that maximizes the user’s utility/rating.
(Adomavicius, Tuzhilin, 2005)
Recommender Systems (RSs) are software tools and techniques that provide suggestions
for items that are most likely of interest to a particular user (Ricci et al., 2015)
• what influences (which features)?
• how (which algorithm)?
References
Adomavicius, G., and Tuzhilin, A. (2005). Toward the next generation of recommender systems: A survey of the state-of-the-art and
possible extensions. IEEE Transactions on Knowledge and Data Engineering, 17(6), 734–749.
Ricci, F., Rokach, L., and Shapira, B. (2015). Recommender Systems: Introduction and Challenges. In Recommender Systems
Handbook (Vol. 54, pp. 1–34). Boston, MA: Springer US.
Marko Tkalčič, RecSys2017SummerSchool-Part1-AffectRecSys 9/64
Decision making
Liking, purchasing, rating, clicking etc. . . actions triggered by decisions
Marko Tkalčič, RecSys2017SummerSchool-Part1-AffectRecSys 10/64
Decision making
Liking, purchasing, rating, clicking etc. . . actions triggered by decisions
One way of looking at recommender systems is the one taken by (Jameson et al., 2015):
we view recommender systems as tools for helping people to make better choices
—not large, complex choices, such as where to build a new airport, but the small- to
medium-sized choices that people make every day: what products to buy, what
documents to read, which people to contact.
References
Jameson, A., Willemsen, M. C., Felfernig, A., de Gemmis, M., Lops, P., Semeraro, G., and Chen, L. (2015). Human Decision Making
and Recommender Systems. In Recommender Systems Handbook (Vol. 54, pp. 611–648). Boston, MA: Springer US.
Marko Tkalčič, RecSys2017SummerSchool-Part1-AffectRecSys 10/64
Decision making models
There are many decision-making models
• ASPECT/ARCADE (Jameson et al.)
• Somatic Markers (Damasio)
• Two Systems (Kahneman and Tversky)
• Multi-system model (Lerner et al.)
References
Jameson, A., Willemsen, M. C., Felfernig, A., de Gemmis, M., Lops, P., Semeraro, G., and Chen, L. (2015). Human Decision Making
and Recommender Systems. In Recommender Systems Handbook (Vol. 54, pp. 611–648). Boston, MA: Springer US.
Damasio, A. (1994) Descartes’ Error: Emotion, Reason, and the Human Brain
Kahneman, D. (2003). A perspective on judgment and choice: mapping bounded rationality. The American Psychologist, 58(9),
697–720.
Lerner, J. S., Li, Y., Valdesolo, P., and Kassam, K. S. (2015). Emotion and Decision Making. Annual Review of Psychology, 66(1),
799–823.
Marko Tkalčič, RecSys2017SummerSchool-Part1-AffectRecSys 11/64
ASPECT/ARCADE models
• ASPECT model of choice-making: based on human-choice patterns
• ARCADE model: strategies to support decision making - i.e. by RS
References
Jameson, A., Willemsen, M. C., Felfernig, A., de Gemmis, M., Lops, P., Semeraro, G., and Chen, L. (2015). Human Decision Making
and Recommender Systems. In Recommender Systems Handbook (Vol. 54, pp. 611–648). Boston, MA: Springer US.
Marko Tkalčič, RecSys2017SummerSchool-Part1-AffectRecSys 12/64
Decision making and emotions - Damasio
• physiological/evolutionary aspect
• emotional processes guide (or bias) behavior, particularly decision-making
• changes in both body and brain states in response to different stimuli
• these physiological signals (or somatic markers) and their evoked emotion are
consciously or unconsciously associated with their past outcomes and bias
decision-making
References
Damasio, A. (1994) Descartes’ Error: Emotion, Reason, and the Human Brain
Marko Tkalčič, RecSys2017SummerSchool-Part1-AffectRecSys 13/64
Kahneman Tversky two systems
• Decision:
• System 1: fast, intuitive, emotion-driven
• system 2: slow, rational
References
Kahneman, D. (2003). A perspective on judgment and choice: mapping bounded rationality. The American Psychologist, 58(9),
697–720.
Marko Tkalčič, RecSys2017SummerSchool-Part1-AffectRecSys 14/64
Kahneman Tversky two systems
Marko Tkalčič, RecSys2017SummerSchool-Part1-AffectRecSys 15/64
Decision making model - Lerner
emotions and personality influence our decisions!
References
Lerner, J. S., Li, Y., Valdesolo, P., and Kassam, K. S. (2015). Emotion and Decision Making. Annual Review of Psychology, 66(1),
799–823.
Marko Tkalčič, RecSys2017SummerSchool-Part1-AffectRecSys 16/64
Personality and preferences
Personality traits (extraverted/introverted, open/conservative etc.) are linked to music
genre preferences (Rentfrow et al, 2003)
References
Rentfrow, P. J., and Gosling, S. D. (2003). The do re mi’s of everyday life: The structure and personality correlates of music
preferences. Journal of Personality and Social Psychology, 84(6), 1236–1256.
Tkalčič, M., Ferwerda, B., Hauger, D., and Schedl, M. (2015). Personality Correlates for Digital Concert Program Notes. In UMAP
2015, Lecture Notes On Computer Science 9146 (Vol. 9146, pp. 364–369).
Marko Tkalčič, RecSys2017SummerSchool-Part1-AffectRecSys 17/64
Emotions are related to other things as well
Why we choose to consume some kind of content?
Marko Tkalčič, RecSys2017SummerSchool-Part1-AffectRecSys 18/64
Emotions are related to other things as well
Why we choose to consume some kind of content?
One of the main reasons why people consume music (Lonsdale, 2011) and films (Oliver,
2008) is emotion regulation.
References
Lonsdale, A. J., and North, A. C. (2011). Why do we listen to music? A uses and gratifications analysis. British Journal of Psychology
(London, England : 1953), 102(1), 108–34. https://doi.org/10.1348/000712610X506831
Oliver, M. B. (2008). Tender affective states as predictors of entertainment preference. Journal of Communication, 58(1), 40–61.
https://doi.org/10.1111/j.1460-2466.2007.00373.x
Marko Tkalčič, RecSys2017SummerSchool-Part1-AffectRecSys 18/64
However, it can get complicated
Why do we like drama, sad films?
• . . . under some circumstances, individuals may choose to view entertainment for
reasons that may not be best described as driven by hedonic motivations but
rather as driven by eudaimonic motivations: greater insight, self reflection, or
contemplations of poignancy or meaningfulness (e.g., what makes life valuable).
• The Hangover
• hedonic quality (comedy)
• no eudaimonic quality
• La vita e’ bella
• hedonic quality (comedy)
• eudaimonic quality
References
Oliver, M. B. (2008). Tender affective states as predictors of entertainment preference. Journal of Communication, 58(1), 40–61.
https://doi.org/10.1111/j.1460-2466.2007.00373.x
Marko Tkalčič, RecSys2017SummerSchool-Part1-AffectRecSys 19/64
What we have today?
Features
• genre
• actors (performers, authors, directors
. . . )
• rating
• timestamp
• location
• price
• year
Algorithms
• Content-based
• Collaborative Filtering
• Knowledge-based
• Hybrid
Marko Tkalčič, RecSys2017SummerSchool-Part1-AffectRecSys 20/64
Are these the right features?
Marko Tkalčič, RecSys2017SummerSchool-Part1-AffectRecSys 21/64
Are these the right features?
Light Darkness
genre emotions
action personality
timestamp
actors
rating
location
Marko Tkalčič, RecSys2017SummerSchool-Part1-AffectRecSys 21/64
Are these the right features?
Thesis: besides using existing data/features and the respective data-driven models,
theory-driven features and models should be investigated to improve recommender
systems.
Let’s look at a couple of sparse examples illustrating the above thesis.
Marko Tkalčič, RecSys2017SummerSchool-Part1-AffectRecSys 22/64
Chomsky-Skinner Debate
Behaviorism
During the first half of the twentieth century, John B. Watson devised methodological
behaviorism, which rejected introspective methods and sought to understand
behavior by only measuring observable behaviors and events (Wikipedia).
Marko Tkalčič, RecSys2017SummerSchool-Part1-AffectRecSys 23/64
Chomsky-Skinner Debate
Behaviorism
During the first half of the twentieth century, John B. Watson devised methodological
behaviorism, which rejected introspective methods and sought to understand
behavior by only measuring observable behaviors and events (Wikipedia).
On language acquisition
• Skinner (behaviorist):
• operant conditioning : child goes through trial-and-error she tries and fails to use correct
language until she succeeds; with reinforcement and shaping provided by the parents
gestures (smiles, attention and approval) which are pleasant to the child.
• there’s no need to understand the underlying hardware
• Chomsky (structuralist):
• operant conditioning could not account for a child’s ability to create or understand an
infinite variety of novel sentences
• language acquisition has an innate structure, it is a function of the human brain.
• there are structures of the brain that control the interpretation and production of speech
• it is important to understand the underlying hardware
References
Andresen, J. (1991). Skinner and Chomsky 30 years later. Or: The return of the repressed . The Behavior Analyst, 14(1), 49–60.
Marko Tkalčič, RecSys2017SummerSchool-Part1-AffectRecSys 23/64
Claudia Perlich, keynote @Recsys 2016
• Q: In the world of deep learning, XGBoost etc. why logistic regression?
• A: I studied neural networks in 1995 . . . downgraded to decision trees in 2004
and won 3 KDD cups with linear models. Personally I find it easier to build up a
model with interesting feature construction. . . I can look under the hood and see
what these things are doing.
https://www.youtube.com/watch?v=1WmqqfXNFZ4&feature=youtu.be&t=3003
Marko Tkalčič, RecSys2017SummerSchool-Part1-AffectRecSys 24/64
Gary Marcus (NYU): let’s put “top down” and “bottom up” knowledge on
equal footing
To get computers to think like humans, we need a new A.I. paradigm, one that places
top down and bottom up knowledge on equal footing. Bottom-up knowledge is the
kind of raw information we get directly from our senses, like patterns of light falling on
our retina. Top-down knowledge comprises cognitive models of the world and how it
works.
References
Gary Marcus, Artificial Intelligence Is Stuck. Here’s How to Move It Forward. New York Times, July 29, 2017
Marko Tkalčič, RecSys2017SummerSchool-Part1-AffectRecSys 25/64
Come again?
Why, again, models-driven approaches complementing data-driven?
• models from other disciplines can be inspiring
• such models can account for information variance
• Marvin Minsky created the first neural net borrowing the concept from neurology
https://en.wikipedia.org/wiki/History_of_artificial_intelligence
Marko Tkalčič, RecSys2017SummerSchool-Part1-AffectRecSys 26/64
But, how can we relate this to recommender systems?
• the gap between existing RS and psychological models is too wide
• psychological models of DM can be
• too generic
• too complex
• hard to implement
Marko Tkalčič, RecSys2017SummerSchool-Part1-AffectRecSys 27/64
But, how can we relate this to recommender systems?
• the gap between existing RS and psychological models is too wide
• psychological models of DM can be
• too generic
• too complex
• hard to implement
• let’s make smaller steps
• measure new features and use our (recsys) algorithms
• emotions
• personality
• let’s shed new light
Marko Tkalčič, RecSys2017SummerSchool-Part1-AffectRecSys 27/64
Table of Contents
Introduction
Why are theory-driven models important?
Models of Emotion and Personality
Marko Tkalčič, RecSys2017SummerSchool-Part1-AffectRecSys 28/64
Personality, Mood and Emotions
Marko Tkalčič, RecSys2017SummerSchool-Part1-AffectRecSys 29/64
Personality, Mood and Emotions
Marko Tkalčič, RecSys2017SummerSchool-Part1-AffectRecSys 30/64
Personality, Mood and Emotions
Marko Tkalčič, RecSys2017SummerSchool-Part1-AffectRecSys 31/64
Personality, Mood and Emotions
Marko Tkalčič, RecSys2017SummerSchool-Part1-AffectRecSys 32/64
Personality, Mood and Emotions
Marko Tkalčič, RecSys2017SummerSchool-Part1-AffectRecSys 33/64
Personality, Mood and Emotions
Marko Tkalčič, RecSys2017SummerSchool-Part1-AffectRecSys 34/64
Personality
• What is personality?
• accounts for individual differences ( = explains the variance in users) in our enduring
emotional, interpersonal, experiential, attitudinal, and motivational styles
Marko Tkalčič, RecSys2017SummerSchool-Part1-AffectRecSys 35/64
Personality
• What is personality?
• accounts for individual differences ( = explains the variance in users) in our enduring
emotional, interpersonal, experiential, attitudinal, and motivational styles
Marko Tkalčič, RecSys2017SummerSchool-Part1-AffectRecSys 35/64
Models of Personality - 4 temperaments
• Four temperaments Hippocratess (cca 400 BC)
• sanguine (pleasure-seeking and sociable)
• choleric (ambitious and leader-like)
• melancholic (analytical and quiet)
• phlegmatic (relaxed and peaceful)
Marko Tkalčič, RecSys2017SummerSchool-Part1-AffectRecSys 36/64
Models of Personality - Big5
• The five factor model (FFM) – Big5:
• Extraversion
• Agreeableness
• Conscientousness
• Neuroticism
• Openness (to new experiences)
The inverse of Neuroticism is sometimes referred to as Emotional Stability,
References
McCraMcCrae, R. R., and John, O. P. (1992). An Introduction to the Five-Factor Model and its Applications. Journal of Personality,
60(2), p175 – 215.
Marko Tkalčič, RecSys2017SummerSchool-Part1-AffectRecSys 37/64
Individual Factors
• Openness to Experience (O),
• high: imaginative, creative people (individualistic, non conforming and are very aware of
their feelings)
• low: down-to-earth, conventional people. (simple and straightforward thinking over
complex, ambiguous and subtle)
• sub-factors are imagination, artistic interest, emotionality, adventurousness, intellect and
liberalism.
• Conscientiousness
• high : prudent, organized
• low: impulsive, disorganized.
• sub-factors are self-efficacy, orderliness, dutifulness, achievement-striving, self-discipline
and cautiousness.
• Extraversion
• high: degree of engagement with the external world, react with enthusiasm and often
have positive emotions
• low: lack of engagement with the external world, quiet, low-key and disengaged in social
interactions.
• sub-factors of E are friendliness, gregariousness, assertiveness, activity level,
excitement-seeking and cheerfulness.
Marko Tkalčič, RecSys2017SummerSchool-Part1-AffectRecSys 38/64
Individual Factors
• Agreeableness
• high : need for cooperation and social harmony
• low : opposite
• subfactors: trust, morality, altruism, cooperation, modesty and sympathy.
• Neuroticism
• high: emotionally reactive. They tend to respond emotionally to relatively neutral stimuli.
They are often in a bad mood, which strongly affects their thinking and decision making
• low: calm, emotionally stable and free from persistent bad mood.
• sub-factors are anxiety, anger, depression, self-consciousness, immoderation and
vulnerability.
Marko Tkalčič, RecSys2017SummerSchool-Part1-AffectRecSys 39/64
Models of personality - others
• RIASEC occupational model:
• Realistic (Doers),
• Investigative (Thinkers),
• Artistic (Creators),
• Social (Helpers),
• Enterprising (Persuaders)
• Conventional (Organizers)
• Bartle gamers model:
• Killers: interfere with the functioning
of the game world or the play
experience of other players
• Achievers: accumulate status tokens by
beating the rules-based challenges of
the game world
• Explorers: discover the systems
governing the operation of the game
world
• Socializers: form relationships with
other players by telling stories within
the game world
• Myers-Briggs Personality Type
• Extraversion (E) or Introversion (I).
• Sensing (S) or Intuition (N).
• Thinking (T) or Feeling (F).
• Judging (J) or Perceiving (P).
• Learning styles (Felder and Silverman
Learning Style Model):
• active/reflective,
• sensing/intuitive,
• visual/verbal,
• sequential/global
• Thomas-Kilmann Conflict Mode
• Assertiveness
• Cooperativeness
There are correlations between these models.
Marko Tkalčič, RecSys2017SummerSchool-Part1-AffectRecSys 40/64
Thomas-Kilmann Conflict Mode
• was developed to measure the conflict resolution styles in groups
COMPETING COLLABORATING
COMPROMISING
AVOIDING ACCOMODATING
COOPERATIVENESS
ASSERTIVENESS
HIGH
HIGH
LOW
LOW
References
Thomas, K. L., and Kilman, R. H. (n.d.). Thomas-Kilman Conflict Mode Instrument.
Marko Tkalčič, RecSys2017SummerSchool-Part1-AffectRecSys 41/64
Domain-specific personality models - tourism
• questionnaire -> factor analysis
• tourist specific personality constructs
• Sun and Chill-out
• Knowledge and Travel
• Independence and History
• Culture and Indulgence
• Social and Sport
• Action and Fun
• Nature and Recreation
• picture-based measuring instruments
• pick and rank 10 images out of a pool
References
Neidhardt, J., Seyfang, L., Schuster, R., and Werthner, H. (2015). A picture-based approach to recommender systems. Information
Technology and Tourism, 15(1), 49–69. https://doi.org/10.1007/s40558-014-0017-5
Marko Tkalčič, RecSys2017SummerSchool-Part1-AffectRecSys 42/64
Measuring the FFM
• Extensive questionnaires (from 5 to several 100s questions)
• BFI: 44 questions
• TIPI : 10 questions
• NEO-IPIP: 300 questions
• For each user u a five tuple b = (b1, b2, b3, b4, b5)
Marko Tkalčič, RecSys2017SummerSchool-Part1-AffectRecSys 43/64
TIPI - Ten-Items Personality Inventory
TIPI: I see myself as (1-7 . . .
agree/disagree):
1. Extraverted, enthusiastic.
2. Critical, quarrelsome.
3. Dependable, self-disciplined.
4. Anxious, easily upset.
5. Open to new experiences,
complex.
6. Reserved, quiet.
7. Sympathetic, warm.
8. Disorganized, careless.
9. Calm, emotionally stable.
10. Conventional, uncreative.
• b1 = q1 + (8 − q6) = Extraversion
• b2 = q2 + (8 − q7) = Agreeableness
• b3 = q3 + (8 − q8) = Conscientiousness
• b4 = q4 + (8 − q9) = Emotional Stability
• b5 = q5 + (8 − q10) = Openness to Experiences
References
Gosling, S. D., Rentfrow, P. J., and Swann, W. B. (2003). A very brief measure of the Big-Five personality domains. Journal of
Research in Personality, 37(6), 504–528. doi:10.1016/S0092-6566(03)00046-1
Marko Tkalčič, RecSys2017SummerSchool-Part1-AffectRecSys 44/64
Let’s measure ourselves
https://goo.gl/forms/qN64cjEaMMlor2F42
Age	
  group Extraversion Agreeableness Conscientiousness Emotional	
  Stability Openness
15	
  to	
  20 Mean 3.79 4.47 4.41 4.61 5.43
SD 1.55 1.22 1.39 1.47 1.17
n	
  =	
   54973 54973 54973 54973 54973
21	
  to	
  30 Mean 3.73 4.5 4.57 4.64 5.49
SD 1.54 1.2 1.39 1.46 1.13
n	
  =	
   40737 40737 40737 40737 40737
31	
  to	
  40 Mean 3.81 4.55 4.77 4.63 5.49
SD 1.54 1.21 1.35 1.42 1.12
n	
  =	
   14752 14752 14752 14752 14752
41	
  to	
  50 Mean 3.85 4.7 4.96 4.72 5.41
SD 1.54 1.18 1.35 1.39 1.17
n	
  =	
   7668 7668 7668 7668 7668
51	
  to	
  60 Mean 3.87 4.89 5.11 4.8 5.39
SD 1.54 1.18 1.31 1.38 1.2
n	
  =	
   3532 3532 3532 3532 3532
61	
  and	
  older Mean 3.85 4.95 5.26 4.92 5.37
SD 1.49 1.17 1.3 1.34 1.26
n	
  =	
   905 905 905 905 905
Male
http://gosling.psy.utexas.edu/scales-weve-developed/ten-item-personality-measure-Marko Tkalčič, RecSys2017SummerSchool-Part1-AffectRecSys 45/64
Emotion vs. Mood vs. Sentiment
Let’s clear some terminology
• Affect : umbrella term for describing the topics of emotion, feelings, and moods
• Emotion:
• brief in duration
• consist of a coordinated set of responses (verbal, physiological, behavioral, and neural
mechanisms)
• triggered
• Mood:
• last longer
• less intense than emotions
• no trigger
• Sentiment:
• towards an object
• positive/negative
Marko Tkalčič, RecSys2017SummerSchool-Part1-AffectRecSys 46/64
Models of Emotions
• Emotions are complex human experiences
• Evolutionary based
• Several definitions, we take with simple models, easy to incorporate in computers:
• Basic emotions
• Dimensional model
• Plutchik wheel
• Geneva Emotion Wheel
Marko Tkalčič, RecSys2017SummerSchool-Part1-AffectRecSys 47/64
Basic Emotions
• Discrete classes model
• Different sets
• Darwin: Expression of emotions in man and animal
• Ekman definition (6 + neutral):
• Happiness
• Anger
• Fear
• Sadness
• Disgust
• Surprise
Marko Tkalčič, RecSys2017SummerSchool-Part1-AffectRecSys 48/64
Dimensional model of Emotions
Three continuous dimensions
• Valence/Pleasure (positive-negative)
Each emotion is a point in the VAD space
Self-Assessment Manikin (SAM)
Marko Tkalčič, RecSys2017SummerSchool-Part1-AffectRecSys 49/64
Dimensional model of Emotions
Three continuous dimensions
• Valence/Pleasure (positive-negative)
• Arousal (high-low )
Each emotion is a point in the VAD space
Self-Assessment Manikin (SAM)
Marko Tkalčič, RecSys2017SummerSchool-Part1-AffectRecSys 49/64
Dimensional model of Emotions
Three continuous dimensions
• Valence/Pleasure (positive-negative)
• Arousal (high-low )
• Dominance (high-low )
Each emotion is a point in the VAD space
Self-Assessment Manikin (SAM)
Marko Tkalčič, RecSys2017SummerSchool-Part1-AffectRecSys 49/64
Dimensional model of Emotions
Three continuous dimensions
• Valence/Pleasure (positive-negative)
• Arousal (high-low )
• Dominance (high-low )
Each emotion is a point in the VAD space
Self-Assessment Manikin (SAM)
Marko Tkalčič, RecSys2017SummerSchool-Part1-AffectRecSys 49/64
Dimensional model of Emotions
Three continuous dimensions
• Valence/Pleasure (positive-negative)
• Arousal (high-low )
• Dominance (high-low )
Each emotion is a point in the VAD space
Self-Assessment Manikin (SAM)
References
Bradley, M. M., and Lang, P. J. (1994). Measuring emotion: the self-assessment manikin and the semantic differential. Journal of
Behavior Therapy and Experimental Psychiatry, 25(1), 49–59.
Marko Tkalčič, RecSys2017SummerSchool-Part1-AffectRecSys 49/64
Circumplex model of Emotions
mapping between basic emotions and dimensional model
References
Posner, J., Russell, J. a, and Peterson, B. S. (2005). The circumplex model of affect: an integrative approach to affective neuroscience,
cognitive development, and psychopathology. Development and Psychopathology, 17(3), 715–734.
https://doi.org/10.1017/S0954579405050340
Marko Tkalčič, RecSys2017SummerSchool-Part1-AffectRecSys 50/64
Plutchik Wheel of emotions
• Robert Plutchik
• eight basic emotions: Joy, Trust, Fear,
Surprise, Sadness, Disgust, Anger and
Anticipation.
• intensity (radius)
• twenty-four Primary, Secondary, and
Tertiary dyads (a feeling composed of
two emotions)
• Primary dyad = one petal apart =
Love = Joy + Trust
• Secondary dyad = two petals apart =
Envy = Sadness + Anger
• Tertiary dyad = three petals apart =
Shame = Fear + Disgust
• Opposite emotions = four petals apart
= Anticipation != Surprise
References
Plutchik, R. (1982). A psychoevolutionary theory of emotions. Social Science Information, 21(4–5), 529–553.
https://doi.org/10.1177/053901882021004003
Marko Tkalčič, RecSys2017SummerSchool-Part1-AffectRecSys 51/64
Geneva Emotion Wheel
• model
• instrument
for
measuring
emotions
References
Sacharin, V., Schlegel, K., and Scherer, K. R. (2012). Geneva Emotion Wheel Rating Study.
Marko Tkalčič, RecSys2017SummerSchool-Part1-AffectRecSys 52/64
Music-specific emotion model - GEMS
• domain-specific emotion model
• Geneva Emotional Music Scale (GEMS)
• 9-factorial model of music-induced emotions
• transcendence
• wonder
• joyful activation
• power
• tension
• sadness
• tenderness
• nostalgia
• peacefulness
References
Zentner, M., Grandjean, D., and Scherer, K. R. (2008). Emotions evoked by the sound of music: characterization, classification, and
measurement. Emotion (Washington, D.C.), 8(4), 494–521. https://doi.org/10.1037/1528-3542.8.4.494
Marko Tkalčič, RecSys2017SummerSchool-Part1-AffectRecSys 53/64
Modeling content with emotions
• user-related emotions
• how does (recommended) content relate to emotions?
• “I want to watch a funny movie tonight”
Marko Tkalčič, RecSys2017SummerSchool-Part1-AffectRecSys 54/64
Modeling movies with emotions
• “A film is - or should be - more like music than like fiction. It should be a
progression of moods and feelings. The theme, what’s behind the emotion, the
meaning, all that comes later.” – Stanley Kubrick
Marko Tkalčič, RecSys2017SummerSchool-Part1-AffectRecSys 55/64
Modeling movies with emotions
• “A film is - or should be - more like music than like fiction. It should be a
progression of moods and feelings. The theme, what’s behind the emotion, the
meaning, all that comes later.” – Stanley Kubrick
• “If my films make one more person miserable, I’ll feel I have done my job.” –
Woody Allen
Marko Tkalčič, RecSys2017SummerSchool-Part1-AffectRecSys 55/64
Modeling movies with emotions
• “A film is - or should be - more like music than like fiction. It should be a
progression of moods and feelings. The theme, what’s behind the emotion, the
meaning, all that comes later.” – Stanley Kubrick
• “If my films make one more person miserable, I’ll feel I have done my job.” –
Woody Allen
• “Through careful manipulation and good storytelling, you can get everybody to
clap at the same time, to laugh at the same time, and to be afraid at the same
time.” – Steven Spielberg
Marko Tkalčič, RecSys2017SummerSchool-Part1-AffectRecSys 55/64
Kurt Vonnegut story arc
Marko Tkalčič, RecSys2017SummerSchool-Part1-AffectRecSys 56/64
Modeling movies with emotions - screenwriting workshop
Source:
https://iedgameresearch.wordpress.com/2014/04/30/noel-mccauley-workshop-staging-emotional-environments/
Marko Tkalčič, RecSys2017SummerSchool-Part1-AffectRecSys 57/64
Modeling movies with emotions
feature-film-length emotion maps
http://yhvh.name/?w=-97&emap=1
Marko Tkalčič, RecSys2017SummerSchool-Part1-AffectRecSys 58/64
Modeling music with emotion
(Juslin et al.) distinguish between three types of music-related emotions:
• expressed emotions
• the composer or performer wants to express
• perceived
• how a listener perceives (but not necessarily feels)
• induced
• truly felt by the listener
References
Juslin, P. N., and Laukka, P. (2004). Expression, Perception, and Induction of Musical Emotions: A Review and a Questionnaire Study
of Everyday Listening. Journal of New Music Research, 33(3), 217–238. https://doi.org/10.1080/0929821042000317813
Marko Tkalčič, RecSys2017SummerSchool-Part1-AffectRecSys 59/64
Music emotion detection from movements
• movements stimulated by music
• captured by mobile phone (raw
acceleration data)
• extract motion features
• linear regressoin predicting GEMS
• R2 between 0.14 and 0.50
References
Irrgang, M., and Egermann, H. (2016). From motion to emotion: Accelerometer data predict subjective experience of music. PLoS
ONE, 11(7), 1–20. https://doi.org/10.1371/journal.pone.0154360
Marko Tkalčič, RecSys2017SummerSchool-Part1-AffectRecSys 60/64
Music emotion recognition (MER)
• perceived emotion = a single point in VA space
• annotations
• features:
• acoustic (loudness, timbre, pitch, rhythm,melody, harmony)
• lyrics
• models:
• regression techniques
• probabilistic (take into account the variance)
Dancing Queen
(Abba)
Civil war (GNR) Suzanne (Leonard
Cohen)
All I have to do is
dream (Everly
Brothers)
References
Wang, J.-C., Yang, Y., and Wang, H. (2016). Affective Music Information Retrieval. In M. Tkalčič, B. De Carolis, M. de Gemmis, A.
Odić, and A. Košir (Eds.), Emotions and Personality in Personalized Services: Models, Evaluation and Applications (pp. 227–261).
Springer.Marko Tkalčič, RecSys2017SummerSchool-Part1-AffectRecSys 61/64
Wrap-up
• Motivation: emotions and personality should be investigated
Marko Tkalčič, RecSys2017SummerSchool-Part1-AffectRecSys 62/64
Wrap-up
• Motivation: emotions and personality should be investigated
• Models of personality and emotions
Marko Tkalčič, RecSys2017SummerSchool-Part1-AffectRecSys 62/64
Wrap-up
• Motivation: emotions and personality should be investigated
• Models of personality and emotions
• Measurement with self-reporting
Marko Tkalčič, RecSys2017SummerSchool-Part1-AffectRecSys 62/64
Next
• unobtrusive measurement
• usage in recommender systems
Marko Tkalčič, RecSys2017SummerSchool-Part1-AffectRecSys 63/64
Questions?
Marko Tkalčič, RecSys2017SummerSchool-Part1-AffectRecSys 64/64

More Related Content

PDF
Affect- and Personality-based Recommender Systems Part II: Acquisition, Usage...
PDF
How To Write A Proposal EssayPaper - Owlcation
PPT
3. What is the principles of Teamwork_Module_V1.0.ppt
PPT
Beyond Buzz - Web 2.0 Expo - K.Niederhoffer & M.Smith
PDF
On serendipity in recommender systems - Haifa RecSoc workshop june 2015
PDF
Design considerations for machine learning system
DOCX
My degree is an EDD in Performance Improvement LeadershipSyste.docx
PPTX
IIR 2017, Lugano Switzerland
Affect- and Personality-based Recommender Systems Part II: Acquisition, Usage...
How To Write A Proposal EssayPaper - Owlcation
3. What is the principles of Teamwork_Module_V1.0.ppt
Beyond Buzz - Web 2.0 Expo - K.Niederhoffer & M.Smith
On serendipity in recommender systems - Haifa RecSoc workshop june 2015
Design considerations for machine learning system
My degree is an EDD in Performance Improvement LeadershipSyste.docx
IIR 2017, Lugano Switzerland

Similar to Affect- and Personality-based Recommender Systems Part I: Motivation, Models (20)

PDF
W4L1_11-667: LARGE LANGUAGE MODELS: METHODS AND APPLICATIONS - Human Evaluati...
PDF
Thinking in System and Scale
PPTX
Dartmouth discussion: What's wrong with "What's wrong with CHAT?"?
DOCX
Soc 156 – Sociology of CommunicationReview Sheet – FinalShor.docx
PPT
R 4.1 big6 intro
PDF
From data to endgame for dissertation and theses: what does it take
PDF
Summary Of Defending Against The Indefensible Essay
PDF
Mapping Patterns Landscape - Overview Results
PPTX
TS4-3: Takumi Sato from Nagoya Institute of Technology
DOCX
Professor Speiser English 28 ITVWeekend College Summer .docx
DOCX
ITS 832Chapter 5From Building a Model to Adaptive Robust.docx
DOCX
Task1-Question Module 7 Discussion ForumExplain the role of i.docx
PDF
Using the Framework of Networks to Enhance Learning and Social Interactions
PDF
Writing Essay Papers
KEY
Thesis Proposal: Understanding Audience Engagement Transmedia
PPT
Interpretations of Socio-Technical Systems: Two Stories and the Narrative of ...
DOCX
The Role of Families and the Community Proposal Template (N.docx
PPTX
Explainable AI is not yet Understandable AI
PDF
Descriptive Essay On My School Library. Online assignment writing service.
PDF
Writing The Perfect Essay (Rule Of 3) By Miss Midwes
W4L1_11-667: LARGE LANGUAGE MODELS: METHODS AND APPLICATIONS - Human Evaluati...
Thinking in System and Scale
Dartmouth discussion: What's wrong with "What's wrong with CHAT?"?
Soc 156 – Sociology of CommunicationReview Sheet – FinalShor.docx
R 4.1 big6 intro
From data to endgame for dissertation and theses: what does it take
Summary Of Defending Against The Indefensible Essay
Mapping Patterns Landscape - Overview Results
TS4-3: Takumi Sato from Nagoya Institute of Technology
Professor Speiser English 28 ITVWeekend College Summer .docx
ITS 832Chapter 5From Building a Model to Adaptive Robust.docx
Task1-Question Module 7 Discussion ForumExplain the role of i.docx
Using the Framework of Networks to Enhance Learning and Social Interactions
Writing Essay Papers
Thesis Proposal: Understanding Audience Engagement Transmedia
Interpretations of Socio-Technical Systems: Two Stories and the Narrative of ...
The Role of Families and the Community Proposal Template (N.docx
Explainable AI is not yet Understandable AI
Descriptive Essay On My School Library. Online assignment writing service.
Writing The Perfect Essay (Rule Of 3) By Miss Midwes
Ad

More from Marko Tkalčič (7)

PDF
Prediction of Music Pairwise Preferences from Facial Expressions
PDF
Affect- and Personality-based Recommender Systems Hands-on: Unobtrusive Acqui...
PDF
201506 umap tkalcic_personality_correlatesdigitalprogramn_otes
PPTX
Affect in recommender systems
PPTX
Addressing the New User Problem with a Personality Based User Similarity Measure
PPTX
Affective User Modeling
PPTX
Affective recommender systems: the role of emotions in recommender systems
Prediction of Music Pairwise Preferences from Facial Expressions
Affect- and Personality-based Recommender Systems Hands-on: Unobtrusive Acqui...
201506 umap tkalcic_personality_correlatesdigitalprogramn_otes
Affect in recommender systems
Addressing the New User Problem with a Personality Based User Similarity Measure
Affective User Modeling
Affective recommender systems: the role of emotions in recommender systems
Ad

Recently uploaded (20)

PPTX
(Ali Hamza) Roll No: (F24-BSCS-1103).pptx
PDF
Data Engineering Interview Questions & Answers Cloud Data Stacks (AWS, Azure,...
PPT
lectureusjsjdhdsjjshdshshddhdhddhhd1.ppt
PPTX
QUANTUM_COMPUTING_AND_ITS_POTENTIAL_APPLICATIONS[2].pptx
PDF
Systems Analysis and Design, 12th Edition by Scott Tilley Test Bank.pdf
PDF
Votre score augmente si vous choisissez une catégorie et que vous rédigez une...
PPTX
retention in jsjsksksksnbsndjddjdnFPD.pptx
PPT
Predictive modeling basics in data cleaning process
PPT
ISS -ESG Data flows What is ESG and HowHow
PDF
annual-report-2024-2025 original latest.
PDF
Introduction to Data Science and Data Analysis
PPTX
AI Strategy room jwfjksfksfjsjsjsjsjfsjfsj
PDF
Introduction to the R Programming Language
PDF
Capcut Pro Crack For PC Latest Version {Fully Unlocked 2025}
PPTX
DS-40-Pre-Engagement and Kickoff deck - v8.0.pptx
PDF
Tetra Pak Index 2023 - The future of health and nutrition - Full report.pdf
PPTX
modul_python (1).pptx for professional and student
PDF
Data Engineering Interview Questions & Answers Batch Processing (Spark, Hadoo...
PPTX
Managing Community Partner Relationships
PDF
Global Data and Analytics Market Outlook Report
(Ali Hamza) Roll No: (F24-BSCS-1103).pptx
Data Engineering Interview Questions & Answers Cloud Data Stacks (AWS, Azure,...
lectureusjsjdhdsjjshdshshddhdhddhhd1.ppt
QUANTUM_COMPUTING_AND_ITS_POTENTIAL_APPLICATIONS[2].pptx
Systems Analysis and Design, 12th Edition by Scott Tilley Test Bank.pdf
Votre score augmente si vous choisissez une catégorie et que vous rédigez une...
retention in jsjsksksksnbsndjddjdnFPD.pptx
Predictive modeling basics in data cleaning process
ISS -ESG Data flows What is ESG and HowHow
annual-report-2024-2025 original latest.
Introduction to Data Science and Data Analysis
AI Strategy room jwfjksfksfjsjsjsjsjfsjfsj
Introduction to the R Programming Language
Capcut Pro Crack For PC Latest Version {Fully Unlocked 2025}
DS-40-Pre-Engagement and Kickoff deck - v8.0.pptx
Tetra Pak Index 2023 - The future of health and nutrition - Full report.pdf
modul_python (1).pptx for professional and student
Data Engineering Interview Questions & Answers Batch Processing (Spark, Hadoo...
Managing Community Partner Relationships
Global Data and Analytics Market Outlook Report

Affect- and Personality-based Recommender Systems Part I: Motivation, Models

  • 1. Affect- and Personality-based Recommender Systems Part I: Motivation, Models Marko Tkalčič, Free University of Bozen-Bolzano ACM Summer School on Recommender Systems 2017 Marko Tkalčič, RecSys2017SummerSchool-Part1-AffectRecSys 1/64
  • 2. Netflix Netflix . . . one of the greatest players in recommender systems! Marko Tkalčič, RecSys2017SummerSchool-Part1-AffectRecSys 2/64
  • 3. Netflix Netflix . . . one of the greatest players in recommender systems! What is Netflix recommending us? Marko Tkalčič, RecSys2017SummerSchool-Part1-AffectRecSys 2/64
  • 4. Netflix Netflix . . . one of the greatest players in recommender systems! What is Netflix recommending us? Movies/films . . . really? Marko Tkalčič, RecSys2017SummerSchool-Part1-AffectRecSys 2/64
  • 5. Netflix Netflix . . . one of the greatest players in recommender systems! What is Netflix recommending us? Movies/films . . . really? “I want to watch a funny movie tonight” Marko Tkalčič, RecSys2017SummerSchool-Part1-AffectRecSys 2/64
  • 6. Netflix Netflix . . . one of the greatest players in recommender systems! What is Netflix recommending us? Movies/films . . . really? “I want to watch a funny movie tonight” Funny is all you want? Marko Tkalčič, RecSys2017SummerSchool-Part1-AffectRecSys 2/64
  • 7. But there’s more!! Marko Tkalčič, RecSys2017SummerSchool-Part1-AffectRecSys 3/64
  • 8. But there’s more!! Question: Can (rating/genre/year/director) summarize that rollercoaster? Thanks to Shlomo Berkovski for the inspiring example from the EMPIRE 2015 keynote. Image source: http://yhvh.name/?w=2646 Marko Tkalčič, RecSys2017SummerSchool-Part1-AffectRecSys 3/64
  • 9. Table of Contents Introduction Why are theory-driven models important? Models of Emotion and Personality Marko Tkalčič, RecSys2017SummerSchool-Part1-AffectRecSys 4/64
  • 10. Who am I? Marko Tkalčič • 2016 - now : assistant professor at Free University of Bozen-Bolzano • 2013 - 2015: postdoc at Johannes Kepler University, Linz • 2011 - 2012: postdoc at University of Ljubljana • 2008 - 2010: PhD student at University of Ljubljana My research explores ways in which psychologically-motivated user characteristics, such as emotions and personality, can be used to improve recommender systems (personalized systems in general). It employs methods such as user studies and machine learning. Marko Tkalčič, RecSys2017SummerSchool-Part1-AffectRecSys 5/64
  • 11. Book, 2016 • Tkalčič, M., Carolis, B. De, Gemmis, M. de, Odić, A., & Košir, A. (Eds.). (2016). Emotions and Personality in Personalized Services. Springer International Publishing. https://doi.org/10.1007/978-3-319-31413-6 • Authors from • Stanford, Cambridge, Imperial College, UCL . . . • topics: • psychological models • acquisition of emotions/personality • personalization techniques • http://www.springer.com/gp/book/9783319314112 Marko Tkalčič, RecSys2017SummerSchool-Part1-AffectRecSys 6/64
  • 12. Goal of this Talk/Learning Outcomes • complements other talks of the Summer School • off the beaten track . . . meant to open new ideas • part of the audience should say • this is BS • this is inspiring Marko Tkalčič, RecSys2017SummerSchool-Part1-AffectRecSys 7/64
  • 13. Goal of this Talk/Learning Outcomes • complements other talks of the Summer School • off the beaten track . . . meant to open new ideas • part of the audience should say • this is BS • this is inspiring We will learn • Part I (Tuesday, 16:30 - 18:30) • why models borrowed from psychology and social sciences are important • models (emotions, personality) Marko Tkalčič, RecSys2017SummerSchool-Part1-AffectRecSys 7/64
  • 14. Goal of this Talk/Learning Outcomes • complements other talks of the Summer School • off the beaten track . . . meant to open new ideas • part of the audience should say • this is BS • this is inspiring We will learn • Part I (Tuesday, 16:30 - 18:30) • why models borrowed from psychology and social sciences are important • models (emotions, personality) • Part II (Thursday, 8:15 - 10:15) • tools for acquiring E&P • usage of E&P in recommender systems Marko Tkalčič, RecSys2017SummerSchool-Part1-AffectRecSys 7/64
  • 15. Table of Contents Introduction Why are theory-driven models important? Models of Emotion and Personality Marko Tkalčič, RecSys2017SummerSchool-Part1-AffectRecSys 8/64
  • 16. What are we predicting in RS? . . . for each user, we want to choose such item that maximizes the user’s utility/rating. (Adomavicius, Tuzhilin, 2005) Marko Tkalčič, RecSys2017SummerSchool-Part1-AffectRecSys 9/64
  • 17. What are we predicting in RS? . . . for each user, we want to choose such item that maximizes the user’s utility/rating. (Adomavicius, Tuzhilin, 2005) Recommender Systems (RSs) are software tools and techniques that provide suggestions for items that are most likely of interest to a particular user (Ricci et al., 2015) Marko Tkalčič, RecSys2017SummerSchool-Part1-AffectRecSys 9/64
  • 18. What are we predicting in RS? . . . for each user, we want to choose such item that maximizes the user’s utility/rating. (Adomavicius, Tuzhilin, 2005) Recommender Systems (RSs) are software tools and techniques that provide suggestions for items that are most likely of interest to a particular user (Ricci et al., 2015) Marko Tkalčič, RecSys2017SummerSchool-Part1-AffectRecSys 9/64
  • 19. What are we predicting in RS? . . . for each user, we want to choose such item that maximizes the user’s utility/rating. (Adomavicius, Tuzhilin, 2005) Recommender Systems (RSs) are software tools and techniques that provide suggestions for items that are most likely of interest to a particular user (Ricci et al., 2015) • what influences (which features)? • how (which algorithm)? References Adomavicius, G., and Tuzhilin, A. (2005). Toward the next generation of recommender systems: A survey of the state-of-the-art and possible extensions. IEEE Transactions on Knowledge and Data Engineering, 17(6), 734–749. Ricci, F., Rokach, L., and Shapira, B. (2015). Recommender Systems: Introduction and Challenges. In Recommender Systems Handbook (Vol. 54, pp. 1–34). Boston, MA: Springer US. Marko Tkalčič, RecSys2017SummerSchool-Part1-AffectRecSys 9/64
  • 20. Decision making Liking, purchasing, rating, clicking etc. . . actions triggered by decisions Marko Tkalčič, RecSys2017SummerSchool-Part1-AffectRecSys 10/64
  • 21. Decision making Liking, purchasing, rating, clicking etc. . . actions triggered by decisions One way of looking at recommender systems is the one taken by (Jameson et al., 2015): we view recommender systems as tools for helping people to make better choices —not large, complex choices, such as where to build a new airport, but the small- to medium-sized choices that people make every day: what products to buy, what documents to read, which people to contact. References Jameson, A., Willemsen, M. C., Felfernig, A., de Gemmis, M., Lops, P., Semeraro, G., and Chen, L. (2015). Human Decision Making and Recommender Systems. In Recommender Systems Handbook (Vol. 54, pp. 611–648). Boston, MA: Springer US. Marko Tkalčič, RecSys2017SummerSchool-Part1-AffectRecSys 10/64
  • 22. Decision making models There are many decision-making models • ASPECT/ARCADE (Jameson et al.) • Somatic Markers (Damasio) • Two Systems (Kahneman and Tversky) • Multi-system model (Lerner et al.) References Jameson, A., Willemsen, M. C., Felfernig, A., de Gemmis, M., Lops, P., Semeraro, G., and Chen, L. (2015). Human Decision Making and Recommender Systems. In Recommender Systems Handbook (Vol. 54, pp. 611–648). Boston, MA: Springer US. Damasio, A. (1994) Descartes’ Error: Emotion, Reason, and the Human Brain Kahneman, D. (2003). A perspective on judgment and choice: mapping bounded rationality. The American Psychologist, 58(9), 697–720. Lerner, J. S., Li, Y., Valdesolo, P., and Kassam, K. S. (2015). Emotion and Decision Making. Annual Review of Psychology, 66(1), 799–823. Marko Tkalčič, RecSys2017SummerSchool-Part1-AffectRecSys 11/64
  • 23. ASPECT/ARCADE models • ASPECT model of choice-making: based on human-choice patterns • ARCADE model: strategies to support decision making - i.e. by RS References Jameson, A., Willemsen, M. C., Felfernig, A., de Gemmis, M., Lops, P., Semeraro, G., and Chen, L. (2015). Human Decision Making and Recommender Systems. In Recommender Systems Handbook (Vol. 54, pp. 611–648). Boston, MA: Springer US. Marko Tkalčič, RecSys2017SummerSchool-Part1-AffectRecSys 12/64
  • 24. Decision making and emotions - Damasio • physiological/evolutionary aspect • emotional processes guide (or bias) behavior, particularly decision-making • changes in both body and brain states in response to different stimuli • these physiological signals (or somatic markers) and their evoked emotion are consciously or unconsciously associated with their past outcomes and bias decision-making References Damasio, A. (1994) Descartes’ Error: Emotion, Reason, and the Human Brain Marko Tkalčič, RecSys2017SummerSchool-Part1-AffectRecSys 13/64
  • 25. Kahneman Tversky two systems • Decision: • System 1: fast, intuitive, emotion-driven • system 2: slow, rational References Kahneman, D. (2003). A perspective on judgment and choice: mapping bounded rationality. The American Psychologist, 58(9), 697–720. Marko Tkalčič, RecSys2017SummerSchool-Part1-AffectRecSys 14/64
  • 26. Kahneman Tversky two systems Marko Tkalčič, RecSys2017SummerSchool-Part1-AffectRecSys 15/64
  • 27. Decision making model - Lerner emotions and personality influence our decisions! References Lerner, J. S., Li, Y., Valdesolo, P., and Kassam, K. S. (2015). Emotion and Decision Making. Annual Review of Psychology, 66(1), 799–823. Marko Tkalčič, RecSys2017SummerSchool-Part1-AffectRecSys 16/64
  • 28. Personality and preferences Personality traits (extraverted/introverted, open/conservative etc.) are linked to music genre preferences (Rentfrow et al, 2003) References Rentfrow, P. J., and Gosling, S. D. (2003). The do re mi’s of everyday life: The structure and personality correlates of music preferences. Journal of Personality and Social Psychology, 84(6), 1236–1256. Tkalčič, M., Ferwerda, B., Hauger, D., and Schedl, M. (2015). Personality Correlates for Digital Concert Program Notes. In UMAP 2015, Lecture Notes On Computer Science 9146 (Vol. 9146, pp. 364–369). Marko Tkalčič, RecSys2017SummerSchool-Part1-AffectRecSys 17/64
  • 29. Emotions are related to other things as well Why we choose to consume some kind of content? Marko Tkalčič, RecSys2017SummerSchool-Part1-AffectRecSys 18/64
  • 30. Emotions are related to other things as well Why we choose to consume some kind of content? One of the main reasons why people consume music (Lonsdale, 2011) and films (Oliver, 2008) is emotion regulation. References Lonsdale, A. J., and North, A. C. (2011). Why do we listen to music? A uses and gratifications analysis. British Journal of Psychology (London, England : 1953), 102(1), 108–34. https://doi.org/10.1348/000712610X506831 Oliver, M. B. (2008). Tender affective states as predictors of entertainment preference. Journal of Communication, 58(1), 40–61. https://doi.org/10.1111/j.1460-2466.2007.00373.x Marko Tkalčič, RecSys2017SummerSchool-Part1-AffectRecSys 18/64
  • 31. However, it can get complicated Why do we like drama, sad films? • . . . under some circumstances, individuals may choose to view entertainment for reasons that may not be best described as driven by hedonic motivations but rather as driven by eudaimonic motivations: greater insight, self reflection, or contemplations of poignancy or meaningfulness (e.g., what makes life valuable). • The Hangover • hedonic quality (comedy) • no eudaimonic quality • La vita e’ bella • hedonic quality (comedy) • eudaimonic quality References Oliver, M. B. (2008). Tender affective states as predictors of entertainment preference. Journal of Communication, 58(1), 40–61. https://doi.org/10.1111/j.1460-2466.2007.00373.x Marko Tkalčič, RecSys2017SummerSchool-Part1-AffectRecSys 19/64
  • 32. What we have today? Features • genre • actors (performers, authors, directors . . . ) • rating • timestamp • location • price • year Algorithms • Content-based • Collaborative Filtering • Knowledge-based • Hybrid Marko Tkalčič, RecSys2017SummerSchool-Part1-AffectRecSys 20/64
  • 33. Are these the right features? Marko Tkalčič, RecSys2017SummerSchool-Part1-AffectRecSys 21/64
  • 34. Are these the right features? Light Darkness genre emotions action personality timestamp actors rating location Marko Tkalčič, RecSys2017SummerSchool-Part1-AffectRecSys 21/64
  • 35. Are these the right features? Thesis: besides using existing data/features and the respective data-driven models, theory-driven features and models should be investigated to improve recommender systems. Let’s look at a couple of sparse examples illustrating the above thesis. Marko Tkalčič, RecSys2017SummerSchool-Part1-AffectRecSys 22/64
  • 36. Chomsky-Skinner Debate Behaviorism During the first half of the twentieth century, John B. Watson devised methodological behaviorism, which rejected introspective methods and sought to understand behavior by only measuring observable behaviors and events (Wikipedia). Marko Tkalčič, RecSys2017SummerSchool-Part1-AffectRecSys 23/64
  • 37. Chomsky-Skinner Debate Behaviorism During the first half of the twentieth century, John B. Watson devised methodological behaviorism, which rejected introspective methods and sought to understand behavior by only measuring observable behaviors and events (Wikipedia). On language acquisition • Skinner (behaviorist): • operant conditioning : child goes through trial-and-error she tries and fails to use correct language until she succeeds; with reinforcement and shaping provided by the parents gestures (smiles, attention and approval) which are pleasant to the child. • there’s no need to understand the underlying hardware • Chomsky (structuralist): • operant conditioning could not account for a child’s ability to create or understand an infinite variety of novel sentences • language acquisition has an innate structure, it is a function of the human brain. • there are structures of the brain that control the interpretation and production of speech • it is important to understand the underlying hardware References Andresen, J. (1991). Skinner and Chomsky 30 years later. Or: The return of the repressed . The Behavior Analyst, 14(1), 49–60. Marko Tkalčič, RecSys2017SummerSchool-Part1-AffectRecSys 23/64
  • 38. Claudia Perlich, keynote @Recsys 2016 • Q: In the world of deep learning, XGBoost etc. why logistic regression? • A: I studied neural networks in 1995 . . . downgraded to decision trees in 2004 and won 3 KDD cups with linear models. Personally I find it easier to build up a model with interesting feature construction. . . I can look under the hood and see what these things are doing. https://www.youtube.com/watch?v=1WmqqfXNFZ4&feature=youtu.be&t=3003 Marko Tkalčič, RecSys2017SummerSchool-Part1-AffectRecSys 24/64
  • 39. Gary Marcus (NYU): let’s put “top down” and “bottom up” knowledge on equal footing To get computers to think like humans, we need a new A.I. paradigm, one that places top down and bottom up knowledge on equal footing. Bottom-up knowledge is the kind of raw information we get directly from our senses, like patterns of light falling on our retina. Top-down knowledge comprises cognitive models of the world and how it works. References Gary Marcus, Artificial Intelligence Is Stuck. Here’s How to Move It Forward. New York Times, July 29, 2017 Marko Tkalčič, RecSys2017SummerSchool-Part1-AffectRecSys 25/64
  • 40. Come again? Why, again, models-driven approaches complementing data-driven? • models from other disciplines can be inspiring • such models can account for information variance • Marvin Minsky created the first neural net borrowing the concept from neurology https://en.wikipedia.org/wiki/History_of_artificial_intelligence Marko Tkalčič, RecSys2017SummerSchool-Part1-AffectRecSys 26/64
  • 41. But, how can we relate this to recommender systems? • the gap between existing RS and psychological models is too wide • psychological models of DM can be • too generic • too complex • hard to implement Marko Tkalčič, RecSys2017SummerSchool-Part1-AffectRecSys 27/64
  • 42. But, how can we relate this to recommender systems? • the gap between existing RS and psychological models is too wide • psychological models of DM can be • too generic • too complex • hard to implement • let’s make smaller steps • measure new features and use our (recsys) algorithms • emotions • personality • let’s shed new light Marko Tkalčič, RecSys2017SummerSchool-Part1-AffectRecSys 27/64
  • 43. Table of Contents Introduction Why are theory-driven models important? Models of Emotion and Personality Marko Tkalčič, RecSys2017SummerSchool-Part1-AffectRecSys 28/64
  • 44. Personality, Mood and Emotions Marko Tkalčič, RecSys2017SummerSchool-Part1-AffectRecSys 29/64
  • 45. Personality, Mood and Emotions Marko Tkalčič, RecSys2017SummerSchool-Part1-AffectRecSys 30/64
  • 46. Personality, Mood and Emotions Marko Tkalčič, RecSys2017SummerSchool-Part1-AffectRecSys 31/64
  • 47. Personality, Mood and Emotions Marko Tkalčič, RecSys2017SummerSchool-Part1-AffectRecSys 32/64
  • 48. Personality, Mood and Emotions Marko Tkalčič, RecSys2017SummerSchool-Part1-AffectRecSys 33/64
  • 49. Personality, Mood and Emotions Marko Tkalčič, RecSys2017SummerSchool-Part1-AffectRecSys 34/64
  • 50. Personality • What is personality? • accounts for individual differences ( = explains the variance in users) in our enduring emotional, interpersonal, experiential, attitudinal, and motivational styles Marko Tkalčič, RecSys2017SummerSchool-Part1-AffectRecSys 35/64
  • 51. Personality • What is personality? • accounts for individual differences ( = explains the variance in users) in our enduring emotional, interpersonal, experiential, attitudinal, and motivational styles Marko Tkalčič, RecSys2017SummerSchool-Part1-AffectRecSys 35/64
  • 52. Models of Personality - 4 temperaments • Four temperaments Hippocratess (cca 400 BC) • sanguine (pleasure-seeking and sociable) • choleric (ambitious and leader-like) • melancholic (analytical and quiet) • phlegmatic (relaxed and peaceful) Marko Tkalčič, RecSys2017SummerSchool-Part1-AffectRecSys 36/64
  • 53. Models of Personality - Big5 • The five factor model (FFM) – Big5: • Extraversion • Agreeableness • Conscientousness • Neuroticism • Openness (to new experiences) The inverse of Neuroticism is sometimes referred to as Emotional Stability, References McCraMcCrae, R. R., and John, O. P. (1992). An Introduction to the Five-Factor Model and its Applications. Journal of Personality, 60(2), p175 – 215. Marko Tkalčič, RecSys2017SummerSchool-Part1-AffectRecSys 37/64
  • 54. Individual Factors • Openness to Experience (O), • high: imaginative, creative people (individualistic, non conforming and are very aware of their feelings) • low: down-to-earth, conventional people. (simple and straightforward thinking over complex, ambiguous and subtle) • sub-factors are imagination, artistic interest, emotionality, adventurousness, intellect and liberalism. • Conscientiousness • high : prudent, organized • low: impulsive, disorganized. • sub-factors are self-efficacy, orderliness, dutifulness, achievement-striving, self-discipline and cautiousness. • Extraversion • high: degree of engagement with the external world, react with enthusiasm and often have positive emotions • low: lack of engagement with the external world, quiet, low-key and disengaged in social interactions. • sub-factors of E are friendliness, gregariousness, assertiveness, activity level, excitement-seeking and cheerfulness. Marko Tkalčič, RecSys2017SummerSchool-Part1-AffectRecSys 38/64
  • 55. Individual Factors • Agreeableness • high : need for cooperation and social harmony • low : opposite • subfactors: trust, morality, altruism, cooperation, modesty and sympathy. • Neuroticism • high: emotionally reactive. They tend to respond emotionally to relatively neutral stimuli. They are often in a bad mood, which strongly affects their thinking and decision making • low: calm, emotionally stable and free from persistent bad mood. • sub-factors are anxiety, anger, depression, self-consciousness, immoderation and vulnerability. Marko Tkalčič, RecSys2017SummerSchool-Part1-AffectRecSys 39/64
  • 56. Models of personality - others • RIASEC occupational model: • Realistic (Doers), • Investigative (Thinkers), • Artistic (Creators), • Social (Helpers), • Enterprising (Persuaders) • Conventional (Organizers) • Bartle gamers model: • Killers: interfere with the functioning of the game world or the play experience of other players • Achievers: accumulate status tokens by beating the rules-based challenges of the game world • Explorers: discover the systems governing the operation of the game world • Socializers: form relationships with other players by telling stories within the game world • Myers-Briggs Personality Type • Extraversion (E) or Introversion (I). • Sensing (S) or Intuition (N). • Thinking (T) or Feeling (F). • Judging (J) or Perceiving (P). • Learning styles (Felder and Silverman Learning Style Model): • active/reflective, • sensing/intuitive, • visual/verbal, • sequential/global • Thomas-Kilmann Conflict Mode • Assertiveness • Cooperativeness There are correlations between these models. Marko Tkalčič, RecSys2017SummerSchool-Part1-AffectRecSys 40/64
  • 57. Thomas-Kilmann Conflict Mode • was developed to measure the conflict resolution styles in groups COMPETING COLLABORATING COMPROMISING AVOIDING ACCOMODATING COOPERATIVENESS ASSERTIVENESS HIGH HIGH LOW LOW References Thomas, K. L., and Kilman, R. H. (n.d.). Thomas-Kilman Conflict Mode Instrument. Marko Tkalčič, RecSys2017SummerSchool-Part1-AffectRecSys 41/64
  • 58. Domain-specific personality models - tourism • questionnaire -> factor analysis • tourist specific personality constructs • Sun and Chill-out • Knowledge and Travel • Independence and History • Culture and Indulgence • Social and Sport • Action and Fun • Nature and Recreation • picture-based measuring instruments • pick and rank 10 images out of a pool References Neidhardt, J., Seyfang, L., Schuster, R., and Werthner, H. (2015). A picture-based approach to recommender systems. Information Technology and Tourism, 15(1), 49–69. https://doi.org/10.1007/s40558-014-0017-5 Marko Tkalčič, RecSys2017SummerSchool-Part1-AffectRecSys 42/64
  • 59. Measuring the FFM • Extensive questionnaires (from 5 to several 100s questions) • BFI: 44 questions • TIPI : 10 questions • NEO-IPIP: 300 questions • For each user u a five tuple b = (b1, b2, b3, b4, b5) Marko Tkalčič, RecSys2017SummerSchool-Part1-AffectRecSys 43/64
  • 60. TIPI - Ten-Items Personality Inventory TIPI: I see myself as (1-7 . . . agree/disagree): 1. Extraverted, enthusiastic. 2. Critical, quarrelsome. 3. Dependable, self-disciplined. 4. Anxious, easily upset. 5. Open to new experiences, complex. 6. Reserved, quiet. 7. Sympathetic, warm. 8. Disorganized, careless. 9. Calm, emotionally stable. 10. Conventional, uncreative. • b1 = q1 + (8 − q6) = Extraversion • b2 = q2 + (8 − q7) = Agreeableness • b3 = q3 + (8 − q8) = Conscientiousness • b4 = q4 + (8 − q9) = Emotional Stability • b5 = q5 + (8 − q10) = Openness to Experiences References Gosling, S. D., Rentfrow, P. J., and Swann, W. B. (2003). A very brief measure of the Big-Five personality domains. Journal of Research in Personality, 37(6), 504–528. doi:10.1016/S0092-6566(03)00046-1 Marko Tkalčič, RecSys2017SummerSchool-Part1-AffectRecSys 44/64
  • 61. Let’s measure ourselves https://goo.gl/forms/qN64cjEaMMlor2F42 Age  group Extraversion Agreeableness Conscientiousness Emotional  Stability Openness 15  to  20 Mean 3.79 4.47 4.41 4.61 5.43 SD 1.55 1.22 1.39 1.47 1.17 n  =   54973 54973 54973 54973 54973 21  to  30 Mean 3.73 4.5 4.57 4.64 5.49 SD 1.54 1.2 1.39 1.46 1.13 n  =   40737 40737 40737 40737 40737 31  to  40 Mean 3.81 4.55 4.77 4.63 5.49 SD 1.54 1.21 1.35 1.42 1.12 n  =   14752 14752 14752 14752 14752 41  to  50 Mean 3.85 4.7 4.96 4.72 5.41 SD 1.54 1.18 1.35 1.39 1.17 n  =   7668 7668 7668 7668 7668 51  to  60 Mean 3.87 4.89 5.11 4.8 5.39 SD 1.54 1.18 1.31 1.38 1.2 n  =   3532 3532 3532 3532 3532 61  and  older Mean 3.85 4.95 5.26 4.92 5.37 SD 1.49 1.17 1.3 1.34 1.26 n  =   905 905 905 905 905 Male http://gosling.psy.utexas.edu/scales-weve-developed/ten-item-personality-measure-Marko Tkalčič, RecSys2017SummerSchool-Part1-AffectRecSys 45/64
  • 62. Emotion vs. Mood vs. Sentiment Let’s clear some terminology • Affect : umbrella term for describing the topics of emotion, feelings, and moods • Emotion: • brief in duration • consist of a coordinated set of responses (verbal, physiological, behavioral, and neural mechanisms) • triggered • Mood: • last longer • less intense than emotions • no trigger • Sentiment: • towards an object • positive/negative Marko Tkalčič, RecSys2017SummerSchool-Part1-AffectRecSys 46/64
  • 63. Models of Emotions • Emotions are complex human experiences • Evolutionary based • Several definitions, we take with simple models, easy to incorporate in computers: • Basic emotions • Dimensional model • Plutchik wheel • Geneva Emotion Wheel Marko Tkalčič, RecSys2017SummerSchool-Part1-AffectRecSys 47/64
  • 64. Basic Emotions • Discrete classes model • Different sets • Darwin: Expression of emotions in man and animal • Ekman definition (6 + neutral): • Happiness • Anger • Fear • Sadness • Disgust • Surprise Marko Tkalčič, RecSys2017SummerSchool-Part1-AffectRecSys 48/64
  • 65. Dimensional model of Emotions Three continuous dimensions • Valence/Pleasure (positive-negative) Each emotion is a point in the VAD space Self-Assessment Manikin (SAM) Marko Tkalčič, RecSys2017SummerSchool-Part1-AffectRecSys 49/64
  • 66. Dimensional model of Emotions Three continuous dimensions • Valence/Pleasure (positive-negative) • Arousal (high-low ) Each emotion is a point in the VAD space Self-Assessment Manikin (SAM) Marko Tkalčič, RecSys2017SummerSchool-Part1-AffectRecSys 49/64
  • 67. Dimensional model of Emotions Three continuous dimensions • Valence/Pleasure (positive-negative) • Arousal (high-low ) • Dominance (high-low ) Each emotion is a point in the VAD space Self-Assessment Manikin (SAM) Marko Tkalčič, RecSys2017SummerSchool-Part1-AffectRecSys 49/64
  • 68. Dimensional model of Emotions Three continuous dimensions • Valence/Pleasure (positive-negative) • Arousal (high-low ) • Dominance (high-low ) Each emotion is a point in the VAD space Self-Assessment Manikin (SAM) Marko Tkalčič, RecSys2017SummerSchool-Part1-AffectRecSys 49/64
  • 69. Dimensional model of Emotions Three continuous dimensions • Valence/Pleasure (positive-negative) • Arousal (high-low ) • Dominance (high-low ) Each emotion is a point in the VAD space Self-Assessment Manikin (SAM) References Bradley, M. M., and Lang, P. J. (1994). Measuring emotion: the self-assessment manikin and the semantic differential. Journal of Behavior Therapy and Experimental Psychiatry, 25(1), 49–59. Marko Tkalčič, RecSys2017SummerSchool-Part1-AffectRecSys 49/64
  • 70. Circumplex model of Emotions mapping between basic emotions and dimensional model References Posner, J., Russell, J. a, and Peterson, B. S. (2005). The circumplex model of affect: an integrative approach to affective neuroscience, cognitive development, and psychopathology. Development and Psychopathology, 17(3), 715–734. https://doi.org/10.1017/S0954579405050340 Marko Tkalčič, RecSys2017SummerSchool-Part1-AffectRecSys 50/64
  • 71. Plutchik Wheel of emotions • Robert Plutchik • eight basic emotions: Joy, Trust, Fear, Surprise, Sadness, Disgust, Anger and Anticipation. • intensity (radius) • twenty-four Primary, Secondary, and Tertiary dyads (a feeling composed of two emotions) • Primary dyad = one petal apart = Love = Joy + Trust • Secondary dyad = two petals apart = Envy = Sadness + Anger • Tertiary dyad = three petals apart = Shame = Fear + Disgust • Opposite emotions = four petals apart = Anticipation != Surprise References Plutchik, R. (1982). A psychoevolutionary theory of emotions. Social Science Information, 21(4–5), 529–553. https://doi.org/10.1177/053901882021004003 Marko Tkalčič, RecSys2017SummerSchool-Part1-AffectRecSys 51/64
  • 72. Geneva Emotion Wheel • model • instrument for measuring emotions References Sacharin, V., Schlegel, K., and Scherer, K. R. (2012). Geneva Emotion Wheel Rating Study. Marko Tkalčič, RecSys2017SummerSchool-Part1-AffectRecSys 52/64
  • 73. Music-specific emotion model - GEMS • domain-specific emotion model • Geneva Emotional Music Scale (GEMS) • 9-factorial model of music-induced emotions • transcendence • wonder • joyful activation • power • tension • sadness • tenderness • nostalgia • peacefulness References Zentner, M., Grandjean, D., and Scherer, K. R. (2008). Emotions evoked by the sound of music: characterization, classification, and measurement. Emotion (Washington, D.C.), 8(4), 494–521. https://doi.org/10.1037/1528-3542.8.4.494 Marko Tkalčič, RecSys2017SummerSchool-Part1-AffectRecSys 53/64
  • 74. Modeling content with emotions • user-related emotions • how does (recommended) content relate to emotions? • “I want to watch a funny movie tonight” Marko Tkalčič, RecSys2017SummerSchool-Part1-AffectRecSys 54/64
  • 75. Modeling movies with emotions • “A film is - or should be - more like music than like fiction. It should be a progression of moods and feelings. The theme, what’s behind the emotion, the meaning, all that comes later.” – Stanley Kubrick Marko Tkalčič, RecSys2017SummerSchool-Part1-AffectRecSys 55/64
  • 76. Modeling movies with emotions • “A film is - or should be - more like music than like fiction. It should be a progression of moods and feelings. The theme, what’s behind the emotion, the meaning, all that comes later.” – Stanley Kubrick • “If my films make one more person miserable, I’ll feel I have done my job.” – Woody Allen Marko Tkalčič, RecSys2017SummerSchool-Part1-AffectRecSys 55/64
  • 77. Modeling movies with emotions • “A film is - or should be - more like music than like fiction. It should be a progression of moods and feelings. The theme, what’s behind the emotion, the meaning, all that comes later.” – Stanley Kubrick • “If my films make one more person miserable, I’ll feel I have done my job.” – Woody Allen • “Through careful manipulation and good storytelling, you can get everybody to clap at the same time, to laugh at the same time, and to be afraid at the same time.” – Steven Spielberg Marko Tkalčič, RecSys2017SummerSchool-Part1-AffectRecSys 55/64
  • 78. Kurt Vonnegut story arc Marko Tkalčič, RecSys2017SummerSchool-Part1-AffectRecSys 56/64
  • 79. Modeling movies with emotions - screenwriting workshop Source: https://iedgameresearch.wordpress.com/2014/04/30/noel-mccauley-workshop-staging-emotional-environments/ Marko Tkalčič, RecSys2017SummerSchool-Part1-AffectRecSys 57/64
  • 80. Modeling movies with emotions feature-film-length emotion maps http://yhvh.name/?w=-97&emap=1 Marko Tkalčič, RecSys2017SummerSchool-Part1-AffectRecSys 58/64
  • 81. Modeling music with emotion (Juslin et al.) distinguish between three types of music-related emotions: • expressed emotions • the composer or performer wants to express • perceived • how a listener perceives (but not necessarily feels) • induced • truly felt by the listener References Juslin, P. N., and Laukka, P. (2004). Expression, Perception, and Induction of Musical Emotions: A Review and a Questionnaire Study of Everyday Listening. Journal of New Music Research, 33(3), 217–238. https://doi.org/10.1080/0929821042000317813 Marko Tkalčič, RecSys2017SummerSchool-Part1-AffectRecSys 59/64
  • 82. Music emotion detection from movements • movements stimulated by music • captured by mobile phone (raw acceleration data) • extract motion features • linear regressoin predicting GEMS • R2 between 0.14 and 0.50 References Irrgang, M., and Egermann, H. (2016). From motion to emotion: Accelerometer data predict subjective experience of music. PLoS ONE, 11(7), 1–20. https://doi.org/10.1371/journal.pone.0154360 Marko Tkalčič, RecSys2017SummerSchool-Part1-AffectRecSys 60/64
  • 83. Music emotion recognition (MER) • perceived emotion = a single point in VA space • annotations • features: • acoustic (loudness, timbre, pitch, rhythm,melody, harmony) • lyrics • models: • regression techniques • probabilistic (take into account the variance) Dancing Queen (Abba) Civil war (GNR) Suzanne (Leonard Cohen) All I have to do is dream (Everly Brothers) References Wang, J.-C., Yang, Y., and Wang, H. (2016). Affective Music Information Retrieval. In M. Tkalčič, B. De Carolis, M. de Gemmis, A. Odić, and A. Košir (Eds.), Emotions and Personality in Personalized Services: Models, Evaluation and Applications (pp. 227–261). Springer.Marko Tkalčič, RecSys2017SummerSchool-Part1-AffectRecSys 61/64
  • 84. Wrap-up • Motivation: emotions and personality should be investigated Marko Tkalčič, RecSys2017SummerSchool-Part1-AffectRecSys 62/64
  • 85. Wrap-up • Motivation: emotions and personality should be investigated • Models of personality and emotions Marko Tkalčič, RecSys2017SummerSchool-Part1-AffectRecSys 62/64
  • 86. Wrap-up • Motivation: emotions and personality should be investigated • Models of personality and emotions • Measurement with self-reporting Marko Tkalčič, RecSys2017SummerSchool-Part1-AffectRecSys 62/64
  • 87. Next • unobtrusive measurement • usage in recommender systems Marko Tkalčič, RecSys2017SummerSchool-Part1-AffectRecSys 63/64