Emotions and dialogue in a peer-production community: the case of Wikipedia
2 likes2,029 views
This document explores the emotional dynamics within the English Wikipedia community, focusing on how editors' emotions correspond with their experience, gender, and the topics they engage with. It includes an analysis of conversational data, user gender identification, and emotional sentiment measurement, revealing trends in emotional congruence and homophily among editors. Results show that more experienced users express higher emotional control, while genders exhibit different emotional expressions in discussions and personal talk pages.
Emotions and dialogue in a peer-production community: the case of Wikipedia
- 1. Emotions and dialogue in a peer-production community: the case of Wikipedia David Laniado Carlos Castillo Barcelona Media Foundation Qatar Computing Research Institute Andreas Kaltenbrunner Mayo Fuster Morell Barcelona Media Foundation Berkman Center, Cambridge, MA, USA August 27th , 2012 WikiSym ’12, Linz, Austria university-logo Laniado, Castillo, Kaltenbrunner, Fuster Morell Emotions and dialogue in a peer-production community 1 / 37
- 2. Outline 1 Introduction 2 Framework of analysis Data acquisition and pre-processing User gender labelling Sentiment analysis 3 Results Emotions and experience Topics and gender Emotional congruence Emotional homophily Emotions in personal (“User talk”) pages 4 Conclusions university-logo Laniado, Castillo, Kaltenbrunner, Fuster Morell Emotions and dialogue in a peer-production community 2 / 37
- 3. Outline 1 Introduction 2 Framework of analysis Data acquisition and pre-processing User gender labelling Sentiment analysis 3 Results Emotions and experience Topics and gender Emotional congruence Emotional homophily Emotions in personal (“User talk”) pages 4 Conclusions university-logo Laniado, Castillo, Kaltenbrunner, Fuster Morell Emotions and dialogue in a peer-production community 3 / 37
- 4. Introduction “Like other aspects of culture, emotions can be seen as an aspect of all social action and social relations. They accompany rational acts as fully as irrational ones, positive experiences as much as negative ones” (Goodwin, 2001) Goal: Shed light on the emotional dimension in a large peer production community focus: English Wikipedia extensive analysis of emotions expressed in talk pages Article talk pages → discussions about how to improve articles User talk pages → a kind of public in-boxes university-logo Laniado, Castillo, Kaltenbrunner, Fuster Morell Emotions and dialogue in a peer-production community 4 / 37
- 5. Research questions 1 How are the emotional styles of editors affected by their level of experience? 2 How are the emotional styles of editors affected by their gender and the topics they choose to work on? 3 How are the emotional expressions affected by interacting with others in comment threads (emotional congruence)? 4 How are the emotional styles of editors related to those of the editors they interact more frequently with (emotional homophily)? 5 How are the emotional styles of editors expressed in their personal spaces (user talk pages)? university-logo Laniado, Castillo, Kaltenbrunner, Fuster Morell Emotions and dialogue in a peer-production community 5 / 37
- 6. Outline 1 Introduction 2 Framework of analysis Data acquisition and pre-processing User gender labelling Sentiment analysis 3 Results Emotions and experience Topics and gender Emotional congruence Emotional homophily Emotions in personal (“User talk”) pages 4 Conclusions university-logo Laniado, Castillo, Kaltenbrunner, Fuster Morell Emotions and dialogue in a peer-production community 6 / 37
- 7. Outline 1 Introduction 2 Framework of analysis Data acquisition and pre-processing User gender labelling Sentiment analysis 3 Results Emotions and experience Topics and gender Emotional congruence Emotional homophily Emotions in personal (“User talk”) pages 4 Conclusions university-logo Laniado, Castillo, Kaltenbrunner, Fuster Morell Emotions and dialogue in a peer-production community 7 / 37
- 8. Dataset: conversations Extracting conversations among editors from the English Wikipedia Articles 3 210 039 Articles with talk page (ATP) 871 485 (27.1%) Total comments in ATP 11 041 246 Comments in ATP linking to policies 460 644 (4.1%) Users who comment articles 350 958 Users with ≥ 100 comments on ATP 12 231 (3.5%) Table: Data extracted from a complete dump of the English Wikipedia, dated March 12th, 2010 university-logo Laniado, Castillo, Kaltenbrunner, Fuster Morell Emotions and dialogue in a peer-production community 8 / 37
- 9. Outline 1 Introduction 2 Framework of analysis Data acquisition and pre-processing User gender labelling Sentiment analysis 3 Results Emotions and experience Topics and gender Emotional congruence Emotional homophily Emotions in personal (“User talk”) pages 4 Conclusions university-logo Laniado, Castillo, Kaltenbrunner, Fuster Morell Emotions and dialogue in a peer-production community 9 / 37
- 10. Identifying user gender ≈ 12 000 users wrote ≥ 100 comments in articles talk pages Gender identified through Wikipedia API for ≈ 2 000 of them Out of the remaining ones, a sample of 1 385 users for manual labelling through crowdsourcing (Crowdflower) university-logo Laniado, Castillo, Kaltenbrunner, Fuster Morell Emotions and dialogue in a peer-production community 10 / 37
- 11. Manual user gender labelling Gender could be identified only for ≈ 50% of users: real name or username (50% of those identified) implicitly stated gender (27% of females, 20% of males) pronoun (15% of females, 10% of males) other indicators: userboxes, pictures, links to personal blogs... Non-admins Admins Total Males 1 087 1 526 2 613 Females 68 97 165 Unknown 6 850 2 603 9 453 Total 8 005 4 226 12 231 Table: Users with ≥ 100 comments by gender and administrator status. Category “unknown” includes: 8 708 users that were not included in the crowd-sourced task university-logo 745 users whose gender could not be identified by evaluators Laniado, Castillo, Kaltenbrunner, Fuster Morell Emotions and dialogue in a peer-production community 11 / 37
- 12. Outline 1 Introduction 2 Framework of analysis Data acquisition and pre-processing User gender labelling Sentiment analysis 3 Results Emotions and experience Topics and gender Emotional congruence Emotional homophily Emotions in personal (“User talk”) pages 4 Conclusions university-logo Laniado, Castillo, Kaltenbrunner, Fuster Morell Emotions and dialogue in a peer-production community 12 / 37
- 13. Measuring the Emotional Content of Discussions Use ANEW list Rates a list of 1060 frequent words on a 9 point scale in three dimensions: Valence: positive vs negative emotions Arousal: strength of the emotions Dominance: feeling in control vs feeling dominated Bradley and Lang. (1999). Affective norms for English words (ANEW) Technical report C-1. The Center for Research in Psychophysiology, University of Florida, FL. university-logo Laniado, Castillo, Kaltenbrunner, Fuster Morell Emotions and dialogue in a peer-production community 13 / 37
- 14. Example messages Table: Example messages with their corresponding Valence, Arousal, and Dominance scores (average over the words contained in each message). V A D Sounds like a good challenge - to be proven or disproven. I’m happy 7.4 5.3 6.2 if it can be shown to go further using closed cubic polynomial solu- tions. The nice thing about these are that they are pretty easy to test numerically . . . –in “Exact trigonometric constants” Seems you have not yet seen female lover after having sex who 5.5 7.0 5.2 do not wish to have sex with the same lover any more :) Once you’ve seen it, you understand very well what war of Venus means compared to war of Mars. –in “House (astrology)” What about the whirlie hazing, the alcohol abuse, the emotional 1.6 5.8 3.5 poverty, the suicide in 1995/6, the biotech plans which were stopped by pitzer protests . . . –in “Harvey Mudd College” university-logo Laniado, Castillo, Kaltenbrunner, Fuster Morell Emotions and dialogue in a peer-production community 14 / 37
- 15. Outline 1 Introduction 2 Framework of analysis Data acquisition and pre-processing User gender labelling Sentiment analysis 3 Results Emotions and experience Topics and gender Emotional congruence Emotional homophily Emotions in personal (“User talk”) pages 4 Conclusions university-logo Laniado, Castillo, Kaltenbrunner, Fuster Morell Emotions and dialogue in a peer-production community 15 / 37
- 16. Outline 1 Introduction 2 Framework of analysis Data acquisition and pre-processing User gender labelling Sentiment analysis 3 Results Emotions and experience Topics and gender Emotional congruence Emotional homophily Emotions in personal (“User talk”) pages 4 Conclusions university-logo Laniado, Castillo, Kaltenbrunner, Fuster Morell Emotions and dialogue in a peer-production community 16 / 37
- 17. Emotions and edit count corr= 0.12, p−value<10−8 1 Male Female mean dominance normalized 0.5 0 −0.5 100 1000 10000 100000 log. of edit count Positive correlation between edit count and dominance university-logo → more experienced users tend to feel “in control” more than newbies Laniado, Castillo, Kaltenbrunner, Fuster Morell Emotions and dialogue in a peer-production community 17 / 37
- 18. Emotions and role Distribution of the ANEW-valence scores of admins vs non-admins valence 4.8 5 5.2 5.4 5.6 5.8 6 6.2 6.4 6.6 6.8 7 0.25 1 Normal Admin 0.9 Normal: mean ± SEM 0.2 Admin: mean ± SEM 0.8 P(Admin|mean(valence)=x) P(Admin|mean(valence)=x) 0.7 0.15 0.6 pdf 0.5 0.1 0.4 0.3 0.05 0.2 0.1 0 0 4.8 5 5.2 5.4 5.6 5.8 6 6.2 6.4 6.6 6.8 7 mean valence Distribution of the ANEW-valence scores of admins vs non-admins mean value and the interval of mean ± the standard error of the mean (SEM) dashed lines indicate probability of being an administrator given the mean valence of university-logo a user (assuming equal sample sizes) Laniado, Castillo, Kaltenbrunner, Fuster Morell Emotions and dialogue in a peer-production community 18 / 37
- 19. Emotions and role Mean (±SEM) of valence and dominance for regular users and administrators valence dominance 5.45 Admin 6.1 Non−admin 5.44 6.08 5.43 6.06 p-values mean dominance mean valence 6.04 5.42 *** p < 0.001 ** p < 0.01 5.41 6.02 * p < 0.05 6 5.4 5.98 5.39 5.96 5.38 All le le All le le ma Ma ma Ma Fe Fe Comments by administrators have higher valence and dominance university-logo No significant difference between female admins and female non-admins Laniado, Castillo, Kaltenbrunner, Fuster Morell Emotions and dialogue in a peer-production community 19 / 37
- 20. Outline 1 Introduction 2 Framework of analysis Data acquisition and pre-processing User gender labelling Sentiment analysis 3 Results Emotions and experience Topics and gender Emotional congruence Emotional homophily Emotions in personal (“User talk”) pages 4 Conclusions university-logo Laniado, Castillo, Kaltenbrunner, Fuster Morell Emotions and dialogue in a peer-production community 20 / 37
- 21. Emotions and gender Words more used by females and males Size accounts for difference in frequency university-logo Laniado, Castillo, Kaltenbrunner, Fuster Morell Emotions and dialogue in a peer-production community 21 / 37
- 22. Emotions and gender Mean (±SEM) of valence and dominance form males and females valence dominance 5.45 Male 6.1 Female 5.44 6.08 5.43 6.06 p-values mean dominance mean valence 6.04 5.42 *** p < 0.001 ** p < 0.01 5.41 6.02 * p < 0.05 6 5.4 5.98 5.39 5.96 5.38 All ins ns All ins ns dm mi m mi a Ad ad Ad n− n− No No Females use words associated to more positive emotions university-logo Difference no more significant when normalizing by article Laniado, Castillo, Kaltenbrunner, Fuster Morell Emotions and dialogue in a peer-production community 22 / 37
- 23. Topics, emotions and gender N≥1 ANEW words; corr=−0.64 (p=0.002) 5.6 Computing Arts 5.5 Philosophy Language 5.4 Health Mathematics 5.3 Belief mean valence Sports 5.2 Agriculture Environment 5.1 Techn. & app. sci. Law 5 Society Business Education 4.9 Culture People 4.8 Science Politics 4.7 Geography and places 0.82 0.84 0.86 0.88 0.9 0.92 0.94 0.96 prop. of male comments History and events Figure: Mean valence for discussions of articles in different topic categories, university-logo vs the proportion of comments written by male editors Laniado, Castillo, Kaltenbrunner, Fuster Morell Emotions and dialogue in a peer-production community 23 / 37
- 24. Links to Wikipedia policies Table: Percentage of messages including links to Wikipedia policies for different classes of users Non-admins Admins All users Males 2.5% 4.9% 3.9% Females 6.2% 12.4% 9.8% Differences between genders are statistically significant with p < 0.01 Comments containing links to Wikipedia policies have higher valence, higher arousal and higher dominance when editors invoke community norms, they tend to do it with a remarkably positive and dominant tone, and with stronger emotional load university-logo Laniado, Castillo, Kaltenbrunner, Fuster Morell Emotions and dialogue in a peer-production community 24 / 37
- 25. Message length and gender Females write longer messages 83 vs 71 words on average Difference is accentuated for female and male administrators 85 vs 68 words on average different leadership style? p-values Differences are statistically significant with p < 0.05 university-logo Laniado, Castillo, Kaltenbrunner, Fuster Morell Emotions and dialogue in a peer-production community 25 / 37
- 26. Outline 1 Introduction 2 Framework of analysis Data acquisition and pre-processing User gender labelling Sentiment analysis 3 Results Emotions and experience Topics and gender Emotional congruence Emotional homophily Emotions in personal (“User talk”) pages 4 Conclusions university-logo Laniado, Castillo, Kaltenbrunner, Fuster Morell Emotions and dialogue in a peer-production community 26 / 37
- 27. Emotional congruence Replies are more positive On average, editors tend to reply with: higher valence: +0.05 higher dominance: +0.04 no statistically significant differences for arousal Users tend to be more positive and dominant when replying, but without recurring to words evoking stronger sentiments Results computed on all comments in article talk pages university-logo Laniado, Castillo, Kaltenbrunner, Fuster Morell Emotions and dialogue in a peer-production community 27 / 37
- 28. Outline 1 Introduction 2 Framework of analysis Data acquisition and pre-processing User gender labelling Sentiment analysis 3 Results Emotions and experience Topics and gender Emotional congruence Emotional homophily Emotions in personal (“User talk”) pages 4 Conclusions university-logo Laniado, Castillo, Kaltenbrunner, Fuster Morell Emotions and dialogue in a peer-production community 28 / 37
- 29. Emotional homophily Mixing patterns: do users interact preferentially with similar users? - By activity: users who write more comments tend to reply preferentially to less active users, end viceversa + By gender: females interact more with other females + By style: users interact more with others expressing similar emotions edges connecting users who have exchanged at least 10 replies red nodes → 15% users expressing higher valence in article discussions black nodes → 15% users expressing lower valence in article discussions size → proportional to the number of connections university-logo Laniado, Castillo, Kaltenbrunner, Fuster Morell Emotions and dialogue in a peer-production community 29 / 37
- 30. Outline 1 Introduction 2 Framework of analysis Data acquisition and pre-processing User gender labelling Sentiment analysis 3 Results Emotions and experience Topics and gender Emotional congruence Emotional homophily Emotions in personal (“User talk”) pages 4 Conclusions university-logo Laniado, Castillo, Kaltenbrunner, Fuster Morell Emotions and dialogue in a peer-production community 30 / 37
- 31. Emotions expressed on personal pages Comments written by users on their own talk page written by females: higher valence, dominance and arousal by male administrators: higher valence and dominance, lower arousal male administrators are positive and dominant, but write less emotional content on the contrary women, including female administrators, express stronger emotions on their personal pages Comments received on personal talk pages more positive emotions than in one’s own comments females receive more positive messages, except for female administrators a supposed gender difference seems to be out-weighed by the type of interactions the administrator tasks bring with them university-logo Laniado, Castillo, Kaltenbrunner, Fuster Morell Emotions and dialogue in a peer-production community 31 / 37
- 32. Outline 1 Introduction 2 Framework of analysis Data acquisition and pre-processing User gender labelling Sentiment analysis 3 Results Emotions and experience Topics and gender Emotional congruence Emotional homophily Emotions in personal (“User talk”) pages 4 Conclusions university-logo Laniado, Castillo, Kaltenbrunner, Fuster Morell Emotions and dialogue in a peer-production community 32 / 37
- 33. Conclusions (1) Wikipedia editors express themselves in general with a positive emotional tone responses tend to be more positive than the comments to which they reply users receive more positive messages than they write on their personal pages Administrators and experienced users play a pivotal role they tend to interact especially with less experienced users they express themselves with a more positive tone university-logo Laniado, Castillo, Kaltenbrunner, Fuster Morell Emotions and dialogue in a peer-production community 33 / 37
- 34. Conclusions (2) Female editors are different in their emotional expression and relationships they work on topics in which the discussions have a more positive tone they receive more positive messages on their personal pages they tend to behave similarly to administrators however, there are some differences in the leadership style females write longer messages, link more often Wikipedia policies, and express stronger emotions in personal spaces Wikipedians of a feather fly together editors communicate more with others having a similar style females interact more with other females university-logo Laniado, Castillo, Kaltenbrunner, Fuster Morell Emotions and dialogue in a peer-production community 34 / 37
- 35. Recommendations Design for positive emotional expressions Besides the neutral point of view, provide spaces for emotional expression, favoring positive exchanges and channeling negative feelings in a non-disctructive manner in particular, build upon the positive tone of user talk pages Encourage experienced editors to contribute to positive emotions leverage contagiousness of emotional styles to “cool down” conflictive discussions positive tone of messages linking to WP policies Gender-aware recruiting it may be advisable that females are invited by other females consider that certain topics may be more attractive to females university-logo seek participation of females where the gender gap is more acute Laniado, Castillo, Kaltenbrunner, Fuster Morell Emotions and dialogue in a peer-production community 35 / 37
- 36. Future work Comparison with other metrics preliminary results on the same data using other lexicons are largely consistent Longitudinal analysis how do emotional styles of editors change over time and with increasing experience? how do emotions in the discussion affect participation? Need for qualitative analysis and human annotation include non-textual emotional aspects such as emoticons, barn stars and virtual gifts cover also less experienced users deal with sarcasm, measure the extent of condescending or paternalistic language in comments addressed at female editors university-logo Laniado, Castillo, Kaltenbrunner, Fuster Morell Emotions and dialogue in a peer-production community 36 / 37
- 37. Main references I Bradley and Lang. Affective norms for English words (ANEW) Technical report C-1. The Center for Research in Psychophysiology, University of Florida, FL, 2012. B. Collier and J. Bear. Conflict, criticism, or confidence: an empirical examination of the gender gap in wikipedia contributions. In Proc. of CSCW, 2012. O. Kucuktunc, B. B. Cambazoglu, I. Weber, and H. Ferhatosmanoglu. A large-scale sentiment analysis for Yahoo! answers. In Proc. of WSDM, 2012. S. K. Lam, A. Uduwage, Z. Dong, S. Sen, D. R. Musicant, L. Terveen, and J. Riedl. WP:clubhouse? an exploration of Wikipedia’s gender imbalance. In Proc. of WikiSym, 2011. D. Laniado, R. Tasso, Y. Volkovich, and A. Kaltenbrunner. When the Wikipedians talk: Network and tree structure of Wikipedia discussion pages. In Proc. of ICWSM, 2011. university-logo Laniado, Castillo, Kaltenbrunner, Fuster Morell Emotions and dialogue in a peer-production community 37 / 37