Visualizing social interactions in Wikipedia - WikiCorp 2018
0 likes598 views
This document describes research on visualizing social interactions and controversies on Wikipedia. Key points: - Researchers developed Contropedia, a tool to analyze controversial elements within Wikipedia articles over time using edit histories. It identifies controversial topics, when they were most disputed, and perspectives from different language versions. - Controversiality is measured by counting disagreeing edits involving specific topics. The tool represents discussions as trees and networks to analyze complexity. - Analysis found political interactions on articles are neutral, while personal talk pages show homophily. Women express more positive emotions and are more relationship-oriented in discussions. - Contropedia aims to increase transparency on knowledge negotiations and perspectives behind published content on Wikipedia.
Visualizing social interactions in Wikipedia - WikiCorp 2018
- 1. Visualizing social interactions in Wikipedia david.laniado@gmail.com French-German Colloquium WikiCorp 2018 10 July, 2018 - Nice
- 2. Wikipedia ● largest repository of human knowledge ● first result for web queries on many topics ● anyone may edit it ● conditions and reflects public opinion
- 3. ● Wikipedia articles are not static pieces of knowledge ● Quality of content is guaranteed less by absence of errors than by their constant improvability ● Editors correct each other so that all significant views on a topic are represented with due weight ● Struggles over content often reflect societal debates, but are not immediately visible to the public
- 4. Wikipedia’s most visible side
- 5. Behind the scenes: talk pages (thousands of comments, 16 archived pages)
- 6. Complete edit history log (thousands of revisions, just in chronological order)
- 7. Part 1 Characterizing discussions in talk pages
- 8. Representing talk pages as discussion trees
- 9. Representing talk pages as discussion trees ● Tree of nested replies ○ Red → root (the article) ○ Blue → structural nodes (talk pages, threads) ○ Green →Anonymous comments ○ Grey → Registered editors’ comments Laniado, D., Tasso, R., Volkovich, Y., & Kaltenbrunner, A. (2011). When the wikipedians talk: Network and tree structure of wikipedia discussion pages. In ICWSM.
- 10. Measuring the complexity of a discussion ● H-index of a discussion tree ○ Balanced depth measure ○ The maximal number h such that there are at least h comments at level (depth) h ○ →there are h subthreads of depth at least h
- 11. Discussions by semantic area # of chains vs # of edits # of users vs # h-index
- 12. Temporal growth of a discussion How to measure the growth of a discussion over time? Comments over time Cumulative growth
- 13. Temporal growth of a discussion A. Kaltenbrunner and D. Laniado (2012). There is No Deadline - Time Evolution of Wikipedia Discussions. WikiSym. Growth rate ∆h (days ) ● Avg # of days for increasing h ○ George W. Bush: ∆h =70.7 days ○ Barack Obama: ∆h =90.2 days ○ Bill Clinton: ∆h =331.9
- 16. Political interactions in Wikipedia ● How does political affiliation affect interactions? ● Political affiliation of 1,390 users who disclosed it with infoboxes ○ 863 Democrats ○ 527 Republicans
- 17. “Jointly they edit”: Neutral mixing in article talk pages ● Reply network from article talk pages: ○ Between Democrats ○ Between Republicans ○ Mixed interactions ➞ Neutral mixing (no homophily) ● In personal talk pages: political homophily ● Conflict (manual labelling): no significant difference for mixed interaction threads J. Neff, D. Laniado, K. Kappler, Y. Volkovich, P. Aragón & A. Kaltenbrunner (2013). Jointly they edit: Examining the impact of community identification on political interaction in wikipedia. PloS one, 8(4), e60584.
- 18. Sentiment analysis of Wikipedia discussions ● Analysis of emotions and language based on three instruments: ○ Affective norms for English words (ANEW) ○ Linguistic Inquiry and Word Count (LIWC) ○ SentiStrength ● Manually labelled user gender ○ 2,613 men, 165 women ○ from sample of users who wrote over 100 comments in articles talk pages ○ gender analysis of emotions and language, to get insights on gender gap
- 19. Emotions and gender Laniado, D., Kaltenbrunner, A., Castillo, C., & Morell, M. F. (2012). Emotions and dialogue in a peer-production community: the case of Wikipedia. WikiSym. ● ANEW words more frequently used by men and women ● Size proportional to the difference in frequency ● Women express more positive emotions ● not significant after normalizing by article
- 20. Topics, emotions and gender ● Mean valence (ANEW) for discussions of articles in different topic categories, vs the proportion of comments written by men
- 21. Language, emotions and leadership style by gender ● Women write longer messages ● Women are more relationship-oriented ○ more personal pronouns, in particular “I”, more social words ● Women are not more insecure ○ Less certainty words, no significant difference for tentativeness and filler words ● Admins are less relationship oriented, but not in the case of women admins ○ Different leadership style Iosub, D., Laniado, D., Castillo, C., Morell, M. F., & Kaltenbrunner, A. (2014). Emotions under discussion: Gender, status and communication in online collaboration. PloS one, 9(8), e104880.
- 22. Relationship orientation (manual inspection) ● Manual classification of 100 comments ● Three main types of comments high in relationship-orientation: ○ inviting comments that explain the edit in a friendly tone, and call for further intervention and collaboration ○ common perspective-building comments that are focused on understanding others and solving debates in a constructive manner ○ appreciative comments that contain positive emotions and celebrate others’ actions ➞ This suggests that relationship-orientation may be conducive to successful collaboration
- 23. Emotional homophily (anger in reply network) ● Disassortativity by activity ○ users who write more comments tend to reply preferentially to less active users, and vice versa ● Assortativity by gender ○ Men interact more with men, women with women ● Assortativity by emotion and language ○ according to all the emotional and linguistic variables tested ○ also in the network of communication on personal talk pages ● edges→at least 10 replies between two editors ● node color →level of anger in user messages, from low to high
- 24. References Daniela Iosub, David Laniado, Carlos Castillo, Mayo Fuster Morell, Andreas Kaltenbrunner (2014). Emotions under Discussion: Gender, Status and Communication in Online Collaboration. PLoS ONE, 9(8): e104880. Jessica J. Neff, David Laniado, Andreas Kaltenbrunner, Karolin Kappler, Yana Volkovich, Pablo Aragón (2013). Jointly they edit: Examining the impact of community identification on political interaction in Wikipedia. PLoS ONE 8(4), e60584. David Laniado, Carlos Castillo, Andreas Kaltenbrunner, Mayo Fuster-Morell (2012). Emotions and dialogue in a peer-production community. WikiSym '12 - 8th International Symposium on Wikis and Open Collaboration. Andreas Kaltenbrunner and David Laniado (2012). There is No Deadline - Time Evolution of Wikipedia Discussions. WikiSym '12 - 8th International Symposium on Wikis and Open Collaboration. David Laniado, Riccardo Tasso, Yana Volkovich and Andreas Kaltenbrunner (2011). When the Wikipedians Talk: Network and Tree Structure of Wikipedia Discussion Pages. ICWSM '11 - 5th International AAAI Conference on Weblogs and Social Media.
- 25. Part 2 Exploring controversies with Contropedia
- 26. Contropedia aims to: ● make a huge volume of information easily accessible ● unveil negotiations behind encyclopaedic content ● allow users to follow the development of individual topics Borra, E., Weltevrede, E., Ciuccarelli, P., Kaltenbrunner, A., Laniado, D., Magni, G., Mauri, M., Rogers, Richards & Venturini, T. (2015). Societal controversies in Wikipedia articles. In Proceedings of CHI.
- 27. Layer view ● which are the most disputed concepts? ● where are controversies located in the article? ● wiki links as focal points ○ the hotter the color, the more controversial the element (wiki link)
- 29. Controversiality measure ● count edits to sentences including a wiki link ● only consider substantive disagreeing edits: ○ substantive -> no vandalism or anti-vandalism ○ disagreeing -> delete some content ● language agnostic approach ○ based only on user activity
- 30. Dashboard view ● ranking of the most controversial elements within the article ● timeline shows when each element was most disputed ● elements that are not present (as wiki links) in the current version of the article are struck through
- 32. Detailed view ● Why was something controversial? ● All the edits involving a specific element ○ edits in chronological order ○ content added and deleted in each edit ● Optionally shown also related comments from the talk page
- 35. How did we get to the current consensus? ● ● To understand how the sentence got to the current formulation, let’s have a look at the edits on ethnic cleansing (by clicking on the element):
- 37. Matching comments from talk pages ● Join the two streams (edit history and talk pages) ● Matching comments and threads from talk pages to entities (wikilinks), directly or through sections and edits ○ comments explicitly mentioning an edit ○ edit summary mentioning a thread from the talk page ○ temporal co-occurrence (edit and comment by a user within a time window) ○ thread title matching section title ○ entity mentioned within a thread
- 38. Comparison of different periods ● Which are the most controversial elements in different periods? ○ selecting specific time windows (before and after 2008)
- 45. How would you word the 2005 audio recording controversy? Trump ... ● “was overheard sharing a number of vulgar sexual remarks over the course of a private conversation” ● “bragged about forcibly kissing women and being permitted to grab women's genitals” ● “talked about kissing and groping women without their consent” ● “jokingly brags to companions that as a celebrity he can and does kiss or groping any woman he's attracted to” ● “appeared to brag about committing sexual assault.”
- 46. Consensus since November 2016
- 48. ● The tool is language agnostic ○ metrics only based on activity, no automatic analysis of language ● It works for any language version ● Eventually using automatic translation of the results And in other language versions?
- 49. Cross-language analysis ● what (and when) was controversial about a topic in different language versions? ● example for article “Israeli-Palestinian conflict” in English, French, Arabic and Hebrew
- 53. (Arabic version, with automatic translation)
- 55. (Hebrew version, with automatic translation)
- 56. Cross-language analysis ● example for article “Homosexuality” in English, Spanish, Portuguese, Italian, Russian
- 62. Cross-language analysis ● example for article “Homosexuality” in English, Spanish, Portuguese, Italian, Russian ○ Catholic church appears only in Italy among the most controversial elements ○ USSR and USA are controversial in the Russian Wikipedia
- 63. Wikipedia as cultural heritage gateway and site ● Anything can be controversial, even cheese... C. Pentzold, E. Weltevrede, M. Mauri, D. Laniado, A. Kaltenbrunner, E. Borra (2017). Digging Wikipedia. The online encyclopedia as digital cultural heritage gateway and site. ACM Journal on Computing and Cultural Heritage (JOCCH), 10 (1), 5, 2017.
- 64. ● Controversy on the origin of Feta (Greece, Macedonia, Bulgaria, Turkey…?)
- 65. ● Spanish vs Catalan Wikipedia ● Different color codes convey different narratives on bullfighting regulation
- 67. Networks of hyperlinks ● Each article represents a concept ● All concepts relevant to an article should link to the corresponding page ● The hyperlink network represents a mind map as an emerging collaborative concept network
- 68. ● Network of hyperlinks around article “1948 Palestinian exodus” ● Node size: degree (# of connections) ● Colors: clusters of densely connected articles ● (functionality not yet integrated in Contropedia)
- 69. Summing up ● what is most controversial? ○ count disagreeing, substantive, edits to a wiki link ○ visualize with layer view and dashboard ● when is it most controversial? ○ controversy timeline ● what is the controversy about? ○ edit history around a specific wiki link
- 70. Contropedia can: ● increase transparency in Wikipedia ● lower barriers to participation ● foster critical digital literacy
- 71. Users and Uses ● Social researchers ● Wikipedians ● Teachers and students ● Journalists ● Citizens
- 72. Work in progress ● fully integrating talk pages ● beyond wiki links: templates, references, images, individual words... ● edit networks, editor camps ● cross-language visualizations ● networks of articles and external references
- 73. Contropedia as a didactic tool ● Showing knowledge as the fruit of continuous negotiations between different points of view ○ Unveil the process behind article creation ● Inspect development of topics within an article ● Compare narratives in different language communities ● Combined with editing Wikipedia and translating articles to the local language
- 74. References E. Borra, E. Weltevrede, P. Ciuccarelli, A. Kaltenbrunner, D. Laniado, G. Magni, M. Mauri, R. Rogers, and T. Venturini. Societal Controversies in Wikipedia Articles, CHI ’15 - Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems, 2015. E. Borra, D. Laniado, E. Weltevrede, M. Mauri, G. Magni, T. Venturini, P. Ciuccarelli, R. Rogers, and A. Kaltenbrunner. A Platform for Visually Exploring the Development of Wikipedia Articles, ICWSM '15 - Proceedings of the 9th International AAAI Conference on Web and Social Media, 2015. F Flöck, D Laniado, F Stadthaus, M Acosta Towards Better Visual Tools for Exploring Wikipedia Article Development – The Use Case of Gamergate Controversy Wikipedia, a Social Pedia: Research Challenges and Opportunities, 2015. C. Pentzold, E. Weltevrede, M. Mauri, D. Laniado, A. Kaltenbrunner, E. Borra (2017) Digging Wikipedia. The online encyclopedia as digital cultural heritage gateway and site ACM Journal on Computing and Cultural Heritage (JOCCH), 10 (1), 5, 2017. and more resources at: http://contropedia.net/
- 75. Play with the Contropedia demo! http://www.contropedia.net/demo It already contains hundreds of articles. Feel free to ask, we can add specific articles you would like to play with in specific languages. david.laniado@gmail.com