![Article creators
• Misunderstand policy
– “I do understand that articles on wikipedia need to be
sourced… it is due to have two [sources] once [our
website goes] live”
• Express high levels of emotion
– “To be honest it's been a real turn off adding articles
to WP and I don't think I will add articles again. So
smile and enjoy.”
• Learn from discussions
– “much as it would break my heart … it is perhaps
sensible that the piece is deleted.”
Net-ethnography
in 8th International Symposium on Wikis and Open Collaboration
(WikiSym 2012)](https://image.slidesharecdn.com/thesis-summary-arguments-about-deleting-wikipedia-content-paris-2013-04-19-130419082954-phpapp01/85/Thesis-summary-arguments-about-deleting-wikipedia-content-paris-2013-04-19-8-320.jpg)
Thesis summary-arguments-about-deleting-wikipedia-content-paris-2013-04-19
- 1. Copyright 2011 Digital Enterprise Research Institute. All rights reserved. Digital Enterprise Research Institute www.deri.ie Enabling Networked Knowledge Arguments about deleting Wikipedia content Jodi Schneider jschneider@pobox.com Vendredi 19th April 2013 1 Télécom ParisTech
- 3. Deletion threatens Wikipedia • 1 in 4 new Wikipedia articles is deleted – within minutes or hours • Demotivating! – 1 in 3 newcomers start by writing a new article – 7X less likely to stay if their article is deleted! • Can we support editor retention?
- 4. Ph.D. case study: argumentative dialogues about deleting Wikipedia articles • Goals: – Understand collaboration & coordination – Identify “pain points” & new IT support opportunities • Approaches: – Net-ethnography • Interviews of community members • Embedded participation • Reading essays, policies, & written dialogues • Analysing article history, user contributions – Content analysis • Departure point: grounded theory or existing categories. With multiple annotators, iteratively refined annotation manual to achieve strong interannotator agreement. • Decision factors (WikiSym 2012) • Walton’s argumentation schemes (CSCW 2013) – Prototyping & iterative design • Design (WikiSym 2012 demo) • User study (reported in dissertation)
- 6. Corpus • Article deletion dialogues from English Wikipedia started on a typical-volume day • 72 dialogues (94 A4 pages)
- 7. Findings: pain points of article deletion • Article creators • Novices visiting or newly joining Wikipedia • No-consensus dialogues
- 8. Article creators • Misunderstand policy – “I do understand that articles on wikipedia need to be sourced… it is due to have two [sources] once [our website goes] live” • Express high levels of emotion – “To be honest it's been a real turn off adding articles to WP and I don't think I will add articles again. So smile and enjoy.” • Learn from discussions – “much as it would break my heart … it is perhaps sensible that the piece is deleted.” Net-ethnography in 8th International Symposium on Wikis and Open Collaboration (WikiSym 2012)
- 9. Novices’ arguments • Structurally different to experts’ arguments • More problematic arguments from novices – Personal preference – Requesting a favor – Analogy to other cases – No harm in keeping an article – Large number of search engine hits Argumentation schemes content analysis in 16th ACM Conference on Computer-Supported Cooperative Work and Social Computing (CSCW 2013)
- 10. No consensus discussions “What works well is simply the community agreeing on a verdict.” Otherwise: • Time-consuming & difficult to judge a case • Same case may get raised repeatedly • Emotional upset is more likely – “messy”, “full of hate and pain” when overturned Net-ethnography & interviews in 8th International Symposium on Wikis and Open Collaboration (WikiSym 2012)
- 11. Articulate criteria Decision factors content analysis in 8th International Symposium on Wikis and Open Collaboration (WikiSym 2012) 4 Factors cover – 91% of comments – 70% of discussions Factor Example (used to justify `keep') Notability Anyone covered by another encyclopedic reference is considered notable enough for inclusion in Wikipedia. Sources Basic information about this album at a minimum is certainly verifiable, it's a major label release, and a highly notable band. Maintenance …this article is savable but at its current state, needs a lot of improvement. Bias It is by no means spam (it does not promote the products). Other I'm advocating a blanket "hangon" for all articles on newly- drafted players
- 12. Use criteria to augment interface Prototype design (RDFa; custom ontology based on FOAF, SIOC) in WikiSym 2012 Demos
- 13. 84% prefer our system “Information is structured and I can quickly get an overview of the key arguments.” “The ability to navigate the comments made it a bit easier to filter my mind set and to come to a conclusion.” “It offers the structure needed to consider each factor separately, thus making the decision easier. Also, the number of comments per factor offers a quick indication of the relevance and the deepness of the decision.” Based on a formative evaluation user study with 20 novice users in dissertation “Enabling reuse of arguments and opinions from online social disputes”
Editor's Notes
- #3: Felipe Ortega via http://www.businessinsider.com/chart-of-the-day-wikipedia-editors-2009-11Wikipedia editors are leaving faster than they can be replaced1 in 3 editors begin by creating a new article7 times as likely to stay if their article is kept
- #4: “only 0.6 percent of those whose articles are met with deletion stayed editing, compared to 4.4 percent of the users whose articles remained”, http://enwp.org/Wikipedia:Wikipedia_ Signpost/2011-04-04/Editor_retention
- #5: Interviews via various means: (skype, IRC, in person)The story: understand the problem (analysis / survey), solve it (define method / tools + analysed criteria), evaluate it (prototype)
- #7: By typical we mean average volume: there are consistently ~500 discussions per week about deleting borderline articles, see our WikiSym paper.
- #9: Mentoring in discussions is effective: Article creators who receive mentoring seem toMake more edits to the articleContinue editingIncrease understanding of policy
- #10: Experts argue from precedentNovices: values, analogy, cause to effectJodi Schneider, KrystianSamp, Alexandre Passant, and Stefan Decker. Arguments about Deletion: How Experience Improves the Acceptability of Arguments in Ad-hoc Online Task Groups. Computer-Supported Cooperative Work and Social Computing (CSCW 2013).
- #12: 3 student annotators (besides me)Iterative refinement of annotation manualGood interannotator agreement
- #14: 20 novice participants used both systems“The ability to navigate the comments made it a bit easier to filter my mind set and to come to a conclusion.”“summarise and, at the same time, evaluate which factor should be considered determinant for the final decision”