Claudia Müller-Birn1• Leonhard Dobusch1 • James D. Herbsleb2
1 Freie Universität Berlin, 2 Carnegie Mellon University
International Conference on Communities and Technologies 2013
July 2nd 2013, Munich (Germany)
Work-to-Rule:
The Emergence of Algorithmic Governance
in Wikipedia
Picture: http://commons.wikimedia.org/wiki/File:Wikimedia_Foundation_Sue_Gardner_Sept_2010.jpg
Quote: http://soc.li/uhtOs4Y
Data: http://meta.wikimedia.org/wiki/Template:Wikimedia_Growth
January 15th, 2013: 12th anniversary of Wikipedia
“The more eyes on an article,
the better it is. That is the
fundamental premise of
Wikipedia, and it explains why
Wikipedia works.”
“More than 1.5 million people
in almost every country have
contributed to Wikipedia’s 23
million articles.”
Aniket Kittur, Bongwon Suh, Bryan A. Pendleton, and Ed H. Chi. 2007. He says, she says: conflict and coordination in Wikipedia. In Proceedings of the SIGCHI Conference on
Human Factors in Computing Systems (CHI '07). ACM, New York, NY, USA, 453-462.
Brian Butler, Elisabeth Joyce, and Jacqueline Pike. 2008. Don't look now, but we've created a
bureaucracy: the nature and roles of policies and rules in wikipedia. In Proceedings of the SIGCHI
Conference on Human Factors in Computing Systems (CHI '08). ACM, New York, NY, USA,
1101-1110.
“the “policyless” ideal that wikis represent is a pipedream”
Cl. Müller-Birn (CC BY-SA 3.0)
Flagged Revisions
Extension
« not registered »
Cl. Müller-Birn (CC BY-SA 3.0)
Flagged Revisions
Extension
• Not registered editors can only indirectly check their contributions.
• The semi-manual system of detecting malicious contributions has been
replaced by an automated workflow.
• Positive faith has been replaced by a negative one.
Cl. Müller-Birn (CC BY-SA 3.0)
German Wikipedia
Cl. Müller-Birn (CC BY-SA 3.0)
German Wikipedia
Cl. Müller-Birn (CC BY-SA 3.0)
Flagged Revisions
Extension
• Bots are autonomous software programs that make edits based on
predefined rules
• Bots help their operators to carry out “mindless, boring and often
reoccurring tasks”
• Bots belong to a special user group with special user rights
• Bots should avoid editing activities outside the article namespace
Cl. Müller-Birn (CC BY-SA 3.0)
Flagged Revisions
Extension
0
1,000,000
2,000,000
2003 2004 2005 2006 2007 2008 2009 2010 2011 2012
boteditcountperyear
Category
Main
Other
Portal
Talk
User
User talk
Wikipedia
editing activities
includes additional
namespaces
editing activities extends
to community space
focus expands further
to project spaces about 45 %
of all edits
are outside
of article
namespace
Cl. Müller-Birn (CC BY-SA 3.0)
check syntax
edit language
add template
update template
add data
update data
archive article
check for spam
set interwiki-link
create content list
link disambiguation
inter-bot-cooperation
create to-do list
send to-do alert
compile statistic
enforce rule
welcome user
support election
Cl. Müller-Birn (CC BY-SA 3.0)
• Content-oriented focus
• Almost 97% of all analyzed bots take over tasks in this area
check syntax
edit language
add template
update template
add data
update data
archive article
check for spam
set interwiki-link
create content list
link disambiguation
inter-bot-cooperation
create to-do list
send to-do alert
compile statistic
enforce rule
welcome user
support election
Cl. Müller-Birn Attribution-ShareAlike 3.0 Unported (CC BY-SA 3.0)
check syntax
edit language
add template
update template
add data
update data
archive article
check for spam
set interwiki-link
create content list
link disambiguation
inter-bot-cooperation
create to-do list
send to-do alert
compile statistic
enforce rule
welcome user
support election
• Task-oriented focus
• 6% of all analyzed bots take over tasks in this area Cl. Müller-Birn (CC BY-SA 3.0)
check syntax
edit language
add template
update template
add data
update data
archive article
check for spam
set interwiki-link
create content list
link disambiguation
inter-bot-cooperation
create to-do list
send to-do alert
compile statistic
enforce rule
welcome user
support election
• Community-oriented focus
• 1% of all analyzed bots take over tasks in this area Cl. Müller-Birn (CC BY-SA 3.0)
What is the guideline about?
“Hi, let’s talk about something different -
expired votes: How should we deal with
them? My vote has been crossed out and I
do not agree with this.”
• An administrator is automatically re-elected if users with voting rights
support this re-election request.
• Either 25 votes within one month or 50 within six months are needed.
• Votes are only valid if they are submitted (by signing a page) in one of the
two possible periods of time .
Cl. Müller-Birn (CC BY-SA 3.0)
“This would be a nice job for a bot.”
“I created a bot request and
ask for feasibility”
• The final community consensus was to delete these obsolete votes and
one bot operator agreed to incorporate this task into her software.
• The requirements of how to implement the bot have not been discussed -
the bot operator just “extended” an existing program.
• A rule is algorithmically translated but its realization is invisible because
the source code is not publicly available.
Cl. Müller-Birn (CC BY-SA 3.0)
2009 2010 2011 2012 2013
Community
0
85
Content
0
7,200
Task
0
340
xqbot’s extending activity focus over time
Cl. Müller-Birn (CC BY-SA 3.0)
• Software features and bots make guidelines and
rules part of the infrastructure and to a certain
extent, makes them harder to change and easier to
enforce.
• The conversion of socially developed rules into
algorithmic ones makes these rules less
transparent.
• Algorithmically translated rules are more scalable
but they do not handle exceptions well.
• The extent and kind of algorithmic governance
depends on the cultural background of a
community.
Let’s summaries
Cl. Müller-Birn (CC BY-SA 3.0)
How can we proceed with these insights?
• We might elaborate more on what rules and guidelines
can and should be translated into the technical
infrastructure in order to help the community to make
this decision.
• We might investigate existing cultural differences in
adapting algorithmic governance mechanisms.
• We might design new features that make the algorithms
in the source code visible and understandable by non-
programmers.
Cl. Müller-Birn (CC BY-SA 3.0)
Aaron Halfaker, R. Stuart Gieger, Jonathan Morgan & John Riedl. (2013). The Rise and Decline of an Open Collaboration System: How Wikipedia's reaction to
sudden popularity is causing its decline. American Behavioral Scientist, 57(5), 664-688.
One example for stepping in the right direction
"http://www.flickr.com/photos/racatumba/93569705/
Thank you for your attention.
Questions?

More Related Content

PPTX
New Metrics for New Media Bay Area CIO IT Executives Meetup
PPTX
Conversation graphs in Online Social Media
PDF
Benchmarking the Privacy-­Preserving People Search
PPT
AvivErlichRavidAOIR5
PPTX
Social network analysis for modeling & tuning social media website
PDF
Social Networks and Social Capital
PPT
topol05
PDF
Community Detection in Social Networks: A Brief Overview
New Metrics for New Media Bay Area CIO IT Executives Meetup
Conversation graphs in Online Social Media
Benchmarking the Privacy-­Preserving People Search
AvivErlichRavidAOIR5
Social network analysis for modeling & tuning social media website
Social Networks and Social Capital
topol05
Community Detection in Social Networks: A Brief Overview

Viewers also liked (17)

PDF
Bringing user-centered design to open source software development
PDF
What and how users read: Transforming reading behavior into valuable feedback...
PPTX
powerpoint
PPT
Texas Sta Rpowerp
PDF
Integrating agile software development and user-centered design
PDF
Rup pemda lobar 2013
PPT
Texas Sta Rpowerp
PPTX
Magazine Review
PDF
Ieee,.Guide.To.The.Software.Engineering.Body.Of.Knowlewdge.2004.Version.Swebo...
PDF
How temporal network analysis can help us to explore existing interrelationsh...
PPTX
The Student Voice Final
PPT
Concurso de Física
PPT
Hall Of Fame
PDF
Japan.R補完計画(Tokyo.R#46LT)
PPT
Kullanılabilirlik 1 tanim
PPTX
Kaizen Sew
PDF
前処理のための前処理(Tokyo.R#45)
Bringing user-centered design to open source software development
What and how users read: Transforming reading behavior into valuable feedback...
powerpoint
Texas Sta Rpowerp
Integrating agile software development and user-centered design
Rup pemda lobar 2013
Texas Sta Rpowerp
Magazine Review
Ieee,.Guide.To.The.Software.Engineering.Body.Of.Knowlewdge.2004.Version.Swebo...
How temporal network analysis can help us to explore existing interrelationsh...
The Student Voice Final
Concurso de Física
Hall Of Fame
Japan.R補完計画(Tokyo.R#46LT)
Kullanılabilirlik 1 tanim
Kaizen Sew
前処理のための前処理(Tokyo.R#45)
Ad

Similar to Work-to-Rule: The Emergence of Algorithmic Governance in Wikipedia (20)

PDF
Quora ML Workshop: Engineering at the Intersection of Productive Efficiency, ...
PDF
Engineering at the intersection of productive efficiency, ideology, and ethic...
PDF
Offenheit als Organisationsprinzip: Offen für Diversität oder Exklusion durch...
PDF
Fehlende Diversität und Autorenschwund in der Wikipedia: 
Ausgrenzung durch g...
PDF
Openness as an Organizing Principle: Open for Diversity or Exclusionary Openn...
PDF
Offen. Divers. Inklusiv. Thinking the Future of Organizations
PDF
Human-Machine Collaboration 
in Networked Information Systems
PDF
Loops of humans and bots in Wikidata
PDF
Repurposing Wikipedia: Wikipedia as data set and analytical device
PPT
Understanding and improving Wikipedia article discussion spaces SAC2011
PDF
Parc floss-wikipedia
ODP
Wikipedia: a model for using the Internet for good
PPT
Defending Wikipedia 2
PPTX
Wikipedia - The most successful encyclopedia in the world
PDF
2013-08 10 evil things - Northeast PHP Conference Keynote
PDF
Wikipedia 101 governance and tech stack
PPTX
Intro to Editing Wikipedia - SCOTUS Editathon at NARA
PPTX
ALIA Wikipedia and libraries
PPTX
World oceansday wiki-snyder2015
PDF
Observing bots in the wild: A quantitative analysis of a large open-source ec...
Quora ML Workshop: Engineering at the Intersection of Productive Efficiency, ...
Engineering at the intersection of productive efficiency, ideology, and ethic...
Offenheit als Organisationsprinzip: Offen für Diversität oder Exklusion durch...
Fehlende Diversität und Autorenschwund in der Wikipedia: 
Ausgrenzung durch g...
Openness as an Organizing Principle: Open for Diversity or Exclusionary Openn...
Offen. Divers. Inklusiv. Thinking the Future of Organizations
Human-Machine Collaboration 
in Networked Information Systems
Loops of humans and bots in Wikidata
Repurposing Wikipedia: Wikipedia as data set and analytical device
Understanding and improving Wikipedia article discussion spaces SAC2011
Parc floss-wikipedia
Wikipedia: a model for using the Internet for good
Defending Wikipedia 2
Wikipedia - The most successful encyclopedia in the world
2013-08 10 evil things - Northeast PHP Conference Keynote
Wikipedia 101 governance and tech stack
Intro to Editing Wikipedia - SCOTUS Editathon at NARA
ALIA Wikipedia and libraries
World oceansday wiki-snyder2015
Observing bots in the wild: A quantitative analysis of a large open-source ec...
Ad

Recently uploaded (20)

PPTX
The various Industrial Revolutions .pptx
PPTX
Web Crawler for Trend Tracking Gen Z Insights.pptx
PDF
NewMind AI Weekly Chronicles – August ’25 Week III
PDF
A Late Bloomer's Guide to GenAI: Ethics, Bias, and Effective Prompting - Boha...
PPT
Module 1.ppt Iot fundamentals and Architecture
PDF
TrustArc Webinar - Click, Consent, Trust: Winning the Privacy Game
PDF
DP Operators-handbook-extract for the Mautical Institute
PDF
ENT215_Completing-a-large-scale-migration-and-modernization-with-AWS.pdf
PDF
Architecture types and enterprise applications.pdf
PPTX
Benefits of Physical activity for teenagers.pptx
PDF
A comparative study of natural language inference in Swahili using monolingua...
PDF
1 - Historical Antecedents, Social Consideration.pdf
PDF
Getting started with AI Agents and Multi-Agent Systems
PPT
What is a Computer? Input Devices /output devices
PPTX
Group 1 Presentation -Planning and Decision Making .pptx
PPTX
MicrosoftCybserSecurityReferenceArchitecture-April-2025.pptx
PDF
WOOl fibre morphology and structure.pdf for textiles
PDF
A review of recent deep learning applications in wood surface defect identifi...
PDF
August Patch Tuesday
PDF
Hindi spoken digit analysis for native and non-native speakers
The various Industrial Revolutions .pptx
Web Crawler for Trend Tracking Gen Z Insights.pptx
NewMind AI Weekly Chronicles – August ’25 Week III
A Late Bloomer's Guide to GenAI: Ethics, Bias, and Effective Prompting - Boha...
Module 1.ppt Iot fundamentals and Architecture
TrustArc Webinar - Click, Consent, Trust: Winning the Privacy Game
DP Operators-handbook-extract for the Mautical Institute
ENT215_Completing-a-large-scale-migration-and-modernization-with-AWS.pdf
Architecture types and enterprise applications.pdf
Benefits of Physical activity for teenagers.pptx
A comparative study of natural language inference in Swahili using monolingua...
1 - Historical Antecedents, Social Consideration.pdf
Getting started with AI Agents and Multi-Agent Systems
What is a Computer? Input Devices /output devices
Group 1 Presentation -Planning and Decision Making .pptx
MicrosoftCybserSecurityReferenceArchitecture-April-2025.pptx
WOOl fibre morphology and structure.pdf for textiles
A review of recent deep learning applications in wood surface defect identifi...
August Patch Tuesday
Hindi spoken digit analysis for native and non-native speakers

Work-to-Rule: The Emergence of Algorithmic Governance in Wikipedia

  • 1. Claudia Müller-Birn1• Leonhard Dobusch1 • James D. Herbsleb2 1 Freie Universität Berlin, 2 Carnegie Mellon University International Conference on Communities and Technologies 2013 July 2nd 2013, Munich (Germany) Work-to-Rule: The Emergence of Algorithmic Governance in Wikipedia
  • 2. Picture: http://commons.wikimedia.org/wiki/File:Wikimedia_Foundation_Sue_Gardner_Sept_2010.jpg Quote: http://soc.li/uhtOs4Y Data: http://meta.wikimedia.org/wiki/Template:Wikimedia_Growth January 15th, 2013: 12th anniversary of Wikipedia “The more eyes on an article, the better it is. That is the fundamental premise of Wikipedia, and it explains why Wikipedia works.” “More than 1.5 million people in almost every country have contributed to Wikipedia’s 23 million articles.”
  • 3. Aniket Kittur, Bongwon Suh, Bryan A. Pendleton, and Ed H. Chi. 2007. He says, she says: conflict and coordination in Wikipedia. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '07). ACM, New York, NY, USA, 453-462.
  • 4. Brian Butler, Elisabeth Joyce, and Jacqueline Pike. 2008. Don't look now, but we've created a bureaucracy: the nature and roles of policies and rules in wikipedia. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '08). ACM, New York, NY, USA, 1101-1110. “the “policyless” ideal that wikis represent is a pipedream”
  • 5. Cl. Müller-Birn (CC BY-SA 3.0)
  • 6. Flagged Revisions Extension « not registered » Cl. Müller-Birn (CC BY-SA 3.0)
  • 7. Flagged Revisions Extension • Not registered editors can only indirectly check their contributions. • The semi-manual system of detecting malicious contributions has been replaced by an automated workflow. • Positive faith has been replaced by a negative one. Cl. Müller-Birn (CC BY-SA 3.0)
  • 10. Flagged Revisions Extension • Bots are autonomous software programs that make edits based on predefined rules • Bots help their operators to carry out “mindless, boring and often reoccurring tasks” • Bots belong to a special user group with special user rights • Bots should avoid editing activities outside the article namespace Cl. Müller-Birn (CC BY-SA 3.0)
  • 11. Flagged Revisions Extension 0 1,000,000 2,000,000 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 boteditcountperyear Category Main Other Portal Talk User User talk Wikipedia editing activities includes additional namespaces editing activities extends to community space focus expands further to project spaces about 45 % of all edits are outside of article namespace Cl. Müller-Birn (CC BY-SA 3.0)
  • 12. check syntax edit language add template update template add data update data archive article check for spam set interwiki-link create content list link disambiguation inter-bot-cooperation create to-do list send to-do alert compile statistic enforce rule welcome user support election Cl. Müller-Birn (CC BY-SA 3.0)
  • 13. • Content-oriented focus • Almost 97% of all analyzed bots take over tasks in this area check syntax edit language add template update template add data update data archive article check for spam set interwiki-link create content list link disambiguation inter-bot-cooperation create to-do list send to-do alert compile statistic enforce rule welcome user support election Cl. Müller-Birn Attribution-ShareAlike 3.0 Unported (CC BY-SA 3.0)
  • 14. check syntax edit language add template update template add data update data archive article check for spam set interwiki-link create content list link disambiguation inter-bot-cooperation create to-do list send to-do alert compile statistic enforce rule welcome user support election • Task-oriented focus • 6% of all analyzed bots take over tasks in this area Cl. Müller-Birn (CC BY-SA 3.0)
  • 15. check syntax edit language add template update template add data update data archive article check for spam set interwiki-link create content list link disambiguation inter-bot-cooperation create to-do list send to-do alert compile statistic enforce rule welcome user support election • Community-oriented focus • 1% of all analyzed bots take over tasks in this area Cl. Müller-Birn (CC BY-SA 3.0)
  • 16. What is the guideline about? “Hi, let’s talk about something different - expired votes: How should we deal with them? My vote has been crossed out and I do not agree with this.” • An administrator is automatically re-elected if users with voting rights support this re-election request. • Either 25 votes within one month or 50 within six months are needed. • Votes are only valid if they are submitted (by signing a page) in one of the two possible periods of time . Cl. Müller-Birn (CC BY-SA 3.0)
  • 17. “This would be a nice job for a bot.” “I created a bot request and ask for feasibility” • The final community consensus was to delete these obsolete votes and one bot operator agreed to incorporate this task into her software. • The requirements of how to implement the bot have not been discussed - the bot operator just “extended” an existing program. • A rule is algorithmically translated but its realization is invisible because the source code is not publicly available. Cl. Müller-Birn (CC BY-SA 3.0)
  • 18. 2009 2010 2011 2012 2013 Community 0 85 Content 0 7,200 Task 0 340 xqbot’s extending activity focus over time Cl. Müller-Birn (CC BY-SA 3.0)
  • 19. • Software features and bots make guidelines and rules part of the infrastructure and to a certain extent, makes them harder to change and easier to enforce. • The conversion of socially developed rules into algorithmic ones makes these rules less transparent. • Algorithmically translated rules are more scalable but they do not handle exceptions well. • The extent and kind of algorithmic governance depends on the cultural background of a community. Let’s summaries Cl. Müller-Birn (CC BY-SA 3.0)
  • 20. How can we proceed with these insights? • We might elaborate more on what rules and guidelines can and should be translated into the technical infrastructure in order to help the community to make this decision. • We might investigate existing cultural differences in adapting algorithmic governance mechanisms. • We might design new features that make the algorithms in the source code visible and understandable by non- programmers. Cl. Müller-Birn (CC BY-SA 3.0)
  • 21. Aaron Halfaker, R. Stuart Gieger, Jonathan Morgan & John Riedl. (2013). The Rise and Decline of an Open Collaboration System: How Wikipedia's reaction to sudden popularity is causing its decline. American Behavioral Scientist, 57(5), 664-688. One example for stepping in the right direction