Topcoder Reflections
MICHAEL CONTRERAS | CROWDSOURCING WEEK GLOBAL | 10 / 25/ 2018
A Decade of Public Sector Crowdsourcing
Topcoder was founded with a simple goal: identify the best
developers in the world through competition.
Over the past decade, Topcoder has been a catalyst for
public sector crowdsourcing, reframing the way the US
Federal Government develops software and conducts
R&D.
Topcoder
Community
Topcoder is home to the
world’s largest network of
Designers, Developers, and
Data Scientists.
The Topcoder Community
includes more than 1.3
million members from every
country around the world.
Community members in
over 190 countries
More engineers than
Microsoft, Facebook,
and Twitter combined
Challenges
per quarter
New members
per week
Paid to the
community
Logins every
90 days
1M+ 5x 1400
1800 $80m 35k
Where we have been . . .
2001
Topcoder launches gamified
platform for computer
scientists to demonstrate
skills and gain notoriety.
First Topcoder Community
member ratings are published.
2010
The NASA Tournament Lab is
formed. Topcoder begins to
work on app dev and
algorithm projects for many
Federal agencies through
Harvard.
2005
NSA begins to sponsor the
Topcoder Open for recruiting
purposes.
Full application development
stack capabilities added in
2006.
2004
Harvard publishes its first
academic study on Topcoder.
First commercial offering for
software components.
2011
Topcoder is onboarded to the
GSA Schedule for Challenges
and Competition Services.
DARPA taps Topcoder to
launch STEM education
community for teens.
2015
Topcoder is awarded a spot
on the NASA Open Innovation
Services contract vehicle
through 2020.
Topcoder and Apple partner to
launch the iOS Community.
2007
First commercial Marathon
Match conducted to develop
an enterprise data science
solution.
Topcoder Community expands
to include UX design.
2017
Topcoder has run challenges
for CMS, I-ARPA, EPA, Air
Force, DOE, VA, PTO, OPM,
USAID
Topcoder and IBM partner to
launch the Cognitive
Community.
2016
Topcoder surpasses 1 million-
member mark.
Topcoder and GE partner to
launch the IoT Community.
Wipro acquires Topcoder to
transform its business and the
IT services industry.
2009
The NSA “Enigma” Marathon
Match is hosted on Topcoder
based on the famous WWII era
decoding of the Enigma
machine, a cryptographic
security system.
01 04 05 07 09 10 11 15 16 17
6
It started with recruiting…
7
8
9
10
11
12
13
14
Algorithms and R&D
16
17
DATA SCIENTISTS
450K
AVERAGE # COMPETITORS
PER CHALLENGE
78
AVERAGE #SUBMISSIONS
PER CHALLENGE
578
Topcoder’s Data Science Community
18 Algorithms and Analytics Sales Enablement
Algorithms and Analytics Overview
Products
● Algorithm Optimization
● Predictive Analytics
● Image and Pattern Recognition
● Objective Optimization
Crowdsourcing Contests
● Marathon Match
● Banner Match
● Mini Marathon Match
● Data Science First-to-
Finish (F2F)
● Data Science Sprint
● Data Visualization
● Data Science Ideation
Objectively scores with an automated scoring function that feeds the Topcoder live leaderboard.
Restricted to the following languages: Java, C++, C#.NET, VB.NET, or Python.
Objectively scores with an automated scoring function that feeds the Topcoder live leaderboard.
Competitors are allowed to use any language and libraries to write their solution.
A shorter duration, more straight-forward version of the Marathon Match. Has the same structure
and rules of a regular Marathon Match but a smaller prize purse for competitors.
Objectively scores with an automated scoring function that feeds the Topcoder live leaderboard.
Restricted to the following languages: Java, C++, C#.NET, VB.NET, or Python. A predetermined
threshold is selected for the match, and once a competitor reaches the desired threshold score
the match ends and the first place winner receives the one prize available.
A series of smaller, more rapid data science challenges with a manual scoring function and no
leaderboard. Competitors are allowed to use any language and libraries to write their solution.
A hybrid project completed by a series of data science tasks to find the unique qualities or
“highlights” in the data, followed by design challenges to visualize the data. The output of this
kind of project is static screens.
Looks to discover or learn new approaches, data, and ideas from the community.
Predictive Toxicology -
Environmental Safety
Predictive Analytics
Every day we come into contact with common pesticides. Which chemicals,
in what dosage, can cause us harm? See how the EPA used crowdsourcing
to deliver more accurate predictive analysis.
Learn More
338 49
a Brand New Approach
discovered, tested, and shared with their scientific community
REGISTRANTS COMPETITORS
804
SUBMISSIONS
3
CHALLENGE DURATION
WEEKS
21
Offering Design and
Development
Outcomes?
22
Topcoder Studio circa 2009
Topcoder Reflections: Observations from a Decade of Crowdsourcing with the Public Sector
24
Application
Wireframe
25
Translate your mobile or web app idea into a structured design brief
that includes a description of the design concept, use case scenarios,
design guidelines, and a low-fidelity mockup of basic UI/UX elements.
What’s Included
Application design brief crowdsourced in the Topcoder Marketplace.
A Topcoder copilot manages all logistics.
What You Receive
Written visual design brief for up to five screens of a single application.
You pay only for the winning brief, not hours, and you own the final
deliverables.
Design Brief
Plan the UI/UX design of your application
Application Wireframe
26
Code
Development
Code
Development
27
Code Development packages make it easy to add the people and skills
you need to round out your team, free valuable time, and accelerate
development.
Crowdsource specific phases of your development process or end-to-
end application builds.
What’s Included
Code or task crowdsourced in the Topcoder Marketplace.
A Topcoder copilot manages all logistics.
What You Receive
Ready-to-deploy code, interface, or service built to your specs. You pay
only for the winning code, not hours, and you own the final deliverables.
Code Development
Enterprise-grade code for every stage of the dev life
cycle
Code Development
POWERED BY
Eight month
challenge
Multiple concurrent
development work streams
Third round
of human
testing
29
New HorizonsVA WORKFORCE DEVELOPMENT
This is footer and you can change the content or delete if you wanted to
30
What is the TCVC?
The TCVC is Topcoder’s Veterans Community
● The VA has 3 projects in motion at Topcoder via NASA - web, analytics & mobile
● Each VA project includes funding for the TCVC
● Topcoder teams with OperationCode, a veterans 503(c) to source members
The TCVC is a place for US Veterans and Active Duty to learn and compete
The TCVC is hosted as a private community within Topcoder’s platform
● Challenges run inside the TCVC are open only to US Veterans and Active Duty
● Challenges types run from “Hello World” style to complex multi-challenge series
● Challenges also come to the TCVC via the VA’s projects as well
● Topcoder challenges can be cross-posted to TCVC with different prizes On veterans.topcoder.com, Vets compete
against each other, mentor each other, code
solutions for paying customers, earn
prizes and payments, and build their
resumes by working on actual customer
solutions.
31
What are the goals of the TCVC?
VA Goals
Cultivate and Prepare veterans for entry into the tech world
● Give veterans the opportunity to PROSUMERS
● Provide opportunity for veterans to demonstrate current, marketable tech skills
Topcoder Goals
Use the TCVC as an incubator for new community members
● Cultivate new members
● Train and grow new copilots who will eventually lead projects in our regular
community
● Create a compelling story for Topcoder
32
Our moment . . .
Army Air Forces
ARPA
BBN
Telenet Inc.
Harvard Topcoder | Wipro
?
37
Special thanks to . . .
Andy LaMora
Clinton Bonner Rashid SialJessie Ford
Mike Morris Brendan Wright

More Related Content

PPT
Top Coder Platform Overview 92209
PPTX
Wowing Customers in the Age of AI and the Gig Economy
PPTX
Top coder
PPTX
Top coder
PPTX
Top coder
PPT
TC Crowdsourcing Presentation
TXT
PDF
[Topcoder] for Developers - Dublin Salesforce User Group
Top Coder Platform Overview 92209
Wowing Customers in the Age of AI and the Gig Economy
Top coder
Top coder
Top coder
TC Crowdsourcing Presentation
[Topcoder] for Developers - Dublin Salesforce User Group

Similar to Topcoder Reflections: Observations from a Decade of Crowdsourcing with the Public Sector (20)

PPTX
Why many data science projects fail
PDF
2 partners ed_kickoff_sirris
PPTX
Opening Up Innovation at NASA - NASA's Open Innovation Toolkit, Experience, a...
PPTX
Opening Up Innovation at NASA (NASA's Open Innovation Toolkit and Experience)
PPTX
Opening Up Innovation at NASA - NASA\CoECI - Steve Rader
PDF
April 2014 building data science keynote at Boston Data Science Meetup - Crow...
PDF
The DATA RING - A canvas for DATA PROJECT
PDF
Resume march 20
PDF
My journey of Innovation 我的創新之旅
PPTX
The Evolution of Open Data
PPTX
Session 01 designing and scoping a data science project
PPTX
Session 01 designing and scoping a data science project
PDF
Tutorial: Social Semantic Web and Crowdsourcing - E. Simperl - ESWC SS 2014
PDF
Kush stats alpha
PDF
Back to Square One: Building a Data Science Team from Scratch
PDF
Oleksii Moskalenko "Continuous Delivery of ML Pipelines to Production"
PDF
Mohamed-Rashad-Resume
PDF
Insemtives swat4ls 2012
PPTX
Crowdsourcing for Online Data Collection
PPTX
Fixing data science & Accelerating Artificial Super Intelligence Development
Why many data science projects fail
2 partners ed_kickoff_sirris
Opening Up Innovation at NASA - NASA's Open Innovation Toolkit, Experience, a...
Opening Up Innovation at NASA (NASA's Open Innovation Toolkit and Experience)
Opening Up Innovation at NASA - NASA\CoECI - Steve Rader
April 2014 building data science keynote at Boston Data Science Meetup - Crow...
The DATA RING - A canvas for DATA PROJECT
Resume march 20
My journey of Innovation 我的創新之旅
The Evolution of Open Data
Session 01 designing and scoping a data science project
Session 01 designing and scoping a data science project
Tutorial: Social Semantic Web and Crowdsourcing - E. Simperl - ESWC SS 2014
Kush stats alpha
Back to Square One: Building a Data Science Team from Scratch
Oleksii Moskalenko "Continuous Delivery of ML Pipelines to Production"
Mohamed-Rashad-Resume
Insemtives swat4ls 2012
Crowdsourcing for Online Data Collection
Fixing data science & Accelerating Artificial Super Intelligence Development
Ad

More from Crowdsourcing Week (20)

PDF
Crowdsourcing à la sbv IMPROVER: the challenge of being your own client
PPTX
Transforming the Global Payments Operation
PDF
Crowdsourced to Outsourced: How online platforms are shaping the future of work
PPTX
Malasya's Experience in Crowd Labour and Sharing Economy
PPTX
LM Industries: Harnessing The Power of Crowdsourced Innovation to Build the F...
PDF
Human Collective Intelligence: the future of corporate innovation
PPTX
9 Ways to Ruin Your Open Innovation Challenge
PPTX
Disruptive Crowdsourcing
PPTX
Accelerating Hardware Development: Ideation and Engineering
PPTX
Attracting and Retaining Top Partners with a Best-in-Class Payments Experience
PPTX
Crowdsourcing Disaster Relief
PPTX
Core + Crowd: Why (and how) crowdsourcing is about to become mainstream
PPTX
Smart and Secure Cities and Communities
PPTX
How Successful Crowdsourcing Depends on asking 'Interesting Questions'
PPTX
Contestant Centered Design: creative approaches to designing competitions
PPTX
How Crypto can Monetize Crowdsourcing
PPTX
A New Report on the State of Open Innovation and What it Means For you
PPTX
Expert Operating System: Business On-Demand
PPTX
Crowdsourcing: Changing the Faces of Innovation at NASA
PPT
Crowdfunding an ICO Without Getting In Trouble
Crowdsourcing à la sbv IMPROVER: the challenge of being your own client
Transforming the Global Payments Operation
Crowdsourced to Outsourced: How online platforms are shaping the future of work
Malasya's Experience in Crowd Labour and Sharing Economy
LM Industries: Harnessing The Power of Crowdsourced Innovation to Build the F...
Human Collective Intelligence: the future of corporate innovation
9 Ways to Ruin Your Open Innovation Challenge
Disruptive Crowdsourcing
Accelerating Hardware Development: Ideation and Engineering
Attracting and Retaining Top Partners with a Best-in-Class Payments Experience
Crowdsourcing Disaster Relief
Core + Crowd: Why (and how) crowdsourcing is about to become mainstream
Smart and Secure Cities and Communities
How Successful Crowdsourcing Depends on asking 'Interesting Questions'
Contestant Centered Design: creative approaches to designing competitions
How Crypto can Monetize Crowdsourcing
A New Report on the State of Open Innovation and What it Means For you
Expert Operating System: Business On-Demand
Crowdsourcing: Changing the Faces of Innovation at NASA
Crowdfunding an ICO Without Getting In Trouble
Ad

Recently uploaded (20)

PDF
TrustArc Webinar - Click, Consent, Trust: Winning the Privacy Game
PDF
Convolutional neural network based encoder-decoder for efficient real-time ob...
PDF
How ambidextrous entrepreneurial leaders react to the artificial intelligence...
PDF
A proposed approach for plagiarism detection in Myanmar Unicode text
PDF
Flame analysis and combustion estimation using large language and vision assi...
PPT
Geologic Time for studying geology for geologist
PDF
Abstractive summarization using multilingual text-to-text transfer transforme...
PDF
From MVP to Full-Scale Product A Startup’s Software Journey.pdf
PDF
A comparative study of natural language inference in Swahili using monolingua...
PPTX
Configure Apache Mutual Authentication
PDF
A review of recent deep learning applications in wood surface defect identifi...
PDF
sustainability-14-14877-v2.pddhzftheheeeee
PDF
Enhancing emotion recognition model for a student engagement use case through...
PDF
Taming the Chaos: How to Turn Unstructured Data into Decisions
PDF
Developing a website for English-speaking practice to English as a foreign la...
PDF
Architecture types and enterprise applications.pdf
PDF
Hybrid horned lizard optimization algorithm-aquila optimizer for DC motor
PDF
sbt 2.0: go big (Scala Days 2025 edition)
PPTX
2018-HIPAA-Renewal-Training for executives
PDF
OpenACC and Open Hackathons Monthly Highlights July 2025
TrustArc Webinar - Click, Consent, Trust: Winning the Privacy Game
Convolutional neural network based encoder-decoder for efficient real-time ob...
How ambidextrous entrepreneurial leaders react to the artificial intelligence...
A proposed approach for plagiarism detection in Myanmar Unicode text
Flame analysis and combustion estimation using large language and vision assi...
Geologic Time for studying geology for geologist
Abstractive summarization using multilingual text-to-text transfer transforme...
From MVP to Full-Scale Product A Startup’s Software Journey.pdf
A comparative study of natural language inference in Swahili using monolingua...
Configure Apache Mutual Authentication
A review of recent deep learning applications in wood surface defect identifi...
sustainability-14-14877-v2.pddhzftheheeeee
Enhancing emotion recognition model for a student engagement use case through...
Taming the Chaos: How to Turn Unstructured Data into Decisions
Developing a website for English-speaking practice to English as a foreign la...
Architecture types and enterprise applications.pdf
Hybrid horned lizard optimization algorithm-aquila optimizer for DC motor
sbt 2.0: go big (Scala Days 2025 edition)
2018-HIPAA-Renewal-Training for executives
OpenACC and Open Hackathons Monthly Highlights July 2025

Topcoder Reflections: Observations from a Decade of Crowdsourcing with the Public Sector

  • 1. Topcoder Reflections MICHAEL CONTRERAS | CROWDSOURCING WEEK GLOBAL | 10 / 25/ 2018 A Decade of Public Sector Crowdsourcing
  • 2. Topcoder was founded with a simple goal: identify the best developers in the world through competition. Over the past decade, Topcoder has been a catalyst for public sector crowdsourcing, reframing the way the US Federal Government develops software and conducts R&D.
  • 3. Topcoder Community Topcoder is home to the world’s largest network of Designers, Developers, and Data Scientists. The Topcoder Community includes more than 1.3 million members from every country around the world.
  • 4. Community members in over 190 countries More engineers than Microsoft, Facebook, and Twitter combined Challenges per quarter New members per week Paid to the community Logins every 90 days 1M+ 5x 1400 1800 $80m 35k
  • 5. Where we have been . . . 2001 Topcoder launches gamified platform for computer scientists to demonstrate skills and gain notoriety. First Topcoder Community member ratings are published. 2010 The NASA Tournament Lab is formed. Topcoder begins to work on app dev and algorithm projects for many Federal agencies through Harvard. 2005 NSA begins to sponsor the Topcoder Open for recruiting purposes. Full application development stack capabilities added in 2006. 2004 Harvard publishes its first academic study on Topcoder. First commercial offering for software components. 2011 Topcoder is onboarded to the GSA Schedule for Challenges and Competition Services. DARPA taps Topcoder to launch STEM education community for teens. 2015 Topcoder is awarded a spot on the NASA Open Innovation Services contract vehicle through 2020. Topcoder and Apple partner to launch the iOS Community. 2007 First commercial Marathon Match conducted to develop an enterprise data science solution. Topcoder Community expands to include UX design. 2017 Topcoder has run challenges for CMS, I-ARPA, EPA, Air Force, DOE, VA, PTO, OPM, USAID Topcoder and IBM partner to launch the Cognitive Community. 2016 Topcoder surpasses 1 million- member mark. Topcoder and GE partner to launch the IoT Community. Wipro acquires Topcoder to transform its business and the IT services industry. 2009 The NSA “Enigma” Marathon Match is hosted on Topcoder based on the famous WWII era decoding of the Enigma machine, a cryptographic security system. 01 04 05 07 09 10 11 15 16 17
  • 6. 6 It started with recruiting…
  • 7. 7
  • 8. 8
  • 9. 9
  • 10. 10
  • 11. 11
  • 12. 12
  • 13. 13
  • 14. 14
  • 16. 16
  • 17. 17 DATA SCIENTISTS 450K AVERAGE # COMPETITORS PER CHALLENGE 78 AVERAGE #SUBMISSIONS PER CHALLENGE 578 Topcoder’s Data Science Community
  • 18. 18 Algorithms and Analytics Sales Enablement Algorithms and Analytics Overview Products ● Algorithm Optimization ● Predictive Analytics ● Image and Pattern Recognition ● Objective Optimization Crowdsourcing Contests ● Marathon Match ● Banner Match ● Mini Marathon Match ● Data Science First-to- Finish (F2F) ● Data Science Sprint ● Data Visualization ● Data Science Ideation Objectively scores with an automated scoring function that feeds the Topcoder live leaderboard. Restricted to the following languages: Java, C++, C#.NET, VB.NET, or Python. Objectively scores with an automated scoring function that feeds the Topcoder live leaderboard. Competitors are allowed to use any language and libraries to write their solution. A shorter duration, more straight-forward version of the Marathon Match. Has the same structure and rules of a regular Marathon Match but a smaller prize purse for competitors. Objectively scores with an automated scoring function that feeds the Topcoder live leaderboard. Restricted to the following languages: Java, C++, C#.NET, VB.NET, or Python. A predetermined threshold is selected for the match, and once a competitor reaches the desired threshold score the match ends and the first place winner receives the one prize available. A series of smaller, more rapid data science challenges with a manual scoring function and no leaderboard. Competitors are allowed to use any language and libraries to write their solution. A hybrid project completed by a series of data science tasks to find the unique qualities or “highlights” in the data, followed by design challenges to visualize the data. The output of this kind of project is static screens. Looks to discover or learn new approaches, data, and ideas from the community.
  • 19. Predictive Toxicology - Environmental Safety Predictive Analytics Every day we come into contact with common pesticides. Which chemicals, in what dosage, can cause us harm? See how the EPA used crowdsourcing to deliver more accurate predictive analysis. Learn More
  • 20. 338 49 a Brand New Approach discovered, tested, and shared with their scientific community REGISTRANTS COMPETITORS 804 SUBMISSIONS 3 CHALLENGE DURATION WEEKS
  • 25. 25 Translate your mobile or web app idea into a structured design brief that includes a description of the design concept, use case scenarios, design guidelines, and a low-fidelity mockup of basic UI/UX elements. What’s Included Application design brief crowdsourced in the Topcoder Marketplace. A Topcoder copilot manages all logistics. What You Receive Written visual design brief for up to five screens of a single application. You pay only for the winning brief, not hours, and you own the final deliverables. Design Brief Plan the UI/UX design of your application Application Wireframe
  • 27. 27 Code Development packages make it easy to add the people and skills you need to round out your team, free valuable time, and accelerate development. Crowdsource specific phases of your development process or end-to- end application builds. What’s Included Code or task crowdsourced in the Topcoder Marketplace. A Topcoder copilot manages all logistics. What You Receive Ready-to-deploy code, interface, or service built to your specs. You pay only for the winning code, not hours, and you own the final deliverables. Code Development Enterprise-grade code for every stage of the dev life cycle Code Development
  • 28. POWERED BY Eight month challenge Multiple concurrent development work streams Third round of human testing
  • 29. 29 New HorizonsVA WORKFORCE DEVELOPMENT This is footer and you can change the content or delete if you wanted to
  • 30. 30 What is the TCVC? The TCVC is Topcoder’s Veterans Community ● The VA has 3 projects in motion at Topcoder via NASA - web, analytics & mobile ● Each VA project includes funding for the TCVC ● Topcoder teams with OperationCode, a veterans 503(c) to source members The TCVC is a place for US Veterans and Active Duty to learn and compete The TCVC is hosted as a private community within Topcoder’s platform ● Challenges run inside the TCVC are open only to US Veterans and Active Duty ● Challenges types run from “Hello World” style to complex multi-challenge series ● Challenges also come to the TCVC via the VA’s projects as well ● Topcoder challenges can be cross-posted to TCVC with different prizes On veterans.topcoder.com, Vets compete against each other, mentor each other, code solutions for paying customers, earn prizes and payments, and build their resumes by working on actual customer solutions.
  • 31. 31 What are the goals of the TCVC? VA Goals Cultivate and Prepare veterans for entry into the tech world ● Give veterans the opportunity to PROSUMERS ● Provide opportunity for veterans to demonstrate current, marketable tech skills Topcoder Goals Use the TCVC as an incubator for new community members ● Cultivate new members ● Train and grow new copilots who will eventually lead projects in our regular community ● Create a compelling story for Topcoder
  • 32. 32
  • 37. 37 Special thanks to . . . Andy LaMora Clinton Bonner Rashid SialJessie Ford Mike Morris Brendan Wright