SlideShare a Scribd company logo
Overview & Workshop Intro
2
Before we start
][
2
amazon
web services
Daniel Gray, Principal Business Development
Amazon Mechanical Turk, Seattle, WA
djgray@amazon.com
T: 310-800-4832
3
Who is Amazon.com? ][
3
44
][Amazon.com’s three businesses
Consumer
Business
Tens of millions of
active customer
accounts
Eight countries:
US, UK, Germany,
Japan, France,
Canada, China, Italy
Seller
Business
Sell on Amazon
websites
Use Amazon
technology for
your own retail
website
Leverage Amazon’s
massive fulfillment
center network
IT Infrastructure
Business
Cloud computing
infrastructure for
hosting web-scale
solutions
Hundreds of
thousands of
registered
customers in over
190 countries
Marketplace
for Work
amazon
web services
5
Our plan for today
][
5
Section I: Overview
1) Level-setting on ‘crowdsourcing’
2) What is Amazon Mechanical Turk
Section II: Workshop Intro
1) Best Practices
2) Workshop
66
More
Technical
More
Business
What is Amazon
Mechanical Turk?
Workshop IntroCrowdsourcing
Best Practices
How technical is it?[ ]
77
1. Crowdsourcing
More
Technical
More
Business Crowdsourcing
What is Amazon
Mechanical Turk?
Workshop Intro
Best Practices
88
Daily Crowdsource
99
Microtasking is…distributed problem solving.
10
Common Use Cases[ ]
10
Data Management
_ Verification
_ Entry & Collection
_ De-dupe
_ Algorithm Training
Annotation
_ Tagging
_ Classification
Content Management
_ Moderation (Photos, Content)
_ Transcription
_ Writing/Editing
Analysis/Research
_ Sentiment
_ Relevance
_ Online Research
1111
2. Amazon Mechanical Turk
More
Technical
More
Business Crowdsourcing
What is Amazon
Mechanical Turk?
Workshop Intro
Best Practices
1212
[ Mechanical
Turk Ecosystem]
• Manual via WebUI
• Automate via API
How to Connect
• Self-serve
• Assisted
i. Partner network
ii. Workshops
How to Engage
• Documentation
• Best Practices
• Developer’s Resources
• Requestor UI Guide (account setup)
• Sandbox
How to get Support
1313
$
Variable expense
Transform fixed human
capital costs into variable
expense
$
Economies of scale
A lower variable expense
than companies can
achieve themselves
Speed and agility
On-demand access to
scalable workforce &
higher productivity
Focus on business
Redeploy valuable
human resources to core
business
Global Reach
500K Workers across
190 countriesScalability
No need to guess
capacity requirements =
avoid waste
Fast Growth
Predictable peaks
On and Off
Variable peaks
Results’ Accuracy
Drive fast, accurate work
results at competitive
prices without long-term
contracts or minimum
commitments

1414
3. Best Practices
More
Technical
More
Business Crowdsourcing
What is Amazon
Mechanical Turk?
Workshop Intro
Best Practices
1515
[ Best Practices ]
1. Worker Accuracy
• Blocking / rejecting
• Qualification
• Curating the workforce
• Masters
• Plurality
• Known answers
• Adjudication
2. Worker ergonomics
3. Marketplace reputation
4. Payment & bonusing
• Pay-work equalization
• Timing
• Bundling
5. Optimization
1616
4. Workshop Intro
More
Technical
More
Business Crowdsourcing
What is Amazon
Mechanical Turk?
Workshop Intro
Best Practices
1717
[ Workshop Intro]
• Strategy
• Business issues & objectives: Establish priorities & goals for the core metrics (input & output) for
measurement and optimization.
• Planning & Design
• Establish blueprints for pilot, with scalability in mind, including:
• Visioning – Assessment of current workflow(s) and success metrics; disintegrating the stages into
Mechanical Turk workflow steps and ensuring they’re optimized for microtasking; integration with and
development of Mechanical Turk workflow(s).
• Scoping & Planning – API-connectivity, address HIT design, Worker instructions and layout,
adjudication strategies, Known Answers, Masters, workflow and best practices implementation.
• Implementation & Launch
• Implement the best practices and designs established in the planning phase; as Advisors, we help to
refine implementation according to the design.
• Continuous Improvement
• Adjust & optimize the prioritized metrics based on BI reporting; observing results, measuring against
goals, root causing gaps, and adjusting:
• Cost efficiency
• Results’ accuracy
• Productivity
18
Thank you
][
18
Daniel Gray, Principal Business Development
Amazon Mechanical Turk, Seattle, WA
djgray@amazon.com
T: 310-800-4832

More Related Content

PPTX
Amazon Mechanical Turk Overview
PPTX
Introduction to Amazon Mechanical Turk
PDF
AWS User Group Berlin - Introduction To Amazon Mechanical Turk
PPTX
AWS Mechanical Turk Office Hours - Jan 2011
PPTX
Social Media Use Cases - Webinar
PPTX
Manage your Workforce - Fall 2010 Requester Product Update
PPTX
SEO + MTurk
PPS
Master Sales Demo
Amazon Mechanical Turk Overview
Introduction to Amazon Mechanical Turk
AWS User Group Berlin - Introduction To Amazon Mechanical Turk
AWS Mechanical Turk Office Hours - Jan 2011
Social Media Use Cases - Webinar
Manage your Workforce - Fall 2010 Requester Product Update
SEO + MTurk
Master Sales Demo

Viewers also liked (12)

PPTX
Amazon Mechanical Turk: Transcription & Captioning for On-line Education & Ac...
PPTX
Seni Budaya-Kebudayaan dari Jawa Timur "REOG PONOROGO"
PDF
E diary - Erasmus Plus KA2 - Italy 2016
PDF
A Study on TQM Development, Performance and Sustenance in Service Industries ...
PPTX
Seni Budaya-Kesenian di jawa barat "Angklung"
PPTX
PPTX
Fisika "Atom, Molekul, dan Ion"
PPTX
Akidah akhlaq - adab berpakaian
ODP
Kepentingan sistem respirasi yang sihat
PPTX
Heat pipes
PDF
Mari membuat-standard-operating-procedures
Amazon Mechanical Turk: Transcription & Captioning for On-line Education & Ac...
Seni Budaya-Kebudayaan dari Jawa Timur "REOG PONOROGO"
E diary - Erasmus Plus KA2 - Italy 2016
A Study on TQM Development, Performance and Sustenance in Service Industries ...
Seni Budaya-Kesenian di jawa barat "Angklung"
Fisika "Atom, Molekul, dan Ion"
Akidah akhlaq - adab berpakaian
Kepentingan sistem respirasi yang sihat
Heat pipes
Mari membuat-standard-operating-procedures
Ad

Similar to Amazon Mechanical Turk: Intro & Overview (20)

PPTX
Mechanical turk increase image sales webinar 040213
PPTX
Mechanical Turk: Increase Image Sales (April 2013 Webinar)
PDF
Amazon mechanical turk intro to govt partners v2
PPTX
Leveraging Mechanical Turk for SEO
PDF
Hands On: Amazon Mechanical Turk - M. Acosta - ESWC SS 2014
PPT
Artificial Artificial Intelligence: Using Amazon Mechanical Turk and .NET to ...
PDF
Amazon mechanical turk intro to bpo's v3
PPTX
AWS featuring Mechanical Turk_for Automotive_2014
PDF
AMT: Requester UI
PDF
AMT: Getting Started
PDF
Mechanical Turk Under the Hood
PPTX
AWS Sydney Summit 2019 Re:Cap
PDF
Beyond Mechanical Turk: An Analysis of Paid Crowd Work Platforms
PDF
Starting your AI/ML project right (May 2020)
PPTX
Drive Digital Transformation Using Machine Learning
PDF
SKL208 - Turbocharge your Business with AI and Machine Learning - Tel Aviv Su...
PDF
Mechanical Turk Demystified: Best practices for sourcing and scaling quality ...
PPT
Best Practices for Sentiment Analysis Webinar
PDF
Why User Behavior Insights? KMWorld Enterprise Search & Discovery 2024
PPT
Cloud to crowd talk to COST Virtual work Bucharest 2014
Mechanical turk increase image sales webinar 040213
Mechanical Turk: Increase Image Sales (April 2013 Webinar)
Amazon mechanical turk intro to govt partners v2
Leveraging Mechanical Turk for SEO
Hands On: Amazon Mechanical Turk - M. Acosta - ESWC SS 2014
Artificial Artificial Intelligence: Using Amazon Mechanical Turk and .NET to ...
Amazon mechanical turk intro to bpo's v3
AWS featuring Mechanical Turk_for Automotive_2014
AMT: Requester UI
AMT: Getting Started
Mechanical Turk Under the Hood
AWS Sydney Summit 2019 Re:Cap
Beyond Mechanical Turk: An Analysis of Paid Crowd Work Platforms
Starting your AI/ML project right (May 2020)
Drive Digital Transformation Using Machine Learning
SKL208 - Turbocharge your Business with AI and Machine Learning - Tel Aviv Su...
Mechanical Turk Demystified: Best practices for sourcing and scaling quality ...
Best Practices for Sentiment Analysis Webinar
Why User Behavior Insights? KMWorld Enterprise Search & Discovery 2024
Cloud to crowd talk to COST Virtual work Bucharest 2014
Ad

Recently uploaded (20)

DOCX
Business Management - unit 1 and 2
PDF
Elevate Cleaning Efficiency Using Tallfly Hair Remover Roller Factory Expertise
PPTX
Amazon (Business Studies) management studies
PDF
pdfcoffee.com-opt-b1plus-sb-answers.pdfvi
PDF
Digital Marketing & E-commerce Certificate Glossary.pdf.................
PPTX
Dragon_Fruit_Cultivation_in Nepal ppt.pptx
PDF
DOC-20250806-WA0002._20250806_112011_0000.pdf
DOCX
Euro SEO Services 1st 3 General Updates.docx
PDF
MSPs in 10 Words - Created by US MSP Network
PDF
COST SHEET- Tender and Quotation unit 2.pdf
PPTX
ICG2025_ICG 6th steering committee 30-8-24.pptx
PPTX
svnfcksanfskjcsnvvjknsnvsdscnsncxasxa saccacxsax
PDF
Ôn tập tiếng anh trong kinh doanh nâng cao
PDF
Stem Cell Market Report | Trends, Growth & Forecast 2025-2034
PPT
Chapter four Project-Preparation material
PDF
SIMNET Inc – 2023’s Most Trusted IT Services & Solution Provider
PDF
Outsourced Audit & Assurance in USA Why Globus Finanza is Your Trusted Choice
PDF
A Brief Introduction About Julia Allison
PPTX
Principles of Marketing, Industrial, Consumers,
PDF
Unit 1 Cost Accounting - Cost sheet
Business Management - unit 1 and 2
Elevate Cleaning Efficiency Using Tallfly Hair Remover Roller Factory Expertise
Amazon (Business Studies) management studies
pdfcoffee.com-opt-b1plus-sb-answers.pdfvi
Digital Marketing & E-commerce Certificate Glossary.pdf.................
Dragon_Fruit_Cultivation_in Nepal ppt.pptx
DOC-20250806-WA0002._20250806_112011_0000.pdf
Euro SEO Services 1st 3 General Updates.docx
MSPs in 10 Words - Created by US MSP Network
COST SHEET- Tender and Quotation unit 2.pdf
ICG2025_ICG 6th steering committee 30-8-24.pptx
svnfcksanfskjcsnvvjknsnvsdscnsncxasxa saccacxsax
Ôn tập tiếng anh trong kinh doanh nâng cao
Stem Cell Market Report | Trends, Growth & Forecast 2025-2034
Chapter four Project-Preparation material
SIMNET Inc – 2023’s Most Trusted IT Services & Solution Provider
Outsourced Audit & Assurance in USA Why Globus Finanza is Your Trusted Choice
A Brief Introduction About Julia Allison
Principles of Marketing, Industrial, Consumers,
Unit 1 Cost Accounting - Cost sheet

Amazon Mechanical Turk: Intro & Overview

  • 2. 2 Before we start ][ 2 amazon web services Daniel Gray, Principal Business Development Amazon Mechanical Turk, Seattle, WA djgray@amazon.com T: 310-800-4832
  • 4. 44 ][Amazon.com’s three businesses Consumer Business Tens of millions of active customer accounts Eight countries: US, UK, Germany, Japan, France, Canada, China, Italy Seller Business Sell on Amazon websites Use Amazon technology for your own retail website Leverage Amazon’s massive fulfillment center network IT Infrastructure Business Cloud computing infrastructure for hosting web-scale solutions Hundreds of thousands of registered customers in over 190 countries Marketplace for Work amazon web services
  • 5. 5 Our plan for today ][ 5 Section I: Overview 1) Level-setting on ‘crowdsourcing’ 2) What is Amazon Mechanical Turk Section II: Workshop Intro 1) Best Practices 2) Workshop
  • 6. 66 More Technical More Business What is Amazon Mechanical Turk? Workshop IntroCrowdsourcing Best Practices How technical is it?[ ]
  • 7. 77 1. Crowdsourcing More Technical More Business Crowdsourcing What is Amazon Mechanical Turk? Workshop Intro Best Practices
  • 10. 10 Common Use Cases[ ] 10 Data Management _ Verification _ Entry & Collection _ De-dupe _ Algorithm Training Annotation _ Tagging _ Classification Content Management _ Moderation (Photos, Content) _ Transcription _ Writing/Editing Analysis/Research _ Sentiment _ Relevance _ Online Research
  • 11. 1111 2. Amazon Mechanical Turk More Technical More Business Crowdsourcing What is Amazon Mechanical Turk? Workshop Intro Best Practices
  • 12. 1212 [ Mechanical Turk Ecosystem] • Manual via WebUI • Automate via API How to Connect • Self-serve • Assisted i. Partner network ii. Workshops How to Engage • Documentation • Best Practices • Developer’s Resources • Requestor UI Guide (account setup) • Sandbox How to get Support
  • 13. 1313 $ Variable expense Transform fixed human capital costs into variable expense $ Economies of scale A lower variable expense than companies can achieve themselves Speed and agility On-demand access to scalable workforce & higher productivity Focus on business Redeploy valuable human resources to core business Global Reach 500K Workers across 190 countriesScalability No need to guess capacity requirements = avoid waste Fast Growth Predictable peaks On and Off Variable peaks Results’ Accuracy Drive fast, accurate work results at competitive prices without long-term contracts or minimum commitments 
  • 14. 1414 3. Best Practices More Technical More Business Crowdsourcing What is Amazon Mechanical Turk? Workshop Intro Best Practices
  • 15. 1515 [ Best Practices ] 1. Worker Accuracy • Blocking / rejecting • Qualification • Curating the workforce • Masters • Plurality • Known answers • Adjudication 2. Worker ergonomics 3. Marketplace reputation 4. Payment & bonusing • Pay-work equalization • Timing • Bundling 5. Optimization
  • 16. 1616 4. Workshop Intro More Technical More Business Crowdsourcing What is Amazon Mechanical Turk? Workshop Intro Best Practices
  • 17. 1717 [ Workshop Intro] • Strategy • Business issues & objectives: Establish priorities & goals for the core metrics (input & output) for measurement and optimization. • Planning & Design • Establish blueprints for pilot, with scalability in mind, including: • Visioning – Assessment of current workflow(s) and success metrics; disintegrating the stages into Mechanical Turk workflow steps and ensuring they’re optimized for microtasking; integration with and development of Mechanical Turk workflow(s). • Scoping & Planning – API-connectivity, address HIT design, Worker instructions and layout, adjudication strategies, Known Answers, Masters, workflow and best practices implementation. • Implementation & Launch • Implement the best practices and designs established in the planning phase; as Advisors, we help to refine implementation according to the design. • Continuous Improvement • Adjust & optimize the prioritized metrics based on BI reporting; observing results, measuring against goals, root causing gaps, and adjusting: • Cost efficiency • Results’ accuracy • Productivity
  • 18. 18 Thank you ][ 18 Daniel Gray, Principal Business Development Amazon Mechanical Turk, Seattle, WA djgray@amazon.com T: 310-800-4832

Editor's Notes

  • #5: To understand Amazon Mechanical Turk, firstit’s helpful to understand Amazon is a company of marketplaces - we have three macro and distinct businesses:1. our Consumer/Retail business (with tens of millions of active customers around the world)2. our Seller business (where third parties sell on Amazon websites, using Amazon technology for their own retail websites, leveraging Amazon’s massive fulfillment center network)3. our IT Infrastructure business (where cloud computing infrastructure for hosting web-scale solutions serves hundreds of thousands of registered customers around the world).Amazon Mechanical Turk is a marketplace for work, and is part of Amazon Web Services (AWS). Like our cloud computing infrastructure which offers computing power, Amazon Mechanical Turk is the infrastructure to access a global, scalable, on-demand workforce for human power.
  • #11: Data management: This includes data or algorithm training…With Mechanical Turk you can quickly develop large sets of training data to train your algorithmsAnalysis/research: perform sentiment analysis, monitor customer feedbackContent management: moderate your collections for appropriateness/accuracy to market, improve search engine-level discoverability by adding descriptions to your collections in addition to tags/keywordsAnnotation: tag collections with keywords, improving classification/categorization. Again, discoverability is critical to image sales - Mechanical Turk Workers can quickly and accurately add attribute descriptions (or tags) to all types of content so that it can be easily indexed and searched. Example Tasks: Generate keywords for an image, advertisement, or website. Identify objects found in an image for easier searching and advertising targeting. Timestamp and tag sections of a video or audio clip based on content.
  • #14: _ Mechanical Turk is a marketplace for work – think of it as the infrastructure layer upon which Partners like Tagasauris offer a technology solution to Requesters (those looking for workers) to access a scalable, global, on-demand Workforce _ Mechanical Turk provides access to the tool required to build workflows and control quality_ Mechanical Turk provides the payment administration (though Amazon Payments)_ …and lastly, it can handle millions of tasks, with web or API-connectivity options_The process includes breaking down a project into tasks, publishing the tasks via HIT’s (human intelligence tasks). Depending on the qualifications set for your tasks, either a defined or undefined group of Workers accept the assignments, submit their results for approval, and the process continues.What are the Key advantages that deliver business benefits: 1. VARIABLE EXPENSE: Lower Costs_ Transform fixed [human capital] costs into variable costs_ Offers a pay-as-you-go model2. GAINECONOMIES OF SCALE: as your collections continue to grow, scale more efficiently3. SPEED AND AGILITY_ On-demand (vs. on the bench) access to a scalable workforce_ Results-focus (vs. time focus) emphasizes productivity; Workers can work in parallel, around the globe, around the clock4. FOCUS ON YOUR BUSINESSWith increased efficiency, productivity, and lower costs—you have more time to focus on critical business objectives, increasing the pace of your innovation._ Experiment Often_ Fail quickly at a low cost= More Innovation5. GLOBAL REACH: 500k Workers across 190 countries6. ACCURACY: drive fast, accurate work results at competitive prices without long term contracts or minimum commitments7. SCALABILITY: avoid waste_ Burst up and down according to the capacity you need, when you need it_ 100% Efficiency_ Independent of your workload type, maintain the flexibility your business demandsThis is the ‘business’ buzz around Mechanical Turk  It’s these business benefits that make Mechanical Turk a strategic initiative for companies like Comscore, Twitter, Linkedin…and the difference between the “disruptors” vs. the “disrupted.”