Introduction to Amazon Mechanical Turk
- 5. Fine print
- 7. Resources TopicsOverview
- 10. Fine print
- 12. Resources What’s it?Amazon Mechanical Turk is a crowdsourcing Internet marketplace that enables computer programmers (Requesters) to coordinate the use of human intelligence to perform tasks which computers are unable to do.
- 19. faster theory/experiment cycle* Mason, W. and Siddarth, S. 2009. “Conducting Behavioral Research on Amazon’s Mechanical Turk.” Yahoo! Research.
- 21. Designing HITs
- 23. Fine print
- 25. Resources Human Intelligence Tasks (HITs)Requesters can specify: task
- 26. keywords
- 27. expiration date
- 28. reward
- 29. time allotted
- 31. location
- 32. approval rating Example HITs: “Identify forward-facing pictures of dogs”“Find and enter a business address”“How attractive are these items?”“Choose the best category for this product”“Read a set of Tweets and decide if they describe an event”
- 33. Designing HITsOptions:Use template or code within Mechanical Turk requires basic HTML Link to external surveyneed some kind of verification input on Mechanical Turk side after Workers complete external survey Designing HITsOptions:Use template or code within Mechanical Turk requires basic HTML Link to external surveyneed some kind of verification input on Mechanical Turk side after workers complete external survey TopicsOverview
- 34. Designing HITs
- 36. Fine print
- 38. Resources Test your HITsUse the Mechanical Turk Requester sandbox view your published your HIT before releasing it to Workers
- 39. check for functionality in all browsers Help Workers work efficientlyDon’t make them scroll and click a lotSet pages to open in new windows
- 40. Manage Worker accuracyReject bad workReward good work with bonusesFilter Workers by qualificationsdesign a test for specific knowledge
- 41. location
- 42. approvals (percentage or number)
- 43. adult content TopicsOverview
- 44. Designing HITs
- 46. Fine print
- 48. Resources Payment information HITs must be prepaid to a Mechanical Turk accountAmazon collects 10% on top of what you pay to Workers
- 49. Amazon collects 10% of any bonuses you grant
- 50. Payment to Workers can be in money or Amazon creditYou’re ultimately only charged for approved HITs, but you’re liable for the amount equal to whatever 100% approved HITs would cost.You’ll have to deal with tax stuff if Workers do enough work for you to meet the IRS threshold for taxable income.Click for prepaid terms and tax information.
- 52. require and check for Adult Content Qualification in order to complete AND preview HITCollect personally identifiable informationAsk for this:e-mail address
- 53. registration at another site
- 54. downloading of softwareClick to view complete policies.
- 56. Designing HITs
- 58. Payments
- 60. Resources Accurate replications of classics Tversky and Kahneman:framing - “Asian disease problem”
- 61. anchoring – “African countries problem”
- 62. conjunction fallacy – “Linda problem”Loewenstein and Prelec:fancy French dinnerKirby and Marakovichyperbolic discountingMore at: http://experimentalturk.wordpress.com/
- 63. ExtensionsEasy cross-cultural researchtasks can be confined to workers living in specific countries
- 64. lots of Indians
- 65. Eriksson and Simpson (2010) TopicsOverview
- 66. Designing HITs
- 68. Payments
- 70. Resources ReferencesAmazon FAQs and tutorialsMechanical Turk FAQ
- 72. Participation AgreementLiterature on Mechanical TurkMason, W. and Siddarth, S. 2009. “Conducting Behavioral Research on Amazon’s Mechanical Turk.” Yahoo! Research.Research using Mechanical TurkSee: http://experimentalturk.wordpress.com/