SlideShare a Scribd company logo
LABELING IMAGES CARNEGIE MELLON UNIVERSITY LUIS VON AHN
9 BILLION MAN-HOURS OF  SOLITAIRE WERE PLAYED IN 2003 EMPIRE STATE BUILDING PANAMA CANAL 7 MILLION MAN-HOURS  (6.8 HOURS OF SOLITAIRE) 20 MILLION MAN-HOURS  (LESS THAN A DAY OF SOLITAIRE)
LABELING IMAGES  WITH WORDS STILL A COMPLETELY OPEN PROBLEM FACE MAN SUPER SEXY
IMAGE SEARCH  ON THE WEB USES FILENAMES AND HTML TEXT
ACCESSIBILITY LESS THAN 10% OF THE WEB IS ACCESSIBLE TO THE  VISUALLY IMPAIRED REASON: MOST IMAGES DON’T HAVE A CAPTION
DESIDERATA A METHOD THAT CAN LABEL ALL IMAGES ON THE WEB  FAST AND CHEAP
HUMAN LABELING THE ESP GAME  COULD LABEL ALL IMAGES ON THE WEB IN 30 DAYS! THE ESP GAME CLEVER
TWO-PLAYER ONLINE GAME PARTNERS DON’T KNOW EACH OTHER AND CAN’T COMMUNICATE OBJECT OF THE GAME:  TYPE THE SAME WORD THE ONLY THING IN COMMON IS  AN IMAGE THE  ESP GAME
PLAYER 1 PLAYER 2 THE  ESP GAME GUESSING:  CAR GUESSING:  BOY GUESSING:  CAR SUCCESS! YOU AGREE ON CAR SUCCESS! YOU AGREE ON CAR GUESSING:  KID GUESSING:  HAT
© 2004 Carnegie Mellon University, all rights reserved. Patent Pending.
MANY PEOPLE PLAY  OVER 20 HOURS A WEEK 3.2 MILLION LABELS  WITH 22,000 PLAYERS THE ESP GAME  IS FUN
LABELING THE  ENTIRE WEB INDIVIDUAL GAMES IN YAHOO! AND MSN AVERAGE OVER 10,000 PLAYERS AT A TIME 5000 PEOPLE  PLAYING SIMULTANEOUSLY CAN LABEL ALL IMAGES ON GOOGLE IN  30 DAYS !
A FEW  MILLION LABELS CAN IMPROVE IMAGE SEARCH CAN BE USED TO  IMPROVE COMPUTER VISION
WHAT ABOUT  CHEATING? IF A PAIR PLAYS TOO FAST, WE DON’T RECORD THE WORDS THEY AGREE ON
WHAT ABOUT  CHEATING? WE GIVE PLAYERS  TEST IMAGES  FOR WHICH WE KNOW ALL THE COMMON LABELS:  WE ONLY STORE A PLAYER’S GUESSES IF THEY SUCCESSFULLY LABEL THE TEST IMAGES
SEARCH  RESULTS
SAMPLE  LABELS SADDAM   MR. WILSON MAN FACE MOUSTACHE BUSH PRESIDENT DUMB YUCK YUCK COMING SOON: MEET YOUR  SOUL MATE  THROUGH THE GAME!
THE ESP GAME GIVES ITS PLAYERS A WEIRD AND BEAUTIFUL SENSE OF ANONYMOUS INTIMACY.  ON THE ONE HAND, YOU HAVE NO IDEA WHO YOUR PARTNER IS.   THE ESP GAME GIVES ITS PLAYERS A WEIRD AND BEAUTIFUL SENSE OF ANONYMOUS INTIMACY. ON THE ONE HAND, YOU HAVE NO IDEA WHO YOUR PARTNER IS.  ON THE OTHER HAND, THE TWO OF YOU ARE BRINGING YOUR MINDS TOGETHER IN A WAY THAT LOVERS WOULD ENVY.  ” THE ESP GAME GIVES ITS PLAYERS A WEIRD AND BEAUTIFUL SENSE OF  ANONYMOUS INTIMACY.   “
MAKES ME THINK “ ” IT’S FUN TRYNG TO GUESS WHAT OTHERS THINK. YOU HAVE TO STEP OUTSIDE OF YOURSELF TO MATCH “ ” IT’S FAST-PACED “ ” HELPS ME LEARN ENGLISH “ ”
BLOGS IN OTHER CROWNING ACHEIVEMENTS,  I HAVEN'T PLAYED IN AT LEAST A WEEK (PROBABLY LONGER), AND I AM STILL ON THE  TOP TEN LIST  OF PLAYERS ON THE ESP GAME. NUMBER EIGHT IF I REMEMBER CORRECTLY.  “ GOD, MY LIFE IS SAD. ”
THERE ARE MANY  FASCINATING  THINGS ABOUT THE ESP GAME:  HOW IT USES A GAME STRUCTURE TO BUILD AN  INDEX OF IMAGES  ON THE WEB WHAT  WORDS BECOME TABOO  FROM FREQUENT USE  HOW INTERESTING SOME OF THE  IMAGES PULLED RANDOMLY  FROM THE WEB ARE “ BUT BY FAR THE MOST INTRIGUING ASPECT OF THE GAME IS HOW OFTEN YOUR RANDOM, UNKNOWN PARTNER IS A COMPLETE IDIOT. ”
REUSING  HUMAN CYCLES
 
WWW. ESPGAME.ORG
MORE SPECIAL THANKS UDI MANBER MONI NAOR LENORE RAMM CHUCK ROSENBERG RONI ROSENFELD DAVID STORK LATANYA SWEENEY SHINGO UCHIHASHI MANUELA VELOSO SPECIAL THANKS TO ADITYA AKELLA JOSH BENALOH GUY BLELLOCH LENORE BLUM MANUEL BLUM ANDREW BORTZ SHUCHI CHAWLA LAURA DABBISH BILL GATES NICK HOPPER SCOTT HUDSON TAKEO KANADE STEVEN KATZ JOHN LANGFORD JITENDRA MALIK SPEAKER’S ENTERTAINMENT PROVIDED BY HOMESTARRUNNER.COM WARDROBE PROVIDED BY THE GAP THIS WORK HAS BEEN AND WILL BE PARTIALLY SUPPORTED BY MANUEL BLUM THE ALADDIN CENTER MICROSOFT RESEARCH
 
SEARCH FOR  FUNNY
THE CHOLESTEROL  POLICE
 

More Related Content

PDF
Search party SXSW
PPTX
2. research(2)
ODP
The Poker Game slideshow
DOCX
Ig1 task 2 analysis work sheet 1
PPTX
Research
PDF
Land On Stage - Final Slide
PPT
Batman arkham asylum
PPTX
3. research
Search party SXSW
2. research(2)
The Poker Game slideshow
Ig1 task 2 analysis work sheet 1
Research
Land On Stage - Final Slide
Batman arkham asylum
3. research

What's hot (19)

PPTX
Elements of storytelling for games
DOCX
Ig1 task 2 analysis work sheet 1
PPT
Secondary Research: Batman Arkham Asylum
PDF
Vladimir Tomko, OK.RU
PPTX
Most anticipated games for PS4 in Singapore (part 3)
PPTX
8. evaluation
PPTX
Unit 1 activity 3 game research
PDF
ゲーム理論BASIC 第22回補足 -シンプルポーカー-
PPTX
Magazine Research
PPTX
Video game release schedule 2016
PDF
Games To Take up On THE TELEPHONE
PPTX
3. research
PPT
Play with a Purpose
DOCX
4. proposal (1)
PPTX
Pubg
PDF
Ps2 God Of War
DOCX
MAX PAYNE GAME SCRIPT
DOCX
Ig1 task 2 analysis work sheet 4
PDF
Most playedgames
Elements of storytelling for games
Ig1 task 2 analysis work sheet 1
Secondary Research: Batman Arkham Asylum
Vladimir Tomko, OK.RU
Most anticipated games for PS4 in Singapore (part 3)
8. evaluation
Unit 1 activity 3 game research
ゲーム理論BASIC 第22回補足 -シンプルポーカー-
Magazine Research
Video game release schedule 2016
Games To Take up On THE TELEPHONE
3. research
Play with a Purpose
4. proposal (1)
Pubg
Ps2 God Of War
MAX PAYNE GAME SCRIPT
Ig1 task 2 analysis work sheet 4
Most playedgames
Ad

Viewers also liked (10)

PPSX
Sexy women
PDF
Zoo weekly thailand 30 september 2013
PPT
Sexy girls
PDF
PowerPoint Porn: 100 Sexy Slides [Safe For Work]
PDF
BERING Prospectus_210x297_082012_04_LowRes
TXT
PPSX
Sol ejercicio3
PPS
Calendario sexy
PPTX
Going Digital
PDF
informe-la-justicia-en-el-peru-cinco-grandes-problemas
Sexy women
Zoo weekly thailand 30 september 2013
Sexy girls
PowerPoint Porn: 100 Sexy Slides [Safe For Work]
BERING Prospectus_210x297_082012_04_LowRes
Sol ejercicio3
Calendario sexy
Going Digital
informe-la-justicia-en-el-peru-cinco-grandes-problemas
Ad

More from sam ran (20)

PPT
michael jackson 1
PPT
michael jckson
PPT
Overview of Business Result Reporting
PPT
Bill Gates and Valentine day
PPT
Manan - Isha
PPT
Beautiful Picture
PPT
ISHA
PPT
web 2.0
PPT
web 2.0
PPT
web 2.0
PPT
web 2.0
PPT
Web 2.0
PPT
Shopping and valentine day
PPT
Happy Valentines Day
PPT
50 ways to a successful marraige
PPT
Valentine Day
PPT
Beautiful Picture
PPT
Startup and Funding - Idea to Billionaire
PPT
Intellectual Property and Trade Mark
PPS
Marraige
michael jackson 1
michael jckson
Overview of Business Result Reporting
Bill Gates and Valentine day
Manan - Isha
Beautiful Picture
ISHA
web 2.0
web 2.0
web 2.0
web 2.0
Web 2.0
Shopping and valentine day
Happy Valentines Day
50 ways to a successful marraige
Valentine Day
Beautiful Picture
Startup and Funding - Idea to Billionaire
Intellectual Property and Trade Mark
Marraige

Recently uploaded (20)

PPTX
Programs and apps: productivity, graphics, security and other tools
PDF
project resource management chapter-09.pdf
PPTX
observCloud-Native Containerability and monitoring.pptx
PPTX
Final SEM Unit 1 for mit wpu at pune .pptx
PDF
Hybrid model detection and classification of lung cancer
PDF
Zenith AI: Advanced Artificial Intelligence
PDF
WOOl fibre morphology and structure.pdf for textiles
PDF
Microsoft Solutions Partner Drive Digital Transformation with D365.pdf
PPTX
Group 1 Presentation -Planning and Decision Making .pptx
PDF
Transform Your ITIL® 4 & ITSM Strategy with AI in 2025.pdf
PDF
Web App vs Mobile App What Should You Build First.pdf
PDF
TrustArc Webinar - Click, Consent, Trust: Winning the Privacy Game
PDF
Architecture types and enterprise applications.pdf
PDF
Video forgery: An extensive analysis of inter-and intra-frame manipulation al...
PPTX
1. Introduction to Computer Programming.pptx
PPTX
Modernising the Digital Integration Hub
PPTX
Tartificialntelligence_presentation.pptx
PDF
DP Operators-handbook-extract for the Mautical Institute
PDF
DASA ADMISSION 2024_FirstRound_FirstRank_LastRank.pdf
PDF
Developing a website for English-speaking practice to English as a foreign la...
Programs and apps: productivity, graphics, security and other tools
project resource management chapter-09.pdf
observCloud-Native Containerability and monitoring.pptx
Final SEM Unit 1 for mit wpu at pune .pptx
Hybrid model detection and classification of lung cancer
Zenith AI: Advanced Artificial Intelligence
WOOl fibre morphology and structure.pdf for textiles
Microsoft Solutions Partner Drive Digital Transformation with D365.pdf
Group 1 Presentation -Planning and Decision Making .pptx
Transform Your ITIL® 4 & ITSM Strategy with AI in 2025.pdf
Web App vs Mobile App What Should You Build First.pdf
TrustArc Webinar - Click, Consent, Trust: Winning the Privacy Game
Architecture types and enterprise applications.pdf
Video forgery: An extensive analysis of inter-and intra-frame manipulation al...
1. Introduction to Computer Programming.pptx
Modernising the Digital Integration Hub
Tartificialntelligence_presentation.pptx
DP Operators-handbook-extract for the Mautical Institute
DASA ADMISSION 2024_FirstRound_FirstRank_LastRank.pdf
Developing a website for English-speaking practice to English as a foreign la...

Sexy Slideshare

  • 1. LABELING IMAGES CARNEGIE MELLON UNIVERSITY LUIS VON AHN
  • 2. 9 BILLION MAN-HOURS OF SOLITAIRE WERE PLAYED IN 2003 EMPIRE STATE BUILDING PANAMA CANAL 7 MILLION MAN-HOURS (6.8 HOURS OF SOLITAIRE) 20 MILLION MAN-HOURS (LESS THAN A DAY OF SOLITAIRE)
  • 3. LABELING IMAGES WITH WORDS STILL A COMPLETELY OPEN PROBLEM FACE MAN SUPER SEXY
  • 4. IMAGE SEARCH ON THE WEB USES FILENAMES AND HTML TEXT
  • 5. ACCESSIBILITY LESS THAN 10% OF THE WEB IS ACCESSIBLE TO THE VISUALLY IMPAIRED REASON: MOST IMAGES DON’T HAVE A CAPTION
  • 6. DESIDERATA A METHOD THAT CAN LABEL ALL IMAGES ON THE WEB FAST AND CHEAP
  • 7. HUMAN LABELING THE ESP GAME COULD LABEL ALL IMAGES ON THE WEB IN 30 DAYS! THE ESP GAME CLEVER
  • 8. TWO-PLAYER ONLINE GAME PARTNERS DON’T KNOW EACH OTHER AND CAN’T COMMUNICATE OBJECT OF THE GAME: TYPE THE SAME WORD THE ONLY THING IN COMMON IS AN IMAGE THE ESP GAME
  • 9. PLAYER 1 PLAYER 2 THE ESP GAME GUESSING: CAR GUESSING: BOY GUESSING: CAR SUCCESS! YOU AGREE ON CAR SUCCESS! YOU AGREE ON CAR GUESSING: KID GUESSING: HAT
  • 10. © 2004 Carnegie Mellon University, all rights reserved. Patent Pending.
  • 11. MANY PEOPLE PLAY OVER 20 HOURS A WEEK 3.2 MILLION LABELS WITH 22,000 PLAYERS THE ESP GAME IS FUN
  • 12. LABELING THE ENTIRE WEB INDIVIDUAL GAMES IN YAHOO! AND MSN AVERAGE OVER 10,000 PLAYERS AT A TIME 5000 PEOPLE PLAYING SIMULTANEOUSLY CAN LABEL ALL IMAGES ON GOOGLE IN 30 DAYS !
  • 13. A FEW MILLION LABELS CAN IMPROVE IMAGE SEARCH CAN BE USED TO IMPROVE COMPUTER VISION
  • 14. WHAT ABOUT CHEATING? IF A PAIR PLAYS TOO FAST, WE DON’T RECORD THE WORDS THEY AGREE ON
  • 15. WHAT ABOUT CHEATING? WE GIVE PLAYERS TEST IMAGES FOR WHICH WE KNOW ALL THE COMMON LABELS: WE ONLY STORE A PLAYER’S GUESSES IF THEY SUCCESSFULLY LABEL THE TEST IMAGES
  • 17. SAMPLE LABELS SADDAM MR. WILSON MAN FACE MOUSTACHE BUSH PRESIDENT DUMB YUCK YUCK COMING SOON: MEET YOUR SOUL MATE THROUGH THE GAME!
  • 18. THE ESP GAME GIVES ITS PLAYERS A WEIRD AND BEAUTIFUL SENSE OF ANONYMOUS INTIMACY. ON THE ONE HAND, YOU HAVE NO IDEA WHO YOUR PARTNER IS. THE ESP GAME GIVES ITS PLAYERS A WEIRD AND BEAUTIFUL SENSE OF ANONYMOUS INTIMACY. ON THE ONE HAND, YOU HAVE NO IDEA WHO YOUR PARTNER IS. ON THE OTHER HAND, THE TWO OF YOU ARE BRINGING YOUR MINDS TOGETHER IN A WAY THAT LOVERS WOULD ENVY. ” THE ESP GAME GIVES ITS PLAYERS A WEIRD AND BEAUTIFUL SENSE OF ANONYMOUS INTIMACY. “
  • 19. MAKES ME THINK “ ” IT’S FUN TRYNG TO GUESS WHAT OTHERS THINK. YOU HAVE TO STEP OUTSIDE OF YOURSELF TO MATCH “ ” IT’S FAST-PACED “ ” HELPS ME LEARN ENGLISH “ ”
  • 20. BLOGS IN OTHER CROWNING ACHEIVEMENTS, I HAVEN'T PLAYED IN AT LEAST A WEEK (PROBABLY LONGER), AND I AM STILL ON THE TOP TEN LIST OF PLAYERS ON THE ESP GAME. NUMBER EIGHT IF I REMEMBER CORRECTLY. “ GOD, MY LIFE IS SAD. ”
  • 21. THERE ARE MANY FASCINATING THINGS ABOUT THE ESP GAME: HOW IT USES A GAME STRUCTURE TO BUILD AN INDEX OF IMAGES ON THE WEB WHAT WORDS BECOME TABOO FROM FREQUENT USE HOW INTERESTING SOME OF THE IMAGES PULLED RANDOMLY FROM THE WEB ARE “ BUT BY FAR THE MOST INTRIGUING ASPECT OF THE GAME IS HOW OFTEN YOUR RANDOM, UNKNOWN PARTNER IS A COMPLETE IDIOT. ”
  • 22. REUSING HUMAN CYCLES
  • 23.  
  • 25. MORE SPECIAL THANKS UDI MANBER MONI NAOR LENORE RAMM CHUCK ROSENBERG RONI ROSENFELD DAVID STORK LATANYA SWEENEY SHINGO UCHIHASHI MANUELA VELOSO SPECIAL THANKS TO ADITYA AKELLA JOSH BENALOH GUY BLELLOCH LENORE BLUM MANUEL BLUM ANDREW BORTZ SHUCHI CHAWLA LAURA DABBISH BILL GATES NICK HOPPER SCOTT HUDSON TAKEO KANADE STEVEN KATZ JOHN LANGFORD JITENDRA MALIK SPEAKER’S ENTERTAINMENT PROVIDED BY HOMESTARRUNNER.COM WARDROBE PROVIDED BY THE GAP THIS WORK HAS BEEN AND WILL BE PARTIALLY SUPPORTED BY MANUEL BLUM THE ALADDIN CENTER MICROSOFT RESEARCH
  • 26.  
  • 27. SEARCH FOR FUNNY
  • 28. THE CHOLESTEROL POLICE
  • 29.