Crowdsourced Object Segmentation with a Game
1 like1,726 views
The document discusses crowdsourced object segmentation through a game, outlining the motivation, experimental setups, results, and ongoing work. It highlights the use of a realistic simulator to analyze object segmentation quality based on user interactions and presents findings from experiments using a large image dataset. Additionally, it notes a successful data collection effort with over 1500 games played by users, contributing to the project’s objectives.
Crowdsourced Object Segmentation with a Game
- 1. Crowdsourced object segmentation with a game Vincent Charvillat Axel Carlier Amaia Salvador Aguilera Xavier Giró i Nieto Oge Marques
- 2. Outline • Motivation • Object Segmentation • Experiments • Results • Conclusions • Ongoing work 2
- 5. Semi-Supervised object segmentation Rough segmentation • B. C. Russell, A. Torralba, K. P. Murphy, and W. T. Freeman. Labelme: A database and web-based tool for image annotation. IJCV, 2008 5
- 6. Semi-Supervised object segmentation • • P. Arbelaez and L. Cohen. Constrained image segmentation from hierarchical boundaries. In CVPR'08, 2008. 2) K. McGuinness and N. E. O'Connor. A comparative evaluation of interactive segmentation algorithms. 6
- 7. Semi-Supervised object segmentation Boring task for users! 7
- 8. 8
- 9. Games with a purpose • J. Steggink and C. Snoek. Adding semantics to image-region annotations with the name-itgame. Multimedia Systems, 2011. • L. von Ahn, R. Liu, and M. Blum. Peekaboom: a game for locating objects in images. In CHI'06, 2006. 9
- 10. Ask’nSeek A. Carlier, O. Marques, and V. Charvillat. Ask'nseek: A new game for object detection and labeling. In ECCV'12 Workshops 2012. 10
- 11. Motivation ? 11
- 12. Outline • Motivation • Object Segmentation • Experiments • Results • Conclusions • Next steps 12
- 13. Constrained parametric min-cuts for automatic object segmentation (CPMC) J. Carreira and C. Sminchisescu. Constrained parametric min-cuts for automatic object segmentation. In CVPR'10, 2010. 13
- 14. Constrained parametric min-cuts for automatic object segmentation 14
- 16. Outline • Motivation • Object Segmentation • Experiments • Results • Conclusions • Ongoing Work 16
- 17. Experiments How many clicks do we need to achieve a certain quality in the segmentation? Test the algorithm for a large image dataset 17
- 18. Pascal VOC2010 1928 images divided in: Train (964) Validation (964) 18
- 20. Simulator • The simulator generates points using the ground truth of the image. 20
- 21. Simulator: Location of clicks S. Goferman, L. Zelnik-Manor, and A. Tal. Context-aware saliency detection. PAMI, 2012. 21
- 23. Outline • Motivation • Object Segmentation • Experiments • Results • Conclusions • Ongoing Work 23
- 24. Jaccard index Measure of similarity between the segmentation result and the ground truth mask 24
- 27. Outline • Motivation • Object Segmentation • Experiments • Results • Conclusions • Ongoing Work 27
- 28. Conclusions • Realistic simulator to process large amounts of data. • Estimation of the expected AVERAGE Jaccard index by clicks. • Inter-class variance of results. 28
- 29. Ongoing Work 29
- 30. Ongoing Work • Image segmentation • CPMC candidates ● ● 30 Label propagation through hierarchical partitions (eg. UCM, BPT…) Grabcut + Superpixels
- 31. Ongoing Work • Data collection • Awarded with $250 in CrowdMM Competition (ACM MM Barcelona 2013). • Already more than 1500 games collected with 100 users More on that in our poster! 31
- 32. Questions, suggestions… Thank you for your attention • Motivation • Object Segmentation • Experiments • Results • Conclusions • Ongoing Work 32