Where should saliency models look next ? (UPC Reading Group)
1 like346 views
The document discusses saliency models, which create probability distribution maps indicating where human observers focus in images, revealing insights into human image understanding. It highlights recent breakthroughs using CNNs, evaluation scores, and the importance of accurately predicting regions containing people, actions, and informative text. The authors conclude that saliency models still lack convergence with human performance, especially in recognizing crucial image areas.
Where should saliency models look next ? (UPC Reading Group)
- 1. WHERE SHOULD SALIENCY MODELS LOOK NEXT ? Zoya Bylinskii, Adrià Recasens, Ali Borji, Aude Oliva, Antonio Torralba, Frédo Durand “Where Should Saliency Models Look Next?”, ECCV2016
- 2. Hello!I am Junting Pan I am here because I love to give presentations. You can find me at junting.pa@gmail.com
- 3. Saliency map Saliency map is a probability distribution map, that describe where human observers look in images. It can provide important clues to human image understanding : - Main focus - Action or event - Participants Regions of interest to human
- 4. 1. Motivation Let’s start with the first set of slides
- 5. Breakthroughs because of …. × Prediction score increment has benn stable for looong time since .. × CNN comes !! × End to end manner : feat. extraction, feat. integration & saliency prediction.
- 6. Breakthroughs because of …. × Prediction score increment has benn stable for looong time since .. × CNN comes !! × End to end manner : feat. extraction, feat. integration & saliency prediction. Evaluation scores have begun to saturate
- 7. “Have saliency models begun to converge on human performance and is saliency a solved problem ?”
- 8. HIGHER LEVEL - Text - Objects of gaze and action - Locations of motion - People in image
- 9. A picture is worth a thousand words A complex idea can be conveyed with just a single still image, namely making it possible to absorb large amounts of data quickly.
- 12. SALICON model CNN applied at 2 different image scales : small & BIG BEST MODELS AT mit benchmark DeepFix FCN built on top of the VGG.
- 13. 10 MOST REPRESENTATIVE images 0,97 of Spearman correlation relative to their ranking on all dataset images
- 14. 3. QUANTIFYING WHERE people & MODELS LOOK IN IMAGES
- 15. Name all image regions under the fixation map 95 percentile threshold 651 regions over 300 images 20 users and 2 MTurk task
- 18. WHAT CAN MODELS GAIN? Gains that model could have if specific region were correctly predicted
- 21. Face saliency is underestimated when faces are small, non-frontal, or not centered in an image Sometimes the actions in a scene are more salient to human observers than the participants, but saliency models can overestimate the relative saliency of the faces
- 22. Not all people are equally important × Assign importance score to each face (using fixation gt and predicted map. × Relative ordering assign by saliency model does not match by the importance given by human fixations.
- 23. Name all image regions under the fixation map GrayWhite Black
- 25. Understanding the text ... × The description of a warning or a book are more informative to observers than the warning or book title.. × Only piece of English text..
- 27. 4. Object of gaze and action
- 28. Objects and action × Objects of gaze and/or action are usually missed × Detecting objects of action remains a problem area for saliency model..
- 30. 4. Conclusion Let’s finish with one slide
- 31. Conclusion Models continue to under-predict crucial image regions containing people, actions, and text. These are the regions with greatest semantic importance in an image, and become essential for saliency applications
- 32. “Have saliency models begun to converge on human performance and is saliency a solved problem ?”
- 33. “NoT YET!”
- 34. THANKS!Any questions? Please contact with the authors :)