Finding media illustrating events
1 like18,561 views
The document discusses finding and organizing media like photos and videos that illustrate real-world events. It describes scraping various event directories and linking the data. Media can be found by searching based on geo-tags, titles, or uploader. Visual analysis is used to prune media results. The goal is to help users explore, annotate and share media related to events. Challenges include interlinking diverse data sources and detecting unscheduled events.
Finding media illustrating events
- 1. Finding Media Illustrating Events Xueliang Liu <xueliang.liu@eurecom.fr> Raphaël Troncy <raphael.troncy@eurecom.fr> Benoit Huet <benoit.huet@eurecom.fr>
- 2. What are Events? Events are observable occurrences grouping People Places Time Experiences documented by Media 19/04/2011 - Finding Media Illustrating Events - ICMR 2011, Trento, Italy -2
- 4. Searching for an event 19/04/2011 - Finding Media Illustrating Events - ICMR 2011, Trento, Italy -4
- 5. Searching for media about an event 19/04/2011 - Finding Media Illustrating Events - ICMR 2011, Trento, Italy -5
- 6. There’s a lot of information out there… 19/04/2011 - Finding Media Illustrating Events - ICMR 2011, Trento, Italy -6
- 7. http://www.flickr.com/photos/mwparenteau/432039783 19/04/2011 - 7
- 8. Organize the mess Event Media Scrape event directories Link the information Find media illustrating events Design the application Interface Linking Open Data cloud diagram, by Richard Cyganiak and Anja Jentzsch. http://lod-cloud.net/ 19/04/2011 - Finding Media Illustrating Events - ICMR 2011, Trento, Italy -8
- 9. Róisín Murphy at Nouveau Casino E0-001-005971169-9 350591 19/04/2011 - Finding Media Illustrating Events - ICMR 2011, Trento, Italy -9
- 10. Representing Events with 19/04/2011 - Finding Media Illustrating Events - ICMR 2011, Trento, Italy - 10
- 11. Interlinking Linking Agents with Freebase, Dbpedia, MusicBrainz Linking Venues with Geonames, Dbpedia, Foursquare (via Uberblic) Linking Events with Last.fm, Upcoming, Eventful Linking Categories with Facebook, Eventful, Upcoming, Zevents, LinkedIn,Eventbrite, TicketMaster Linking Users with Social Graph API 19/04/2011 - Finding Media Illustrating Events - ICMR 2011, Trento, Italy - 11
- 12. Róisín Murphy at Nouveau Casino 19/04/2011 - Finding Media Illustrating Events - ICMR 2011, Trento, Italy - 12
- 13. Media explicitly associated with the event APIs Machine tags “lastfm:events” 4790 photos, 263 1.7 million images over videos over 110 events 108.000 events 19/04/2011 - Finding Media Illustrating Events - ICMR 2011, Trento, Italy - 13
- 14. Representing Media with Media Ontology 19/04/2011 - Finding Media Illustrating Events - ICMR 2011, Trento, Italy - 14
- 15. How much data is there? Event Agent Location Photos User Last.fm 57,258 50,150 16,471 1,425,318 18,542 Upcoming 13,114 0 7,330 347,959 4,518 Eventful 37,647 6,543 14,576 0 0 Total 108,019 56,693 38,377 1,773,277 23,060 1,248,021 geo-tagged photos by propagating information from events! 19/04/2011 - Finding Media Illustrating Events - ICMR 2011, Trento, Italy - 15
- 16. How fast media are uploaded? 19/04/2011 - Finding Media Illustrating Events - ICMR 2011, Trento, Italy - 16
- 17. Finding more media that illustrate an event A. Compute the bounding box area of a venue B. Retrieve all media geo-tagged in this area C. Retrieve all media with a similar title D. Prune the results with visual analysis E. Extend the result set with all media from the same uploader 19/04/2011 - Finding Media Illustrating Events - ICMR 2011, Trento, Italy - 17
- 18. A. Bounding box of Nouveau Casino? 19/04/2011 - Finding Media Illustrating Events - ICMR 2011, Trento, Italy - 18
- 19. B. 74 photos taken in this area this day 19/04/2011 - Finding Media Illustrating Events - ICMR 2011, Trento, Italy - 19
- 20. C. 85 additional photos with a similar title 19/04/2011 - Finding Media Illustrating Events - ICMR 2011, Trento, Italy - 20
- 21. D. 6 photos after visual pruning 19/04/2011 - Finding Media Illustrating Events - ICMR 2011, Trento, Italy - 21
- 22. How is the visual pruning performed? Model dataset: photo id + photo geo Testing dataset: similar title Low-level features used: Color moments, Gabor texture, Edge histogram L1 distance on the K-nearest neighbors Threshold Min L1 distance between two model image pairs Conservative approach 19/04/2011 - Finding Media Illustrating Events - ICMR 2011, Trento, Italy - 22
- 23. E. 66 photos after uploader heuristics hellerpop DustGraph / Stefan cartoixa 13 photos 46 photos 19/04/2011 - Finding Media Illustrating Events - ICMR 2011, Trento, Italy - 23
- 24. Same process for videos 1 video (id) 3 videos (geo) 26 videos (title) Visual pruning performed on key frames Nb positive > 50% 19/04/2011 - Finding Media Illustrating Events - ICMR 2011, Trento, Italy - 24
- 25. How illustrated are the events? Query By ID Query By Geo Query By Title Visual Pruning Heuristic Photos 5 74 (74) 85 (85) 6 (6) 66 (66) Videos 1 3 (0) 23 (0) 13 (0) - (title) Videos 10 (10) (title+venue) 20 events Model dataset: 785 photos Testing dataset: 1766 photos (1573 positive, 193 negative) Results: 439 photos (99% precision, 28% recall) 19/04/2011 - Finding Media Illustrating Events - ICMR 2011, Trento, Italy - 25
- 26. User Interface 19/04/2011 - Finding Media Illustrating Events - ICMR 2011, Trento, Italy - 26
- 27. Conclusions Method for finding media illustrating scheduled events Search media with machine and geo tags Search media with title and normal tags Prune visually and retrieve all media from confirmed users Challenge: do not necessarily trust the geo-coordinates Event-based approach for users to explore, annotate and share media UX can help semantics, semantics can help UX Outstanding challenges in interlinking and curating the data Detecting events from social media activity 19/04/2011 - Finding Media Illustrating Events - ICMR 2011, Trento, Italy - 27
- 28. http://www.slideshare.net/troncy 19/04/2011 - Finding Media Illustrating Events - ICMR 2011, Trento, Italy - 28