Cluster based landmark and event detection for tagged photo collections
1 like1,263 views
The document presents a framework for detecting landmarks and events from tagged photo collections using a cluster-based approach. It details the steps involving the creation of a photo similarity graph, clustering methods, and landmark detection with a focus on enhancing geospatial coherence and tagging precision. The results indicate significant improvements in classification performance and accuracy through the use of tag features.
![#users / #photos baseline features
[2 years, 50 users / 120 photos]
[1 day, 2 users / 10 photos]
Quack et al., CIVR 2008 duration](https://image.slidesharecdn.com/clusterbasedlandmarkeventdetection-111203113333-phpapp01/85/Cluster-based-landmark-and-event-detection-for-tagged-photo-collections-18-320.jpg)
Cluster based landmark and event detection for tagged photo collections
- 1. Cluster-based Landmark and Event Detection on Tagged Photo Collections Symeon Papadopoulos, Christos Zigkolis, Yiannis Kompatsiaris, Athena Vakali
- 2. user generated content creates new opportunities
- 3. real-world depicted in users’ online collections
- 4. potential for many insights into what people see, do and like need new tools for content organization
- 6. clusters landmarks + events landmark event
- 8. + + photos tags geo
- 9. overview 1 2 landmark landmark event 4 3
- 10. step 1: create photo similarity graph 1 2 landmark landmark event 4 3
- 11. SURF SIFT visual similarity casa mila, la pedrera tag similarity co-occurrence latent semantic indexing
- 12. step 2: use graph to cluster the photos 1 2 landmark landmark event 4 3
- 13. the concept of node structure neighborhood of node v + node itself = structure of node v v v v N(v) v Γ(v)
- 14. the concept of structural similarity (1) v u Γ(v) ∩ Γ(u) structural similarity between nodes v and u Γ(v) Γ(u)
- 15. the concept of structural similarity (2) high structural similarity photo cluster 1 C A B photo cluster 2 low structural similarity
- 16. # edges complexity O (km m) graph-based clustering average node degree # dimensions # clusters k-means clustering O (I C n D) # iterations # nodes O (n2 log n) hierarchical agglomerative clustering
- 17. step 3: detect landmarks & events 1 2 landmark landmark event 4 3
- 18. #users / #photos baseline features [2 years, 50 users / 120 photos] [1 day, 2 users / 10 photos] Quack et al., CIVR 2008 duration
- 19. Landmark Tags additional features Event Tags
- 20. step 4: post-process landmark clusters 1 2 landmark landmark event 4 3
- 21. cluster merging based on proximity
- 22. cluster tag filtering CLUSTER TAGS helado tropical barcelona cielos spain field park güell jaume oller park sclupture el beso low frequency tags generic tags
- 23. results
- 24. 207,750 photos 7,768 users 33,959 unique tags compare graph-based vs. k-means clustering user study geospatial coherence high geospatial coherence low geospatial coherence
- 25. user study VISUAL precision recall κ-statistic graph-based 1.000 0.110 1.000 k-means 0.806 0.324 0.226 TAG precision recall κ-statistic graph-based 0.950 0.182 0.820 k-means 0.848 0.307 0.564
- 26. geospatial coherence VISUAL radius std. deviation graph-based 357 m 1.18 km k-means 2.4 km 1.73 km TAG graph-based 456 m 1.15 km k-means 767 m 1.76 km
- 27. classification performance 16% - 23% improvement thanks to tag features
- 28. landmark localization accuracy sagrada familia, cathedral, catholic 15.2m la pedrera, casa mila 31.8m parc guell 9.6m boqueria, market, mercado, ramblas 82.1m camp nou, fc barcelona, nou camp 18.7m
- 29. event category composition music, concert, gigs, dj 43.1% conference, presentation 6.5% local traditional, parades 4.6% racing, motorbikes, f1 3.3%
- 30. clusttour www.clusttour.gr twitter.com/clusttour facebook.com/clusttour