Exploiting User Comments for Audio-visual Content Indexing and Retrieval (ECIR'13)
0 likes767 views
The document discusses the use of user comments for indexing and retrieving audio-visual content, highlighting the challenges posed by noise, such as uninformative comments and spam. It presents methodologies for automatic term extraction, burst detection in commenting activity, and vocabulary regularization, emphasizing the potential gains in retrieval performance by effectively leveraging user-generated content. The conclusion suggests that addressing inherent noise in comments can enhance content annotation and retrieval efficiency.
![Bursts
• Peaks in commenting activity may contain interesting information
[External]:
Actor wins
an award
User Comments for Content Indexing and Retrieval 13](https://image.slidesharecdn.com/ecir2013-eickhoff-130403060121-phpapp01/85/Exploiting-User-Comments-for-Audio-visual-Content-Indexing-and-Retrieval-ECIR-13-13-320.jpg)
![Bursts
• Peaks in commenting activity may contain interesting information
[Internal]:
Controversial
comment
User Comments for Content Indexing and Retrieval 14](https://image.slidesharecdn.com/ecir2013-eickhoff-130403060121-phpapp01/85/Exploiting-User-Comments-for-Audio-visual-Content-Indexing-and-Retrieval-ECIR-13-14-320.jpg)
![Generalized Burst Detection
• Kleinberg [1] measured bursts per term
• We need a more general representation of activity peaks
[1] John Kleinberg. Bursty and Hierarchical Structure in Streams, 2003
User Comments for Content Indexing and Retrieval 15](https://image.slidesharecdn.com/ecir2013-eickhoff-130403060121-phpapp01/85/Exploiting-User-Comments-for-Audio-visual-Content-Indexing-and-Retrieval-ECIR-13-15-320.jpg)
Exploiting User Comments for Audio-visual Content Indexing and Retrieval (ECIR'13)
- 1. Exploiting User Comments for Audio-visual Content Indexing and Retrieval Carsten Eickhoff, Wen Li and Arjen P. de Vries March 25, 2013 Delft University of Technology Challenge the future
- 2. Overview • Introduction and statistics • Harnessing user comments for content indexing • Dealing with noise • Retrieval experiments User Comments for Content Indexing and Retrieval 2
- 3. Example User Comments for Content Indexing and Retrieval 3
- 4. Content Annotation • Audio-visual content retrieval relies on textual meta data • Author-provided titles and descriptions are often not enough • Collaborative tagging can provide more information User Comments for Content Indexing and Retrieval 4
- 5. Available Annotation Sources • Tagging content is a tedious task • To make it more interesting, tagging is sometimes integrated in games and reputation schemes • Still, 58% of a 10,000-video sample from YouTube are annotated with less than 140 characters of text each • At the same time, comment threads are massive… User Comments for Content Indexing and Retrieval 5
- 6. Automatic term extraction You will get kissed on the nearest possible Friday by the love of your omg i luv that stuff life.Tomorrow will be the best day of your life.However,if you don't post this comment to at least 3 videos,you will die within 2 days.Now uv started reading dis dunt stop… lol luv it luv Cute snoopy User Comments for Content Indexing and Retrieval 6
- 7. Types of Noise 1. Uninformative comments omg i luv that stuff User Comments for Content Indexing and Retrieval 7
- 8. Types of Noise 1. Uninformative comments You will get kissed on the nearest possible Friday by the love of your life.Tomorrow will be the best day 2. Unrelated comments (incl. spam) of your life.However,if you don't post this comment to at least 3 videos,you will die within 2 days.Now uv started reading dis dunt stop… User Comments for Content Indexing and Retrieval 8
- 9. Types of Noise 1. Uninformative comments OMG YEAH 2. Unrelated comments (incl. spam) LOL1!1!!! i luv that part u like 3. Misspellings and chat speak robot chicken? User Comments for Content Indexing and Retrieval 9
- 10. Types of Noise 1. Uninformative comments 2. Unrelated comments (incl. spam) Snoopy est si mignon!! 3. Misspellings and chat speak 4. Foreign language utterances User Comments for Content Indexing and Retrieval 10
- 11. LM-based Keyword extraction • Find those terms that have a locally higher likelihood of occurrence than globally in the collection • Similar notion as tf/idf but within the LM framework User Comments for Content Indexing and Retrieval 11
- 12. Bursts • Peaks in commenting activity may contain interesting information User Comments for Content Indexing and Retrieval 12
- 13. Bursts • Peaks in commenting activity may contain interesting information [External]: Actor wins an award User Comments for Content Indexing and Retrieval 13
- 14. Bursts • Peaks in commenting activity may contain interesting information [Internal]: Controversial comment User Comments for Content Indexing and Retrieval 14
- 15. Generalized Burst Detection • Kleinberg [1] measured bursts per term • We need a more general representation of activity peaks [1] John Kleinberg. Bursty and Hierarchical Structure in Streams, 2003 User Comments for Content Indexing and Retrieval 15
- 16. Burst and Cause • Capturing bursts seems to help • But we also need its cause • A mixture of language models accounts for burst and pre- burst term likelihoods User Comments for Content Indexing and Retrieval 16
- 17. Vocabulary Regularization • Currently: Discriminative terms are good • As a result: Misspellings and non-English terms are recommended • Wikipedia can help identify such cases: Snoopy User Comments for Content Indexing and Retrieval 17
- 18. Vocabulary Regularization • Currently: Discriminative terms are good • As a result: Misspellings and non-English terms are recommended • Wikipedia can help identify such cases: Yeah!!1% Wait, that’s not a word… User Comments for Content Indexing and Retrieval 18
- 19. Data Set • 10,000 YouTube videos crawled in 2009/10 • 20 seed queries, following “related videos” link • 4.7 M user comments • On average 360 comments per video (σ = 984) User Comments for Content Indexing and Retrieval 19
- 20. Retrieval experiments • TREC-style retrieval experiment • 40 manually constructed topics • Pooled top 10 results evaluated via crowdsourcing • BM25F models with fields per source (title, description, etc.) User Comments for Content Indexing and Retrieval 20
- 21. Retrieval performance User Comments for Content Indexing and Retrieval 21
- 22. Retrieval performance User Comments for Content Indexing and Retrieval 22
- 23. Retrieval performance User Comments for Content Indexing and Retrieval 23
- 24. Retrieval performance • 40% gain in MAP User Comments for Content Indexing and Retrieval 24
- 25. Retrieval performance • 40% gain in MAP User Comments for Content Indexing and Retrieval 25
- 26. Experiments under Sparsity • 58% of all video descriptions are shorter than 140 characters • 50% of all titles are shorter than 35 characters • We limit our corpus to videos with short titles and/or descriptors • This affects 77% of all videos in our sample… User Comments for Content Indexing and Retrieval 26
- 27. Retrieval performance (sparse) User Comments for Content Indexing and Retrieval 27
- 28. Retrieval performance (sparse) • 54% gain in MAP User Comments for Content Indexing and Retrieval 28
- 29. Closing the Circle User Comments for Content Indexing and Retrieval 29
- 30. Conclusion • User comments can enhance content annotation if we deal with the domain-inherent noise appropriately • Modeling commenting activity bursts, we can find informative on-topic comments • Through the use of Wikipedia, misspellings and foreign language utterances can be reliably identified User Comments for Content Indexing and Retrieval 30
- 31. Future Directions • Additional regularization resources (e.g., Delicious, WordNet) • New domains (e.g., social media streams linked to TV) • Content-aware term extraction • Cold start problem • Cross-language ability User Comments for Content Indexing and Retrieval 31
- 32. Thank You! User Comments for Content Indexing and Retrieval 32