Media Fragments Indexing using Social Media
2 likes1,411 views
This document discusses a media fragment indexing framework aimed at improving the searchability of multimedia resources by utilizing social media, particularly Twitter, to acquire annotations for media fragments. It surveys the implementation of media fragment URIs on major video sharing platforms and outlines a methodology for indexing these fragments through tweets. The findings indicate that while only a few platforms implement the standard, they contribute to a significant portion of video sharing on the web, with future work proposed to enhance the indexing and annotation processes.
Media Fragments Indexing using Social Media
- 1. Media Fragment Indexing Using Social Media Yunjia Li1, Raphael Troncy2, Mike Wald1 and Gary Wills1 1School of Electronics and Computer Science University of Southampton, UK 2EURECOM, Sophia Antipolis, France, 1
- 2. Agenda • Media Fragments • Media Fragment Indexing Framework • Survey on Media Fragment URI Implementations on Video Sharing Platforms • Indexing Media Fragments Using Twitter • Conclusions and Future Work 2
- 3. Media Fragment • Denote the inside content of multimedia resources • Dimensions defined in the Media Fragment URI 1.0 spec – Temporal dimension http://example.org/test.mp4#t=3,7 – Spatial dimension (a rectangle area) http://example.org/test.mp4#xywh=120,240,180,240 3
- 4. Current Situation • Multimedia uploading, sharing, tagging is easy • Searching a complete multimedia resource on major search engines is easy • But searching multimedia resource at a fine-grained level on major search engines is difficult – Availability of annotations: limited amount of annotations linked to media fragments – SEO problem: • The landing page is not search-engine-friendly • Everything is on the same page and the notion of media fragment is not explicitly embedded in HTML 4
- 6. Google’s Ajax Content Crawler • The Crawler is designed to index Ajax content • Replace token “#!” in URLs with “_escaped_fragment_” 6 *Diagram from https://developers.google.com/webmasters/ajax- crawling/docs/getting-started
- 7. Key Ideas • The fragment information must be included in the URL – Syntax: W3C Media Fragment 1.0 Specification • Prepare two sets of pages for every media fragment – original landing page for end-users – a snapshot page for SEO • Landing page keeps the original user interaction – Highlight media fragments on opening • SEO page – ONLY includes annotations of the media fragment – Embed rich snippet 7
- 8. The Solution 8 Server Crawler 1: 1: Submit pretty URL replay/1#!t=3,7 to the crawler 2: 2: Crawler asks server for replay/1?_escaped_fragment_=t=3,7 Terrace Theater 3: Snapshot page Snapshot/1?_escaped_frag ment_=t=3,7 3: Redirect the request to the snapshot page generated by the server. The snapshot page only contains annotations and Microdata for “#t=3,7”, Terrace Theater Linked Data Landing page replay/1#!t=3,7 Terrace Theater replay/1#!t=3,7 4: 4: The snapshot page is returned to the crawler with URL replay/1#!t=3,7 5: Terrace Theater 5: A user searches keyword “Terrace Theater” 6: replay/1#!t=3,7 6: Google includes replay/1#!t=3,7 in the search results 7: 7: The user click the link and ask for the document at replay/1#!t=3,7 8: 8: The server returns the landing page containing both “Terrace Theater” and “Linked Data” 9: 9: The landing page highlights the media fragment by start playing from 3s to 7s
- 9. Discussion • The Media Fragment Indexing Framework solved the SEO problem of media fragments • The scalability of such method largely relies on whether there are large number of annotations linked to media fragments • Looking for media fragment annotations? – Timed-text, transcript, speech recognition – Manual annotations on each video sharing platforms – Social Media (Twitter) 9
- 10. Survey on Media Fragment URI Implementation 10
- 11. Media Fragments and Social Media • The deep-linking function • A Media Fragment URL can be embedded in a Tweet • Text of the Tweet is the annotation to the URL • Get annotations by filtering Tweets that have MF URIs 11
- 12. Filter Tweets by Media Fragment URIs • Problem: – Any URL in Tweet is potentially a MF URI – Too many false-positive cases http://example.org/1234#t=23 http://example.org/1234?t=23 http://example.org/1234?track=23 – They could all be MF URIs, need to be identified manually • Work around: – Identify platforms (partially-)implementing MF URI – Only filter Tweets containing URLs from those domains 12
- 13. Survey Methodology • Find a list of video sharing platforms – http://en.wikipedia.org/wiki/List_of_video_hosting_services – 59 websites are targeted in the survey – Some of them have access restrictions • Go through each website manually to see whether they provide deep-linking function, such as: – Social sharing button from a certain time point – Deep-linking option in right click menu 13
- 14. Survey Results (1) • 9 websites partially-implemented MFURI – 56.com, Dailymotion, Hulu, Vbox7, Viddler, vimeo, Tudou, Youku and YouTube • They use different syntax to encode temporal dimension – Most of them use URI query, except YouTube & Vimeo – Parameter name: “start”, “t”, “st”, etc – Only Hulu implemented the end time • Only YouTube partially implemented spatial dimension – This is an external function implemented by Clickberry https://clickberry.tv/video/6dafe30e-dcb8-44b8-8190-32be8249a297 14
- 15. Survey Results (2) • Only 9 websites partially-implemented MFURI, however: – Those websites have covered most videos shared on the web – eBizMBA report: http://www.ebizmba.com/articles/video-websites • Select filter keywords based on the survey results: – Twitter is banned in China, so 56.com, Tudou and Youku are ignored – Hulu has access restriction outside U.S. • Filter keywords: “YouTube”, “Dailymotion”, “Vbox7”, “Vimeo” and “Viddler” 15
- 16. Indexing Media Fragments Using Twitter 16
- 17. Twitter Media Fragment Indexer • Collect Tweets filtered by the keywords • Extract MF URIs in Tweets, parse the media fragment information • Use Media Fragment Indexing Framework to publish Tweets as media fragment annotations • Embed rich snippet in the snapshot pages • Create sitemap for Google to crawl the snapshot pages • User searches keywords in the Tweet in Google and the link will lead to the video with corresponding start time 17
- 19. Indexing Results (1) • Monitor 50-hour non-stop Twitter stream • Filter phrase: “youtube, dailymotion, vimeo, vbox7, viddler” • 5,779,858 Tweets examined, 5,269,742 contain URLs • 32,754 Tweets contain MF URIs, 32796 MF URIs in total • Media Fragment URIs shared in each website: 19 Website No. of MFURIs % YouTube 32,666 99.604 Dailymotion 101 0.308 Vbox7 0 0 Viddler 0 0 Vimeo 29 0.088
- 20. Indexing Results (2) • 13,088 distinct videos are found • 17,854 distinct MF URIs for sitemap – Many Tweets share the same video, but different fragments – Many retweets – Some video are not available in UK • 17,479 URLs (97.9%) in the sitemap have been indexed by Google • Only 775 URLs are indexed as VideoObject even though all rich snippets are embedded in all snapshot pages 20
- 21. Demo • Search “Chris Eppstein” • As a result, this landing page will be opened and the video start playing from the time indicated in the Tweet containing keywords “Chris Eppstein” 21
- 22. Conclusions and Future Work 22
- 23. Conclusions • Introduced Media Fragment Indexing Framework • Propose the using of social media to acquire more annotations to media fragments • Survey the MF URI implementation on major video sharing platforms • Twitter Media Fragment Indexer – Monitor Tweet Stream and automatically create media fragment annotations – Index media fragments in Google – YouTube is the most important domain to share media fragments on Twitter 23
- 24. Future Work • How valid tweets could be served as media fragment annotations – many noisy and unrelated text – many re-tweets • Experiment on larger scale (billions of tweets and continuous monitoring) • Expand the methodology to other media fragment annotations, such as timed-text • Extract named entities from tweets and further link media fragments to the Linked Data Cloud 24
- 25. Questions? 25