Hyper Video Browser Search and Hyperlinking in Broadcast Media
0 likes537 views
Eurecom's Hyper Video Browser is designed to enhance user access to multimedia content by integrating advanced search and hyperlinking capabilities during video playback. It employs multimodal content analysis for better navigation through video collections and offers dynamic hyperlinks related to currently viewed media segments. The system demonstrated strong performance in the MediaEval 2014 search and hyperlinking task, showcasing its relevance and efficiency in handling large media repositories.
![EURECOM – Campus SophiaTech,
450 Route des Chappes, 06410 Biot FRANCE
www.eurecom.fr
Experiments
Dataset : MediaEval Search and Hyperlinking 2013 task dataset:
Search task: 50 queries in the known-item framework:
Query text: “What does a ball look like when it hits the wall
during Squash”
Visual Cues: “ball hitting a wall in slow motion”
Hyperlinking task: 30 anchors in the ad-hoc framework:
Anchor is defined by start and end time in a video, i.e. a
media fragment which can be represented with all available
features (transcript, context (larger media fragment), visual
concepts for shots, metadata)
Hyper Video Browser
Search and Hyperlinking in Broadcast Media
Maria Eskevich, Huynh Nguyen, Mathilde Sahuguet, Benoit Huet
benoit.huet@eurecom.fr
Objectives
Experimental results
Conclusion
Framework
• Build an integrated search/browsing interface enhanced with further hyperlinking feature during playout.
• Develop multiple navigation means though the content including multi-faceted search and browsing interface for navigating in video collections and the
dynamic suggestion of hyperlinks related to a media fragment content, rather than the entire video, being viewed.
• Utilize the state-of-the-art multimodal content analysis and indexing techniques, at multiple temporal granularity.
The Hyper Video Browser endeavors to facilitate user access to multimedia content.
The novel media browsing interface provides effective skimming using categorized tag-clouds, allows to perform textual and visual searches (using
natural language queries, and their combination with textual description of requested visual content), and offers hyperlinks to media containing related
content dynamically while playing the selected video.
The system performed among the best of MediaEval Search and Hyperlinking task, indicating its relevance and performance accuracy.
ACM MULTIMEDIA 2015
26 - 30 October 2015, Brisbane, Australia
Broadcast Media: BBC content
Manually transcribed
subtitles
Metadata:
title, cast, description, broadcast time
Shot segmentation,
keyframes
Visual content analysis:
Scene segmentation
Concept detection
on shot level
Lucene/Solr:
Indexing/Retrieval on shot level
Media database:
76 214 fragments
Webservice:
(HTML5/AJAX/PHP)
Videos:
2323 items/1697 hours
Search/Browse Hyperlinking
User Interface
Shot segmentation,
keyframe extraction
Result list:
Media fragments level
• Clickable animated tag-
cloud;
• Search field for free text
query;
• Choice of facets: Cast,
Description, Title and
Visual Concepts;
• Visual entry point: media
fragments corresponding
to a keyword randomly
selected from the tag-cloud:
keyframe and metadata.
Access to the content via:
• List of hyperlinks whose content
relates to the content of the video
fragment currently playing (on the
left): keyframe and metadata;
• Dynamically displayed textual
description of the program and of the
current fragment;
• Bi-colored temporal stripe that gives
direct access to individual media
fragments and provides insight upon
the program structure;
• Full length video.
Motivation
Massive amounts of digital media is being produced and consumed daily on the Internet. This requires technologies that allow efficient access to the
content of a media repository through an intuitive and easy-to-use interface, while based on the state-of-the-art retrieval techniques.
References
[1] E. Apostolidis, V. Mezaris, M. Sahuguet, B. Huet, B. Cervenkova, D. Stein, S. Eickeler, J. L. Redondo Garcia, R. Troncy, and L. Pikora. Automatic fine-grained hyperlinking of videos within a closed collection
using scene segmentation. In ACMMM 2014, 22nd ACM International Conference on Multimedia, Orlando, Florida, USA, 11 2014.
[2] M. Eskevich, R. Aly, D. N. Racca, R. Ordelman, S. Chen, and G. J. Jones. The Search and Hyperlinking Task at MediaEval 2014. In Proceedings of MediaEval 2014 Workshop, Barcelona, Catalunya, Spain, 2014.
[3] H. Le, Q. Bui, B. Huet, B. Cervenkova, J. Bouchner, E. E. Apostolidis, F. Markatopoulou, A. Pournaras, V. Mezaris, D. Stein, S. Eickeler, and M. Stadtschnitzer. LinkedTV at MediaEval 2014 Search and Hyperlinking
Task. In Proceedings of MediaEval 2014 Workshop, Barcelona, Catalunya, Spain, 2014.
[4] P. Sidiropoulos, V. Mezaris, I. Kompatsiaris, H. Meinedo, M. Bugalho, and I. Trancoso. Temporal video segmentation to scenes using high-level audiovisual features. IEEE Trans. Circuits Syst. Video Techn.,
21(8):1163-1177, 2011.
Search Hyperlinking
Access to the content via:](https://image.slidesharecdn.com/poster-demo-hypervideobrowserbh-151030002256-lva1-app6891/85/Hyper-Video-Browser-Search-and-Hyperlinking-in-Broadcast-Media-1-320.jpg)
Hyper Video Browser Search and Hyperlinking in Broadcast Media
- 1. EURECOM – Campus SophiaTech, 450 Route des Chappes, 06410 Biot FRANCE www.eurecom.fr Experiments Dataset : MediaEval Search and Hyperlinking 2013 task dataset: Search task: 50 queries in the known-item framework: Query text: “What does a ball look like when it hits the wall during Squash” Visual Cues: “ball hitting a wall in slow motion” Hyperlinking task: 30 anchors in the ad-hoc framework: Anchor is defined by start and end time in a video, i.e. a media fragment which can be represented with all available features (transcript, context (larger media fragment), visual concepts for shots, metadata) Hyper Video Browser Search and Hyperlinking in Broadcast Media Maria Eskevich, Huynh Nguyen, Mathilde Sahuguet, Benoit Huet benoit.huet@eurecom.fr Objectives Experimental results Conclusion Framework • Build an integrated search/browsing interface enhanced with further hyperlinking feature during playout. • Develop multiple navigation means though the content including multi-faceted search and browsing interface for navigating in video collections and the dynamic suggestion of hyperlinks related to a media fragment content, rather than the entire video, being viewed. • Utilize the state-of-the-art multimodal content analysis and indexing techniques, at multiple temporal granularity. The Hyper Video Browser endeavors to facilitate user access to multimedia content. The novel media browsing interface provides effective skimming using categorized tag-clouds, allows to perform textual and visual searches (using natural language queries, and their combination with textual description of requested visual content), and offers hyperlinks to media containing related content dynamically while playing the selected video. The system performed among the best of MediaEval Search and Hyperlinking task, indicating its relevance and performance accuracy. ACM MULTIMEDIA 2015 26 - 30 October 2015, Brisbane, Australia Broadcast Media: BBC content Manually transcribed subtitles Metadata: title, cast, description, broadcast time Shot segmentation, keyframes Visual content analysis: Scene segmentation Concept detection on shot level Lucene/Solr: Indexing/Retrieval on shot level Media database: 76 214 fragments Webservice: (HTML5/AJAX/PHP) Videos: 2323 items/1697 hours Search/Browse Hyperlinking User Interface Shot segmentation, keyframe extraction Result list: Media fragments level • Clickable animated tag- cloud; • Search field for free text query; • Choice of facets: Cast, Description, Title and Visual Concepts; • Visual entry point: media fragments corresponding to a keyword randomly selected from the tag-cloud: keyframe and metadata. Access to the content via: • List of hyperlinks whose content relates to the content of the video fragment currently playing (on the left): keyframe and metadata; • Dynamically displayed textual description of the program and of the current fragment; • Bi-colored temporal stripe that gives direct access to individual media fragments and provides insight upon the program structure; • Full length video. Motivation Massive amounts of digital media is being produced and consumed daily on the Internet. This requires technologies that allow efficient access to the content of a media repository through an intuitive and easy-to-use interface, while based on the state-of-the-art retrieval techniques. References [1] E. Apostolidis, V. Mezaris, M. Sahuguet, B. Huet, B. Cervenkova, D. Stein, S. Eickeler, J. L. Redondo Garcia, R. Troncy, and L. Pikora. Automatic fine-grained hyperlinking of videos within a closed collection using scene segmentation. In ACMMM 2014, 22nd ACM International Conference on Multimedia, Orlando, Florida, USA, 11 2014. [2] M. Eskevich, R. Aly, D. N. Racca, R. Ordelman, S. Chen, and G. J. Jones. The Search and Hyperlinking Task at MediaEval 2014. In Proceedings of MediaEval 2014 Workshop, Barcelona, Catalunya, Spain, 2014. [3] H. Le, Q. Bui, B. Huet, B. Cervenkova, J. Bouchner, E. E. Apostolidis, F. Markatopoulou, A. Pournaras, V. Mezaris, D. Stein, S. Eickeler, and M. Stadtschnitzer. LinkedTV at MediaEval 2014 Search and Hyperlinking Task. In Proceedings of MediaEval 2014 Workshop, Barcelona, Catalunya, Spain, 2014. [4] P. Sidiropoulos, V. Mezaris, I. Kompatsiaris, H. Meinedo, M. Bugalho, and I. Trancoso. Temporal video segmentation to scenes using high-level audiovisual features. IEEE Trans. Circuits Syst. Video Techn., 21(8):1163-1177, 2011. Search Hyperlinking Access to the content via: