Survey of Semantic Media Annotation Tools - towards New Media Applications with Linked Media
9 likes4,424 views
The document discusses the potential of linked media and the importance of semantic media annotation tools for new media applications. It highlights the need for common representations of media structure and content, as well as the interoperability of media ontologies across systems. Additionally, it surveys various existing annotation tools that support online media and their functionalities.
Survey of Semantic Media Annotation Tools - towards New Media Applications with Linked Media
- 1. Television Linked To The Web www.linkedtv.eu Lyndon Nixon, MODUL University Vienna GmbH Raphael Troncy, EURECOM Survey of Semantic Media Annotation Tools for the Web: towards New Media Applications with Linked Media 25 May 2014 LiME Workshop ESWC2014
- 2. 2 www.linkedtv.eu Overview Why Linked Media? What is Linked Media? Requirements on online media annotation Survey of existing tools LinkedTV services and applications for Linked Media Summary and Outlook Survey of Semantic Media Annotation Tools for the Web: Towards new Media Applications with Linked Media
- 3. 3 www.linkedtv.eu Why Linked Media? Survey of Semantic Media Annotation Tools for the Web: Towards new Media Applications with Linked Media „It is growing at more than 20% per annum, fuelled by increased demands for new programming and the huge saving it represents compared with shooting new footage. Interactive technology and the Internet will further contribute to the growth of the market as it makes stock footage cheaper and easier to locate and license.“ - http://moneyam.uk-wire.com/cgi-bin/articles/200201020827103514P.html
- 4. 4 www.linkedtv.eu Why Linked Media? Survey of Semantic Media Annotation Tools for the Web: Towards new Media Applications with Linked Media Dublin Core Metadata Initiative (DCMI) Metadata Encoding and Transmission Standard (METS)
- 5. 5 www.linkedtv.eu Why Linked Media? Survey of Semantic Media Annotation Tools for the Web: Towards new Media Applications with Linked Media
- 6. 6 www.linkedtv.eu What is Linked Media? Survey of Semantic Media Annotation Tools for the Web: Towards new Media Applications with Linked Media • What is being described? • The media object – what is it? Video uploaded to YouTube 2011 ZDF documentary broadcast 1980 Rembrandt self-portrait from 1639 Date? Is Rembrandt a subject of the whole video? Or only a specific temporal segment of the video? Is the whole frame important or only the part with Rembrandt‘s face?
- 7. 7 www.linkedtv.eu What is Linked Media? Survey of Semantic Media Annotation Tools for the Web: Towards new Media Applications with Linked Media Media fragment players: http://ninsuna.elis.ugent.be/MFPlayer/html5 http://linkedtv.eurecom.fr/mediafragmentplayer Emerging support in Web browsers: Firefox, Safari, Chrome
- 8. 8 www.linkedtv.eu What is Linked Media? The object of annotation • What is the description of the media? "Rembrandt" "Rembrandt self- portrait" from 1639 Survey of Semantic Media Annotation Tools for the Web: Towards new Media Applications with Linked Media
- 9. 9 www.linkedtv.eu What is Linked Media? Digital identifiers for non- addressable resources • “URIs for concepts” since the Semantic Web • Linked Data has popularized the notion Survey of Semantic Media Annotation Tools for the Web: Towards new Media Applications with Linked Media
- 10. 10 www.linkedtv.eu What is Linked Media? 1. Web media descriptions need a common representation of media structure 2. Web media descriptions need a common representation of media content 3. Web media descriptions need to use a media ontology which supports description of both the structure and content of media 4. The descriptions of media in terms of common representations of structure and content are the basis for deriving links across media on the Web (Linked Media) Survey of Semantic Media Annotation Tools for the Web: Towards new Media Applications with Linked Media
- 11. 11 www.linkedtv.eu Requirements on online media annotation Survey of Semantic Media Annotation Tools for the Web: Towards new Media Applications with Linked Media “Just as Web browsers could never have been successful without a core consensus on the Web page mark up (HTML), Web media can not fulfil its potential without a core consensus on the available descriptive metadata.” (1) tool or service is online and allows to annotate media of different types via URL (2) media can be annotated in terms of its parts and the media fragmentation follows a non-proprietary approach supported by other tools (3) media can be annotated by concepts where a shared understanding is possible across systems (4) media ontology used for the generated descriptions is interoperable across systems
- 12. 12 www.linkedtv.eu Survey: overview of annotation tools Survey of Semantic Media Annotation Tools for the Web: Towards new Media Applications with Linked Media EuropeanaConnect Media Annotation Suite (http://dme.arcs.ac.at/annotation/) "Developed in the EuropeanaConnect project, it supports image,audio and video. Both DBPedia and Geonames resources can be annotation targets, and are suggested from free text or location references respectively. Annotations can be exported as RDF using a tool- specific vocabulary."
- 13. 13 www.linkedtv.eu Survey: overview of annotation tools Survey of Semantic Media Annotation Tools for the Web: Towards new Media Applications with Linked Media ImageSnippets (http://www.imagesnippets.com/) Open Video Annotation (http://www.openvideoannotation.o rg/)
- 14. 14 www.linkedtv.eu Survey: overview of annotation tools Survey of Semantic Media Annotation Tools for the Web: Towards new Media Applications with Linked Media Annotorius (http://annotorious.github.io/) Pundit and SemTube (http://thepund.it and http://www.semedia.dibet.univpm.it/semtube/)
- 15. 15 www.linkedtv.eu Survey: overview of annotation tools Survey of Semantic Media Annotation Tools for the Web: Towards new Media Applications with Linked Media Web based; open videos from online or the repository; term suggestion; export annotations as file or to repository. ConnectME annotation tool https://github.com/lyndonnixon/ annotationtool Contact lyndon.nixon@modul.ac.at GNU GPL 3.0 license
- 16. 16 www.linkedtv.eu LinkedTV: background Survey of Semantic Media Annotation Tools for the Web: Towards new Media Applications with Linked Media 40% of TV viewers are using a companion device alongside the TV program.* * J. Abreu, P. Almeida, B. Teles, and M. Reis. Viewer behaviors and practices in the (new) television environment. In Proceedings of the 11th European Conference on Interactive TV and Video, EuroITV '13. http://www.linkedtv.eu Ever saw something on TV and wanted to know more about it, but didn‘t even know how to search for it?
- 17. 17 www.linkedtv.eu LinkedTV technology Survey of Semantic Media Annotation Tools for the Web: Towards new Media Applications with Linked Media “...schilderij van Jan Sluijters....” Video object and word detection dbpedia.org/resource/Jan_Sluyters Connection to concepts Paintings by Jan Sluijters Selection of related concepts Selection of related content Presentation engine
- 18. 18 www.linkedtv.eu Annotation task: automatic approaches Survey of Semantic Media Annotation Tools for the Web: Towards new Media Applications with Linked Media http://nerd.eurecom.fr
- 19. 19 www.linkedtv.eu Media Fragment + Open Annotation + NERD 08.11. 2012 Survey of Semantic Media Annotation Tools for the Web: Towards new Media Applications with Linked Media
- 20. 20 www.linkedtv.eu Annotation task: automatic approaches Survey of Semantic Media Annotation Tools for the Web: Towards new Media Applications with Linked Media
- 21. 21 www.linkedtv.eu Annotation task: automatic approaches Survey of Semantic Media Annotation Tools for the Web: Towards new Media Applications with Linked Media http://linkedtv.eurecom.fr/tv2rdf/
- 22. 22 www.linkedtv.eu Annotation task: manual approaches Survey of Semantic Media Annotation Tools for the Web: Towards new Media Applications with Linked Media The LinkedTV Editor Tool will have a public code release in late 2014 (planned GNU GPL 3.0), but a test installation can be used at http://editortool.linkedtv.eu (Enquiries: jblom@beeldengeluid.nl)
- 23. 23 www.linkedtv.eu Enrichment task: associating media Survey of Semantic Media Annotation Tools for the Web: Towards new Media Applications with Linked Media Edward Snowden asks for asylum in Russia (04 / 07 / 2013)
- 24. 24 www.linkedtv.eu Enrichment task: associating media Survey of Semantic Media Annotation Tools for the Web: Towards new Media Applications with Linked Media In which Russian airport is he exactly? LSCOM:Face LSCOM:Building ?
- 25. 25 www.linkedtv.eu Enrichment task: associating media Survey of Semantic Media Annotation Tools for the Web: Towards new Media Applications with Linked Media List of Relevant Named Entities (1) Named Entity (2) Filtering and Ranking b) Expanded Entities b) Re-ranked Entities a) Entities from Video
- 26. 26 www.linkedtv.eu Enrichment task: associating media Survey of Semantic Media Annotation Tools for the Web: Towards new Media Applications with Linked Media http://linkedtv.project.cwi.nl/news/ mainscreen.html http://linkedtv.project.cwi.nl/news/ Second Screen @ Main Screen @
- 27. 27 www.linkedtv.eu Outlook Survey of Semantic Media Annotation Tools for the Web: Towards new Media Applications with Linked Media Tool Input Fragment support Conceptual annotation Ontology Europeana Connect Audio, image or video. By URL Spatial & temporal (SVG) DBPedia and GeoNames Own vocabulary ImageSnippets Image only. By URL Spatial. Free to import from LOD or create own LIO ontology OpenVideo Annotation Video. Unclear if support URL Only temporal in the demo, unknown syntax Free text or tags Open Annotation Model Annotorius Image only. By URL. Spatial, proprietary syntax DBPedia No ontology (JS data model) Pundit and SemTube Image and video. By URL Spatial, proprietary syntax Freebase. Terms can be imported. Own RDF vocabulary ConnectME Video. By URL Spatial & temporal using MF URIs DBPedia and Geonames ConnectME Ontology LinkedTV TV2RDF and Editor Tool Video. By URL (Editor Tool uses resources on the LinkedTV platform) Spatial & temporal using MF URIs DBPedia. Editor tool suggests DBPedia but allows any URI. LinkedTV Ontology