RS at Search and Hyperlinking of Television Content Task
- 1. JOANNEUM RESEARCH Forschungsgesellschaft mbH DIGITAL Institute for Information and Communication Technologies Steyrergasse 17 8010 Graz, Austria Tel. +43 316 876-5000 Fax +43 316 876 - 5010 digital@joanneum.at www.joanneum.at/digital www.icosole.eu Acknowledgements: The ICoSOLE project receives funding from the European Union’s Seventh Framework Programme (FP7/2007-2013) under grant agreement n° 610370. digpos322 014 Approach JRS at Hyperlinking of Television Content Task Generate basic links based on text and metadata q Inputs: subtitles or ASR transcripts, metadata (synopsis) q Text preprocessing: remove punctuation, normalize capitalization, remove stop words/very short words q Obtain set T of terms found in DBpedia q Add synonyms ST for terms T from WordNet q Select connected entities CT for the terms in T from FreeBase q For geographic locations Tg T (as identified by FreeBase), select related geographic entities GTg from GeoNames q Weight for matching as follows: Experiments Werner Bailer, Harald Stiegler q For multiple occurrences per segment, determine total weight as Total similarity between segments v1, v2 determined as q Use fixed segments of 20 seconds Use of visual concepts (in some runs) q Treat as text annotations q Concepts only or subtitles/ASR plus concepts Rerank based of visual matches q Use VLAT signatures of SIFT descriptors (from DOG points) q Every 5th frame (using one field when interlacing artifacts are present) q Rerank top 50 segments Fig. 1: MAP of submitted runs Eight pairs of runs q Pair: one with anchor segment, one with context segment (+/- 3 minutes) q 4 runs: subtitles and the different text resources q Visual reranking for subtitles and LIMSI q Visual concepts with subtitles and visual concepts only Results q Text only methods are very similar to those used in 2014: results are overall worse ICoSOLE q More differences between anchors q More differences between different text resources q Large difference between anchor only and context for some runs q Overall, context is only slightly better q Visual reranking adds slight improvement q Visual concepts: cannot be treated like other annotations, not sufficient on their own, no improvement when added to text