Project demonstrator

Deliverable 7.4 Project demonstrator v1
Lyndon Nixon, STI International
07th
October 2013
Work Package 7: Dissemination
LinkedTV
Television Linked To The Web
Integrated Project (IP)
FP7-ICT-2011-7. Information and Communication Technologies
Grant Agreement Number 287911

Project demonstrator v1 D7.4
© LinkedTV Consortium, 2013 2/41
Dissemination level1 PU
Contractual date of delivery 30th
September 2013
Actual date of delivery 7th
October 2013
Deliverable number D7.4
Deliverable name Project demonstrator v1
File LinkedTV_D7.4.docx
Nature Demonstrator
Status & version Final
Number of pages 41
WP contributing to the
deliverable
7
Task responsible STI International
Other contributors LinkedTV consortium partners
Author(s) Lyndon Nixon
Reviewer Benoit Huet
EC Project Officer Thomas Küpper
Keywords Dissemination
Abstract (for dissemination) LinkedTV is a project implementing innovative technology and
services at every step of the media workflow, in order to
enable a seamless interlinking of Web and TV content for the
consumer. This technology needs to be seen and to be
communicated, hence besides implementation also
dissemination is important for the project R&D. This
demonstrator collects all project R&D results into a single
document, reflecting our online listings of tools, services and
1 • PU = Public
• PP = Restricted to other programme participants (including the Commission Services)
• RE = Restricted to a group specified by the consortium (including the Commission Services)
• CO = Confidential, only for members of the consortium (including the Commission Services))

demos which are kept continually up to date. .
Table of contents
1 The LinkedTV demonstrator of Tools & Services........................... 5
1.1 History of the document...........................................................................................5
2 LinkedTV demos................................................................................ 6
2.1 Scenarios.................................................................................................................7
2.1.1 Linked Cultural Heritage TV.......................................................................7
2.1.2 LinkedTV News application........................................................................8
2.2 Interactive Video Player...........................................................................................9
2.2.1 Multiscreen Toolkit.....................................................................................9
2.2.2 NERD video viewer..................................................................................10
2.2.3 Gesture recognition interface...................................................................10
2.3 Media Analysis.......................................................................................................11
2.3.1 Keyword Extraction Tool..........................................................................11
2.3.2 Shot Segmentation ..................................................................................12
2.3.3 Object Re-detection .................................................................................13
2.3.4 LinkedTV REST Service ..........................................................................13
2.3.5 Face Detection.........................................................................................14
2.4 Media annotation ...................................................................................................14
2.4.1 LinkedTV Editor Tool ...............................................................................14
2.5 Linked media (media interlinking)..........................................................................15
2.5.1 NERD Platform ........................................................................................15
2.5.2 SemiTags.................................................................................................15
2.5.3 Targeted Hypernym Discovery (THD)......................................................16
2.5.4 Metadata Conversion Tool.......................................................................17
2.5.5 Live Topic Generation from Event Streams.............................................18
2.5.6 Tracking and Analyzing The 2013 Italian Election...................................19
2.5.7 Grab your Favorite Video Fragment: Interact with a Kinect and Discover
Enriched Hypervideo................................................................................20
2.5.8 Linked Services Infrastructure .................................................................21

2.6 Personalisation ......................................................................................................22
2.6.1 Content and Concept Filtering Demonstrator ..........................................22
2.6.2 LinkedTV User Model Editor LUME.........................................................23
2.6.3 LinkedTV Semantic Filtering....................................................................23
2.6.4 General Analytics INterceptor (GAIN)......................................................24
2.6.5 EasyMiner................................................................................................25
2.6.6 Context Detection ....................................................................................25
2.6.7 Attention Tracker......................................................................................26
3 LinkedTV tools and services .......................................................... 27
3.1 LinkedTV Platform .................................................................................................28
3.2 Linked media .........................................................................................................29
3.2.1 NERD.......................................................................................................29
3.2.2 SemiTags.................................................................................................30
3.2.3 THD..........................................................................................................30
3.2.4 Linked Hypernyms Dataset......................................................................31
3.2.5 Wikifier Evaluation Framework for GATE ................................................32
3.2.6 Linked Services Infrastructure (LSI).........................................................33
3.3 Personalisation ......................................................................................................34
3.3.1 EasyMiner................................................................................................34
3.3.2 GAIN ........................................................................................................34
3.3.3 LinkedTV User Model Editor (LUME).......................................................35
3.3.4 LinkedTV Semantic Filtering (LSF)..........................................................35
3.3.5 LinkedTV User Model Ontology (LUMO) .................................................36
3.3.6 f-PocketKRHyper .....................................................................................37
3.3.7 Linked Profiler..........................................................................................37
3.4 Media analysis.......................................................................................................38
3.4.1 Mixture Subclass Discriminant Analysis (MSDA).....................................38
3.4.2 Keyword Extraction Tool..........................................................................39
4 Summary and outlook..................................................................... 40

1 The LinkedTV demonstrator of Tools & Services
This deliverable presents the LinkedTV demonstrator of project-implemented Tools and
Services.
LinkedTV is a project implementing innovative technology and services at every step of the
media workflow, in order to enable a seamless interlinking of Web and TV content for the
consumer. This technology needs to be seen and to be communicated, hence besides
implementation also dissemination is important for the project R&D. This demonstrator
collects all project R&D results into a single document, reflecting our online listings of tools,
services and demos which are kept continually up to date: Web demos or videos published at
http://www.linkedtv.eu/demos and public APIs or software projects which are announced at
http://www.linkedtv.eu/demos-materials/tools-and-services/ .
1.1 History of the document
Table 1: History of the document
Date Version Name Comment
09.08.2013 V0.1 Lyndon Nixon Created initial structure
27.09.2013 V0.9 Lyndon Nixon Created deliverable based on all inputs collected
for LinkedTV demos and LinkedTV tools and
services.
03.10.2013 V0.95 Lyndon Nixon Updates from QA: corrected structure & formatting
for better clarity and reference to project work
packages.
07.10.2013 V1.0 Lyndon Nixon Final document

2 LinkedTV demos
In case anyone was wondering what we all have been doing in LinkedTV since two years,
the best way to get a real insight into all the work that has been achieved would be to take a
look at our Online Demos page2… there we proudly list – currently – twenty six demos
which have been produced within this project.
 2 scenario demonstrators, showing the current public demos available for our
Linked Cultural Heritage and Interactive News TV scenarios;
 3 interactive video player demonstrators, with the project’s main toolkit for
multiscreen interactive video, and demos of interactivity enabled in front end players
by named entity recognition (NER) and gesture recognition, respectively;
 5 media analysis demonstrators, showing keyword extraction, shot segmentation,
object re-detection, face detection and the project’s REST service which aggregates
all this functionality plus automatic speech recognition and speaker detection!
 8 media interlinking demonstrators, covering the path from entity extraction to
connecting media based on those respective extracted entities;
 A media annotation demonstrator, where a Web based editor tool allows manual
correction and completion of the automatic results from the above media analysis and
interlinking;
 7 personalisation demonstrators, from manually modeling user preferences to
learning those preferences over time by capturing user behaviour and mining their
system interactions, as well as filtering the LinkedTV annotations and enrichments for
the user based on the resulting user model.
For sure this is not everything that is being done in LinkedTV, simply other tools or services
are not yet mature or stable enough for public demonstration – but they will be! We hope you
enjoy the demos and get a feeling for the rich and innovative functionalities of LinkedTV.
The above news item was published September 27, 2013 on the LinkedTV project
webpage.
The demonstrators are collected into 6 categories, listed below:
 The scenario demonstrators which are the latest online demos from LinkedTV work
package 6 of the Interactive News (RBB) and Hyperlinked Documentary (Sound &
Vision) use cases;
 The video player demonstrators which show LinkedTV client technology for accessing
enriched videos and interacting with the video and its enrichment content (LinkedTV
work packages 3 and 5);
 Media analysis demonstrators (LinkedTV work package 1);
2 http://www.linkedtv.eu/demos

 Media interlinking demonstrators (LinkedTV work package 2);
 Media annotation, focused on the work done on a LinkedTV Editor Tool;
 Media personalisation (LinkedTV work package 4).
2.1 Scenarios
2.1.1 Linked Cultural Heritage TV
Description
This scenario developed by the Netherlands Institute for Sound and Vision is focused on
cultural heritage. It uses material from the program Tussen Kunst & Kitsch (similar to the
BBC’s Antiques Roadshow) courtesy of Dutch public broadcaster AVRO3. In the show,
people take art objects with them to be assessed by an expert. The objects brought in
provide the possibility to add relevant information on questions like Who, When, What,
Where and Subject, all related to art history and cultural heritage. This first demo has been
showcased at several conferences in 2012.
Homepage
Linked Cultural Heritage TV: http://bit.ly/ZxNIQ7
Responsible partner
Beeld en Geluid (Sound and Vision)
Contact person
Lotte Belice Baltussen
3 http://www.avro.nl

2.1.2 LinkedTV News application
Description
LinkedTV News is a second screen application for tablets that acts as a companion to
viewers when watching the news broadcasts. Its main goal is to enrich television newscasts
by integrating them with other media thus, integrating and concentrating the different
activities related to getting informed about the news in one interactive multiple screen, and
potentially mobile experience. It is designed to accommodate two viewing modes in terms of
interaction: a lean back mode and a lean forward mode.
The news application is the basis for the realisation of the LinkedTV scenario “Interactive
News” (with the partner RBB). This is the current prototype UI and is not fully functional.
Homepage
LinkedTV News: http://linkedtv.project.cwi.nl/news/
Responsible partner
CWI
Contact person
Michiel Hildebrand

2.2 Interactive Video Player
2.2.1 Multiscreen Toolkit
Description
The Multiscreen Toolkit enables rapid prototyping of multiscreen applications, allowing
developers and designers to focus on their concept ideas, rather than having to deal with
synchronization and communication between screens. Support and default solutions are
provided for sharing and notifications between screen, and functionalities are available for
different interface options such as touch screens and traditional remotes.
The toolkit is used in LinkedTV for prototyping and implementing a 2nd screen application,
which enables viewing and exploring the enrichments related to a TV program on a
touchscreen tablet. The application also supports social interaction between viewers while
watching a program.
Homepage
Multiscreen toolkit: http://www.noterik.nl/products/mulitscreen_tootlkit_video
Responsible partner
Noterik BV4
Contact person
Daniel Ockeloen
4 http://www.noterik.nl/

2.2.2 NERD video viewer
Description
The NERD video viewer demonstrates the functionality of the NERD tool by performing entity
extraction on a given YouTube or DailyMotion video and showing the results in a Web
interface. Entities are highlighted in the video transcript and are linked to explanatory
information from the Web.
Homepage
NERD video viewer: http://linkedtv.eurecom.fr/nerdviewer/
Responsible partner
EURECOM
Contact
Raphaël Troncy
2.2.3 Gesture recognition interface
Description
A set of predefined gestures can be recognized in the interactive video player through this
gesture recognition interface (play, pause, next, previous, etc.)
Responsible partner
UMONS
Contact person
Matei Mancas

2.3 Media Analysis
2.3.1 Keyword Extraction Tool
Description
This online demo performs keyword extraction on German and Dutch text. The screen is split
in two parts (for German and Dutch respectively). The form enables users to fill a text box
with some text and indicate a file name, and to submit this file for analysis. The file is then
indexed (it may take longer before the file gets uploaded and indexed). After indexing 20 top
keywords extracted from the file are displayed on top of the screen. For the keyword
extraction, the algorithm employs proper Part of Speech taggers for German and Dutch, also
making feasible the identification of key-phrases. Under the form, a list of already uploaded
and indexed files is shown. By clicking on the file name, the keywords extracted from this file
along with the text of the file are displayed. Moreover, the user can update the text of existing
files and re-submit them for analysis. The uploaded files can be deleted by clicking the cross
symbol [X] at the end of each filename. The preloaded documents have been built from
content provided by LinkedTV partners’ RBB (RBB Aktuell) and Sound & Vision (Tussen
Kunst & Kitsch, AVRO).
Homepage
Keyword Extraction Tool: http://ner.vse.cz/KeywordExtraction
The system is free to use subject to user registration, contact Tomas Kliegr.
Responsible partner
University of Economics Prague
Contact person
Ivo Lašek

2.3.2 Shot Segmentation
Description
This video demo presents the results of the shot segmentation algorithm on one video of the
news show scenario (Rundfunk Berlin-Brandenburg’s; RBB Aktuell) and one video of the
documentary scenario (Sound & Vision; Tussen Kunst & Kitsch, AVRO). The objective of this
algorithm is to segment a video into shots, i.e., sequences of consecutive frames captured
without interruption by a single camera, by performing shot boundary detection. The
transition between two successive shots can be abrupt (where, one frame belongs to a shot
and the following frame belongs to the next shot) or gradual (where, two shots are combined
using chromatic, spatial or spatial-chromatic production effects which gradually replace one
shot by another). The algorithm performs both abrupt and gradual transition detection.
However, for the videos of the news show scenario only abrupt transitions have been
considered, since gradual transitions are rarely used. In contrast, for the videos of the
documentary scenario, where the use of production effects (e.g., fade in/out, dissolve, wipe)
is a common approach, gradual transition detection have been performed. The results are
presented in the form of subtitles in the videos, by indicating the starting point of each
detected shot.
The video shot segmentation and concept detection demonstrator was developed
by CERTH-ITI5 as part of the MediaMixer EU FP7 CSA Project (http://www.mediamixer.eu),
using video analysis algorithms developed in LinkedTV. In this demo, the LinkedTV analysis
algorithms are applied to lecture videos, coming from the videolectures.net collection. The
videos are automatically segmented into shots, and then 37 concept detectors are applied to
each shot, revealing the shots’ visual content. These analysis results enable the user to
search by concept and access at the shot level the lecture videos.
Homepage
http://multimedia.iti.gr/mediamixer/demonstrator.html
Responsible partner
CERTH
Contact person
Vasilieos Mezaris
5 http://www.iti.gr/iti/index.html

2.3.3 Object Re-detection
Description
This video demo presents the results of the object re-detection algorithm on a video from the
documentary scenario (Sound & Vision; Tussen Kunst & Kitsch, AVRO). Object re-detection
aims at finding occurrences of specific objects in a single video or a collection of still images
and videos. The algorithm takes as input a picture (query image) of a manually specified
object of interest by the user, who marks this object on one frame of the video with a
bounding box. Then, this picture is compared against consecutive or non-consecutive frames
of the video and the instances of the depicted object are automatically detected and marked
with a bounding box. In this video demo, the detected re-occurrences of the object of interest
are indicated by a green rectangle around them. The object re-detection algorithm is robust
against a range of scale and rotation operations and partial occlusion. However, in some
cases, extremely different viewing conditions (due to major modifications in scale and/or
rotation), under which the object’s re-appearance takes place, lead to significant change of
the visual information, and thus detection failure.
Responsible partner
CERTH
Contact person
Vasilieos Mezaris
2.3.4 LinkedTV REST Service
Description
This web-based REST Service integrates the LinkedTV techniques for audio, visual and
textual analysis of multimedia content. Specifically, this service performs Automatic Speech
Recognition (ASR) and Speaker Identification on the audio channel, Shot Segmentation,
Concept Detection, Object Re-detection and Face Detection and Tracking on the visual
channel and Keyword Extraction on the video’s subtitles or meta-data, or using the output of
the ASR analysis.
Responsible partner
Fraunhofer IAIS
Contact person
Daniel Stein

2.3.5 Face Detection
Description
This video demos present the results of the face detection algorithm applied on a S&V video.
When a face is detected, the algorithm demarcates it with a bounding box. Face detection is
performed by applying Haar-like cascade classifiers, combined with skin color detection, to
every frame of the video sequence. This method performs well on images, and we adapted it
to videos in order to create face tracks: we use spatio-temporal information to link matching
faces, and perform a linear interpolation to smooth the results.
Responsible partner
EURECOM
Contact person
Mathilde Sahuguet
2.4 Media annotation
2.4.1 LinkedTV Editor Tool
Description
The Editor Tool is developed in LinkedTV to allow for visualisation of the annotations and
enrichments generated for a video, and their manual correction and completion within the
Web browser.
Responsible partner
Sound and Vision
Contact person
Jaap Blom

2.5 Linked media (media interlinking)
2.5.1 NERD Platform
Description
NERD aggregates several named entity recognition services into a single API and Web
interface. It is used in LinkedTV to process the annotations generated by the Video Analysis
step and extract named entities which are identified unambiguously using Semantic Web
URIs (Linked Data). In this demo, we show:
 Apply Named Entity recognition on any text, in different languages including Dutch
and German
 Apply Named Entity recognition on timed text, and re-temporal alignment of the
named entity in the video … with a video player showcasing the results
 A personalized dashboard for a logged-in user which enables to monitor his NERD
activity
Homepage
NERD Platform: http://nerd.eurecom.fr/
Responsible partner
EURECOM
Contact person
Raphaël Troncy
2.5.2 SemiTags
Description
SemiTags performs Named Entity Recognition on Dutch and German text. It has been
incorporated into the NERD interface (see above).
Homepage
SemiTags: http://ner.vse.cz/SemiTags/app/index
Responsible partner
Contact person
Ivo Lašek

2.5.3 Targeted Hypernym Discovery (THD)
Description
THD performs Named Entity and Entity Recognition and classification on English, Dutch and
German text and disambiguates the entities to Wikipedia articles. Entities are also assigned
types from Dbpedia6 and YAGO7 ontologies providing semantic interoperability. In addition to
DBpedia and YAGO, the system uses the Linked Hypernyms Dataset8 as the underlying
knowledge base, which makes THD produce results complimentary to those produced by
wikifiers based only on DBpedia or YAGO. A unique feature of THD is the possibility to
extract the type of the entity from live Wikipedia using on-demand hypernym discovery.
Homepage
Targeted Hypernym Discovery: http://entityclassifier.eu/
Screencast
THD Screencast: http://entityclassifier.eu/thd/screencasts/
Responsible partner
Contact person
Tomas Kliegr
6 http://dbpedia.org
7 http://www.mpi-inf.mpg.de/yago-naga/yago/
8 http://ner.vse.cz/datasets/linkedhypernyms

2.5.4 Metadata Conversion Tool
Description
The Metadata Conversion Tool is the primary component to generate the RDF based
semantic descriptions of the media. It uses other components such as NERD (see above) to
process the different legacy metadata it receives (including the outputs of the EXMARaLDA
tool above), and output a RDF description conform to the LinkedTV ontology
(http://www.linkedtv.eu/ontology) where fragments of the annotated video are linked to
Semantic Web URIs (Linked Data). In this demo, we show:
 Automatic conversion into RDF of legacy metadata attached to video content, while
keeping provenance information
 Automatic conversion into RDF of WP1 analysis results performed on this video
content, while keeping provenance information
 Automatic interlinking of common resources with LOD resources
 Automatic push of the resulting metadata in the LinkedTV Platform
 Useful SPARQL queries to show what can then be retrieved
Homepage
Metadata Conversion Tool: http://linkedtv.eurecom.fr/tv2rdf
Responsible partner
EURECOM
Contact person
Raphaël Troncy

2.5.5 Live Topic Generation from Event Streams
Description
Social platforms constantly record streams of heterogeneous data about human’s activities,
feelings, emotions and conversations opening a window to the world in real-time. Trends can
be computed but making sense out of them is an extremely challenging task due to the
heterogeneity of the data and its dynamics making often short-lived phenomena. We develop
a framework which collects microposts shared on social platforms that contain media items
as a result of a query, for example a trending event. It automatically creates different visual
storyboards that reflect what users have shared about this particular event. More precisely it
leverages on: i) visual features from media items for near-deduplication, and ii) textual
features from status updates to interpret, cluster, and visualize media items. The prototype is
publicly available at http://mediafinder.eurecom.fr.
Homepage
Live Topic Generation from Event Streams: http://mediafinder.eurecom.fr/
Responsible partner
EURECOM
Contact person
Raphaël Troncy

2.5.6 Tracking and Analyzing The 2013 Italian Election
Description
Social platforms open a window to what is happening in the world right now: fragmented
pieces of heterogeneous data, such as (micro-)posts and media items, are posted by people
that share their feelings or their activities related to events. Such an information is worth to
be analyzed in order to get the big picture of an event from the crowd point of view. In this
paper, we present a general framework to capture and analyze micro-posts containing media
items relevant to a search term. We describe the results of an experiment that consists in
collecting fresh social media posts (posts containing media items) from numerous social
platforms in order to generate the story of the “2013 Italian Election” from the crowd point of
view. Items are grouped in meaningful time intervals that are further analyzed through
deduplication, clusterization, and visual representation. The final output is a storyboard that
provides a satirical summary of the elections as perceived by the crowd. The system is
publicly available at http://mediafinder.eurecom.fr/story/elezioni2013
Homepage
Tracking and Analyzing The 2013 Italian Election:
http://mediafinder.eurecom.fr/story/elezioni2013
Responsible partner
EURECOM
Contact person
Raphaël Troncy

2.5.7 Grab your Favorite Video Fragment: Interact with a Kinect and Discover
Enriched Hypervideo
Description
In this demonstration, we propose an approach for enriching the user experience when
watching television using a second screen device. The user can control the video program
being watched using a Kinect and can grab, at any time, a fragment from this video. Then,
we perform named entity recognition on the subtitles of this video fragment in order to spot
relevant concepts. Entities are used to gather information from the Linked Open Data cloud
and to discover what the vox populi says about this program. This generates media galleries
that enrich the seed video fragments grabbed by the user who can then navigate this
enriched content on a second screen device.
Responsible partner
EURECOM
Contact person
Raphaël Troncy

2.5.8 Linked Services Infrastructure
Description
LSI makes use of Web APIs for online media platforms such as Flickr or YouTube, defining
mapping rules between the semantic query (in terms of a Linked Data resource) and the
structural API query to the non-semantic Web API. From semantic to structural query is
called “lowering” while the transformation of the structured resource (usually JSON or XML)
to a semantic result (RDF) is called “lifting”. The use of mapping rules means that – provided
the Web API does not change – media can be retrieved from that source repeatedly with the
actual mapping only needing to be defined once. Media resource matches from different
APIs is collected in parallel, while a local store of metadata of media resources relevant to a
known, expected concept has been added to improve retrieval speed. LSI returns a list of
matching media resources in RDF, with additional metadata for each media resource which
could be used in subsequent ranking and selection of ‘most relevant’ matches.
Homepage
Linked Services Infrastructure: http://production.sti2.org/lsi
Responsible partner
STI
Contact person
Lyndon Nixon

2.6 Personalisation
2.6.1 Content and Concept Filtering Demonstrator
Description
This web demonstrator serves as the entry point for the content and concept filtering services
provided by the f-PocketKRHyper reasoned, developed by CERTH-ITI for LinkedTV.
Functionalities supported by this demonstrator include the user creating or updating a
preference profile in a designated ontology formalization, receiving recommended content
from a plurality of content items available to the system based on his/her profile, and
receiving recommended concepts based on the propagation of his/her interests on the
LinkedTV personalization concept space (LUMO). Additionally, the user may review the
content available to the system and upload/update the semantic description of content items.
The web demo is supported by a video presentation of its functionalities.
Homepage
To be announced soon9
Responsible partner
CERTH-ITI
Contact person
Dimitrios Panagiotou, Dorothea Tsatsou, Vasileios Mezaris
9 http://www.linkedtv.eu/demos-materials/online-demos

2.6.2 LinkedTV User Model Editor LUME
Description
The LinkedTV User Model Editor LUME provides an intuitive user interface for end users of
LinkedTV to build and manage their user models. It is implemented as a web application and
accessible over the Web (with authentication). For this purpose it provides RESTful web
services for further integration with other LinkedTV components, in particular with the
LinkedTV video player.
Homepage
LUME: http://data.linkedtv.eu:8081/LUME3-war/user.xhtml
(For the user name and password, please contact Fraunhofer IAIS.)
Responsible partner
Fraunhofer IAIS
Contact person
Rüdiger Klein
2.6.3 LinkedTV Semantic Filtering
Description
LSF is the implementation of the LinkedTV Semantic Filtering. It provides an efficient and
scalable system to filter media fragment annotations (MFA)1 and enriched media content
(eMFA) using personalized user models in a context-sensitive way. The core of LSF is a
graph matching algorithm which correlates the active user interest model (aUIM) and the
(enriched) media fragment annotations (eMFA).
Homepage
LSF: http://data.linkedtv.eu:8081/LUME3-war/user.xhtmlhttp://data.linkedtv.eu:8081/LUME3-
war/user.xhtml - _blank
(For the user name and password, please contact Fraunhofer IAIS.)
Responsible partner
Fraunhofer IAIS
Contact person
Rüdiger Klein

2.6.4 General Analytics INterceptor (GAIN)
Description
GAIN is a stack of web applications and services for capturing and preprocessing user
interactions with semantically described content. GAIN outputs a set of instances in tabular
form (fixed-length vectors) suitable for further processing with generic machine-learning
algorithms.
Within LinkedTV, GAIN is as a component of a “SMART-TV” recommender system. Content
interacted with is automatically described with DBpedia types using a Named Entity
Recognition (NER) system THD, and user interest is determined based on collected interest
clues.
Homepage
General Analytics Interceptor: http://inbeat.eu/demo/
Screencast
Screencast: http://inbeat.eu/screencasts/
Responsible partner
Contact person
Jaroslav Kuchař

2.6.5 EasyMiner
Description
EasyMiner is a web-based rule learning system, producing decision rules and association
rules. Its user interface resembles a web search engine, the user poses a query in the form
of a pattern of a rule. The data are uploaded via a csv file or accessed as a remote database
table. The user can use automatic data preprocessing facility, or define the preprocessing
manually. Easy Miner can work with multi-valued attributes, supports negations and
conjunctions in the rule and multiple interest measures, which can be used as constraints,
including support, confidence, lift and chi-square. The discovered rules can be exported to a
business rules system (GUHA AR PMML or Drools DRL format). EasyMiner has a built-in
reporting.
Homepage
EasyMiner: http://easyminer.eu/
Screencast
Screencast: http://easyminer.eu/screencasts
Responsible partner
Contact person
Tomas Kliegr
2.6.6 Context Detection
Description
Contextual features (the number of people for the moment) are extracted by using a RGBD
camera. Those features are then sent to GAIN through the player server which will identify
the videos ID and video time when the change in the number of people occurs.
Responsible partner
UMONS
Contact person
Matei Mancas

2.6.7 Attention Tracker
Description
The viewer head direction is extracted by using a RGBD camera and it is sent to GAIN
through the player server which will identify the videos ID and video time when the change in
attention occurs. If the viewer is looking towards the TV screen, the approximate coordinates
(+/-10cm) are also sent along with the user ID.
Responsible partner
UMONS
Contact person
Matei Mancas

3 LinkedTV tools and services
The LinkedTV project endeavours to make available to external parties the tools and
services it develops within the research work. While some tools and services are very
specific in their implementation to the LinkedTV workflow and data models, others may be
more immediately re-usable in other contexts. Certainly, the individual tools and services
generally support one specific functionality that is required for LinkedTV, whether it be
performing media analysis, annotation, interlinking, personalisation or management.
Similarly, some tools and services are open source / public license and we can provide
details of the API endpoints or download links for their code; others are subject to the IP of
the respective partner. API or software usage is permitted – at least for non-commercial
purposes – if the link is provided here, please check the documentation for the precise
license under which the tool or service is provided. Other tools and services are listed here
without public links, meaning that partners are willing to grant access when contacted by e-
mail.
LinkedTV lists, at the time of writing, 16 tools and services at http://www.linkedtv.eu/demos-
materials/tools-and-services/, which is not the complete number of tools and services used in
the project – however some must be kept private for now, either due to license restrictions or
because they are using partner data which cannot be made public.
 The LinkedTV Platform provides a Web interface, SPARQL endpoint and Faceted
Search;
 For our Linked Media work, discover tools for supporting the media annotation and
hyperlinking;
 For the personalisation work, several services support user model creation, evolution,
management and application for filtering the media annotations and hyperlinks;
 For media analysis, we can release an algorithm for event detection in video and a
Web service for keyword extraction from text.
The above news item was published September 30, 2013 on the LinkedTV project
webpage.
Public tools and services have been split into four categories: the LinkedTV Platform as a
central integrating service for all other components (work package 5), the Linked Media
components which handle media annotation and hyperlinking (work package 2), the
personalisation components (work package 4) and the media analysis components (work
package 1).

3.1 LinkedTV Platform
Description
The LinkedTV Platform is a RDF Repository (Triple/Quad Store) based on OpenLink Virtuoso
for the permanent storage of all RDF data for annotated media fragments which are
produced by the different LinkedTV components and services. Access to the RDF data is
provided through a direct SPARQL endpoint as well as a custom REST API based on the
Elda Linked Data API (http://code.google.com/p/elda/)
Download / API
http://data.linkedtv.eu/ (Web interface)
http://data.linkedtv.eu/sparql (SPARQL endpoint)
http://data.linkedtv.eu/fct/ (Faceted Search)
The API access and documentation are currently private.
Responsible partner
Condat
Contact person
Jan Thomsen

3.2 Linked media
3.2.1 NERD
Description
A Framework compiling 10 NER extractors including: AlchemyAPI, DBpedia Spotlight,
Extractiv, Lupedia, OpenCalais, Saplo, SemiTags, Wikimeta, Yahoo! Content Analysis
Framework and Zemanta (Evri is no longer available). NERD detects automatically the
language of the text being analyzed. NERD provides an individual dashboard to monitor
logged in user activity.
Download / API
http://nerd.eurecom.fr/api/
Endpoint Documentation
http://nerd.eurecom.fr/documentation
Presentation / tutorial
http://www.slideshare.net/giusepperizzo/nerd-11974343
Responsible partner
EURECOM
Contact
Raphaël Troncy

3.2.2 SemiTags
Description
It is a Named Entity Recognition and Disambiguation Web Service
Download / API
http://ner.vse.cz/SemiTags/rest/v1/recognize
http://ner.vse.cz/SemiTags/app/static/developers
Responsible partner
University of Economics, Prague
Contact person
Ivo Lašek
3.2.3 THD
Description
THD is advanced hypernym discovery tool. Its goal is not to find all hypernyms in the corpus
but rather to find hypernyms for a specific entity. THD also has the advantage of requiring no
training and uses up-to-date on-line resources to find hypernyms in real time.
Download / API
http://entityclassifier.eu/thd/api/v1/extraction
http://www.entityclassifier.eu/thd/docs/
Responsible partner
Contact person
Tomas Kliegr

3.2.4 Linked Hypernyms Dataset
Description
This Linked Hypernym dataset attaches entity articles in English, German and Dutch
Wikipedia with a DBpedia resource or a DBpedia ontology concept as their type. The types
are hypernyms mined from articles’ free text using hand-crafted lexicosyntactic patterns. The
dataset contains 2.8 million entity-type assignments, out of which nearly 2.5 million are novel
with respect to DBpedia and 2 million w.r.t. Yago 2s and DBpedia.
Download / API
http://ner.vse.cz/datasets/linkedhypernyms/
Documentation
http://ner.vse.cz/datasets/linkedhypernyms/dataset-description/
Responsible partner
Contact person
Tomas Kliegr

3.2.5 Wikifier Evaluation Framework for GATE
Description
GATE plugins for evaluation of Wikipedia-based NER systems (also known as Wikifiers),
which provide links to Wikipedia articles describing recognized entities, and classify those
entities with types from the DBpedia Ontology. The plugins come with two evaluation
datasets. The News dataset – consists of a small number of standard-length news articles
freely available from the BBC10 and New York Times11. The dataset is a derivative of the
originally published WEKEX12 dataset. This dataset is released under the Creative Commons
BY-SA 3.013 license.
Download / API
http://ner.vse.cz/datasets/evaluation/
Documentation
http://ner.vse.cz/datasets/evaluation/tools/
Responsible partner
Contact person
Milan Dojchinovski
10 http://www.bbc.com/
11 http://www.nytimes.com/
12 http://nerd.eurecom.fr/ui/evaluation/wekex2011-goldenset.tar.gz
13 http://creativecommons.org/licenses/by-sa/3.0/

3.2.6 Linked Services Infrastructure (LSI)
Description
The REST API of the Linked Services Infrastructure provides a programmatic means to
integrate LSI functionality into other tools and services. The LSI takes a Linked Data URI and
returns a set of relevant media resources from the open Web.
Endpoint
http://production.sti2.org/lsi/api/invoke
http://production.sti2.org/lsi/docs/index.html
Responsible partner
STI International
Contact person
Lyndon Nixon

3.3 Personalisation
3.3.1 EasyMiner
Description
EasyMiner is a web interface to the association rule mining system LISp-Miner. Its user
interface resembles a web search engine. Easy Miner discovers multi-valued attributes,
supports full range of logical connectives and 19 interest measures. An experimental
relevance feedback module is used to filter rules based on previous user interactions.
Download / API
http://sewebar-demo.lmcloud.vse.cz/izi-miner/
http://easyminer.eu/
Presentation / tutorial
http://easyminer.eu/feature-tutorial
Responsible partner
Contact person
Tomas Kliegr
3.3.2 GAIN
Description
Framework for capturing and preprocessing user interaction and implicit feedback
Endpoint
API access is not yet public
http://inbeat.eu/gain/docs/
Responsible partner
Contact person
Tomas Kliegr

3.3.3 LinkedTV User Model Editor (LUME)
Description
The User Model Editor provides a RESTful API for further integration with other LinkedTV
components, in particular with the LinkedTV video player.
Endpoint
http://data.linkedtv.eu:8081/LUME3-war/service
Responsible partner
Fraunhofer IAIS
Contact person
Rüdiger Klein
3.3.4 LinkedTV Semantic Filtering (LSF)
Description
The LinkedTV Semantic Filter provides a RESTful API for further integration with other
LinkedTV components, in particular with the LinkedTV video player.
Endpoint
http://data.linkedtv.eu:8081/LUME3-war/service
Responsible partner
Fraunhofer IAIS
Contact person
Rüdiger Klein

3.3.5 LinkedTV User Model Ontology (LUMO)
Description
The LUMO ontology is developed within the LinkedTV project to semantically represent user-
pertinent information, in order to enable semantic personalization and contextualization (i.e.
filtering) of concepts and content in the networked media domain. LUMO is currently
accompanied by a separate mappings ontology, modelling mappings of LUMO to several
existing ontologies and vocabularies.
Homepage
http://mklab.iti.gr/project/lumo
Download / API
LUMO: http://data.linkedtv.eu/ontologies/lumo
LUMO mappings: http://data.linkedtv.eu/ontologies/lumo_mappings/
LUMO: http://data.linkedtv.eu/ontologies/lumo/lumo_doc/index.html
LUMO mappings:
http://data.linkedtv.eu/ontologies/lumo_mappings/lumo_mappings_doc/index.html
Responsible partner
CERTH-ITI
Contact person
Dorothea Tsatsou, Vasileios Mezaris

3.3.6 f-PocketKRHyper
Description
f-PocketKRHyper is a general purpose fuzzy semantic reasoner, which for the purposes of
LinkedTV performs content filtering by semantically matching a user profile against a set of
available content items, and concept filtering by expanding the interests of a user along the
LinkedTV personalization concept space (LUMO).
Download / API
http://mklab.iti.gr/project/f-pocketkrhyper
http://mklab.iti.gr/project/f-pocketkrhyper
Responsible partner
CERTH-ITI
Contact person
3.3.7 Linked Profiler
Description
Linked Profiler is a semantic profile engineering API, responsible for receiving input by
captured or explicitly declared user interests and (a) unifying it under a common
personalisation-centric vocabulary LUMO and, based on the uniform input (b) learning a user
profile over time and content consumption through its subcomponent Simple Learner. It
ultimately produces a fuzzy semantic user profile that can be received by the LinkedTV
filtering tools.
Download / API
http://mklab.iti.gr/project/linkedtv-profiling-tools
http://mklab.iti.gr/project/linkedtv-profiling-tools
Responsible partner
CERTH-ITI
Contact person

3.4 Media analysis
3.4.1 Mixture Subclass Discriminant Analysis (MSDA)
Description
MSDA is a supervised dimensionality reduction technique that overcomes the nonlinearity
problem of LDA by deriving a subclass data division and the respective lower dimensional
subspace suitable for sample discrimination. In the particular implementation the negentopy
increment is used to implement a nongaussianity criterion for deriving the best subclass
partition. Moreover, the nearest neighbor method is used in the derived subspace for
classifying unlabeled observations. This method is generic and can be used in a very wide
range of learning problems; in LinkedTV, it is used primarily as part of our solutions to the
event detection problem.
Download / API
http://www.iti.gr/~bmezaris/downloads.html
http://www.iti.gr/~bmezaris/downloads.html
Responsible partner
CERTH-ITI
Contact person
Nikolaos Gkalelis, Vasileios Mezaris

3.4.2 Keyword Extraction Tool
Description
Keyword extraction is a Web application and Web service, which performs keyword
extraction from English, German and Dutch text. The system is based on the standard TF-
IDF approach, which is extended with language-specific POS tagging and noun-phrase
chunking for identification of key phrases.
Download / API
https://ner.vse.cz/KeywordExtraction/rest/v2
https://ner.vse.cz/KeywordExtraction/developers.html
Responsible partner
UEP
Contact person
Tomas Kliegr

4 Summary and outlook
This deliverable lists all LinkedTV demos and LinkedTV tools and services publicly
announced via the project website. Both demos and tools & services are grouped by type of
functionality, using the technical work packages 1-5 as a basis for the grouping. The
LinkedTV Platform is the integrating layer for all server-side components. An overview of the
entire ecosystem of LinkedTV tools and services has been created in the form of an
illustration (below), which is introduced and explained in the deliverable D5.4 Final
Integration Platform. Apart from those tools and services which cannot be made public due to
licensing or copyright restrictions, LinkedTV has taken care to list information and links for all
tools and services developed and used within the project online at its website. This
deliverable provides a written recording of the status of those online lists as at the month 24
milestone.
Figure 1: The LinkedTV Ecosystem (from Deliverable 5.4)

We also must acknowledge that there are LinkedTV results which cannot be immediately
made public, either due to existing licensing restrictions on (part of the) code being used or
due to limitations to public dissemination requested by the holders of the copyright on seed
video content (RBB, or AVRO in the Sound & Vision scenario). Addressing the former issue,
we have sought to provide public information in the form of videos or presentations for tools
and services which cannot be publicly made available as software downloads or public APIs.
Thus interested persons can still learn about the LinkedTV result and contact the responsible
persons if they are interested in access, according to the applied software license or API
terms & conditions. In the latter, we encourage LinkedTV partners to also ensure the
availability of results using open video contents which can be publicly disseminated online,
while appreciating that for the purposes of the LinkedTV work the implementation using RBB
or AVRO content takes precedence. As reflected in the table below, all LinkedTV
functionalities are now well presented to interested external organisations, with a number of
“live” online demos complementing otherwise “passive” information (video, slidesets). The
number of public tools and services is slightly lower, with some still needing to be kept
private for now. They are however publicly disseminated by pre-recorded demonstrators and
can also be potentially used for live demos at events.
Category LinkedTV WP
Public tools &
services
Interactive demos
(e.g. Web UI based)
Passive demos
(e.g. video
recording, slides)
LinkedTV
Platform &
Player
3,5 3 1 2
Media analysis 1 2 2 5
Media
annotation and
hyperlinking
2 6 7 5
Personal-
isation
4 6 4 5
Scenarios 6 N/A 1 1

Project demonstrator

More Related Content

What's hot (8)

Similar to Project demonstrator (20)

More from LinkedTV (13)

Recently uploaded (20)

Project demonstrator