A pragmatic view on Semantic Technologies

Invited Talk:
A pragmatic view on Semantic Technologies
Roberto García, Universitat de Lleida, Spain
International Semantic Intelligence Conference (ISIC 2021)
New Delhi, India - February 25, 2021

Presentation https://rhizomik.net/~roberto
International Semantic Intelligence Conference, ISIC 2021 New Delhi, India February 25-27,2021 2
https://en.wikipedia.org/wiki/Eating_your_own_dog_food

Motivation & Outline
• Illustrate that even
“a little semantics goes a long way”
James Hendler, circa 1997
https://www.cs.rpi.edu/~hendler/LittleSemanticsWeb.html
• Do so through applications using semantic technologies
I have participated in:
• Game of Thrones
• Example project I use in the classroom
• MedISys Plant Health Threats
• Media Monitoring project for the European Food Safety Authority
• InVID Social Media Verification
• European research project about media verification and reuse for
journalistic purposes

Example 1: Classroom Project
• Example project to show what is expected from
the project they should deliver at the end
• Motivation:
application that supports readers of Game of
Thrones books (especially those that have seen
the TV series)
• Characters, houses they are loyal too, books they appear
in, picture showing series actor playing the character,...
• Added value by using semantic technologies:
• Reduced cost by integrating multiple existing data sources
• CSV, SPARQL, Web pages,…
• Facilitate the development of apps that allow exploring the
data
• Ease conceptualisation and maintenance by reusing
existing vocabularies (ontologies)

Reuse Existing Data
• Kaggle dataset (https://www.kaggle.com/mylesoneill/game-of-thrones)
• character-deaths.csv
• Structure: name, allegiance, death year,...
nobility (1: true | 0: false), appear in book (1: true | 0: false)

RobertBaratheon
allegiances
deathYear
1
bookOfDeath
47
4
deathChapter
1
appearsInBook5
0
0
0
0
appearsInBook4
appearsInBook3
appearsInBook2
appearsInBook1
Reuse Existing Data
• Structure overview:
1
gender
bookIntroChapter
1
nobility
298
House Baratheon
image

Vocabularies Reuse
• Ontology Design
• Reuse: FOAF, DBPedia Ontology, OWL, RDFS,...
dbo:Person
dbo:Noble
rdfs:subClassOf
dbo:Book
dbo:FictionalCharacter
dbp:gender
dbo:deathDate
dbo:lastAppearance
dbo:allegiance
foaf:Image
foaf:Person
foaf:name
foaf:depiction
owl:equivalentClass
xsd:string
xsd:date
appearsIn
appearsNotIn
bookIntroChapter
deathChapter
xsd:string
xsd:integer
domain
domain
domain
domain
domain
domain
domain
range
range
range
range
range
range
range

RobertBaratheon
bookOfDeath
47
4
deathChapter
1
dbr:A_Game_of_Thrones
gender
bookIntroChapter
dbr:A_Clash_of_Kings
dbr:A_Storm_of_Swords
dbr:A_Feast_for_Crows
appearsNotIn
appearsIn
Data & Vocabularies Reuse
• Make some semantics explicit:
1
nobility
appearsNotIn
dbr:A_Game_of_Thrones deathYear
dbr:A_Dance_with_Dragons
allegiances
298
House Baratheon
appearsNotIn
appearsNotIn
image

RobertBaratheon
bookOfDeath
47
4
deathChapter
Male
dbp:gender
bookIntroChapter
dbr:A_Clash_of_Kings
dbr:A_Storm_of_Swords
dbr:A_Feast_for_Crows
appearsNotIn
appearsIn
Data & Vocabularies Reuse
• Make some semantics explicit:
rdf:type
appearsNotIn
appearsNotIn
appearsNotIn
deathYear
dbr:A_Dance_with_Dragons
allegiances
298
House Baratheon
image
dbo:Noble

SPARQL WHERE
SPARQL CONSTRUCT

Semantic ETL
• Semantics-powered ETL (Extract-Transform-Load)
• SPARQL CONSTRUCT
PREFIX dbo: <http://dbpedia.org/ontology/>
PREFIX dbr: <http://dbpedia.org/resource/>
PREFIX dbp: <http://dbpedia.org/property/>
PREFIX foaf: <http://xmlns.com/foaf/0.1/>
PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>
CONSTRUCT {
?character a dbo:FictionalCharacter ;
foaf:title ?title ;
rdfs:comment ?comment .
}
WHERE {
?character a dbo:FictionalCharacter ;
dbo:series dbr:Game_of_Thrones ;
dbp:title ?title ;
rdfs:comment ?comment .
FILTER(LANG(?comment) IN ("en", "es"))
}
http://yasgui.org/short/B1IFUPXBG

RobertBaratheon
“298”^^xsd:gYear
dbo:deathDate
dbo:lastAppearance
47
4
deathChapter
bookIntroChapter
Automatic Semantic Integration
rdf:type
House Baratheon
“Robert Baratheon”@en
foaf:name
rdf:type
dbo:FictionalCharacter
dbr:Robert_Baratheon
owl:sameAs
dbo:allegiance
rdf:type
dbo:Noble
foaf:depiction
“Protector of the Realm”
foaf:title
“…es un personaje ficticio…”@es
rdfs:comment
“…is a fictional character…”@en

Tabular to Semantic Data
• Generate unique identifiers as URIs
• Independent from data source
• Table to RDF triples (subject –predicate object/literal)
• Rows
correspond to the same subject identified by URI
• Columns
correspond to subject properties
• Cells
correspond to objects (relationships) or literals (attributes)
• objets, sujects, and properties replace tabular value with URI
• Jon Snow  http://mydomain.org/persons/Jon_Snow
• literales, text and make data type explicit if available
• 299  "299"^^<http://www.w3.org/2001/XMLSchema#gYear>

Tabular to Semantic Data
• Example for one row:
@PREFIX : <http://mydomain.org/persons/> .
@PREFIX families: <http://mydomain.org/families/> .
@PREFIX got: <http://mydomain.org/got-ontology/> .
@PREFIX foaf: <http://xmlns.com/foaf/0.1/> .
@PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> .
@PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> .
@PREFIX dbo: <http://dbpedia.org/ontology/> .
@PREFIX dbp: <http://dbpedia.org/property/> .
@PREFIX dbr: <http://dbpedia.org/resource/> .
...
:Robert_Baratheon rdf:type dbo:Nobel, dbo:FictionalCharacter ;
rdfs:label "Robert Baratheon"@en ;
foaf:name "Robert Baratheon"@en ;
dbo:allegiance families:House_Baratheon ;
dbp:genre "Male" ;
got:deathChapter "47"^^<http://www.w3.org/2001/XMLSchema#integer> ;
owl:sameAs dbr:Robert_Baratheon ;
...

Automate Transformation
• Many tools Tabular or Relational DB to RDF
• For tabular: OpenRefine

Additional Sources
• Character’s pictures https://www.hbo.com/game-of-thrones/cast-and-crew
• Get picture, plus characters’ names to reconcile with
previous RDF data using OpenRefine
• Automatic integration using same URI for characters

Final Result
• Automatic User Interface
to explore the data
• Generation driven by
data structure
https://rhizomer.rhizomik.net/datasets/got

Example 2: EFSA Project
• Multilingual Ontology for Plant Health Threats Media Monitoring
• Development and testing of the media monitoring tool MedISys for
the early identification and reporting of existing and emerging
plant health threats
• Timing (duration): January 2014 – June 2016 (2.5 years)
• Funding: EFSA
• Objectives:
• Collate new and appropriate media information sources
• Multilingual ontology for the global identification of emerging new
plant health threats to be appended to MedISys
• English, Spanish, Italian, French, Dutch, German, Portuguese, Russian, Chinese and Arabic
• Develop and test strategies to monitor re-emerging plant health
threats on global and regional scale

Proposed Approach
• Approach based on the use of semantics and ontologies
• Ontology: key component of the developed system that structures and
provides knowledge about plant health threats
• Knowledge captured from existing sources and experts
• Guides applications for
• Knowledge capture
• Indirect sources search
• Terms translation
• Media monitoring
categories generation
An ontology is a formal, explicit specification of a shared conceptualisation.
is
means
implies expressed in
terms of
Abstract model of
portion of world
Machine-readable
and understandable
Based on a
consensus
Concepts,
properties,...

Ontology Generation
• Ontology Skeleton
• Collected 140 pests/diseases from EPPO Alerts, 2000/29-1-
A-1 and EU Emergency Control Measures
• 117 linked to UniProt Taxonomy:
• Taxonomical information, scientific/common/other names,…
• 47 linked also to Wikipedia
• Common names in
multiple languages

Ontology Generation
• Plant Health Threats Ontology
• Enrich ontology with affected crops, hosts, vectors, symptoms
expressions…

Ontology Enrichment
• Plant Health Threats Ontology
• All concepts linked to labels in different languages
• Extract as keywords for MedISys or Web search filters,…
• Example: “Maladie de Pierce” OR ( “grapevine” AND
“sharpshooter” )
Xylella fastidiosa
Gammaproteobacteria
Nerium oleander,
Prunus salicina, Medicago
sp., Sorghum halepense,…
Homalodisca coagulata,
Graphocephala sp.,
Oncometopia sp.,
Draeculacephala sp.,…
Grapevine, Citrus, Olive,
Almond, Peach, Coffee,…
subClassOf
vector
host
crop
“Pierce's disease”, “Citrus
variegated chlorosis” en
“Maladie de Pierce” fr
“葉緣焦枯病菌” zn
“Glassy-winged sharpshooter”,
“Spittlebugs”, “Froghoppers”,
“Planthoppers”,… en
“vite” it,… …

Ontology Enrichment
• Ontology Editor
• Assist experts during the knowledge capture process
23
International Semantic Intelligence Conference, ISIC 2021 New Delhi, India February 25-27,2021

Assisted Knowledge Capture
• Ontology Editor – forms with assistance

Assisted Knowledge Capture
• Ontology Editor - autocomplete

Example 3: InVID Project
• H2020 project InVID, In Video Veritas
• Verification of Social Media Video Content for the News
Industry
• https://www.invid-project.eu
• Reuse of User Generated Video from Social Media for
journalistic purposes
• Discovering social media about current events
• Video verification to avoid fake news
• Request reuse, check licensing, negotiate terms, sign
agreements,… even economic compensation

Objectives
• Sophisticated models for copyright information:
• Rights status
• Reuse terms
• Negotiation
• Copyright agreements
• Trust and confidence on rights statements
• Potentially legally binding
(time stamp, signatures, tamper proof,…)
• Proposed approach:
• Semantic Web: rich information modelling and reasoning
• Blockchain: immutable and accountable information storage

Copyright Ontology
• Copyright knowledge representation
• Copyright Ontology based on the
fundamental ontological distinctions:
• Abstract: intangible
• Process: happens,
temporal stages
(action, event,…)
• Object: can be defined
independent of time
(includes digital objects)
Victor Hugo’s
Les Misérables
Abstract
Objects
Processes

Copyright Ontology
• Also capture the dynamic parts of
the copyright value chain
• Actions performed
by value chain participants
• Plus consumer actions:
• Buy, Attend, Access,
Play, Tune,…
• Plus licensing actions:
• Agree/Disagree
• Transfer, Attribute,…
Victor Hugo’s
Les Misérables
Creator Actor Producer Broadcaster User
Motion Picture
Script
Adaptation Performance
manifest perform record
Communication
broadcast
adapt
Literary Work
tune
29

Copyright Ontology
• Model full details of an action,
its dimensions like as a verb in
a sentence (roles):
• who performs it,
what is manipulated,
when, where…
Role Kind Main Role Description
who schema:agent
The direct performer or driver of the
action (animate or inanimate)
schema:participant
Other co-agents that participated in the
action indirectly, for instance a recipient
what schema:object
The object upon which the action is
carried out
schema:result The result produced in the action
where schema:location Where an action takes place
schema:fromLocation
The original location of the object or the
agent before the action
schema:toLocation
The final location of the object or the
agent after the action
when schema:startTime
When the action started or the time it is
expected to start
schema:endTime
When the action finished or the time it is
expected to end
pointInTime
The point in time when the action
happens
duration
The amount of time the action requires to
complete
with schema:instrument
The object that helps the agent perform
the action
why aim The reason or objective of the action
how manner The way the action is carried out
if condition
Something that must hold or happen
before the action starts
then consequence
Something that must hold or happen after
the action is completed
Who?
What?
When? Where?
30

Check UGV Rights Status
https://rights.invid.udl.cat
31

Request Reuse
Current video, plus
all future YouTube videos by content owner or
in any social network linked to InVID profile
32

License Reasoning
• Streamline licensing
• License to organisation or everyone
• License future videos
• Semantic representation
of agreements
• Semantic queries to check
previous agreements
• Including territories,
timeframes
or revocations
InVID Rights
Management
Rights
Database
JSON-LD
Semantic
Repository
Semantic
Copyright
Management
33

Store Agreement
• JSON-LD serialisation of a
Reuse Agreement:
• Grants any member of the Daily Planet
permission to republish a YouTube
video whose owner is the Google user
{
"@context": {
"@vocab": "http://invid.udl.cat/ontology/",
"cro": "http://rhizomik.net/ontologies/copyrightonto.owl#",
"schema": "http://schema.org/"
},
"@id": "…/reuseAgreements/1", "@type": "cro:Agree",
"cro:when": "2019-02-16T15:15:00Z",
"cro:who": [
{
"@id": ”…/inVIDUsers/1", "@type": "schema:Person",
"schema:name": "Clark Kent",
"schema:email": "journalist@invid-project.eu",
"schema:memberOf": {
"@id": "…/organizations/1",
"@type": "schema:Organization",
"schema:name": "Daily Planet"
}
},
{
"@id": "…/contentOwners/1", "@type": "schema:Person",
"username": "user”, "schema:email": "user@gmail.com",
"schema:name": "Google User"
} ],
"cro:what": {
"@id": "…/reuseTerms/1", "@type": "cro:MakeAvailable",
"schema:startTime": "2019-03-01T10:44:00Z",
"schema:endTime": "2019-05-01T10:44:00Z",
"cro:who": { "@id": "…/organizations/1" },
"cro:what": {
"@id": "…/youTubeVideos/_5l7vn1QdKM", "@type": "YouTubeVideo",
"user": {
"@id": "…/youTubeChannels/MyChannel", "@type": "YouTubeChannel",
"contactURL": "http://www.youtube.com/channel/MyChannel/about",
"contentOwner": { "@id": "…/contentOwners/1" }
}
}
}
}
34

License Reasoning

SPARQL for License Reasoning
• SPARQL standard for semantic
queries
• Check intended reuse against
existing agreements
• Encapsulate complexities minimising
implementation cost
• Flexibility and scalability
• Example:
• Active agreements, not disagreed,
with Make Available term
• what: restricted to the YouTube
video _5l7vn1QdKM
• startTime: 2019-11-15
• who: is InVIDUser 2, Organization 1
or any organization InVIDUser 2 is a
member of
• where: Spain or a region Spain is
contained in
PREFIX …
SELECT DISTINCT ?isAuthorized ?why
WHERE {
?agree rdf:type cro:Agree ;
cro:what ?term ; cro:when ?agreeDate
FILTER ( xsd:dateTime( ?agreeDate) <= now() )
OPTIONAL {
?disagree rdf:type cro:Disagree ;
cro:what ?term ; cro:when ?disagreeDate
FILTER ( xsd:dateTime( ?disagreeDate) <= now() )
}
BIND((bound(?agree) && (!bound(?disagree))) AS ?isAuthorized)
BIND(if(bound( ?disagree), ?disagree, ?agree) AS ?why)
?term rdf:type cro:MakeAvailable .
?term cro:what <…/youTubeVideos/_5l7vn1QdKM> .
?term schema:startTime ?start FILTER ("2019-11-15" >= ?start)
…
{
{ ?term cro:who <…/inVIDUsers/2>}
UNION
{ ?term cro:who <…/organizations/1>}
UNION
{ ?term cro:who ?organization .
<…/inVIDUsers/2> schema:memberOf ?organization }
}
{
{ ?term cro:where "Spain"}
UNION
{ ?term cro:where ?regionName .
?region rdfs:label ?regionName .
?country rdfs:label "Spain" .
?country (schema:containedInPlace)+ ?region
}
}
}
36

Trustful Agreements
• Use Ethereum Smart Contracts
• Blockchain as a global shared computer
• Immutable transactions (executed in all nodes)
• Encode rules guaranteed to execute
• Smart contract keeps track of semantic agreements
• Participants digitally sign negotiation steps, last by both (agreement)
• Identity management using uPort mobile app
• Self-Sovereign Identities (e.g. email attestations)
• Transaction signing: scan QR code
• Optional: remuneration using cryptocurrency wallet
https://www.uport.me
Rights
Database
InVID Rights
Management
Distributed
Ledger
Agreement
Time Stamping
Accountability
Auditability
Tamper Proof
Identify
Attestations

Thank you for your attention
Questions?
More details:
http://rhizomik.net/~roberto
?

A pragmatic view on Semantic Technologies

More Related Content

What's hot (20)

Similar to A pragmatic view on Semantic Technologies (20)

More from Roberto García (20)

Recently uploaded (20)

A pragmatic view on Semantic Technologies

Editor's Notes