Different Semantic Perspectives for Question Answering Systems

NLP & Semantic Computing Group
N L P
Different Semantic Perspectives for
Hybrid Question Answering Systems
Andre Freitas
University of Passau
OKBQA, Jeju, 2016

http://www.slideshare.net/andrenfreitas
These slides:

Outline
 Multiple Perspectives of Semantic Representation
 Lightweight Semantic Representation
 Knowledge Graph Extraction from Text
 Answering Queries with
Knowledge Graphs
 Reasoning
 Take-away Message

Multiple Perspectives of
Semantic Representation

QA & Semantics
• Question Answering is about managing semantic
representation, extraction, selection
trade-offs.
• And it is about integrating multiple components
in a complex approach.
• Semantic best-effort, systems tolerant to
noisy, inconsistent, vague, data.
5

 “Most semantic models have dealt with particular types of
constructions, and have been carried out under very simplifying
assumptions, in true lab conditions.”
 “If these idealizations are removed it is not clear at all that modern
semantics can give a full account of all but the simplest
models/statements.”
Formal World Real World
Baroni et al. 2013
Semantics for a Complex World
6

Why Not RDF?
• Follows a more “database-type” of
representation perspective.
• Gap towards representing text.

Choices of Semantic Representation
• Logical
• Frames: verbs | nouns
• Binary relations: binary | n-ary
• Named entities
• Language Models
• Syntactic structures
• Bag-of-words
Concept-level
representation
Background
knowledge
Extraction
complexity
8

Information Extraction
• Logical
• Frames:
 verbs | nouns
• Binary relations:
 binary | n-ary
• Named entities
• Syntactic Structures & LMs
• Bag-of-words
• Semantic parsing
• Semantic role labeling
• Relation extraction:
– closed/open
• Named entity recognition
• Syntactic/N-gram Parsing
• Indexing9

Information Extraction
• Logical
• Frames:
 verbs | nouns
• Binary relations:
 binary | n-ary
• Named entities
• Syntactic Structures & LMs
• Bag-of-words
• Semantic parsing
• Semantic role labeling
• Relation extraction:
– closed/open
• Named entity recognition
• Syntactic/N-gram Parsing
• Indexing
Use all of them!

Representation focal points
• Types of knowledge to focus at the
representation:
 Facts vs Definitions vs Opinions
 Temporality
 Spatiality
 Modality
 Polarity
 Rhetorical structures
 …
11

Lightweight Semantic
Representation

Objective
• Provide a lightweight knowledge
representation model which:
 Can represent textual discourse information.
• Maximizes the capture of textual
information.
 Is convenient to extract from text.
 Is convenient to access (query and browse).
13

Lightweight Semantic Representation
Representing Texts as Contextualized Entity-Centric
Linked Data Graphs, WebS 2013

Representation Assumptions
• Data integration:
 Named entities (instances)
 Abstract classes (unary predicates)
• Rich taxonomical structures.
• Context representation as a first class citizen.
• Open vocabulary.
• Word instead of sense/concept.
15

16

Representation of Complex Relations
General Electric Company, or GE , is an American multinational conglomerate
corporation incorporated in Schenectady , New York
17

Data Integration points
Named entities are lower
entropy integration points
Pivot
points18

Named entities are also
low entropy entry points
for answering queries
Pivot
points19

Also abstract classes …
Pivot
points20

They are also a very
convenient way to
represent.
Pivot
points21

22

Taxonomy Extraction
 Are predicates with more complex compositional patterns which
describe sets.
 Parsing complex nominals.
American multinational conglomerate corporation
On the Semantic Representation and Extraction of Complex
Category Descriptors, NLDB 2014
multinational conglomerate corporation
corporation
conglomerate corporation
is a
is a
is a
Pivot
points

24

Context Representation
Reification as a first class
representation element
25

Context Representation
Temporality, spatiality,
modality, rhetorical relations
…
26

Rhetorical Structures using Reification
• cause:
 e.g. “because scraping the bottom with a metal utensil will scratch
the surface.”
• circumstance
 e.g. “After completing your operating system reinstallation,”
• concession
 e.g. “Although the hotel is situated adjacent to a beach,”
• condition
 e.g. “If you can break the $ 1000 dollar investment range,”
• contrast
 e.g. “but you can do better with 2.4ghz or 900mhz phones.”
• purpose
 e.g.“in order for the rear passengers to get in the vehicle.”
• …
27

28

Open Vocabulary
Temporality, spatiality,
modality, rhetorical relations
…
29

Open Vocabulary
• Easier to extract but difficult to consume.
• We pay the price at query time.
• How to operate over a large-scale semantically
heterogeneous knowledge-graphs?
30

31

Words instead of Senses
• Motivation: Disambiguation is a tough problem.
• Sense granularity can be, at many situations,
arbitrary (too context dependent).
• We treat a word as a superposition of senses,
almost in a “quantum mechanical sense”.
32

SenseSuperposition
Coecke et al. (2010): Category theory and
Lambek calculus.

Revisited RDF (for Representing Texts)
• Data Model Types: Instance, Class, Property…
• RDFS: Taxonomic representation.
• Reification for contextual relations (subordinations).
• Blank nodes for n-ary relations.
• Triple.
• Labels over URIs.
34

Abstract Meaning Representations – AMR,
Maximal Use of PropBank Frame Files
Alternative Representations

Distributional
Semantics

Distributional Semantic Models
 Semantic Model with low acquisition effort
(automatically built from text)
Simplification of the representation
 Enables the construction of comprehensive
commonsense/semantic KBs
 What is the cost?
Some level of noise
(semantic best-effort)
Limited semantic model
37

Distributional Semantics as Commonsense Knowledge
Commonsense is here
θ
car
dog
cat
bark
run
leash
Semantic Approximation is here
38

I find it rather odd that people are already trying to tie the
Commission's hands in relation to the proposal for a
directive, while at the same calling on it to present a Green
Paper on the current situation with regard to optional and
supplementary health insurance schemes.
I find it a little strange to now obliging the Commission to a
motion for a resolution and to ask him at the same time to
draw up a Green Paper on the current state of voluntary
insurance and supplementary sickness insurance.
=?
Beyond Single Word Vector Models:
Compositionality

Compositional Semantics
 Can we extend DS to account for the meaning of
phrases and sentences?
 Compositionality: The meaning of a complex
expression is a function of the meaning of its
constituent parts.

Compositional Semantics
Words in which the
meaning is directly
determined by their
distributional behaviour
(e.g., nouns).
Words that act as functions
transforming the
distributional profile of
other words (e.g., verbs,
adjectives, …).
41

Compositional-Distributional Semantics

Recursive Neural Networks for
Structure Prediction
43

New Model: Recursive Neural Tensor Network
• Goal: Function that composes two vectors.
• More expressive than any other RNN so far.
44 Socher et al.

Socher et al.

Compositional-distributional model for
Categories
46

Embedding
Knowledge Graphs
47

The vector space is
segmented48
Dimensional reduction
mechanism!
A Distributional Structured Semantic Space for
Querying RDF Graph Data, IJSC 2012

Compositional-distributional model for
paraphrases
A Compositional-Distributional Semantic Model for
Searching Complex Entity Categories, *SEM (2016)

Knowledge Graph Extraction from Text

Graphene

Graph Extraction Pipeline
Text
Transformation
N-ary
Relation
Extraction
Text
Simplification
Graph
Serialization
Taxonomy
Extraction
Storage
RST
Classification
ML-based
Rule-based
Rule-based
ML-based
52

Minimalistic Text Transformations
Text
Transformation
N-ary
Relation
Extraction
Text
Simplification
Graph
Serialization
Taxonomy
Extraction
Storage
RST
Classification
ML-based
Rule-based
Rule-based
ML-based
53

Minimalistic Text Transformations
• Co-reference Resolution
 Pronominal co-references.
• Passive
 We have been approached by the investment banker.
 The investment banker approached us.
• Genitive modifier
 Malaysia's crude palm oil output is estimated to have
risen.
 The crude palm oil output of Malasia is estimated to
have risen. 54

Text Simplification
Text
Transformation
N-ary
Relation
Extraction
Text
Simplification
Graph
Serialization
Taxonomy
Extraction
Storage
RST
Classification
ML-based
Rule-based
Rule-based
ML-based
55

Text Simplification for KG
Extraction
“Defeating Republican nominee Mitt Romney,
Obama, who was the first African American to hold
the office, was reelected president in November
2012.”
 relations are spread across clauses
 relations are presented in non-canonical form
56

Text Simplification for KG
Extraction
• Insertion of a text simplification step
 Obama was reelected president in November 2012.
 Obama was the first African American to hold the office.
 Obama was defeating Mitt Romney.
 Mitt Romney was Republican nominee.
57

Syntax-driven sentence
simplification approach
Task:
• Reduce the linguistic complexity of a text while retaining the
original information/meaning using a set of syntax-based
rewrite operations (deletion, insertion, reordering, sentence
splitting).
Idea:
• Simplify a sentence by separating out components that supply
only secondary information into simpler stand-alone context
sentences, thus yielding one or more reduced core sentences.

Approach
• Linguistic analysis of sentences from the English Wikipedia
to identify constructs which provide only secondary information:
• non-restrictive relative clauses
• non-restrictive and restrictive appositive phrases
• participial phrases offset by commas
• adjective and adverb phrases delimited by punctuation
• particular prepositional phrases
• lead noun phrases
• intra-sentential attributions
• parentheticals
• conjoined clauses with specific features
• particular punctuation
•Rule-based simplification rules.
Improving Relation Extraction by Syntax-based
Sentence Simplification (2016)

N-ary Relation Extraction
Text
Transformation
N-ary
Relation
Extraction
Text
Simplification
Graph
Serialization
Taxonomy
Extraction
Storage
RST
Classification
Rule-based
Rule-based
ML-based
60
OpenIE, University of Washington

Taxonomy Extraction
Text
Transformation
N-ary
Relation
Extraction
Text
Simplification
Graph
Serialization
Taxonomy
Extraction
Storage
RST
Classification
Rule-based
Rule-based
ML-based
61
Representation and Extraction of Complex
Category Descriptors, NLDB 2014

RST Classification
Text
Transformation
N-ary
Relation
Extraction
Text
Simplification
Graph
Serialization
Taxonomy
Extraction
Storage
RST
Classification
Rule-based
Rule-based
ML-based
62

Rhetorical Structure Extraction
63
TEXT-LEVEL RST-STYLE DISCOURSE PARSER (Feng and Hirst, 2012)
Structure classification Relation classification

Answering Queries with
Knowledge Graphs

Now our graph supports semantic
approximations as a first-class operation

Approach Overview
Query Planner
Ƭ-Space
(embedding graphs)
Wikipedia
Commonsense
knowledge
RDF
Explicit Semantic
Analysis
Core semantic approximation &
composition operations
Query AnalysisQuery Query Features
Query Plan
66

Core Principles
 Minimize the impact of Ambiguity, Vagueness,
Synonymy.
 Address the simplest matchings first (semantic pivoting).
 Semantic Relatedness as a primitive operation.
 Distributional semantics models as commonsense
knowledge representation.
 Lightweight syntactic constraints.
67

• Step 2: Query NER
 Rules-based: POS Tag + IDF
Who is the daughter of Bill Clinton married to?
(PROBABLY AN INSTANCE)
Query Pre-Processing
(Question Analysis)
68

• Step 3: Determine answer type
 Rules-based.
Who is the daughter of Bill Clinton married to?
(PERSON)
(Question Analysis)
69

• Transform natural language queries into a
pseudo-logical form.
“Who is the daughter of Bill Clinton married to?”
(Question Analysis)
70

(Question Analysis)
Bill Clinton daughter married to
(INSTANCE)
Person
ANSWER
TYPE
QUESTION FOCUS
71
• Step 5: Determine the query pattern
 Rules based.
• Remove stop words.
• Merge words into entities.
• Reorder structure from core entity position.

• Step 5: Determine the query pattern
 Rules based.
• Remove stop words.
• Merge words into entities.
• Reorder structure from core entity position.
(Question Analysis)
Bill Clinton daughter married to
(INSTANCE)
Person
(PREDICATE) (PREDICATE) Query Features
72

• Map query features into a query plan.
• A query plan contains a sequence of:
 Search operations.
 Navigation operations.
Query Planning
(INSTANCE) (PREDICATE) (PREDICATE) Query Features
 (1) INSTANCE SEARCH (Bill Clinton)
 (2) DISAMBIGUATE ENTITY TYPE
 (3) GENERATE ENTITY FACETS
 (4) p1 <- SEARCH RELATED PREDICATE (Bill Clintion, daughter)
 (5) e1 <- GET ASSOCIATED ENTITIES (Bill Clintion, p1)
 (6) p2 <- SEARCH RELATED PREDICATE (e1, married to)
 (7) e2 <- GET ASSOCIATED ENTITIES (e1, p2)
 (8) POST PROCESS (Bill Clintion, e1, p1, e2, p2)
Query Plan
73

Core Entity Search
Bill Clinton daughter married to Person
:Bill_Clinton
Query:
KB:
Entity search
74

Distributional Semantic Search
:Bill_Clinton
Query:
:Chelsea_Clinton
:child
:Baptists
:religion
:Yale_Law_School
:almaMater
...
(PIVOT ENTITY)
(ASSOCIATED
TRIPLES)
75
KB:

:Bill_Clinton
Query:
:Chelsea_Clinton
:child
:Baptists
:religion
:Yale_Law_School
:almaMater
...
sem_rel(daughter,child)=0.054
sem_rel(daughter,child)=0.004
sem_rel(daughter,alma mater)=0.001
Which properties are semantically related to ‘daughter’?
76
KB:

:Bill_Clinton
Query:
:Chelsea_Clinton
:child
77
KB:

:Bill_Clinton
Query:
:Chelsea_Clinton
:child
(PIVOT ENTITY)
78
KB:

:Bill_Clinton
Query:
:Chelsea_Clinton
:child
:Mark_Mezvinsky
:spouse
79
KB:
Note the lazy
disambiguation

80

What is the highest mountain?
Second Query Example
(CLASS) (OPERATOR) Query Features
mountain - highest PODS
81

Entity Search
Mountain highest
:Mountain
Query:
:typeOf
(PIVOT ENTITY)
82
KB:

Extensional Expansion
Mountain highest
:Mountain
Query:
:Everest
:typeOf
(PIVOT ENTITY)
:K2:typeOf
...
83
KB:

Distributional Semantic Matching
Mountain highest
:Mountain
Query:
:Everest
:typeOf
(PIVOT ENTITY)
:K2:typeOf
...
:elevation
:location
...
:deathPlaceOf
84
KB:

Get all numerical values
Mountain highest
:Mountain
Query:
:Everest
:typeOf
(PIVOT ENTITY)
:K2:typeOf
...
:elevation
:elevation
8848 m
8611 m
85
KB:

Apply operator functional definition
Mountain highest
:Mountain
Query:
:Everest
:typeOf
(PIVOT ENTITY)
:K2:typeOf
...
:elevation
:elevation
8848 m
8611 m
SORT
TOP_MOST
86
KB:

Results
87

StarGraph
• Open source NoSQL platform for building and
interacting with large and sparse knowledge
graphs.
• Semantic approximation as a built-in operation.
• Scalable query execution performance.
88

Heuristics for the selection of the
semantic pivot is critical!
• Discussed here just superficially:
 Information-theoretical justification.
How hard is the Query? Measuring the Semantic Complexity of Schema-
Agnostic Queries, IWCS (2015).
Schema-agnositc queries over large-schema databases: a distributional
semantics approach, PhD Thesis (2015).
On the Semantic Mapping of Schema-agnostic Queries: A Preliminary
Study, NLIWoD (2015).
89

Indra
 Multilingual platform for experimentation with
different word vector models.
"Indra's net" is the net of the Vedic god Indra, whose net hangs over
his palace on Mount Meru, the axis mundi of Hindu cosmology and
Hindu mythology. Indra's net has a multifaceted jewel at each vertex,
and each jewel is reflected in all of the other jewels.
In the Avatamsaka Sutra, the image of "Indra's net" is used to describe
the interconnectedness of the universe.

Indra

Bridging Structured & Unstructured Data
• NER + Text + Passage Retrieval Ranking
 Simple and powerful QA basis.
• Lazy disambiguation.
94

Treo Answers Jeopardy Queries (Video)

Ranking Candidate Answers
• But what if there are multiple candidate answers!
Q: Who was Queen Victoria’s second son?
• Answer Type: Person
• Passage:
The Marie biscuit is named after Marie Alexandrovna, the daughter of
Czar Alexander II of Russia and wife of Alfred, the second son of Queen
Victoria and Prince Albert
96Dan Jurafky’s slides

Ranking Candidate Answers
• But what if there are multiple candidate answers!
Q: Who was Queen Victoria’s second son?
• Answer Type: Person
• Passage:
97Dan Jurafky’s slides

Feature Engineering
followed by a ‘,’ followed by an
apposition
Who was Queen Victoria’s second son?
contains an entity in
the query
has a four-word
overlap
type =
PERSON
matches AnswerType
98

Propositionalisation
e0 followedBy(,) followedByAppositionContainingQueryEntities() answer …
Alfred true true true
… … …
passage
entity (e0)
entity (en)
…
answer
99

Reasoning for Text Entailment

Beyond Word Vector Models
give birth
mother
car
θ
Distributional semantics can
give us a hint about the
concepts’ semantic proximity...
...but it still can’t tell us what
exactly the relationship
between them is
give birth
mother
???

Beyond Word Vector Models
give birth
mother
???
give birth
mother
???

Beyond Word Vector Models:
Intensional Reasoning
 Representing structured intensional-level
knowledge.
 Creation of an intensional-level reasoning
model.
104

Commonsense Reasoning
 Selective (focussed) reasoning
 - Selecting the relevant facts in the context of the
inference
 Reducing the search space.
Scalability

Extended WordNet (XWN)

Commonsense Data (ConceptNet)
http://conceptnet5.media.mit.edu/
107

Distributional semantic relatedness as a
Selectivity Heuristics
Distributional
heuristics
108
target
source
answer

Distributional
heuristics
109
target
source
answer

Distributional
heuristics
110
target
source
answer

John Smith EngineerInstance-level
occupation
Does John Smith have a degree?

A Distributional Semantics Approach for
Selective Reasoning on Commonsense Graph
Knowledge Bases, NLDB (2015).

Bringing it into the Real World

Semeval 2017

Take-away Message
• Choosing the sweet-spot in terms of semantic
representation is critical for the construction of robust
QA systems.
 Work at a word-based representation instead of a sense
representation.
 Text simplification/clausal disembedding critical for
relation extraction.
 Need for a standardized semantic representation for
relations extracted from texts.

Take-away Message
• Text entailment:
 Intensional-level reasoning.
 Natural logic.
 Distributional semantics.
• Distributional semantics:
 Robust, language-agnostic semantic matching.
 Selective reasoning over commonsense KBs.

Different Semantic Perspectives for Question Answering Systems

More Related Content

What's hot (20)

Viewers also liked (12)

Similar to Different Semantic Perspectives for Question Answering Systems (20)

More from Andre Freitas (13)

Recently uploaded (20)

Different Semantic Perspectives for Question Answering Systems