Natural language processing: word senses and relations

Requirements for Representation.
Expressiveness: Expressiveness refers to the ability of the representation
system to capture a wide range of meanings, nuances, and contexts.
• Rich Vocabulary: The system should have a rich set of symbols to describe
various entities, actions, properties, and relationships.
• Complex Structures: It should support the representation of complex
structures such as nested and hierarchical relationships.
• Context Sensitivity: The representation should capture context to
accurately reflect meaning (e.g., the difference between "bank" in "river
bank" and "financial bank").
• Ambiguity Resolution: It should handle and resolve ambiguities in natural
language.

Formalism: Formalism ensures that the representation has a well-
defined syntax and semantics, allowing for unambiguous interpretation
and processing.
• Clear Syntax: The rules for forming valid expressions should be precise
and unambiguous.
• Defined Semantics: Each expression should have a clear meaning,
defined in a way that machines can interpret consistently.
• Standardization: Use of standard representation languages (e.g., First-
Order Logic, Description Logic) can help in maintaining consistency
and interoperability.

Inference: Inference capability allows the system to derive new
information from existing knowledge, which is essential for reasoning
and decision-making.
• Logical Deduction: The system should support logical inference
mechanisms such as modus ponens, modus tollens, and syllogism.
• Probabilistic Inference: For handling uncertainty, the system might
need to support probabilistic reasoning (e.g., Bayesian networks).
• Temporal Reasoning: For applications involving time, the system
should support reasoning about temporal relationships and events.

Propositional Logic
1. "If it is raining, then the ground is wet." - P → Q
2. "Either the system is down, or the network is slow." - P ∨ Q
3. "If the user is logged in and the session is active, then the user can access the dashboard." - (P ∧
Q) → R
4. "The application will fail unless it is updated." - ¬Q → P or P ∨ Q
5. "If the file is not found, an error message is displayed." - ¬P → Q6. "The server is running if and
only if the power is on." - P ↔ Q
7. "If the temperature is above 30°C, then the air conditioner turns on." - P → Q
8. "If the database is backed up, the system will restart, and the logs will be cleared." - P → (Q ∧ R)
9. "If the user presses the submit button, then the form is validated and the data is sent." - P → (Q ∧
R)
10. "The document is either saved or discarded." - P ∨ Q

First-Order Logic (FOL),
• First-Order Logic (FOL), also known as Predicate Logic or First-Order
Predicate Calculus, is a formal system used to express statements
about objects, their properties, and their relationships. FOL extends
propositional logic by incorporating quantifiers and predicates,
making it much more expressive.

Key Components of First-Order Logic
Terms:
• Constants: Represent specific objects in the domain (e.g., a, b,
Raju).
• Variables: Represent any object in the domain (e.g., x, y, z).
• Functions: Map a set of objects to another object
• e.g., ‘fatherOf(Raju)’
Predicates: Represent properties or relationships between objects. For
example,
• Student(Raju) means “Raju is a student," and Likes(Raju, Pizza) means
"John likes pizza."

Logical Connectives:
• Conjunction (∧): P ∧ Q means "P and Q."
• Disjunction (∨): P ∨ Q means "P or Q."
• Negation (¬): ¬P means "not P."
• Implication (→): P → Q means "if P then Q."
• Biconditional (↔): P ↔ Q means "P if and only if Q."

Quantifiers:
• Universal Quantifier (∀): ∀x P(x) means "for all x, P(x) is true."
• Existential Quantifier (∃): ∃x P(x) means "there exists an x such that
P(x) is true."

• "All humans are mortal."
• "Some students are brilliant." –
• "No dogs can fly.“
• "If a person is a parent, then they have a child." -
• "There is a cat that is black."
• "Every student loves some book." –
• "Someone likes everyone." -
• "If an animal is a bird, then it can fly." -
• "There exists a unique number that is even."
• "For every number, there is a greater number." -

Natural language processing: word senses and relations

First-Order Logic
Chapter 8.1-8.3
CMSC 471
Adapted from slides by
Tim Finin and
Marie desJardins.
Some material adopted from notes
by Andreas Geyer-Schulz

Outline
• First-order logic
• Properties, relations, functions, quantifiers, …
• Terms, sentences, axioms, theories, proofs, …
• Extensions to first-order logic
• Logical agents
• Reflex agents
• Representing change: situation calculus, frame problem
• Preferences on actions
• Goal-based agents

First-order logic
• First-order logic (FOL) models the world in terms of
• Objects, which are things with individual identities
• Properties of objects that distinguish them from other
objects
• Relations that hold among sets of objects
• Functions, which are a subset of relations where there is
only one “value” for any given “input”
• Examples:
• Objects: Students, lectures, companies, cars ...
• Relations: Brother-of, bigger-than, outside, part-of, has-
color, occurs-after, owns, visits, precedes, ...
• Properties: blue, oval, even, large, ...
• Functions: father-of, best-friend, second-half, one-more-
than ...

User provides
• Constant symbols, which represent individuals in the
world
• Mary
• 3
• Green
• Function symbols, which map individuals to individuals
• father-of(Mary) = John
• color-of(Sky) = Blue
• Predicate symbols, which map individuals to truth
values
• greater(5,3)
• green(Grass)
• color(Grass, Green)

FOL Provides
• Variable symbols
• E.g., x, y, foo
• Connectives
• Same as in PL: not (), and (), or (), implies (), if
and only if (biconditional )
• Quantifiers
• Universal x or (Ax)
• Existential x or (Ex)

Sentences are built from terms and atoms
• A term (denoting a real-world individual) is a constant
symbol, a variable symbol, or an n-place function of n terms.
x and f(x1, ..., xn) are terms, where each xi is a term.
A term with no variables is a ground term
• An atomic sentence (which has value true or false) is an n-
place predicate of n terms
• A complex sentence is formed from atomic sentences
connected by the logical connectives:
P, PQ, PQ, PQ, PQ where P and Q are sentences
• A quantified sentence adds quantifiers  and 
• A well-formed formula (wff) is a sentence containing no
“free” variables. That is, all variables are “bound” by
universal or existential quantifiers.
(x)P(x,y) has x bound as a universally quantified variable, but y is
free.

Quantifiers
• Universal quantification
• (x)P(x) means that P holds for all values of x in the
domain associated with that variable
• E.g., (x) dolphin(x)  mammal(x)
• Existential quantification
• ( x)P(x) means that P holds for some value of x in the
domain associated with that variable
• E.g., ( x) mammal(x)  lays-eggs(x)
• Permits one to make a statement about some object
without naming it

Quantifiers
• Universal quantifiers are often used with “implies” to form “rules”:
(x) student(x)  smart(x) means “All students are smart”
• Universal quantification is rarely used to make blanket statements
about every individual in the world:
(x)student(x)smart(x) means “Everyone in the world is a student and is
smart”
• Existential quantifiers are usually used with “and” to specify a list of
properties about an individual:
(x) student(x)  smart(x) means “There is a student who is smart”
• A common mistake is to represent this English sentence as the FOL
sentence:
(x) student(x)  smart(x)
• But what happens when there is a person who is not a student?

Quantifier Scope
• Switching the order of universal quantifiers does not change the
meaning:
• (x)(y)P(x,y) ↔ (y)(x) P(x,y)
• Similarly, you can switch the order of existential quantifiers:
• (x)(y)P(x,y) ↔ (y)(x) P(x,y)
• Switching the order of universals and existentials does change
meaning:
• Everyone likes someone: (x)(y) likes(x,y)
• Someone is liked by everyone: (y)(x) likes(x,y)

Connections between All and Exists
We can relate sentences involving  and 
using De Morgan’s laws:
(x) P(x) ↔ (x) P(x)
(x) P ↔ (x) P(x)
(x) P(x) ↔  (x) P(x)
(x) P(x) ↔ (x) P(x)

Quantified inference rules
• Universal instantiation
• x P(x)  P(A)
• Universal generalization
• P(A)  P(B) …  x P(x)
• Existential instantiation
• x P(x) P(F)  skolem constant F
• Existential generalization
• P(A)  x P(x)

Universal instantiation
(a.k.a. universal elimination)
• If (x) P(x) is true, then P(C) is true, where C is any constant in the
domain of x
• Example:
(x) eats(Ziggy, x)  eats(Ziggy, IceCream)
• The variable symbol can be replaced by any ground term, i.e., any
constant symbol or function symbol applied to ground terms only

Existential instantiation
(a.k.a. existential elimination)
• From (x) P(x) infer P(c)
• Example:
• (x) eats(Ziggy, x)  eats(Ziggy, Stuff)
• Note that the variable is replaced by a brand-new
constant not occurring in this or any other sentence in
the KB
• Also known as skolemization; constant is a skolem
constant
• In other words, we don’t want to accidentally draw
other inferences about it by introducing the constant
• Convenient to use this to reason about the unknown
object, rather than constantly manipulating the
existential quantifier

Existential generalization
(a.k.a. existential introduction)
• If P(c) is true, then (x) P(x) is inferred.
• Example
eats(Ziggy, IceCream)  (x) eats(Ziggy, x)
• All instances of the given constant symbol are replaced by the new
variable symbol
• Note that the variable symbol cannot already exist anywhere in the
expression

Translating English to FOL
Every gardener likes the sun.
x gardener(x)  likes(x,Sun)
You can fool some of the people all of the time.
x t person(x) time(t)  can-fool(x,t)
You can fool all of the people some of the time.
x t (person(x)  time(t) can-fool(x,t))
x (person(x)  t (time(t) can-fool(x,t))
All purple mushrooms are poisonous.
x (mushroom(x)  purple(x))  poisonous(x)
No purple mushroom is poisonous.
x purple(x)  mushroom(x)  poisonous(x)
x (mushroom(x)  purple(x))  poisonous(x)
There are exactly two purple mushrooms.
x y mushroom(x)  purple(x)  mushroom(y)  purple(y) ^ (x=y)  z
(mushroom(z)  purple(z))  ((x=z)  (y=z))
Clinton is not tall.
tall(Clinton)
X is above Y iff X is on directly on top of Y or there is a pile of one or more other
objects directly on top of one another starting with X and ending with Y.
x y above(x,y) ↔ (on(x,y)  z (on(x,z)  above(z,y)))
Equivalent
Equivalent

Example: A simple genealogy KB by FOL
• Build a small genealogy knowledge base using FOL that
• contains facts of immediate family relations (spouses, parents, etc.)
• contains definitions of more complex relations (ancestors, relatives)
• is able to answer queries about relationships between people
• Predicates:
• parent(x, y), child(x, y), father(x, y), daughter(x, y), etc.
• spouse(x, y), husband(x, y), wife(x,y)
• ancestor(x, y), descendant(x, y)
• male(x), female(y)
• relative(x, y)
• Facts:
• husband(Joe, Mary), son(Fred, Joe)
• spouse(John, Nancy), male(John), son(Mark, Nancy)
• father(Jack, Nancy), daughter(Linda, Jack)
• daughter(Liz, Linda)
• etc.

• Rules for genealogical relations
• (x,y) parent(x, y) ↔ child (y, x)
(x,y) father(x, y) ↔ parent(x, y)  male(x) (similarly for mother(x,
y))
(x,y) daughter(x, y) ↔ child(x, y)  female(x) (similarly for son(x,
y))
• (x,y) husband(x, y) ↔ spouse(x, y)  male(x) (similarly for wife(x,
y))
(x,y) spouse(x, y) ↔ spouse(y, x) (spouse relation is symmetric)
• (x,y) parent(x, y)  ancestor(x, y)
(x,y)(z) parent(x, z)  ancestor(z, y)  ancestor(x, y)
• (x,y) descendant(x, y) ↔ ancestor(y, x)
• (x,y)(z) ancestor(z, x)  ancestor(z, y)  relative(x, y)
(related by common ancestry)
(x,y) spouse(x, y)  relative(x, y) (related by marriage)
(x,y)(z) relative(z, x)  relative(z, y)  relative(x, y) (transitive)
(x,y) relative(x, y) ↔ relative(y, x) (symmetric)
• Queries
• ancestor(Jack, Fred) /* the answer is yes */
• relative(Liz, Joe) /* the answer is yes */
• relative(Nancy, Matthew)
/* no answer in general, no if under closed world assumption
*/
• (z) ancestor(z, Fred)  ancestor(z, Liz)

Semantics of FOL
• Domain M: the set of all objects in the world (of
interest)
• Interpretation I: includes
• Assign each constant to an object in M
• Define each function of n arguments as a mapping Mn => M
• Define each predicate of n arguments as a mapping Mn => {T, F}
• Therefore, every ground predicate with any instantiation will have a
truth value
• In general there is an infinite number of interpretations because
|M| is infinite
• Define logical connectives: ~, ^, , =>, <=> as in PL
• Define semantics of (x) and (x)
• (x) P(x) is true iff P(x) is true under all interpretations
• (x) P(x) is true iff P(x) is true under some interpretation

• Model: an interpretation of a set of sentences such that
every sentence is True
• A sentence is
• satisfiable if it is true under some interpretation
• valid if it is true under all possible interpretations
• inconsistent if there does not exist any interpretation under which
the sentence is true
• Logical consequence: S |= X if all models of S are also
models of X

Axioms, definitions and theorems
•Axioms are facts and rules that attempt to capture all
of the (important) facts and concepts about a domain;
axioms can be used to prove theorems
•Mathematicians don’t want any unnecessary (dependent)
axioms –ones that can be derived from other axioms
•Dependent axioms can make reasoning faster, however
•Choosing a good set of axioms for a domain is a kind of design
problem
•A definition of a predicate is of the form “p(X) ↔ …”
and can be decomposed into two parts
•Necessary description: “p(x)  …”
•Sufficient description “p(x)  …”
•Some concepts don’t have complete definitions (e.g., person(x))

More on definitions
• A necessary condition must be satisfied for a statement to be
true.
• A sufficient condition, if satisfied, assures the statement’s truth.
• Duality: “P is sufficient for Q” is the same as “Q is necessary for P.”
• Examples: define father(x, y) by parent(x, y) and male(x)
• parent(x, y) is a necessary (but not sufficient) description
of
father(x, y)
• father(x, y)  parent(x, y)
• parent(x, y) ^ male(x) ^ age(x, 35) is a sufficient (but not
necessary) description of father(x, y):
father(x, y)  parent(x, y) ^ male(x) ^ age(x, 35)
• parent(x, y) ^ male(x) is a necessary and sufficient
description of father(x, y)
parent(x, y) ^ male(x) ↔ father(x, y)

More on definitions
P(x)
S(x)
S(x) is a
necessary
condition of P(x)
(x) P(x) => S(x)
S(x)
P(x)
S(x) is a
sufficient
condition of P(x)
(x) P(x) <= S(x)
P(x)
S(x)
S(x) is a
necessary and
sufficient
condition of P(x)
(x) P(x) <=> S(x)

Higher-order logic
• FOL only allows to quantify over variables, and
variables can only range over objects.
• HOL allows us to quantify over relations
• Example: (quantify over functions)
“two functions are equal iff they produce the same value for
all arguments”
f g (f = g)  (x f(x) = g(x))
• Example: (quantify over predicates)
r transitive( r )  (xyz) r(x,y)  r(y,z)  r(x,z))
• More expressive, but undecidable. (there isn’t an
effective algorithm to decide whether all sentences are valid)
• First-order logic is decidable only when it uses predicates with only one
argument.

Expressing uniqueness
• Sometimes we want to say that there is a single,
unique object that satisfies a certain condition
• “There exists a unique x such that king(x) is true”
• x king(x)  y (king(y)  x=y)
• x king(x)  y (king(y)  xy)
• ! x king(x)
• “Every country has exactly one ruler”
• c country(c)  ! r ruler(c,r)
• Iota operator: “ x P(x)” means “the unique x such
that p(x) is true”
• “The unique ruler of Freedonia is dead”
• dead( x ruler(freedonia,x))

Notational differences
• Different symbols for and, or, not, implies, ...
•         
• p v (q ^ r)
• p + (q * r)
• etc
• Prolog
cat(X) :- furry(X), meows (X), has(X, claws)
• Lispy notations
(forall ?x (implies (and (furry ?x)
(meows ?x)
(has ?x claws))
(cat ?x)))

Logical agents for the Wumpus World
Three (non-exclusive) agent architectures:
• Reflex agents
• Have rules that classify situations, specifying how to react to each possible
situation
• Model-based agents
• Construct an internal model of their world
• Goal-based agents
• Form goals and try to achieve them

A simple reflex agent
• Rules to map percepts into observations:
b,g,u,c,t Percept([Stench, b, g, u, c], t)  Stench(t)
s,g,u,c,t Percept([s, Breeze, g, u, c], t)  Breeze(t)
s,b,u,c,t Percept([s, b, Glitter, u, c], t)  AtGold(t)
• Rules to select an action given observations:
t AtGold(t)  Action(Grab, t);
• Some difficulties:
• Consider Climb. There is no percept that indicates the
agent should climb out – position and holding gold are not
part of the percept sequence
• Loops – the percept will be repeated when you return to a
square, which should cause the same response (unless we
maintain some internal model of the world)

Representing change
• Representing change in the world in logic can be
tricky.
• One way is just to change the KB
• Add and delete sentences from the KB to reflect
changes
• How do we remember the past, or reason about
changes?
• Situation calculus is another way
• A situation is a snapshot of the world at some
instant in time
• When the agent performs an action A in
situation S1, the result is a new situation
S2.

Situation calculus
• A situation is a snapshot of the world at an interval of time during which
nothing changes
• Every true or false statement is made with respect to a particular situation.
• Add situation variables to every predicate.
• at(Agent,1,1) becomes at(Agent,1,1,s0): at(Agent,1,1) is true in situation (i.e., state)
s0.
• Alternatively, add a special 2nd-order predicate, holds(f,s), that means “f is true in
situation s.” E.g., holds(at(Agent,1,1),s0)
• Add a new function, result(a,s), that maps a situation s into a new situation as
a result of performing action a. For example, result(forward, s) is a function
that returns the successor state (situation) to s
• Example: The action agent-walks-to-location-y could be represented by
• (x)(y)(s) (at(Agent,x,s)  onbox(s))  at(Agent,y,result(walk(y),s))

Deducing hidden properties
• From the perceptual information we obtain in situations, we can infer
properties of locations
l,s at(Agent,l,s)  Breeze(s)  Breezy(l)
l,s at(Agent,l,s)  Stench(s)  Smelly(l)
• Neither Breezy nor Smelly need situation arguments because pits and
Wumpuses do not move around

Deducing hidden properties II
• We need to write some rules that relate various
aspects of a single world state (as opposed to across
states)
• There are two main kinds of such rules:
• Causal rules reflect the assumed direction of causality in the
world:
(l1,l2,s) At(Wumpus,l1,s)  Adjacent(l1,l2)  Smelly(l2)
( l1,l2,s) At(Pit,l1,s)  Adjacent(l1,l2)  Breezy(l2)
Systems that reason with causal rules are called model-
based reasoning systems
• Diagnostic rules infer the presence of hidden properties
directly from the percept-derived information. We have
already seen two diagnostic rules:
( l,s) At(Agent,l,s)  Breeze(s)  Breezy(l)
( l,s) At(Agent,l,s)  Stench(s)  Smelly(l)

Representing change:
The frame problem
• Frame axioms: If property x doesn’t change as a result of applying
action a in state s, then it stays the same.
• On (x, z, s)  Clear (x, s) 
On (x, table, Result(Move(x, table), s)) 
On(x, z, Result (Move (x, table), s))
• On (y, z, s)  y x  On (y, z, Result (Move (x, table), s))
• The proliferation of frame axioms becomes very cumbersome in complex
domains

The frame problem II
• Successor-state axiom: General statement that
characterizes every way in which a particular predicate
can become true:
• Either it can be made true, or it can already be true and not
be changed:
• On (x, table, Result(a,s)) 
[On (x, z, s)  Clear (x, s)  a = Move(x, table)] 
[On (x, table, s)  a  Move (x, z)]
• In complex worlds, where you want to reason about
longer chains of action, even these types of axioms are
too cumbersome
• Planning systems use special-purpose inference methods to
reason about the expected state of the world at any point in
time during a multi-step plan

Qualification problem
• Qualification problem:
• How can you possibly characterize every single effect of an action, or every
single exception that might occur?
• When I put my bread into the toaster, and push the button, it will become
toasted after two minutes, unless…
• The toaster is broken, or…
• The power is out, or…
• I blow a fuse, or…
• A neutron bomb explodes nearby and fries all electrical components, or…
• A meteor strikes the earth, and the world we know it ceases to exist, or…

Ramification problem
• Similarly, it’s just about impossible to characterize every side effect of every action, at every
possible level of detail:
• When I put my bread into the toaster, and push the button, the bread will become toasted after two
minutes, and…
• The crumbs that fall off the bread onto the bottom of the toaster over tray will also become toasted, and…
• Some of the aforementioned crumbs will become burnt, and…
• The outside molecules of the bread will become “toasted,” and…
• The inside molecules of the bread will remain more “breadlike,” and…
• The toasting process will release a small amount of humidity into the air because of evaporation, and…
• The heating elements will become a tiny fraction more likely to burn out the next time I use the toaster, and…
• The electricity meter in the house will move up slightly, and…

Knowledge engineering!
• Modeling the “right” conditions and the “right” effects at the “right”
level of abstraction is very difficult
• Knowledge engineering (creating and maintaining knowledge bases
for intelligent reasoning) is an entire field of investigation
• Many researchers hope that automated knowledge acquisition and
machine learning tools can fill the gap:
• Our intelligent systems should be able to learn about the conditions and
effects, just like we do!
• Our intelligent systems should be able to learn when to pay attention to, or
reason about, certain aspects of processes, depending on the context!

Preferences among actions
• A problem with the Wumpus world knowledge base that we have
built so far is that it is difficult to decide which action is best among a
number of possibilities.
• For example, to decide between a forward and a grab, axioms
describing when it is OK to move to a square would have to mention
glitter.
• This is not modular!
• We can solve this problem by separating facts about actions from
facts about goals. This way our agent can be reprogrammed just by
asking it to achieve different goals.

• The first step is to describe the desirability of actions independent of
each other.
• In doing this we will use a simple scale: actions can be Great, Good,
Medium, Risky, or Deadly.
• Obviously, the agent should always do the best action it can find:
(a,s) Great(a,s)  Action(a,s)
(a,s) Good(a,s)  (b) Great(b,s)  Action(a,s)
(a,s) Medium(a,s)  ((b) Great(b,s)  Good(b,s))  Action(a,s)
...

• We use this action quality scale in the following way.
• Until it finds the gold, the basic strategy for our agent
is:
• Great actions include picking up the gold when found and
climbing out of the cave with the gold.
• Good actions include moving to a square that’s OK and
hasn't been visited yet.
• Medium actions include moving to a square that is OK and
has already been visited.
• Risky actions include moving to a square that is not known
to be deadly or OK.
• Deadly actions are moving into a square that is known to
have a pit or a Wumpus.

Goal-based agents
• Once the gold is found, it is necessary to change
strategies. So now we need a new set of action values.
• We could encode this as a rule:
• (s) Holding(Gold,s)  GoalLocation([1,1]),s)
• We must now decide how the agent will work out a
sequence of actions to accomplish the goal.
• Three possible approaches are:
• Inference: good versus wasteful solutions
• Search: make a problem with operators and set of states
• Planning: to be discussed later

Natural language processing: word senses and relations

More Related Content

Similar to Natural language processing: word senses and relations (20)

Recently uploaded (20)

Natural language processing: word senses and relations