02 knowledge-based systems

28.09.2019
Knowledge Engineering

Knowledge Based
Systems
Knowledge Engineering - Natig Vahabov 2

“Knowledge is of no
values unless you put it
into practice”
Anton Chekhov
Russian playwright, short-story writer

Agenda
• Knowledge-based systems
• Types of KBS
– Expert systems
– Neural networks
– Case-based reasoning
– Genetic algorithms
– Intelligent agents

What are KBSs?
• A knowledge based system is a system that uses
artificial intelligence techniques in problem-solving
processes to support human decision-making,
learning, and action
• Two central components of KBSs are
– Knowledge base
• Consists of a set of facts and a set of rules, frames, or
procedures
– Inference engine
• Responsible for the application of knowledge base to the
problem on hand.

KBS Flowchart

Types of KBSs
• Expert systems
• Neural networks
• Case-based reasoning
• Genetic algorithms
• Intelligent agents

Expert Systems

Expert System
‘An expert system is a computer system that
emulates, or acts in all respects, with the
decision-making capabilities of a human
expert’.
Prof Edward Feigenbaum

ES as part of AI

Expert Systems
• An expert system is a computer program designed to
emulate the problem-solving behavior of an expert in
a specific knowledge domain
• In order to qualify as an expert system, a system must
have the capability of explaining or justifying its
conclusions
• A system which can explain its reasoning process is
said to demonstrate meta-knowledge (knowledge
about its own knowledge)

Early Expert Systems
• DENDRAL – used in chemical mass
spectroscopy to identify chemical constituents
• MYCIN – medical diagnosis of illness
• DIPMETER – geological data analysis for oil
• PROSPECTOR – geological data analysis for
minerals
• XCON/R1 – configuring computer systems

ES Applications (Hayes-Roth Table)
Category Problem addressed Examples
Interpretation
Inferring situation
descriptions from sensor
data
Hearsay (speech
recognition), PROSPECTOR
Prediction
Inferring likely
consequences of given
situations
Preterm Birth Risk
Assessment
Diagnosis
Inferring system
malfunctions from
observables
CADUCEUS, MYCIN, PUFF,
Mistral, Eydenet, Kaleidos
Design
Configuring objects under
constraints
Dendral, Mortgage Loan
Advisor, R1 (DEC VAX
Configuration), SID
(DEC VAX 9000 CPU)
Planning Designing actions
Mission Planning for
Autonomous Underwater
Vehicle

ES Applications (Hayes-Roth Table)
Category Problem addressed Examples
Monitoring
Comparing observations to
plan vulnerabilities
REACTOR
Debugging
Providing incremental
solutions for complex
problems
SAINT, MATHLAB,
MACSYMA
Repair
Executing a plan to
administer a prescribed
remedy
Toxic Spill Crisis
Management
Instruction
Diagnosing, assessing, and
repairing student behavior
SMH.PAL, Intelligent Clinical
Training, STEAMER
Control
Interpreting, predicting,
repairing, and monitoring
system behaviors
Real Time Process Control,
Space Shuttle Mission
Control

Basic functions of ES

Problem Domain vs
Knowledge Domain
• An expert’s knowledge is specific to one
problem domain – medicine, finance, science,
engineering, etc.
• The expert’s knowledge about solving specific
problems is called the knowledge domain.
• The problem domain is always a superset of
the knowledge domain.

Problem Domain vs
Knowledge Domain

Elements of an Expert System
• User interface – mechanism by which user
and system communicate
• Exploration facility – explains reasoning of
expert system to user
• Working memory – global database of facts
used by rules
• Inference engine – makes inferences deciding
which rules are satisfied and prioritizing

• Agenda – a prioritized list of rules created by
the inference engine, whose patterns are
satisfied by facts or objects in working
memory.
• Knowledge acquisition facility – automatic way
for the user to enter knowledge in the system
bypassing the explicit coding by knowledge
engineer.

Languages, Shells, and Tools
• Expert system languages are post-third
generation
• Programming languages, LISP and PROLOG,
are typically used in expert systems
implementation, in particular Artificial
intelligence applications
• Expert system languages (e.g. CLIPS) focus on
ways to represent knowledge
– CLIPS is an expert system shell developed by NASA

Development of an ES
• The knowledge engineer establishes a dialog
with the human expert to elicit knowledge
• The knowledge engineer codes the knowledge
explicitly in the knowledge base
• The expert evaluates the expert system and
gives a critique to the knowledge engineer

Development of an ES

The Role of AI
• An algorithm is an ideal solution guaranteed
to yield a solution in a finite amount of time.
• When an algorithm is not available or is
insufficient, we rely on artificial intelligence
(AI).
• Expert system relies on inference – we accept
a “reasonable solution.”

Uncertainty
• Both human experts and expert systems must
be able to deal with uncertainty
• It is easier to program expert systems with
shallow knowledge than with deep knowledge
• Shallow knowledge – based on empirical and
heuristic knowledge
• Deep knowledge – based on basic structure,
function, and behavior of objects

Dealing with Uncertainty
• Certainty
– If A is false, then ⌐ A is true
• Uncertainty
– If A is false, then B is true with probability P
– Introducing Probabilistic Reasoning
– Doctor examines a patient, and X symptoms exist
but some of them missing, so doctor can conclude:
a patient has disease X with probability P

General Methods of Inferencing
• Forward chaining – reasoning from facts to the
conclusions resulting from those facts – best
for prognosis, monitoring, and control
– primarily data-driven
• Backward chaining – reasoning in reverse from
a hypothesis, a potential conclusion to be
proved to the facts that support the
hypothesis – best for diagnosis problems
– primarily goal driven

Production Rules
• Knowledge base is also called production
memory
• Production rules can be expressed in IF-THEN
pseudocode format
– IF you are hungry THEN eat
• In rule-based systems, the inference engine
determines which rule antecedents are
satisfied by the facts

Production Systems
• Rule-based expert systems – most popular
type today
• Knowledge is represented as multiple rules
that specify what should/not be concluded
from different situations
• Forward chaining
– start w/facts and use rules do draw conclusions/take actions.
• Backward chaining
– start w/hypothesis and look for rules that allow hypothesis to
be proven true

Advantages of ES
• Consistency
– Consistency is the main benefit of an ES. Since it is
a computer-based system, all the knowledge or
logic are programmed into it. If it meets the same
situation, it will make the same decision again and
again. Because always the decision will be made
based on some rules and logic.
• Availability
– The multiple users can access an ES
simultaneously and get responses immediately

Advantages of ES
• Logic
– In an ES, all of the rules, conditions and their
understandings on the way to making a decision or
a choice are always clear because it was
programmed in.
• Persistence/Accessibility
– The ESs are always available. They can be accessed
anytime 24*7. It is one of the important
advantages of an expert system over the human
experts.

Disadvantages of ES
• Data integrity
– The ESs need to be updated manually, because
they all itself don’t learn.
• Time & Cost
– The time and cost required to buy or set up an ES
are very high.
• Specific
– The ESs are generally developed for a specific
domain, on the other hand a human expert can be
specialized in more than one area

Disadvantages of ES
• Emotionless
– The human experts have a sort of awareness about
the situation, that means how they feel, how they
effective in the situation. But the ESs have no
awareness about any situation that they face
• Commonsense
– An ES has to go through by following the rules and
regulations as they programmed, so it can’t provide
a solution for the totally new kind of problem

Neural Network

Neural Network
• Neural networks represent a brain metaphor for
information processing. Neural computing refers to a
pattern recognition methodology for machine
learning. The resulting model from neural computing
is often called an artificial neural network (ANN) or
neural network (NN)
• Due to their ability to learn from the data, their
nonparametric nature (i.e., no rigid assumptions), and
their ability to generalize, neural networks have been
shown to be promising in many forecasting and
business classification applications

Neural Network

Key developments in history
of NN
• 1943 − It has been assumed that the concept of neural network
started with the work of physiologist, Warren McCulloch, and
mathematician, Walter Pitts, when in 1943 they modeled a
simple neural network using electrical circuits in order to
describe how neurons in the brain might work
• 1949 − Donald Hebb’s book, The Organization of Behavior, put
forth the fact that repeated activation of one neuron by another
increases its strength each time they are used
• 1958 − A learning method for McCulloch and Pitts neuron model
named Perceptron was invented by Rosenblatt
• 1960 − Bernard Widrow and Marcian Hoff developed models
called "ADALINE" and “MADALINE”

Key developments in history
of NN
• 1961 − Rosenblatt made an unsuccessful attempt but proposed
the “backpropagation” scheme for multilayer networks
• 1969 − Multilayer perceptron (MLP) was invented by Minsky and
Papert
• 1971 − Kohonen developed Associative memories
• 1976 − Stephen Grossberg and Gail Carpenter developed
Adaptive resonance theory
• 1985 − Boltzmann machine was developed by Ackley, Hinton, and
Sejnowski.
• 1986 − Rumelhart, Hinton, and Williams introduced Generalised
Delta Rule.
• 1988 − Kosko developed Binary Associative Memory (BAM) and
also gave the concept of Fuzzy Logic in ANN.

Structure of a Biological
Neural Network

Elements of BNN
• The human brain is composed of special cells called
neurons
• Neural network elements
– Nucleus
• The central processing portion of a neuron
– Soma
• The main body of the neuron in which the cell nucleus is contained
– Dendrite
• The part of a biological neuron that provides inputs to the cell
– Axon
• An outgoing connection (i.e., terminal) from a biological neuron
– Synapse
• The connection (where the weights are) between processing elements in
a neural network

Artificial Neural Network
• An ANN model emulates a biological neural network.
• Neural concepts are usually implemented as software
simulations of the massive parallel processes that
involve processing elements (also called artificial
neurons) interconnected in a network structure.
• Connections between neurons have an associated
weight.
• Each neuron calculates a weighted sum of the
incoming neuron values, transforms this input, and
passes on its neural value as the input to subsequent
neurons or external outputs.

Artificial Neural Network

BNN vs ANN

Learning in ANN
• Supervised learning
– the training data you feed to the algorithm includes the
desired solutions, called labels
– Algortihms: Regression, Classification
• Unsupervised learning
– the training data is unlabeled, the system tries to learn
without a teacher
– Algorithms: Clustering, Anomaly and novelty detection,
Visualization and dimensionality reduction, Association
rule learning
!will cover deep in later

Learning in ANN
• Semi-supervised learning
– some algorithms can deal with partially labeled training data,
usually a lot of unlabeled data and a little bit of labeled data
– Google Photos
• Reinforcement learning
– an area of machine learning concerned with how software
agents ought to take actions in an environment so as to
maximize some notion of cumulative reward
– AlphaGo beat the world champion Ke Jie at the game of Go in
2017
!will cover deep in later

Characteristics of ANN
• Adaptive learning
• Self-organization
• Error tolerance
• Real-time operation
• Parallel information processing

Case-Based Reasoning

Case-Based Reasoning
• A case has two parts: a problem and a solution
• Cases represent experience: they record how a
problem was solved in the past
• CBR is a methodology in which knowledge and
inferences are derived from historical cases. It is
based on the premise that new problems are often
similar to previously encountered problems and, past
solutions may be used in the current situations
• CBR is particularly applicable to problems in which
the domain is not understood well enough for a
robust statistical model or system of equations to be
formulated

Process of CBR
• Retrieve
– Given a target problem, retrieve the most similar cases
• Reuse
– Map the solution and reuse the best old solution to solve the
current case
• Revise
– Test the solution and, if necessary, revise the old case to
come up with the solution
• Retain
– After the solution has been successfully adapted to the target
problem, store the resulting experience as a new case

Process of CBR

Similarity Computation
• Cases are ranked according to their similarity
based on the similarity of each feature
• The degree of similarity can be expressed by a
real number between 0 (not similar) and 1
(identical)
• The importance of different features may be
different. In that case, similarity is computed
by weighted average

CBR Examples
• Intelligent customer support and sales support
• Retrieval of tour packages from travel catalogs
• Conflict resolution in air traffic control
• Conceptual building design aid
• Conceptual design aid for electronic devices
• Medical diagnosis
• Aircraft troubleshooting
• Heuristic retrieval of legal knowledge
• Computer supported conflict resolution through
negotiation or mediation

CBR Pros/Cons
• Advantages
– Improved knowledge acquisition
– Reduced development time
– Easier explanation
– Learning over time
• Disadvantages
– Storing of cases in the KB
– Implicit link between problem and solution
– Access and retrieval speed

Genetic Algorithms

Genetic Algorithms
• Programs that attempt to find optimal
solutions to problems by conceptually
following steps inspired by the biological
processes of evolution
• The method learns by producing offspring
that are better and better, as measured by a
fitness-to-survive function, until an optimal or
near-optimal solution is obtained.

Genetic Algorithm Applications
• Genetic algorithms provide a set of efficient,
domain-independent search heuristics for a
broad spectrum of applications including
– Dynamic process control
– Complex design of engineering structures
– Scheduling
– Transportation and routing
– Layout and circuit design
– Telecommunications
– Discovery of new connectivity typologies

GA Fundamentals
• Chromosome
– A candidate solution for a genetic algorithm
• Fitness function
– A measure of the objective to be obtained
• Generation
– An iteration of the genetic algorithmic process in
which candidate solutions are combined to
produce offspring

Processes within GA
• Reproduction
– Through reproduction, genetic algorithms produce new
generations of improved solutions by selecting parents with
higher fitness ratings or by giving such parents a greater
probability of being contributors and by using random
selection
• Crossover
– The combining of parts of two superior solutions by a genetic
algorithm in an attempt to produce an even better solution
• Mutation
– A genetic operator that causes a random change in a
potential solution

Genetic Algorithm Process

Genetic Algorithm Parameters
• Some parameters must be set for the genetic
algorithm
– Number of initial solutions to generate
– Number of offspring to generate
– Number of parents and offspring to keep for the next
generation
– Mutation probability
– Probability distribution of crossover point occurrence
• Their values are dependent on the problem being
solved and are usually determined through trial and
error

GA Benefits/Limitations
• Genetic algorithms are particularly useful for complex
problems that require rapid development of set of
good solutions
• Limitations
– Not all problems can be framed in the mathematical manner
that genetic algorithms demand
– Development of a genetic algorithm is complex
– In some situations, the “genes” from a few comparatively
highly fit (but not optimal) individuals may come to dominate
the population, causing it to converge on a local maximum
– Most genetic algorithms rely on random number generators
that produce different results each time the model runs

Knowledge-based
Intelligent Agents

Intelligent Agents
• A computer program that carries out a set of
operations on behalf of a user or another program,
with some degree of autonomy, and in doing so,
employs some knowledge or representation of the
user’s goals or desires.
• Agents in various forms
– Software agents, wizards, software daemons, e-mail agents
(mail bots), web browsing assisting agents, intelligent search
agents (Web robots, spiders), Internet softbots, network
management and monitoring agents, e-commerce agents

Intelligent Agents

Features of Intelligent Agents
• Reactivity
– Agents perceive their environment and respond in a timely
fashion to changes that occur in it
• Proactiveness
– Agents are able to exhibit goal-directed behavior by taking
initiative
• Social ability
– Agents are capable of interacting with other agents in order
to satisfy their design objectives
• Autonomy
– Agents must have control over their own actions and be able
to work and launch actions independently of the user or
other actors

Why use Intelligent Agents
• The Gartner Group findings on information overload:
– The amount of data collected by large enterprises doubles
every year.
– Knowledge workers can analyze only about 5% of this data.
– Information overload reduces our decision-making
capabilities by 50 percent.
• A major value of intelligent agents is that they are
able to assist in searching through all the data
• Intelligent agents save time by making decisions
about what is relevant to the user as well as by
automating routine tasks

Architecture of an IA
• Knowledge Level
– The most abstract level: describe agent by saying what it
knows
– Example: A taxi agent might know that the Golden Gate
Bridge connects San Francisco with the Marin County.
• Logical Level
– The level at which the knowledge is encoded into sentences
– Example: Links(GoldenGateBridge, SanFrancisco,
MarinCounty).
• Implementation Level
– The physical representation of the sentences in the logical
level
– Example: ‘(links goldengatebridge sanfrancisco marincounty)

How smart is an IA
Intelligence levels
• Level 0 - Agents retrieve documents for a user under
straight orders
• Level 1 - Agents provide a user-initiated searching
facility for finding relevant Web pages
• Level 2 - Agents maintain users’ profiles
• Level 3 - Agents have a learning and deductive
component to help a user who cannot formalize a
query or specify a target for a search

IA vs ES
• Agents and expert systems are similar in that they
both intend to incorporate domain knowledge to
automate decision making
• They are different in the following aspects:
– Classic ES are not coupled to any environment in which they
act; they act through a user as a middle man. Agents can
actively search information from the environment in which
they reside
– ES are not generally capable of reactive and proactive
behavior
– ES are not generally equipped with social ability in the sense
of cooperation, coordination, and negotiation.

Internet-Based Software
Agents
• Nine major application areas
– Assisting in workflow and administrative management
– Collaborating with other agents and people
– Supporting e-commerce
– Supporting desktop applications
– Assisting in information access and management, including
searching and FAQs
– Processing e-mail and messages
– Controlling and managing network access
– Managing systems and networks
– Creating user interfaces, including navigation (browsing)

Tasks to tackle for IA
• Learning
• Performance
• Multi agents
• Cost justification
• Security and privacy
• Ethical issues
• Acceptance

References
• Russell, Stuart; Norvig, Peter (1995). Artificial Intelligence: A
Modern Approach
• Lenat, Douglas B., and Randall Davis. "Knowledge-based systems
in artificial intelligence." New York: McGrav-Hill. Nev (1982)
• Jackson, Peter, and Peter Jackson. Introduction to expert
systems. Vol. 2. Reading, MA: Addison-Wesley, 1990
• Aamodt, Agnar, and Enric Plaza. "Case-based reasoning:
Foundational issues, methodological variations, and system
approaches." AI communications 7.1 (1994): 39-59.
• Wooldridge, Michael, and Nicholas R. Jennings. "Intelligent
agents: Theory and practice." The knowledge engineering
review 10.2 (1995): 115-152.

02 knowledge-based systems

More Related Content

What's hot (20)

Similar to 02 knowledge-based systems (20)

Recently uploaded (20)

02 knowledge-based systems