Introduction to knowledge discovery

Introduction to
Knowledge Engineering
Dr. Albert Fornells Herrera
http://www.linkedin.com/in/afornells
Introduction to Knowledge Engineering - Albert Fornells

1. Definition
2. Process of Knowledge Engineering
3. References
2
Outline

 What is Knowledge Engineering?
 The process of building intelligent knowledge-based systems
 Build a KBS is still more art than engineering
 Knowledge is incomplete, uncertain and heuristic
 Complex to estimate the nature and the amount of knowledge needed
 An incremental and prototyped approach is the way to incrementally evolve
the system from its conception until the final version
 Six iterative basic phases (Waterman, 1986; Durkin, 1994):
1. Problem assessment
2. Data and knowledge acquisition
3. Development of a prototype system
4. Development of a complete system
5. Evaluation and revision of the system
6. Integration and maintenance of the system
3
Knowledge Engineering

4
The process of Knowledge Engineering
Problem assessment
Data and knowledge
acquisition
Development of a
prototype system
Development of a
complete system
Evaluation and revision of
the system
Integration and
maintenance of the system
Determine the characteristics of the problem
Identify the main participants in the project
Specify the project’s objective
Determine the resources needed for building the system
Collect and analyze data and knowledge
Make key concepts of the system design more explicit
Choose a tool for building an Intelligent system
Transform data and represent the knowledge
Design and implement a prototype system
Test the prototype with test cases
Prepare a detailed design for a full-scale system
Collect additional data and knowledge
Develop the user interface
Implement the complete system
Evaluate the system against the performance criteria
Revise the system as necessary
Make arrangements for technology transfer
Establish an effective maintenance program
Requirements
Concepts
Structure
KBS ‘beta’
Reformulation
Redesign
Tuning
KBS tested
Maintenance
Assessment
Redesign
Iterations

 Goal: (1) Determine the characteristics of the problem; (2) Identify the participants
of the project; (3) Specify the objectives of the project and (4) Determine what
resources are needed for building the system.
 The problem characterization implies:
 Determine the problem type:
 Diagnosis. Inferring malfunctions of an object from its behavior and recommending solutions
 Selection. Recommending the best option from a list of possible alternatives
 Prediction. Predicting the future behavior of an object from its behavior in the past
 Classification. Assigning an object to one of the defined classes
 Clustering. Dividing an heterogeneous group of objects into homogeneous subgroups
 Optimization. Improving the quality of solutions until an optimal one is found
 Control Governing the behavior of an object to meet specified requirements in real-time
 Input and output variables and their interactions
 The form and content of the solution.
5
KE process: Problem assessment

 Identify the participants of the project. There are two critical participants:
 The knowledge engineer (a person capable of designing, building and testing an
intelligent system)
 The domain expert (a knowledgeable person capable of solving problems in a
specific area or domain)
 Specify the project’s objectives
 Gaining a competitive edge, improving the quality of decisions, reducing labor costs,
and improving the quality of products and services, etc
 Determine what resources are needed for building the system
 Computer facilities
 Development software
 Knowledge and data sources (human experts, textbooks, manuals, web sites,
databases and examples)
 Money
6

 Toy problem: Found X.
 Do you think that a KBS is needed?
 X=sqrt(3^2+4^2)=5
 Maybe it is not necessary
 But maybe experts do not think the same because they misunderstand the
questions.
 They apply computer vision techniques for text recognition.
 Who is wrong?
7
4cm
3cm
x
4cm
3cm
x
Here is

 Goal: To obtain further understanding of the problem domain by (1) collecting and
analyzing both data and knowledge and (2) making key concepts of the system’s
design more explicit
 The choice of the system building tool depends on the acquired data. Data is
often collected from different sources and thus can be of different types.
 Continuous variables vs variables divided into several ranges vs normalized (i.e. 0 - 1)
 Symbolic (textual) data vs numerical data vs heterogeneous data
 Imprecise and noisy data vs well-defined and clean data
 As a result, the data must be transformed, or massaged, into the form useful for
a particular tool. There are three important issues that must be resolved before
massaging the data (Berry and Linoff, 1997)
 Incompatible data
 Inconsistent data
 Missing data
8
KE process: Data and knowledge acquisition

 Knowledge acquisition is an inherently iterative process based on:
 Start with reviewing documents and reading books, papers and manuals related to
the problem domain in order to become familiar with the problem
 Collect further knowledge through interviewing the domain expert. The expert is
asked to identify four or five typical cases, describe how each case is solved and
explain, or ‘think out loud’, the reasoning behind each solution (Russell, 2002)
 Study and analyze the acquired knowledge, and repeat the entire process again
 However, extracting knowledge from a human expert is a difficult process – it is
often called the ‘knowledge acquisition bottleneck’.
 Quite often experts are unaware of the amount of knowledge they have and the
problem-solving strategy they use, or are unable to verbalize it.
 Experts may also provide us with irrelevant, incomplete or inconsistent information
 Example of cheese factory (Donald Michie, 1982)
 The data and knowledge acquired should enable us to describe the problem-
solving strategy at the most abstract level, but any detailed analysis has to be
done before evaluating the prototype
9
KE process: Data and knowledge acquisition

 Goal: Create an intelligent system – or, rather, a small version of it – and test it
with a number of test cases.
 What is a prototype?
 A prototype system can be defined as a small version of the final system
 It is designed to test how well we understand the problem, or in other words, to make
sure that the problem-solving strategy, the tool selected for building a system, and
techniques for representing acquired data and knowledge are adequate to the task
 It also provides us with an opportunity to actively engage the domain expert in the
system’s development
 What is a test case?
 Once it is built, the prototype’s performance needs to be tested (usually together with
the domain expert)
 The domain expert takes an active part in testing the system and, as a result,
becomes more involved in the system’s development
 What should we do if we have made a bad choice of the system-building tool?
 We should throw the prototype away and start the prototyping phase over again
10
KE process: Development of a prototype system

 Goal: Once prototype is assessed, we develop a plan, schedule and budget for
the complete system, and also clearly define the system’s performance.
 Main milestones:
 Adding data and knowledge to the system (i.e. collect core rules, additional
historical examples, etc.)
 Develop the user interface for delivering information to a user. Some system
focuses on explaining its reasoning process and justify its advice. Others represent
data in graphical forms.
 The development is like an evolutionary process where, as the project needs,
new data and knowledge are collected and added to the system.
11
KE process: Development of a complete system

 Goal: To evaluate an intelligent system is, in fact, to assure that the system
performs the intended task to the user’s satisfaction
 Intelligent systems are designed to solve problems that quite often do not have
clearly defined ‘right’ and ‘wrong’ solutions
 A formal evaluation of the system is normally accomplished with the test cases
selected by the user
 The system’s performance is compared against the performance criteria that were
agreed at the end of the prototyping phase
 The evaluation often reveals the system’s limitations and weaknesses, so it is
revised and relevant development phases are repeated
 There are specific methodological procedures to assess the performance
(Ian Witten, 2005)
 X Stratified cross-validation
 Leave-one out
 Statistical tests are a common way for comparing the performance of the
approach with respect to experts and also with respect to others approaches
12
KE process: Evaluation and revision

 Goal: It involves integrating the system into the environment where it will
operate and establishing an effective maintenance program
 What means integration?
 Integration means interfacing the new intelligent system with existing systems within
an organization and arranging for technology transfer
 What means establish a maintenance program?
 Intelligent systems are knowledge-based systems, and because knowledge evolves
over time, the system has to be able to be upgraded
 The organization that uses the system should have in-house expertise to maintain
and modify the system
 Is maintenance forever?
 No, a big change in the knowledge domain could imply to throw away the intelligent
system and start from the beginning.
13
KE process: Integration and maintenance

14
Summary: Roadmap for tackling a problem
Knowledge
Problem
Analysis
Data
Analysis
Data
Processing
Modeling
Evaluation
Production

Problem: I want to develop an intelligent system that can help me to fix
malfunctions of my computer.
Describe, from a global point of view, how will you tackle the process of Knowledge
Engineering?
1. Problem assessment
2. Data and knowledge acquisition
3. Development of a prototype system
4. Development of a complete system
5. Evaluation and revision of the system
6. Integration and maintenance of the system
15
Exercise: Will a KBS work for my problem?

 A. J. González y D. D. Dankel, The Engineering of Knowledge Based Systems.
Theory and Practice. Prentice Hall, 2000.
16
References

Introduction to knowledge discovery

More Related Content

What's hot (20)

Viewers also liked (17)

Similar to Introduction to knowledge discovery (20)

Recently uploaded (20)

Introduction to knowledge discovery