Concept Learning with types and benefits

What is Concept Learning?
• Concept learning is a process of acquiring knowledge or understanding about
specific concepts or categories from examples. In essence, it's the ability of a
machine learning model to identify patterns and generalize from data in order to
classify new instances correctly.
• Training Data Collection: Concept learning begins with a set of examples that
belong to different categories or classes. Each example is represented by a set of
features that describe its characteristics.
• Hypothesis Space: The concept learning algorithm operates within a hypothesis
space, which is a set of possible hypotheses or rules that can be used to classify
instances. The goal of the algorithm is to find the hypothesis that best fits the
training data and can generalize to new, unseen data.
• Learning Process: The algorithm uses the training data to adjust its hypotheses or
rules in order to classify instances correctly. It aims to identify patterns,
relationships, and rules that differentiate between different classes.

What is Concept Learning?
• Generalization: Once the algorithm has learned from the training data, it should be able to
generalize its understanding of the concept to new, unseen instances. This means it should be
able to accurately classify instances it hasn't encountered during training.
• Evaluation: The effectiveness of the concept learning algorithm is typically evaluated using
testing data or cross-validation techniques. This ensures that the algorithm's learned concept
generalizes well and doesn't simply memorize the training data (overfitting).
• Concept learning is crucial in various machine learning tasks, such as classification and pattern
recognition. The process can be straightforward for simple concepts, but it becomes more
complex as the concept involves intricate relationships among features or when dealing with
noisy or ambiguous data.

What is the Aim of Concept Learning?
Concept learning aims to find a function or rule that truly represents the particular
concept being learned.
The function must be a true representation of the concept so that it can be able to
make accurate classifications of unseen data. By “true representation”, it means that
the function must be able to approximate the true value of a target concept.
The target concept refers to what we’re trying to classify.

How Concept Learning works :
Suppose that the bank wants to classify customers according to five features:
•Gender
•Age
•Income
•Dependents
•Loan Amount
Each row in the training set represents a single applicant (known as an instance).
Each applicant has five features with differing values, and has the possible outcomes,
“Yes” or “No,” depending on the values of those features.

Each customer is either a negative or positive example.
The applicants whose loan application is accepted are the positive examples, while
the applicants whose loan application is rejected are the negative examples.

Representation of Hypotheses:
Each hypothesis in the hypothesis space depends on a certain number of features. In our case,
each hypothesis depends on two features.
Each feature has a value associated with it, and each value is represented using a constraint. A
constraint is an indication of the importance of each feature in each hypothesis.
There are three types of constraints to represent the values of each feature. These are
• the single-value constraint,
• the specific constraint, and
• the general constraint.
A hypothesis is often represented as a vector of constraints.
For example, suppose that the bank prefers applicants older than 18, with an income greater
than 4000, regardless of the applicant’s gender, the loan amount, or the number of dependents.
This hypothesis would be represented as: < “?”, >18, “>4K”, “?”, “?”>

< “?”, >18, “>4K”, “?”, “?”>
It is a vector of constraints, where the constraints can be interpreted as follows:
•The gender of the applicant doesn’t matter, and so gender is represented using a
general constraint. Having a general constraint means that applicants of any gender
are a positive example. The same applies to the loan amount and the number of
dependents.
•Income has to be greater than 4500, and age has to be greater than 18 for the
applicant to be accepted. Income and age are thus represented as single-value
constraints. A single-value constraint means that only applicants of age greater than
18 or income higher than 4000 are positive examples.
Two more types of hypotheses are used in searching for the best hypothesis. These
are the most general hypothesis and the most specific hypothesis.

• The Most General Hypothesis is named as such because it uses the general
constraint for every feature in the hypothesis. The most general hypothesis is thus
denoted as follows:
< “?”, “?”, “?”, “?”, “?”>
• This hypothesis implies that any value is acceptable for any feature and that each
applicant is a positive example.
• The Most Specific Hypothesis uses the specific constraint for every feature in the
hypothesis. The most specific hypothesis is represented as follows:
< “0”, “0”, “0”, “0”, “0”>
• This hypothesis implies that none of the features’ values are acceptable and that
none of the applicants is a positive example.

Example:
Let’s assume there are two features F1 and F2 with F1 has A and B as
possibilities and F2 as X and Y as possibilities.
F1 – > A, B
F2 – > X, Y
Instance Space: (A, X), (A, Y), (B, X), (B, Y) – 4 Examples
Hypothesis Space: (A, X), (A, Y), (A, ø), (A, ?), (B, X), (B, Y), (B, ø), (B, ?), (ø,
X), (ø, Y), (ø, ø), (ø, ?), (?, X), (?, Y), (?, ø), (?, ?) – 16
Semantically Distinct Hypothesis Space: (A, X), (A, Y), (A, ?), (B, X), (B,
Y), (B, ?), (?, X), (?, Y), (?, ?), (ø, ø) – 10
? --Accept anything
Ø -- Does not accept

Hypothesis Space
Similarly there are 5 . 4 . 4 . 4 . 4 . 4 = 5120 syntactically distinct hypotheses within
H.

Notice, however, that every hypothesis containing one or more “ø” symbols represents
the empty set of instances; that is, it classifies every instance as negative.
Therefore, the number of semantically distinct hypotheses is only 1 + (4 . 3 . 3 . 3 . 3 . 3)
= 973.
Our EnjoySport example is a very simple learning task, with a relatively small, finite
hypothesis space.

General-to-Specific Ordering of Hypotheses
To illustrate the general-to-specific ordering, consider the two hypotheses
h1 = (Sunny, ?, ?, Strong, ?, ?)
h2 = (Sunny, ?, ?, ?, ?, ?)
Now consider the sets of instances that are classified positive by hl and by h2.
Because h2 imposes fewer constraints on the instance, it classifies more instances as
positive.
In fact, any instance classified positive by h1 will also be classified positive by h2.
Therefore, we say that h2 is more general than h1.
For any instance x in X and hypothesis h in H, we say that x satisjies h if and only if
h(x) = 1.
We define the more_general_than_or_equale_to relation in terms of the sets of
instances that satisfy the two hypotheses.

Finding the Best Hypothesis:
There are many methods for searching for the best hypothesis. One such method is
the Find-S method.
The Find-S algorithm helps search for the best hypothesis (called the maximally
specific hypothesis). The idea behind this method is to compare feature values in the
most specific hypothesis to those of each positive example in the training set.

The algorithm starts by searching for each positive example.
What it’s looking for is whether the value of the feature in the positive example is the
same as the corresponding value of the feature in the hypothesis.
If the values are the same, then the algorithm will move on to the next positive
example. If the values are different, then the algorithm will change the value of the
feature in the most specific hypothesis to that of a general constraint, “?”.
The algorithm continues this process until it reaches the last positive example in the
training set. Then this leads to the maximally specific constraint.
Find-S Algorithm

Steps of the Find-S Algorithm
1. Let h be the most specific hypothesis.
2. For each positive example in the training set and each feature in the
examples, if the value of the feature equals the corresponding value
of the most specific hypothesis, do nothing.
3. Otherwise, replace the hypothesis value with the general constraint,
“?”.
Find-S Algorithm

Example of the Find-S Algorithm
Suppose that you want to play a sport and want to decide on which day you enjoy the
sport. Each day has six features, sky, temperature, humidity, wind, water, and forecast,
as shown in the training set:

To start with the Find-S algorithm, choose any positive example in the training
dataset and initialize it as the most specific hypothesis (Let’s choose row 1):
• Compare the values of the features to the first positive training example in row
1:
• The values of the training example and the most specific hypothesis are the same, so we
do nothing. We move on to the next positive training example (in row 2), and compare it to
the most specific hypothesis:
The values of the features for humidity are different, so we replace the feature in S2
with “?”. So now we have:

• Row 3 is a negative example, so we ignore it. We then move on to the next positive
training example (in row 4), and compare it to the most specific hypothesis, S2:
The values of the features for water are different, so we replace the feature in S2
with “?”. So now we have:
So now we have reached the last positive example, and we have the maximally
specific hypothesis:
<Sunny, Warm, ?, Strong, ?, Same>

Concept Learning Applied to the Real World?
Concept learning answers many business questions and enables organizations to take
appropriate steps in their business. It helps organizations make quick and accurate
classifications with large amounts of data.
Spam Filtering
•Answers the question: Is this email spam or not spam?
•It uses an email’s characteristics to determine whether the email will be in spam or not.
Customer Purchasing
•Answers the question: Is the customer likely to buy or not buy?
•A customer’s characteristics, such as past purchase behavior, are used to determine whether
they will buy or not.
School Admissions
•Answers the question: Is the student likely to pass or fail?
•A student’s characteristics, such as academic scores, determine whether they will pass or fail.
Medical Diagnoses
•Answers the question: Is the patient likely to have the disease or not?
•A patient’s characteristics, such as symptoms, age, and medical history, are used to
determine whether they have a certain disease or not.

Candidate Elimination
Algorithm

Candidate Elimination
Algorithm
The Candidate-Elimination algorithm computes the version space containing all
hypotheses from H that are consistent with an observed sequence of training
examples.
• It begins by initializing the version space to the set of all hypotheses in H; that is,
by initializing the G boundary set to contain the most general hypothesis in H
G0 = { <?, ?, ?, ?, ?, ?> }
and initializing the S boundary set to contain the most specific hypothesis
S0 = { <0, 0, 0, 0, 0, 0> }
• These two boundary sets delimit the entire hypothesis space, because every other
hypothesis in H is both more general than S0 and more specific than G0.
• As each training example is considered, the S and G boundary sets are generalized
and specialized, respectively, to eliminate from the version space any hypotheses
found inconsistent with the new training example.
• After all examples have been processed, the computed version space contains all

Step1: Load Data set
Step2: Initialize General Hypothesis and Specific Hypothesis.
Step3: For each training example
Step4: If example is positive example
make specific hypothesis more general [FIND-S]
if attribute_value == hypothesis_value:
Do nothing
else:
replace attribute value with '?' (Basically
generalizing it)
Step5: If example is Negative example
make general hypothesis more specific.
CEA steps

Algorithmic steps:
Initially : G = [[?, ?, ?, ?, ?, ?], [?, ?, ?, ?, ?, ?], [?, ?, ?, ?, ?, ?],
[?, ?, ?, ?, ?, ?], [?, ?, ?, ?, ?, ?], [?, ?, ?, ?, ?, ?]]
S = [Null, Null, Null, Null, Null, Null]
For instance 1 : <'sunny','warm','normal','strong','warm ','same'> and positive
output.
G1 = G
S1 = ['sunny','warm','normal','strong','warm ','same’]
For instance 2 : <'sunny','warm','high','strong','warm ','same'> and positive output.
G2 = G
S2 = ['sunny','warm',?,'strong','warm ','same']

For instance 3 : <'rainy','cold','high','strong','warm ','change'> and negative
output.
G3 = [['sunny', ?, ?, ?, ?, ?], [?, 'warm', ?, ?, ?, ?], [?, ?, ?, ?, ?, ?],
[?, ?, ?, ?, ?, ?], [?, ?, ?, ?, ?, ?], [?, ?, ?, ?, ?, 'same’]]
S3 = S2
For instance 4 : <'sunny','warm','high','strong','cool','change'> and positive
output.
G4 = G3
S4 = ['sunny','warm',?,'strong', ?, ?]
At last, by synchronizing the G4 and S4 algorithm produce the output.
Output :
G = [['sunny', ?, ?, ?, ?, ?], [?, 'warm', ?, ?, ?, ?]]
S = ['sunny','warm',?,'strong', ?, ?]

Concept Learning with types and benefits

More Related Content

Similar to Concept Learning with types and benefits (20)

More from mousmiin (6)

Recently uploaded (20)

Concept Learning with types and benefits