7 decision tree

By
Dr.Ramkumar.T
ramkumar.thirunavukarasu@vit.ac.in
Decision Tree
1

Decision Tree
• Relatively fast compared to other classification
models
• Obtain similar and sometimes better accuracy
compared to other models
• Simple and easy to understand
• Can be converted into simple and easy to
understand classification rules
2

Decision Tree
• The tree has three types of nodes.
• A root node - that has no incoming edges
and one or more outgoing edges.
• Internal nodes - Each of which has exactly
one incoming edge and two or more
outgoing edges.
• Leaf or terminal nodes - Each of which
has exactly one incoming edge and no
outgoing edges.
4

How Decision Tree Works ?
• Classifying a test record is straightforward once a
decision tree has been constructed.
• Starting from the root node, we apply the test
condition to the record and follow the appropriate
branch based on the outcome of the test.
• This will lead us either to another internal node,
for which a new test condition is applied, or to a
leaf node.
• The class label associated with the leaf node is
then assigned to the record.
5

Requirements of Decision Tree
• Attribute-value description - object or case must
be expressible in terms of a fixed collection of
properties or attributes (e.g., hot, mild, cold).
• Predefined classes (target values)- the target
function has discrete output values (Boolean or
multiclass)
• Sufficient data- Enough training cases should be
provided to learn the model.
6

How Decision Tree Inducted?
• Tree is constructed in a top-down recursive
divide-and-conquer manner
• At start, all the training examples are at the root
• Attributes are categorical (if continuous-valued,
they are discretized in advance)
• Attributes are selected on the basis of a heuristic
or statistical measure (e.g., information gain)
• Examples are partitioned recursively based on
selected attributes
7

Decision Tree Induction – Measures
• Entropy or Information Theory - One of the techniques
for selecting an attribute to split a node.
• “If you have uncertainty – you have Information”
• Information is defined as “-pilogpi” where „pi‟ is the
probability of some event.
• Information of any event that is likely to have several
possible outcomes is given by
• Information Gain – It is the measure of how good an attribute
is for predicting the class of each of the training data.
8

DT – Measures (ID3 Algorithm)
• Expected information (entropy) needed to
classify a tuple in D:
• Information needed (after using A to split D
into v partitions) to classify D:
 Information gained by branching on
attribute A
9
(D)
Entropy
Entropy(D)
Gain(A) A



10
Gain Ratio Measure (C4.5)
 Information gain measure is biased towards attributes with a
large number of values
 C4.5 (a successor of ID3) uses gain ratio to overcome the
problem (normalization to information gain)
GainRatio(A) = Gain(A)/SplitEntropy(A)
 Ex.
Gain_ratio(income) = 0.029/0.926 = 0.031
 The attribute with the maximum gain ratio is selected as the
splitting attribute
)
|
|
|
|
(
log
|
|
|
|
)
( 2
1 D
D
D
D
D
py
SplitEntro
j
v
j
j
A 

 

926
.
0
)
14
4
(
log
14
4
)
14
6
(
log
14
6
)
14
4
(
log
14
4
)
( 2
2
2 







D
py
SplitEntro A

7 decision tree

More Related Content

What's hot (19)

Similar to 7 decision tree (20)

Recently uploaded (20)

7 decision tree