Post pruning
- 1. Post pruning methods Mojtaba amiri M.S of artificial intelligence
- 2. Approaches • top-down approach : proceed from the root towards the leaves of tree • bottom-up approach : starting the analysis from the leaves and finishing with the root node
- 3. Based methods 1. Reduced Error Pruning 2. Pessimistic error pruning 3. Error complexity pruning 4. Minimum Error pruning 5. Cost based pruning 6. Iterative growing and pruning algorithm
- 4. Reduced Error Pruning • This method was proposed by Quinlan . • This method considers each of the decision nodes in the tree to be candidates for pruning, consist of removing the subtree rooted at that node, making it a leaf node . • The available data is divided into three parts: training examples , validation examples for pruning , a set of test examples for estimate of accuracy over future unseen examples • Next step calculating new tree accuracy
- 5. Reduced Error Pruning • The problems related to the method of reduced error pruning are basically two: 1. The use of a pruning set distinct from the training set is inadequate when a small number of observations are available 2. the parts of the original tree that correspond to special cases outside the test set may be lost after pruning. Therefore trees pruned via that pruning method may fail in correctly classifying exceptional cases.
- 6. Reduced Error Pruning • Pros : linear computational complexity • Cons : over pruning , if the test set is much smaller than the training set
- 7. Pessimistic error pruning • This pruning method, proposed by Quinlan • it avoids using an independent pruning set • The misclassification rate, estimated by means of the training set • If results to be optimistic , thus it always happens that pruning operations based on the same data set produce trees that are larger than it is needed .
- 8. Pessimistic error pruning • Pros : This method has a linear complexity in the number of leaves • Cons: the worst case is that in which the tree has not to be pruned at all
- 9. Error complexity pruning • It finds error complexity at each node . 1. Calculating the error cost of the node 2. r(t) is error rate of a node 3. p(t) is probability of occurrence of a node 4. If node t was not pruned then error cost of subtree T , rooted at t: 5. Then , The error complexity of the node
- 10. Error complexity pruning • The method consists of following steps: 1. a ( error complexity ) is computed for each node. 2. the minimum a node is pruned. 3. the above is repeated and a forest of pruned tree is formed. 4. the tree with best accuracy is selected.
- 11. Minimum Error pruning • This method was developed by Niblett and Brotko • bottom-up approach • seeks a single tree that minimizes the expected error rate on an independent data set • If it is predicted that all future examples will be in class c, the following equation is used to predict the expected error rate of pruning at node t : 1. K : # of class 2. nt : # of examples in node t 3. nt,c : # of examples assigned to class c in node t
- 12. Minimum Error pruning • The method consists of following steps 1. At each non leaf node in the tree, calculate expected error rate if that subtree is pruned. 2. Calculate the expected error rate for that node if subtree is not pruned. 3. If pruning the node leads to greater expected error rate, then keep the subtree; otherwise, prune it .
- 13. Cost based pruning • In this method not only an error rate is considered at each node but also a cost is considered . • That is for pruning decision tree error rate and cost of deciding selection of one or more class-label attribute is considered .
- 14. Iterative growing and pruning algorithm • Gelfand et al • These goals are reached by splitting the data set into two subsets • then by repeatedly growing and pruning a tree on different subsets • a tree is grown by using the first subset • then it is pruned by using the second subset