Classification & preduction
2 likes1,196 views
This document discusses classification and prediction methods including decision trees, naive Bayes classification, and linear regression. It provides examples of each method and explains key concepts like the two step classification process, attribute selection measures for decision trees, and using linear regression to predict an output variable from an input variable.
Classification & preduction
- 1. Classification and prediction Prepared By - Mr. Nilesh Magar
- 2. •What Is Classification? •Example •Two Step Process: Learning Step: Training set made up of DB tuples & their associated class labels- Classification Rule or Decision Tree or mathematical Formulae Classification Step: •Supervised Learning: •Accuracy of the classifier: Prepared By - Mr. Nilesh Magar
- 3. Classifier Class label Attribute Prepared By - Mr. Nilesh Magar
- 4. Decision Tree •Between 1970-1980 J. Rose Quinlan, a researcher in Machine Learning developed a decision tree algorithm known as ID3 (Iterative dichotomiser), C4.5 is the succesor of ID3. •CART(Classification & Regression tree is also developed during the same period which describe the generation of binary tree. • Flowchart like tree structure- root, Node, Branch, leaf node. •How are decision trees used for classification? Prepared By - Mr. Nilesh Magar
- 5. 3 Attribute Selection methods 3 Termination Condition 3 Splitting scenarios Prepared By - Mr. Nilesh Magar
- 6. Splitting Scenarios 1) A is Discrete value 2) A is continuous Valued 3) Discrete Value but Binary tree must be produced Prepared By - Mr. Nilesh Magar
- 7. Termination Condition : Recursive 1. All of the tuples in partition D (represented at node N) belong to the same class (steps 2 and 3), or 2. There are no remaining attributes on which the tuples may be further partitioned (step 4). In this case, majority voting is employed (step 5). This involves converting node N into a leaf and labeling it with the most common class in D. Alternatively, the class distribution of the node tuples may be stored. 3. There are no tuples for a given branch, that is, a partition Dj is empty (step 12). In this case, a leaf is created with the majority class in D (step 13). Prepared By - Mr. Nilesh Magar
- 8. Attribute Selection Measures: 1. Information Gain: 2. Gain Ratio: 3. Gini Index Prepared By - Mr. Nilesh Magar
- 9. Performance: •Quite simple, suitable for relatively small data sets •Large real-world databases? •Training tuples should reside in main memory Issues: •Over fitting Tree pruning 1. Pre-pruning 2. Post-pruning Prepared By - Mr. Nilesh Magar
- 10. Bayes Classification Method •Statistical Classifier •They use to predict class membership probability •Based on Bayes’ Theorm •Naïve •It assumes “effects of an attribute value on a given class is independent of the value of the other attributes” – class condition independence •The name bayes is taken from the name thomas Bayes who did early work in probability and decision theoryduring 18th century. Prepared By - Mr. Nilesh Magar
- 11. Bayesian Theorem •Let X is data tuple “evidence” & H is hypothesis that X belongs to specific class. •Determine P(H|X): •Posterior probability: P(H|X), tuple X contains customers attribute age=35 & salary=40,000 , H customer will buy a computer. •Prior Probability: P(H) •P(X|H) : •P(X) •Bayesian Theorem: P(H|X) = P(X|H) P(H) / P(X) Prepared By - Mr. Nilesh Magar
- 12. Naïve Bayesian classifier: Suppose there are m classes, C1, C2, …..,Cm. Given a tuple, X, the classifier will predict that X belongs to the class having the highest posterior probability, conditioned on X. X belong to Ci If & only if P(Ci|X)>P(Cj|X) for 1<= j <= m, j!=I So Bayes theorem is P(Ci|X) = P(X|Ci) P(Ci) / P(X) As P(X) is constant for all classes so only P(X|Ci) P(Ci) need to be maximize, If class prior probability is not known then P(C1) = P(C2) = …… = P(Cm) so only P(X|Ci) need to maximize. But maximization of P(X|Ci) is computationally expensive so we will apply Class conditional independence, Prepared By - Mr. Nilesh Magar
- 13. Example: Prepared By - Mr. Nilesh Magar
- 14. Prepared By - Mr. Nilesh Magar
- 15. Prepared By - Mr. Nilesh Magar
- 16. Prediction Regression Analysis Can be used to model the relationship between 2 variables. Predictor Variable: The values of the predictor variables are known. Response variable: The response variable is what we want to predict. Linear regression: y = b+wx; y = w0+w1x Prepared By - Mr. Nilesh Magar
- 17. Example Animal height (feet) weight (lbs) Animal1 9 300 Animal2 8.78 295 Animal3 9.6 312 Animal4 8.09 280 Animal5 5 200 Animal6 5.5 250 Animal7 5.42 230 Animal8 5.75 250 Prepared By - Mr. Nilesh Magar
- 18. Given the above data, we compute = 7.15 and = 264.7 (9-7.15)(300–264.7)+(8.78–7.15)(295–264.7)+(9.6–7.15)(312–264.7)+………+(5.75-7.15)(250–264.7) W1= (9 – 7.15)2 + (8.78 – 7.15 ) 2 +……… (5.75-7.15) 2 = 19.35337 Let w 0 = 264.7 – (19.35337)(7.15) = 126.3234 y = 126.3234 + 19.35337x. Using this equation, we can predict that the Animal with 8 feet height can have 281.1504 lbs weight.( 126.3234 + 19.35337(8)) Prepared By - Mr. Nilesh Magar
- 19. Subjects 1) U.M.L. 2) P.P.L. 3) D.M.D.W. 4) O.S. 5) Programming Languages 6) RDBMS Mr. Nilesh Magar Lecturer at MIT, Kothrud, Pune. 9975155310. Prepared By - Mr. Nilesh Magar
- 20. Thank You Prepared By - Mr. Nilesh Magar