Unit3_Classification_BAYES_Machine_Learning.pdf
- 1. Course - Machine Learning Course code-IT 312 Topic- Types of ML(Unit-II) Sanjivani Rural Education Society’s Sanjivani College of Engineering, Kopargaon-423603 (An Autonomous Institute Affiliated to Savitribai Phule Pune University, Pune) NAAC ‘A’Grade Accredited, ISO 9001:2015 Certified Department of Information Technology (NBA Accredited) Dr.R.D.Chintamani Asst. Prof. 1
- 2. ML- Unit-III CLASSIFICATION Department of IT Unit-III-CLASSIFICATION • Syllabus • Naïve Bayes Classifier, Model Assumptions, Probability estimation • Required data processing, M-estimates, Feature selection: Mutual • information, Classifier 2
- 3. ML- Unit-III CLASSIFICATION Department of IT Naïve Bayes Classifier • Naïve Bayes algorithm is a supervised learning algorithm, which is based on Bayes theorem and used for solving classification problems. • It is mainly used in text classification that includes a high-dimensional training dataset. • Naïve Bayes Classifier is one of the simple and most effective Classification algorithms which helps in building the fast machine learning models that can make quick predictions. • It is a probabilistic classifier, which means it predicts on the basis of the probability of an object. 3
- 4. ML- Unit-III CLASSIFICATION Department of IT Why is it called Naïve Bayes? 4 • Naïve: It is called Naïve because it assumes that the occurrence of a certain feature is independent of the occurrence of other features. Such as if the fruit is identified on the bases of color, shape, and taste, then red, spherical, and sweet fruit is recognized as an apple. Hence each feature individually contributes to identify that it is an apple without depending on each other. • Bayes: It is called Bayes because it depends on the principle of Bayes' Theorem.
- 5. ML- Unit-III CLASSIFICATION Department of IT Bayes' Theorem: 5 • used to determine the probability of a hypothesis with prior knowledge. It depends on the conditional probability. P(A|B) is Posterior probability: Probability of hypothesis A on the observed event B. P(B|A) is Likelihood probability: Probability of the evidence given that the probability of a hypothesis is true.
- 6. ML- Unit-III CLASSIFICATION Department of IT Bayes' Theorem: 6 P(A) is Prior Probability: Probability of hypothesis before observing the evidence. P(B) is Marginal Probability: Probability of Evidence.
- 7. ML- Unit-III CLASSIFICATION Department of IT Working of Naïve Bayes' Classifier: 7 Suppose we have a dataset of weather conditions and corresponding target variable "Play". So using this dataset we need to decide that whether we should play or not on a particular day according to the weather conditions. So to solve this problem, we need to follow the below steps: • Convert the given dataset into frequency tables. • Generate Likelihood table by finding the probabilities of given features. • Now, use Bayes theorem to calculate the posterior probability.
- 8. ML- Unit-III CLASSIFICATION Department of IT Problem: If the weather is sunny, then the Player should play or not? 8
- 9. ML- Unit-III CLASSIFICATION Department of IT Frequency table for the Weather Conditions: 9 Outlook Play 0 Rainy Yes 1 Sunny Yes 2 Overcast Yes 3 Overcast Yes 4 Sunny No 5 Rainy Yes 6 Sunny Yes 7 Overcast Yes 8 Rainy No 9 Sunny No 10 Sunny Yes 11 Rainy No 12 Overcast Yes 13 Overcast Yes
- 10. ML- Unit-III CLASSIFICATION Department of IT Frequency table for the Weather Conditions: 10 Weather Yes No Overcast 5 0 Rainy 2 2 Sunny 3 2 Total 10 5
- 11. ML- Unit-III CLASSIFICATION Department of IT Likelihood table weather condition: 11 Weather No Yes Overcast 0 5 5/14= 0.35 Rainy 2 2 4/14=0.29 Sunny 2 3 5/14=0.35 All 4/14=0.29 10/14=0.71
- 12. ML- Unit-III CLASSIFICATION Department of IT Applying Bayes 'theorem: 12 P(Yes|Sunny)= P(Sunny|Yes)*P(Yes)/P(Sunny) P(Sunny|Yes)= 3/10= 0.3 P(Sunny)= 0.35 P(Yes)=0.71 So P(Yes|Sunny) = 0.3*0.71/0.35= 0.60 P(No|Sunny)= P(Sunny|No)*P(No)/P(Sunny) P(Sunny|NO)= 2/4=0.5 P(No)= 0.29 P(Sunny)= 0.35 So P(No|Sunny)= 0.5*0.29/0.35 = 0.41 So as we can see from the above calculation that P(Yes| Sunny)>P(No|Sunny) Hence on a Sunny day, Player can play the game.
- 13. ML- Unit-III CLASSIFICATION Department of IT Advantages and Disadvantages of Naïve Bayes Classifier: 13 Advantages • Naïve Bayes is one of the fast and easy ML algorithms to predict a class of datasets. • It can be used for Binary as well as Multi-class Classifications. • It performs well in Multi-class predictions as compared to the other Algorithms. • It is the most popular choice for text classification problems. Disadvantages of Naïve Bayes Classifier: • Naive Bayes assumes that all features are independent or unrelated, so it cannot learn the relationship between features.
- 14. ML- Unit-III CLASSIFICATION Department of IT Advantages and Disadvantages of Naïve Bayes Classifier: 14 Advantages • Naïve Bayes is one of the fast and easy ML algorithms to predict a class of datasets. • It can be used for Binary as well as Multi-class Classifications. • It performs well in Multi-class predictions as compared to the other Algorithms. • It is the most popular choice for text classification problems. Disadvantages of Naïve Bayes Classifier: • Naive Bayes assumes that all features are independent or unrelated, so it cannot learn the relationship between features.
- 15. ML- Unit-III CLASSIFICATION Department of IT Applications of Naïve Bayes Classifier: 15 • It is used for Credit Scoring. • It is used in medical data classification. • It can be used in real-time predictions because Naïve Bayes Classifier is an eager learner. • It is used in Text classification such as Spam filtering and Sentiment analysis.
- 16. ML- Unit-III CLASSIFICATION Department of IT Types of Naïve Bayes Model: 16 • Gaussian: The Gaussian model assumes that features follow a normal distribution. This means if predictors take continuous values instead of discrete, then the model assumes that these values are sampled from the Gaussian distribution. • Multinomial: The Multinomial Naïve Bayes classifier is used when the data is multinomial distributed. It is primarily used for document classification problems, it means a particular document belongs to which category such as Sports, Politics, education, etc. The classifier uses the frequency of words for the predictors. • Bernoulli: The Bernoulli classifier works similar to the Multinomial classifier, but the predictor variables are the independent Booleans variables. Such as if a particular word is present or not in a document. This model is also famous for
- 17. ML- Unit-III CLASSIFICATION Department of IT Types of Naïve Bayes Model: 17 • Gaussian: The Gaussian model assumes that features follow a normal distribution. This means if predictors take continuous values instead of discrete, then the model assumes that these values are sampled from the Gaussian distribution. • Multinomial: The Multinomial Naïve Bayes classifier is used when the data is multinomial distributed. It is primarily used for document classification problems, it means a particular document belongs to which category such as Sports, Politics, education, etc. The classifier uses the frequency of words for the predictors. • Bernoulli: The Bernoulli classifier works similar to the Multinomial classifier, but the predictor variables are the independent Booleans variables. Such as if a particular word is present or not in a document. This model is also famous for