Ensemble modeling and Machine Learning
0 likes690 views
This document discusses ensemble modeling techniques. It begins with an introduction to ensemble models and their advantages over single models in reducing biases, variability, and inaccuracies. It then explains how ensemble models work by combining the predictions from multiple machine learning models. Common ensemble methods like bagging and boosting are described, along with the mathematics of reducing bias, variance, and noise. Bagging is explained in more detail, including the bagging algorithm and an example of bagging ensembles using R. The document concludes by outlining topics to cover in subsequent sections, such as boosting, comparing bagging and boosting, and gradient boosting machines.
Ensemble modeling and Machine Learning
- 1. Ensemble Modeling Step Up Analytics July 16, 2017 Ganesh S Step Up Analytics 1 / 12
- 2. Road Map Introduction Ensemble models and possible drawback/s of single speciļ¬c model How ensemble models works and example Frequently used ensemble methods and mathematics Bagging and Bagging Algorithm Bagging ensembles using R Comparison of result Continue with. . . Ganesh S Step Up Analytics 2 / 12
- 3. Introduction Many of you might studied and practiced diļ¬erent classiļ¬cation as well regression algorithms. Also, many a time modeler uses a model at a time. Ever wondered what would happen if we could combine more than one classiļ¬cation model? Whether resulting combo might more accurate or less variant? Will answer these questions shortly Ganesh S Step Up Analytics 3 / 12
- 4. Ensemble models and possible drawback/s of single speciļ¬c model Ensembles are the answers to these questions It is the process of running two or more related but diļ¬erent machine learning models and then synthesizing the results into single predictive or machine learning model It can have biases Presence of high variability Outright inaccuracies Ganesh S Step Up Analytics 4 / 12
- 5. How ensemble models works and example of ensemble Producing a distribution called a simple ML model on the subset of original data Combining the distribution in one aggregated model Random Forest It is the group of multiple decision trees which built on diļ¬erent sample data,evaluates diļ¬erent factors and/or weight common variables diļ¬erently. Ganesh S Step Up Analytics 5 / 12
- 6. How ensembles works Figure: Working of Ensembles Ganesh S Step Up Analytics 6 / 12
- 7. Frequently used ensemble methods and mathematics Bagging Boosting Distance between predated (y) and actual (y) should be less. (y ā y) = Bias + Variance + Noise Bias - The average distance between predictions. Variance - Variability in the predictions. Noise - Lower bound on the prediction error that the predictor can achieve. If we want to minimize(y ā y) we have to minimize above three. Ganesh S Step Up Analytics 7 / 12
- 8. Bagging and Bagging Algorithm Bagging stands for Bootstrapped Aggregation Bagging is the way to decrease variance of your prediction by generating additional training data from the original data with diļ¬erent combination and replications Bagging Algorithm 1. Samples(with replacement) are repeatedly taken from the data set, so that each record has an equal record has an equal probability of being selected, and each sample is of the same size as the original training data set. These are bootstrapped samples. 2. Train the model and record the predictions for each sample. 3. Bagging ensembles will be deļ¬ned as the class with most votes or the average of prediction made. Ganesh S Step Up Analytics 8 / 12
- 9. Bagging Ensembles using R Small case study using R, How ensemble bagging works! Data Source is UCI data repository - Car Evaluation Data Set Regression models is used Bagging Bagging in R Ganesh S Step Up Analytics 9 / 12
- 10. Results of Bagging Figure: Working of Ensembles Ganesh S Step Up Analytics 10 / 12
- 11. Continue with... Boosting and Boosting in R Bagging and Boosting case study in python Bagging-Boosting comparison Famous GBM(Gradient Boosting Method) GBM in R as well in Python with case study Ganesh S Step Up Analytics 11 / 12
- 12. Thank You !!! Ganesh S Step Up Analytics 12 / 12