Regression analysis and its type
- 1. MAHARSHI DAYANAND SARASWATI UNIVERSITY , AJMER TYPES OF REGRESSION ANALYSIS SUBMITTED BY, EKTA BAFNA M.TECH 2nd SEM. SUBMITTED TO, SIR PRAMOD RATHORE
- 2. TABLE OF CONTENTS Introduction of regression Analysis Terminologies Linear Regression Polynomial Regression Support Vector Regression Decision Tree Regression Ridge Regression Lasso Regression Logistic Regression
- 3. REGRESSION ANALYSIS IS A FORM OF PREDICTIVE MODELLING TECHNIQUE WHICH INVESTIGATES THE RELATIONSHIP BETWEEN A DEPENDENT (TARGET) AND INDEPENDENT VARIABLE (S) (PREDICTOR). THIS TECHNIQUE IS USED FOR FORECASTING, TIME SERIES MODELLING AND FINDING THE CAUSAL EFFECT RELATIONSHIP BETWEEN THE VARIABLES. REGRESSION IS A SUPERVISED LEARNING TECHNIQUE WHICH HELPS IN FINDING THE CORRELATION BETWEEN VARIABLES AND ENABLES US TO PREDICT THE CONTINUOUS OUTPUT VARIABLE BASED ON THE ONE OR MORE PREDICTOR VARIABLES.
- 4. DEPENDENT VARIABLE INDEPENDENT VARIABLE OUTLIERS UNDERFITTING AND OVERFITTING Terminologies Related to the Regression Analysis:
- 6. LINEAR REGRESSION : • Linear regression is a statistical regression method which is used for predictive analysis. •If there is only one input variable (x), then such linear regression is called simple linear regression. And if there is more than one input variable, then such linear regression is called multiple linear regression. mathematical equation for Linear regression: Y= aX+b Here, Y = dependent variables (target variables), X= Independent variables (predictor variables), a and b are the linear coefficients
- 7. Polynomial Regression: •Polynomial Regression is a type of regression which models the non-linear dataset using a linear model. •It is similar to multiple linear regression, but it fits a non-linear curve between the value of x and corresponding conditional values of y. •Suppose there is a dataset which consists of datapoints which are present in a non- linear fashion, so for such case, linear regression will not best fit to those datapoints. To cover such datapoints, we need Polynomial regression. Polynomial regression equation Y= b0+b1x+ b2x2+ b3x3+.....+ bnxn
- 8. Support Vector Regression: Support Vector Machine is a supervised learning algorithm which can be used for regression as well as classification problems. So if we use it for regression problems, then it is termed as Support Vector Regression. Below are some keywords which are used in Support Vector Regression: Kernel: It is a function used to map a lower- dimensional data into higher dimensional data. Hyperplane: In general SVM, it is a separation line between two classes, but in SVR, it is a line which helps to predict the continuous variables and cover most of the datapoints. Boundary line: Boundary lines are the two lines apart from hyperplane, which creates a margin for datapoints. Support vectors: Support vectors are the datapoints which are nearest to the hyperplane and opposite class.
- 9. Decision Tree Regression: •Decision Tree regression builds a tree-like structure in which each internal node represents the "test" for an attribute, each branch represent the result of the test, and each leaf node represents the final decision or result. •Decision Tree is a supervised learning algorithm which can be used for solving both classification and regression problems. •It can solve problems for both categorical and numerical data Random forest is one of the most powerful supervised learning algorithms which is capable of performing regression as well as classification tasks. With the help of Random Forest regression, we can prevent Overfitting in the model by creating random subsets of the dataset. g(x)= f0(x)+ f1(x)+ f2(x)+....
- 10. Ridge Regression: •Ridge regression is one of the most robust versions of linear regression in which a small amount of bias is introduced so that we can get better long term predictions. •Ridge regression is a regularization technique, which is used to reduce the complexity of the model. It is also called as L2 regularization. The equation for ridge regression will be: A general linear or polynomial regression will fail if there is high collinearity between the independent variables, so to solve such problems, Ridge regression can be used.
- 11. Lasso Regression: •Lasso regression is another regularization technique to reduce the complexity of the model. •It is similar to the Ridge Regression except that penalty term contains only the absolute weights instead of a square of weights. •Since it takes absolute values, hence, it can shrink the slope to 0, whereas Ridge Regression can only shrink it near to 0. •It is also called as L1 regularization. The equation for Lasso regression will be:
- 12. Logistic Regression •Logistic regression is another supervised learning algorithm which is used to solve the classification problems. In classification problems, we have dependent variables in a binary or discrete format such as 0 or 1 , Yes or No, True or False. •Logistic regression uses sigmoid function or logistic function which is a complex cost function. The function can be represented as: f(x)= Output between the 0 and 1 value. x= input to the function e= base of natural logarithm.