![BDA- Unit-II Regression Department of IT
Simple Linear Regression With scikit-learn
25
• Step 2: Provide data
• The second step is defining data to work with. The inputs (regressors, 𝑥)
and output (response, 𝑦) should be arrays or similar objects. This is the
simplest way of providing data for regression:
• >>> x = np.array([5, 15, 25, 35, 45, 55]).reshape((-1, 1))
• >>> y = np.array([5, 20, 14, 32, 22, 38])
• Now, you have two arrays: the input, x, and the output, y. You should call
.reshape() on x because this array must be two-dimensional, or more
precisely, it must have one column and as many rows as necessary. That’s
exactly what the argument (-1, 1) of .reshape() specifies.](https://image.slidesharecdn.com/unit2regression-240517095730-f75e77d2/85/Unit2_Regression-ADVANCED-ANALYTICAL-THEORY-AND-METHODS-USING-PYTHON-25-320.jpg)
![BDA- Unit-II Regression Department of IT
Simple Linear Regression With scikit-learn
26
• This is how x and y look now:
• >>> x
• array([[ 5],
• [15],
• [25],
• [35],
• [45],
• [55]])
• >>> y
• array([ 5, 20, 14, 32, 22, 38])
• As you can see, x has two dimensions, and x.shape is (6, 1), while y has a
single dimension, and y.shape is (6,).](https://image.slidesharecdn.com/unit2regression-240517095730-f75e77d2/85/Unit2_Regression-ADVANCED-ANALYTICAL-THEORY-AND-METHODS-USING-PYTHON-26-320.jpg)
![BDA- Unit-II Regression Department of IT
Simple Linear Regression With scikit-learn
31
• When you’re applying .score(), the arguments are also the predictor x and
response y, and the return value is 𝑅².
• The attributes of model are .intercept_, which represents the coefficient 𝑏₀,
and .coef_, which represents 𝑏₁:
• >>> print(f"intercept: {model.intercept_}")
• intercept: 5.633333333333329
• >>> print(f"slope: {model.coef_}")
• slope: [0.54]
• The code above illustrates how to get 𝑏₀ and 𝑏₁. You can notice that
.intercept_ is a scalar, while .coef_ is an array.](https://image.slidesharecdn.com/unit2regression-240517095730-f75e77d2/85/Unit2_Regression-ADVANCED-ANALYTICAL-THEORY-AND-METHODS-USING-PYTHON-31-320.jpg)
Unit2_Regression, ADVANCED ANALYTICAL THEORY AND METHODS USING PYTHON
- 1. Course – Big Data Analytics (Professional Elective-II) Course code-IT314B Unit-II- ADVANCED ANALYTICAL THEORY AND METHODS USING PYTHON Sanjivani Rural Education Society’s Sanjivani College of Engineering, Kopargaon-423603 (An Autonomous Institute Affiliated to Savitribai Phule Pune University, Pune) NAAC ‘A’ Grade Accredited, ISO 9001:2015 Certified Department of Information Technology (NBA Accredited) Mr. Rajendra N Kankrale Asst. Prof. 1
- 2. BDA- Unit-II Regression Department of IT Unit-II- ADVANCED ANALYTICAL THEORY AND METHODS USING PYTHON • Syllabus 2 ADVANCED ANALYTICAL THEORY AND METHODS USING PYTHON Introduction to Scikit-learn, Installations, Dataset, matplotlib, filling missing values, Regression and Classification using Scikit-learn Association Rules: FP growth, Regression: Linear Regression, Logistic Regression, Classification: Naïve Bayes classifier
- 3. BDA- Unit-II Regression Department of IT Unit-II- Regression 3
- 4. BDA- Unit-II Regression Department of IT Unit-II- Regression • Motivation • Regression estimates the relationship between the target and the independent variable. • It is used to find the trends in data. • It helps to predict real/continuous values. • By performing the regression, we can confidently determine the most important factor, the least important factor, and how each factor is affecting the other factors. 4
- 5. BDA- Unit-II Regression Department of IT Linear Regression • Linear Regression is a supervised machine learning algorithm. • It tries to find out the best linear relationship that describes the data you have. • It assumes that there exists a linear relationship between a dependent variable and independent variable(s). • The value of the dependent variable of a linear regression model is a continuous value i.e. real numbers. 5
- 6. BDA- Unit-II Regression Department of IT Representing Linear Regression Model • Linear regression model represents the linear relationship between a dependent variable and independent variable(s) via a sloped straight line • The sloped straight line representing the linear relationship that fits the given data best is called as a regression line. • It is also called as best fit line. 6
- 7. BDA- Unit-II Regression Department of IT Types of Linear Regression- 1. Simple Linear Regression 2. Multiple Linear Regression 7
- 8. BDA- Unit-II Regression Department of IT Simple Linear Regression For simple linear regression, the form of the model is- Y = β0 + β1X 8 Here, Y is a dependent variable. X is an independent variable. β0 and β1 are the regression coefficients. β0 is the intercept or the bias that fixes the offset to a line. β1 is the slope or weight that specifies the factor by which X has an impact on Y.
- 9. BDA- Unit-II Regression Department of IT Simple Linear Regression There are following 3 cases possible- Case-01: β1 < 0 It indicates that variable X has negative impact on Y. If X increases, Y will decrease and vice-versa. 9
- 10. BDA- Unit-II Regression Department of IT Simple Linear Regression Case-02: β1 = 0 • It indicates that variable X has no impact on Y. • If X changes, there will be no change in Y. 10
- 11. BDA- Unit-II Regression Department of IT Simple Linear Regression 11 Case-03: β1 > 0 It indicates that variable X has positive impact on Y. If X increases, Y will increase and vice-versa.
- 12. BDA- Unit-II Regression Department of IT Multiple Linear Regression- 12 In multiple linear regression, the dependent variable depends on more than one independent variables. For multiple linear regression, the form of the model is- Y = β0 + β1X1 + β2X2 + β3X3 + …… + βnXn Here, Y is a dependent variable. X1, X2, …., Xn are independent variables. β0, β1,…, βn are the regression coefficients. βj (1<=j<=n) is the slope or weight that specifies the factor by which Xj has an impact on Y.
- 13. BDA- Unit-II Regression Department of IT Evaluation metrics for a linear regression model Mean Squared Error (MSE) The most common metric for regression tasks is MSE. It has a convex shape. It is the average of the squared difference between the predicted and actual value. Since it is differentiable and has a convex shape, it is easier to optimize. MSE penalizes large errors. 13
- 14. BDA- Unit-II Regression Department of IT Evaluation metrics for a linear regression model 14
- 15. BDA- Unit-II Regression Department of IT Evaluation metrics for a linear regression model Mean Absolute Error (MAE) This is simply the average of the absolute difference between the target value and the value predicted by the model. Not preferred in cases where outliers are prominent. MAE does not penalize large errors. 15
- 16. BDA- Unit-II Regression Department of IT Evaluation metrics for a linear regression model 16
- 17. BDA- Unit-II Regression Department of IT Evaluation metrics for a linear regression model Root Mean Squared Error(RMSE) As RMSE is clear by the name itself, that it is a simple square root of mean squared error. 17
- 18. BDA- Unit-II Regression Department of IT Evaluation metrics for a linear regression model 18
- 19. BDA- Unit-II Regression Department of IT Evaluation metrics for a linear regression model R-squared explains to what extent the variance of one variable explains the variance of the second variable. In other words, it measures the proportion of variance of the dependent variable explained by the independent variable. R squared is a popular metric for identifying model accuracy. It tells how close are the data points to the fitted line generated by a regression algorithm. A larger R squared value indicates a better fit. This helps us to find the relationship between the independent variable towards the dependent variable. 19
- 20. BDA- Unit-II Regression Department of IT Evaluation metrics for a linear regression model • SSE is the sum of the square of the difference between the actual value and the predicted value • SST is the total sum of the square of the difference between the actual value and the mean of the actual value. • yi is the observed target value, ŷi is the predicted value, and y-bar is the mean value, m represents the total number of observations. 20
- 21. BDA- Unit-II Regression Department of IT Evaluation metrics for a linear regression model 21
- 22. BDA- Unit-II Regression Department of IT Evaluation metrics for a linear regression model 22 • R² score ranges from 0 to 1. The closest to 1 the R², the better the regression model is. If R² is equal to 0, the model is not performing better than a random model. If R² is negative, the regression model is erroneous. • A small MAE suggests the model is great at prediction, while a large MAE suggests that your model may have trouble in certain areas. MAE of 0 means that your model is a perfect predictor of the outputs. • If you have outliers in the dataset then it penalizes the outliers most and the calculated MSE is bigger. So, in short, It is not Robust to outliers which were an advantage in MAE.
- 23. BDA- Unit-II Regression Department of IT Simple Linear Regression With scikit-learn 23 • You’ll start with the simplest case, which is simple linear regression. There are five basic steps when you’re implementing linear regression: 1. Import the packages and classes that you need. 2. Provide data to work with, and eventually do appropriate transformations. 3. Create a regression model and fit it with existing data. 4. Check the results of model fitting to know whether the model is satisfactory. 5. Apply the model for predictions.
- 24. BDA- Unit-II Regression Department of IT Simple Linear Regression With scikit-learn 24 • Step 1: Import packages and classes • The first step is to import the package numpy and the class LinearRegression from sklearn.linear_model: • >>> import numpy as np • >>> from sklearn.linear_model import LinearRegression
- 25. BDA- Unit-II Regression Department of IT Simple Linear Regression With scikit-learn 25 • Step 2: Provide data • The second step is defining data to work with. The inputs (regressors, 𝑥) and output (response, 𝑦) should be arrays or similar objects. This is the simplest way of providing data for regression: • >>> x = np.array([5, 15, 25, 35, 45, 55]).reshape((-1, 1)) • >>> y = np.array([5, 20, 14, 32, 22, 38]) • Now, you have two arrays: the input, x, and the output, y. You should call .reshape() on x because this array must be two-dimensional, or more precisely, it must have one column and as many rows as necessary. That’s exactly what the argument (-1, 1) of .reshape() specifies.
- 26. BDA- Unit-II Regression Department of IT Simple Linear Regression With scikit-learn 26 • This is how x and y look now: • >>> x • array([[ 5], • [15], • [25], • [35], • [45], • [55]]) • >>> y • array([ 5, 20, 14, 32, 22, 38]) • As you can see, x has two dimensions, and x.shape is (6, 1), while y has a single dimension, and y.shape is (6,).
- 27. BDA- Unit-II Regression Department of IT Simple Linear Regression With scikit-learn 27 • Step 3: Create a model and fit it • The next step is to create a linear regression model and fit it using the existing data. • Create an instance of the class LinearRegression, which will represent the regression model: • >>> model = LinearRegression() • This statement creates the variable model as an instance of LinearRegression. You can provide several optional parameters to LinearRegression:
- 28. BDA- Unit-II Regression Department of IT Simple Linear Regression With scikit-learn 28 • This statement creates the variable model as an instance of LinearRegression. You can provide several optional parameters to LinearRegression: • fit_intercept is a Boolean that, if True, decides to calculate the intercept 𝑏₀ or, if False, considers it equal to zero. It defaults to True. • normalize is a Boolean that, if True, decides to normalize the input variables. It defaults to False, in which case it doesn’t normalize the input variables. • copy_X is a Boolean that decides whether to copy (True) or overwrite the input variables (False). It’s True by default. • n_jobs is either an integer or None. It represents the number of jobs used in parallel computation. It defaults to None, which usually means one job. - 1 means to use all available processors. • Your model as defined above uses the default values of all parameters.
- 29. BDA- Unit-II Regression Department of IT Simple Linear Regression With scikit-learn 29 • It’s time to start using the model. First, you need to call .fit() on model: • >>> model.fit(x, y) • LinearRegression() • With .fit(), you calculate the optimal values of the weights 𝑏₀ and 𝑏₁, using the existing input and output, x and y, as the arguments. In other words, .fit() fits the model. It returns self, which is the variable model itself. That’s why you can replace the last two statements with this one: • >>> model = LinearRegression().fit(x, y) • This statement does the same thing as the previous two. It’s just shorter.
- 30. BDA- Unit-II Regression Department of IT Simple Linear Regression With scikit-learn 30 • Step 4: Get results • Once you have your model fitted, you can get the results to check whether the model works satisfactorily and to interpret it. • You can obtain the coefficient of determination, 𝑅², with .score() called on model: • >>> r_sq = model.score(x, y) • >>> print(f"coefficient of determination: {r_sq}") • coefficient of determination: 0.7158756137479542
- 31. BDA- Unit-II Regression Department of IT Simple Linear Regression With scikit-learn 31 • When you’re applying .score(), the arguments are also the predictor x and response y, and the return value is 𝑅². • The attributes of model are .intercept_, which represents the coefficient 𝑏₀, and .coef_, which represents 𝑏₁: • >>> print(f"intercept: {model.intercept_}") • intercept: 5.633333333333329 • >>> print(f"slope: {model.coef_}") • slope: [0.54] • The code above illustrates how to get 𝑏₀ and 𝑏₁. You can notice that .intercept_ is a scalar, while .coef_ is an array.