Multiple Regression with two Independent Variable- by Dr. Vikramjit Singh.pdf
0 likes31 views
The document elaborates on multiple regression analysis, explaining its purpose of predicting a dependent variable using two or more independent variables. It includes key formulas and calculations for regression coefficients, intercepts, and the coefficient of determination (r²), as well as applying these concepts to a dataset for practical understanding. The document also provides a case study with example calculations to illustrate the methodology of regression analysis.
Multiple Regression with two Independent Variable- by Dr. Vikramjit Singh.pdf
- 1. Image Source : https://commons.wikimedia.org/ Multiple Regression with two Independent Variable By- Vikramjit Singh,PhD
- 2. Multiple Regression Analysis : What it is ??
- 3. Multiple Regression • Multiple Regression is an extension of simple linear regression. • It is used when we try to predict the value of a dependent variable(target or criterion variable) based on two or more independent variables(predictor or explanatory variables) • Take for example if we want to predict the ‘scores obtained’ based upon ‘no of study hours’ and ‘no of classes attended’ then it is the case of multiple regression.
- 4. Equation of Multiple Regression 𝑌 = 𝛽0 ± 𝛽1 𝑋1 ± 𝛽2 𝑋2 ± ⋯ 𝛽𝑛 𝑋𝑛 ± 𝜀 Where 𝑋1,, 𝑋2…𝑋𝑛= independent variables 𝑌= dependent variable 𝛽1 = The regression coefficient of variable X1 𝛽2 = The regression coefficient of variable X2 𝛽𝑛= The regression coefficient of variable Xn 𝛽0= The intercept point of the regression line and the Y axis. 𝜀= Residual or error terms
- 5. Simple Regression Model in two IV 𝑌 = 𝛽0 ± 𝛽1 𝑋1 ± ±𝛽2 𝑋2 ± 𝜀 Where 𝑥= independent variable 𝑦= dependent variable 𝛽1 = The regression coefficient of variable 𝑋1 𝛽2 = The regression coefficient of variable 𝑋2 𝛽0= The intercept point of the regression line and the Y axis 𝜀= Residual or error terms
- 6. Simple Regression Model in two IV 𝑌 = 𝛽0 ± 𝛽1 𝑋1 ± 𝛽2 𝑋2 ± 𝜀 Or DATA = FIT + RESIDUAL
- 7. Using method of deviation for solving regression equation Let us understand the basic symbols and equations to be used ഥ 𝑌 = The Mean of Dependent Variable 𝑌 𝑋1 = The Mean of Independent Variable 𝑋1 𝑋2 = The Mean of Independent Variable 𝑋2 𝛴𝑦 = 𝛴 𝑌 − ത 𝑌 , 𝛴𝑥1 = 𝛴 𝑋1 − 𝑋1 , 𝛴𝑥2 = 𝛴 𝑋2 − 𝑋2
- 8. Using method of deviation for solving regression equation Let us understand the basic symbols and equations to be used – 𝛴𝑥1𝑦 = 𝛴(𝑥1 ∗ 𝑦 ), 𝛴𝑥2𝑦 = 𝛴(𝑥2 ∗ 𝑦 ), 𝛴𝑥1𝑥2= 𝛴(𝑥1 ∗ 𝑥2) 𝛴𝑥1 2 = Sum of square of 𝑥1 , 𝛴𝑥2 2 = Sum of square of 𝑥2
- 9. The regression coefficient and Intercept 𝛽1 = (𝛴𝑥1 𝑦)𝛴𝑥2 2−(𝛴𝑥2 𝑦)(𝛴𝑥1 𝑥2 ) 𝛴𝑥1 2 𝛴𝑥2 2 − 𝛴𝑥1 𝑥2 2 𝛽2 = (𝛴𝑥2 𝑦)𝛴𝑥1 2−(𝛴𝑥1 𝑦)(𝛴𝑥1 𝑥2 ) 𝛴𝑥1 2 𝛴𝑥2 2 − 𝛴𝑥1 𝑥2 2 𝛽0 = ഥ 𝑌 − 𝛽1𝑋1 − 𝛽2𝑋2
- 10. Suppose We have the following dataset with dependent variable Y and two independent Variables X1 and X2 . Y X1 X2 155 62 25 215 78 11 179 70 20 140 60 22 200 72 14 159 67 24 192 71 15 212 75 14
- 11. Y X1 X2 𝑦= 𝑌 − ത 𝑌 𝑥1 = 𝑋1 − 𝑋1 𝑥2 = 𝑋2 − 𝑋2 𝑥1 ∗ 𝑦 𝑥2 ∗ 𝑦 𝑥1 ∗ 𝑥2 𝑥1 2 𝑥2 2 155 62 25 -26.5 -7.375 6.875 195.4375 -182.188 -50.7031 54.39063 47.26563 215 78 11 33.5 8.625 -7.125 288.9375 -238.688 -61.4531 74.39063 50.76563 179 70 20 -2.5 0.625 1.875 -1.5625 -4.6875 1.171875 0.390625 3.515625 140 60 22 -41.5 -9.375 3.875 389.0625 -160.813 -36.3281 87.89063 15.01563 200 72 14 18.5 2.625 -4.125 48.5625 -76.3125 -10.8281 6.890625 17.01563 159 67 24 -22.5 -2.375 5.875 53.4375 -132.188 -13.9531 5.640625 34.51563 192 71 15 10.5 1.625 -3.125 17.0625 -32.8125 -5.07813 2.640625 9.765625 212 75 14 30.5 5.625 -4.125 171.5625 -125.813 -23.2031 31.64063 17.01563 1452 555 145 0 0 0 1162.5 -953.5 -200.375 263.875 194.875 ത 𝑌= 181.5 𝑋1= 69.375 𝑋2= 18.125 Calculating the coefficient of regression
- 12. Calculating the coefficient of regression We Have- 𝛴𝑥1𝑦 = 1162.5 𝛴𝑥2𝑦 = −953.5 𝛴𝑥1𝑥2=−200.375 𝛴𝑥1 2 = 263.875 𝛴𝑥2 2 = 194.875 𝛽1 = (𝛴𝑥1 𝑦)𝛴𝑥2 2−(𝛴𝑥2 𝑦)(𝛴𝑥1 𝑥2 ) 𝛴𝑥1 2 𝛴𝑥2 2 − 𝛴𝑥1𝑥2 2 𝛽1 = 1162.5 𝑋 194.875−(−953.5) 𝑋 (−200.375) 263.875 𝑋194.875 −(−200.375)2 𝛽1= 35484.63 11272.5 𝛽1= 3.147893
- 13. Calculating the coefficient of regression We Have- 𝛴𝑥1𝑦 = 1162.5 𝛴𝑥2𝑦 = −953.5 𝛴𝑥1𝑥2=−200.375 𝛴𝑥1 2 = 263.875 𝛴𝑥2 2 = 194.875 𝛽2 = (𝛴𝑥2 𝑦)𝛴𝑥1 2−(𝛴𝑥1 𝑦)(𝛴𝑥1 𝑥2 ) 𝛴𝑥1 2 𝛴𝑥2 2 − 𝛴𝑥1𝑥2 2 𝛽2 = −953.5 𝑋 263.875−(1162.5) 𝑋 (−200.375) 263.875 𝑋194.875 −(−200.375)2 𝛽2= −18668.9 11272.5 𝛽2= -1.65614
- 14. Calculating the coefficient of regression We Have- ഥ 𝑌 = 181.5 𝛽1= 3.147893 𝑋1 = 69.375 𝛽2 = -1.65614 𝑋2 = 18.125 𝛽0 = ഥ 𝑌 − 𝛽1𝑋1 − 𝛽2𝑋2 𝛽0 = 181.5 − 3.148 𝑋 69.375 − −1.656 𝑋 18.125 𝛽0 = 181.5 − 218.3925 − (−30.015) 𝛽0= -6.7425
- 15. Substituting the values of 𝛽0, 𝛽1and 𝛽2 into the regression model we can get the equation of line of best fit , Thus Estimate Dependent Variable Value= 𝑌 𝑌 = -6.7425 + 3.147893 𝑋1- 1.65614 𝑋2 Interpretation – 𝛽0 (Constant/Intercept) = -6.7425 indicates when both the predictor variable are zero then the value of y is -6.7425 𝛽1 (Regression Coefficient) = 3.147893 indicates that as 𝑋1increases by one unit there is an increase of 3.147893 in Y assuming 𝑋2 is held constant. 𝛽2 (Regression Coefficient) = - 1.65614 indicates that as𝑋2increases by one unit there is a decrease of 1.65614 in Y assuming 𝑋1 is held constant.
- 16. Calculating Estimated Score ( 𝑌) coefficient of determination(R2) - ANOVA for Multiple Regression Cases Y X1 X2 𝑌 𝑌- ത 𝑌 ( 𝑌- ത 𝑌)2 Y− 𝑌 (Y − 𝑌)𝟐 Y − ത 𝑌 (Y − ത 𝑌)2 1 155 62 25 147.0234 -34.4766 1188.638 7.976634 63.62669 -26.5 702.25 2 215 78 11 220.5756 39.07561 1526.904 -5.57561 31.08747 33.5 1122.25 3 179 70 20 180.4872 -1.01279 1.025744 -1.48721 2.211794 -2.5 6.25 4 140 60 22 145.696 -35.804 1281.926 -5.696 32.44442 -41.5 1722.25 5 200 72 14 196.7198 15.21984 231.6434 3.280164 10.75948 18.5 342.25 6 159 67 24 164.419 -17.081 291.7616 -5.41897 29.36525 -22.5 506.25 7 192 71 15 191.9158 10.4158 108.489 0.084197 0.007089 10.5 110.25 8 212 75 14 206.1635 24.66352 608.289 5.836485 34.06456 30.5 930.25 Total 1452 555 145 1453 1.000315 5238.677 -1.00031 203.5667 0 5442 SSreg SSres SStot
- 17. Equation of ANOVA table for Multiple Regression Sources of Variation Sum of Squares Df Mean Square F Regression ( 𝑌- ത 𝑌)2 K(no. of variables being estimated ) -1 SSreg/2 MSreg/MSres Residual (Y − 𝑌)𝟐 N- K SSres/(N-3) Total (Y − ത 𝑌)2 N-1
- 18. Equation of ANOVA table for Simple Linear Regression Sources of Variation Sum of Squares Df Mean Square F Regression 5238.677 2 5238.677/2 =2619.339 2619.339 /40.73334 = 64.30 Residual 203.5667 5 203.5667/5 =40.73334 Total 5442 7
- 19. Calculating coefficient of Determination(R2) R2 = 𝐸𝑥𝑝𝑙𝑎𝑖𝑛𝑒𝑑 𝑉𝑎𝑟𝑖𝑎𝑡𝑖𝑜𝑛 𝑇𝑜𝑡𝑎𝑙 𝑉𝑎𝑟𝑖𝑎𝑡𝑖𝑜𝑛 = 𝑅𝑒𝑔𝑟𝑒𝑠𝑠𝑖𝑜𝑛 𝑆𝑢𝑚 𝑜𝑓 𝑆𝑞𝑢𝑎𝑟𝑒(𝑆𝑆𝑅) 𝑇𝑜𝑡𝑎𝑙 𝑆𝑢𝑚 𝑜𝑓 𝑆𝑞𝑢𝑎𝑟𝑒(𝑆𝑆𝑇) = 5238.677 5442 = 0.962638 We can say that 96.26 % of the variation in the dependent variable Y is explained by variable X1 and X2.
- 20. Calculating adjusted (R2) Adj R2 = 1- 𝑁−1 𝑁−𝐾 X (1-R2) = 1- 7 5 (1-0.962638) = 1-1.4 * 0.037362 = 0.947693 We can say with adjustment that 94.77 % of the variation in the dependent variable Y is explained by variable X1 and X2.
- 21. One more Example….. A study is conducted involving 10 students to investigate the relationship and affects of revision time and lecture attendance on exam performance. Perform Regression analysis and interpret the results. Student Exam Score(Y) Revision Time (X1) Lecture Attendance (X2) 1 40 6 4 2 44 10 4 3 46 12 5 4 48 14 7 5 52 16 9 6 58 18 12 7 60 22 14 8 68 24 20 9 74 26 21 10 80 32 24
- 22. Y X1 X2 𝑦= 𝑌 − ത 𝑌 𝑥1 = 𝑋1 − 𝑋1 𝑥2 = 𝑋2 − 𝑋2 𝑥1 ∗ 𝑦 𝑥2 ∗ 𝑦 𝑥1 ∗ 𝑥2 𝑥1 2 𝑥2 2 40 6 4 -17 -12 -8 204 136 96 144 64 44 10 4 -13 -8 -8 104 104 64 64 64 46 12 5 -11 -6 -7 66 77 42 36 49 48 14 7 -9 -4 -5 36 45 20 16 25 52 16 9 -5 -2 -3 10 15 6 4 9 58 18 12 1 0 0 0 0 0 0 0 60 22 14 3 4 2 12 6 8 16 4 68 24 20 11 6 8 66 88 48 36 64 74 26 21 17 8 9 136 153 72 64 81 80 32 24 23 14 12 322 276 168 196 144 570 180 120 0 0 0 956 900 524 576 504 ത 𝑌= 57 𝑋1=18 𝑋2=12 Calculating the coefficient of regression
- 23. Calculating the coefficient of regression We Have- 𝛴𝑥1𝑦 = 956 𝛴𝑥2𝑦 = 900 𝛴𝑥1𝑥2=524 𝛴𝑥1 2 = 576 𝛴𝑥2 2 = 504 𝛽1 = (𝛴𝑥1 𝑦)𝛴𝑥2 2−(𝛴𝑥2 𝑦)(𝛴𝑥1 𝑥2 ) 𝛴𝑥1 2 𝛴𝑥2 2 − 𝛴𝑥1𝑥2 2 𝛽1 = 956 𝑋 504−900 𝑋 524 576 𝑋 504 −(524)2 𝛽1= 0.65
- 24. Calculating the coefficient of regression We Have- 𝛴𝑥1𝑦 = 956 𝛴𝑥2𝑦 = 900 𝛴𝑥1𝑥2=524 𝛴𝑥1 2 = 576 𝛴𝑥2 2 = 504 𝛽2 = (𝛴𝑥2 𝑦)𝛴𝑥1 2−(𝛴𝑥1 𝑦)(𝛴𝑥1 𝑥2 ) 𝛴𝑥1 2 𝛴𝑥2 2 − 𝛴𝑥1𝑥2 2 𝛽2 = 900 𝑋 576−956 𝑋 524 576 𝑋 504 −(524)2 𝛽2= 1.11
- 25. Calculating the coefficient of regression We Have- ഥ 𝑌 = 57 𝛽1= 0.65 𝑋1 = 18 𝛽2 = 1.11 𝑋2 = 12 𝛽0 = ഥ 𝑌 − 𝛽1𝑋1 − 𝛽2𝑋2 𝛽0 = 57 − 0.65 𝑋 18 − 1.11 𝑋 12 𝛽0= 31.98
- 26. Substituting the values of 𝛽0, 𝛽1and 𝛽2 into the regression model we can get the equation of line of best fit , Thus Estimate Dependent Variable Value= 𝑌 𝑌 = 31.98 + 0.65 𝑋1+1.11 𝑋2 Interpretation – 𝛽0 (Constant/Intercept) = 31.98 indicates when both the predictor variable are zero then the value of Y is 31.98 𝛽1 (Regression Coefficient) = 0.65 indicates that as 𝑋1increases by one unit there is an increase of 0.65 in Y assuming 𝑋2 is held constant. 𝛽2 (Regression Coefficient) = 1.11 indicates that as 𝑋2increases by one unit there is a increase of 1.11 in Y assuming 𝑋1 is held constant.
- 27. Calculating Estimated Score ( 𝑌) coefficient of determination(R2) - ANOVA for Multiple Regression Cases Y X1 X2 𝑌 𝑌- ത 𝑌 ( 𝑌- ത 𝑌)2 Y− 𝑌 (Y − 𝑌)𝟐 Y − ത 𝑌 (Y − ത 𝑌)2 1 40 6 4 40.32 -16.68 278.2224 -0.32 0.1024 -17 289 2 44 10 4 42.92 -14.08 198.2464 1.08 1.1664 -13 169 3 46 12 5 45.33 -11.67 136.1889 0.67 0.4489 -11 121 4 48 14 7 48.85 -8.15 66.4225 -0.85 0.7225 -9 81 5 52 16 9 52.37 -4.63 21.4369 -0.37 0.1369 -5 25 6 58 18 12 57 0 0 1 1 1 1 7 60 22 14 61.82 4.82 23.2324 -1.82 3.3124 3 9 8 68 24 20 69.78 12.78 163.3284 -1.78 3.1684 11 121 9 74 26 21 72.19 15.19 230.7361 1.81 3.2761 17 289 10 80 32 24 79.42 22.42 502.6564 0.58 0.3364 23 529 Total 570 180 120 570 0 1620.47 -0 13.6704 0 1634 SSreg SSres SStot
- 28. Equation of ANOVA table for Multiple Regression Sources of Variation Sum of Squares Df Mean Square F Regression ( 𝑌- ത 𝑌)2 K(no. of variables being estimated ) -1 SSreg/2 MSreg/MSres Residual (Y − 𝑌)𝟐 N- K SSres/(N-3) Total (Y − ത 𝑌)2 N-1
- 29. Equation of ANOVA table for Simple Linear Regression Sources of Variation Sum of Squares Df Mean Square F Regression 1620.47 2 1620.47/2 =810.235 810.235 /1.953 = 414.87 Residual 13.67 7 13.67/7 =1.953 Total 1634 9
- 30. Calculating coefficient of Determination(R2) R2 = 𝐸𝑥𝑝𝑙𝑎𝑖𝑛𝑒𝑑 𝑉𝑎𝑟𝑖𝑎𝑡𝑖𝑜𝑛 𝑇𝑜𝑡𝑎𝑙 𝑉𝑎𝑟𝑖𝑎𝑡𝑖𝑜𝑛 = 𝑅𝑒𝑔𝑟𝑒𝑠𝑠𝑖𝑜𝑛 𝑆𝑢𝑚 𝑜𝑓 𝑆𝑞𝑢𝑎𝑟𝑒(𝑆𝑆𝑅) 𝑇𝑜𝑡𝑎𝑙 𝑆𝑢𝑚 𝑜𝑓 𝑆𝑞𝑢𝑎𝑟𝑒(𝑆𝑆𝑇) = 1620.47 1634 = 0.99172 We can say that 99.17 % of the variation in the dependent variable (Marks) Y is explained by variable X1(revision time) and X2(Lecture attendance)
- 31. Calculating adjusted (R2) Adj R2 = 1- 𝑁−1 𝑁−𝐾 X (1-R2) = 1- 9 7 (1-0.9917) = 0.989 We can say with adjustment that 98.9 % of the variation in the dependent variable Y is explained by independent variables X1 and X2.
- 32. Thanks E-mail: drvsinghonline@gmail.com Mob: +91-9438574139 © V. Singh © V. Singh