M1 regression metrics_middleschool
- 1. AI Club
- 3. How good a model is: Predictive performance • Predictive performance in Classification: Accuracy • Predictive performance in Regression ?
- 4. How good a model is: Predictive performance • Predictive performance in Classification: Accuracy • Predictive performance in Regression ? • What can you use to tell you how good your regression AI is?
- 5. Predictive performance in regression • Predictive performance in Classification: Accuracy • Predictive performance in Regression: • Calculate the error = (Predicted – True) If the correct value is 10 and AI predicted 11 Error = 11-10 = 1
- 6. Predictive performance in regression • Predictive performance in Classification: Accuracy • Predictive performance in Regression: • Calculate the error = (Predicted – True) If the correct value is 10 and AI predicted 9 Error = ?
- 7. Predictive performance in regression • Predictive performance in Classification: Accuracy • Predictive performance in Regression: • Calculate the error = (Predicted – True) If the correct value is 10 and AI predicted 9 Error = 9-10 = -1?
- 8. Predictive performance in regression • Predictive performance in Classification: Accuracy • Predictive performance in Regression: • Calculate the error = (Predicted – True) If the correct value is 10 and AI predicted 9 Absolute Value Error = Absolute Value(-1) = 1
- 9. Predictive performance in regression • Predictive performance in Classification: Accuracy • Predictive performance in Regression: • Calculate the error = (Predicted – True) In case of accuracy, if the AI got 6 correct answers out of 10, its accuracy = 60% Similarly, if the AI got errors of 1.2,5.1,2,3,6,1,4.3.. etc for 10 questions, what is the overall error?
- 10. Predictive performance in regression • Predictive performance in Classification: Accuracy • Predictive performance in Regression: • Calculate the error = (Predicted – True) In case of accuracy, if the AI got 6 correct answers out of 10, its accuracy = 60% Similarly, if the AI got errors of 1.2,5.1,2,3,6,1,4.3.. etc for 10 questions, what is the overall error? Over all error is Average(1.2,5.1,2,3,4,1,4.3 …)
- 11. Predictive performance in regression • Exercise: • True Value: 5, Predicted Value: 3 • True Value: 7, Predicted Value: 10 Predictive performance in regression
- 12. Predictive performance in regression • Example: • True Value: 5, Predicted Value: 3 -> Error: 2 • True Value: 7, Predicted Value: 10 -> Error: -3 Predictive performance in regression
- 13. Predictive performance in regression • Example: • True Value: 5, Predicted Value: 3 -> Absolute Error: 2 • True Value: 7, Predicted Value: 10 -> Absolute Error: 3 Predictive performance in regression
- 14. Predictive performance in regression • Example: • True Value: 5, Predicted Value: 3 -> Absolute Error: 2 • True Value: 7, Predicted Value: 10 -> Absolute Error: 3 • Mean Absolute Error: (3+2)/2 Predictive performance in regression
- 15. Predictive performance in regression • Example: • True Value: 5, Predicted Value: 3 -> Absolute Error: 2 • True Value: 7, Predicted Value: 10 -> Absolute Error: 3 • Mean Absolute Error: (3+2)/2 • Another way to measure error • True Value: 5, Predicted Value: 3 -> Square Error: 4 • True Value: 7, Predicted Value: 10 -> Square Error: 9 Predictive performance in regression
- 16. Predictive performance in regression • Example: • True Value: 5, Predicted Value: 3 -> Absolute Error: 2 • True Value: 7, Predicted Value: 10 -> Absolute Error: 3 • Mean Absolute Error: (3+2)/2 • Another way to measure error • True Value: 5, Predicted Value: 3 -> Square Error: 4 • True Value: 7, Predicted Value: 10 -> Square Error: 9 • Mean Square Error: (9+4)/2 Predictive performance in regression
- 17. Predictive performance in regression • Example: • True Value: 5, Predicted Value: 3 -> Absolute Error: 2 • True Value: 7, Predicted Value: 10 -> Absolute Error: 3 • Mean Absolute Error: (3+2)/2 • Another way to measure error • True Value: 5, Predicted Value: 3 -> Square Error: 4 • True Value: 7, Predicted Value: 10 -> Square Error: 9 • Mean Square Error: (9+4)/2 • Root Mean Square Error: root((9+4)/2) Predictive performance in regression
- 18. Summary: How to Measure Regression • In regression, you have to compare the number you get from the AI with the correct number • Lower difference is better • Its called Root Mean Square Error (RMSE) • Take the difference (Prediction – Right answer) • Square it so that it is always positive (Prediction – Right Answer) * (Prediction – Right Answer) • Take the average (Mean) • Take the square root