MLSEV Virtual. My first BigML Project
0 likes234 views
The document outlines the process of building a predictive model for credit delinquency, specifically to identify customers who may default on payments. It discusses defining the problem, the data involved, model evaluation, and importance of minimizing risk while predicting defaults. The final model chosen is a simple 15-node decision tree that balances performance and accuracy.
MLSEV Virtual. My first BigML Project
- 1. 2nd edition
- 2. #MLSEV My First BigML Model Mercè Martín VP of Applications, BigML
- 3. #MLSEV • Lots of decisions • Lots of potentially related variables • Uncertain correlations ML CAN HELP Do I really need a model?
- 4. #MLSEV We decide the actionNew data arrives The model labels it Maybe I could use a model…
- 6. #MLSEV Credit delinquency I WANT TO MINIMIZE RISK BY PREDICTING DEFAULTS https://www.kaggle.com/c/GiveMeSomeCredit
- 9. #MLSEV Defining the real question When do I consider a customer is in default? When the customer misses payments? What if the customer pays late? What is the maximum delinquency that you allow?
- 10. #MLSEV Defining the contest goal Predicting who will be 90 days past due or worse to act only on them
- 11. #MLSEV And now…
- 12. #MLSEV The First Decision https://bigml.com/accounts/register MLSEV 1-month FREE BOOSTED
- 13. #MLSEV The Data Dictionary Variable Name Description Type SeriousDlqin2yrs Person experienced 90 days past due delinquency or worse Y/N RevolvingUtilizationOfUnsecuredLine s Total balance on credit cards and personal lines of credit except real estate and no installment debt like car loans divided by the sum of credit limits percentag e age Age of borrower in years integer NumberOfTime30- 59DaysPastDueNotWorse Number of times borrower has been 30-59 days past due but no worse in the last 2 years. integer DebtRatio Monthly debt payments, alimony,living costs divided by monthy gross income percentag e MonthlyIncome Monthly income real NumberOfOpenCreditLinesAndLoans Number of Open loans (installment like car loan or mortgage) and Lines of credit (e.g. credit cards) integer NumberOfTimes90DaysLate Number of times borrower has been 90 days or more past due. integer NumberRealEstateLoansOrLines Number of mortgage and real estate loans including home equity lines of credit integer NumberOfTime60- 89DaysPastDueNotWorse Number of times borrower has been 60-89 days past due but no worse in the last 2 years. integer NumberOfDependents Number of dependents in family excluding themselves (spouse, children etc.) integer 10 predictors
- 14. #MLSEV The Data
- 15. #MLSEV The Source How to interpret your data? • Field types • Locale (decimals) • Missing tokens • Text / Items parsing
- 16. #MLSEV The Dataset How is data distributed? • Histograms • Statistics • Number of missings • Number of errors
- 17. #MLSEV And now… The Model
- 18. #MLSEV The Model What insights will the model extract? • Patterns • Importance • and…
- 19. #MLSEV The Prediction What label corresponds to this loan? • Predictions (labels) • Confidence • Explanations
- 22. #MLSEV And now… The Evaluation
- 23. #MLSEV The Evaluation Do predictions match the real values? Hey! Great accuracy!!! right?
- 24. #MLSEV I wish to make a complaint!
- 25. #MLSEV The Evaluation Do predictions match the real values? • Positive class: 1 1 / 1 Predicted / Actual TP FN 0 / 1 FP TN 1 / 0 0 / 0
- 26. #MLSEV The Costs Predicting who will be 90 days past due or worse to act only on them • Always remember the goal TO MINIMIZE COST WE SHOULD MAXIMIZE THE RECALL • And the costs of failing!!! Unbalanced
- 27. #MLSEV And now… Model Tuning
- 28. #MLSEV Compensating unbalance The percentage of examples of the class we are interested is very low Increasing their frequency could help the model to learn better
- 29. #MLSEV Choosing according to Costs THE BALANCED MODEL WORKS BETTER vs. Unbalanced Balanced
- 32. #MLSEV Automating tuning Smart search for the best performing configuration
- 33. #MLSEV And the winner is… A simple decision tree!!! • 15-node • balanced • pruned