Machine Learning Interpretability
2 likes1,085 views
This document discusses machine learning interpretability. It defines interpretation as giving explanations to humans for machine learning models and decisions. It notes that humans create, are affected by, and demand explanations for decision systems. The document outlines different techniques for model interpretability including intrinsically interpretable models, post-hoc interpretability techniques that provide explanations for black box models, and model-specific and model-agnostic techniques. It provides examples like partial dependence plots, individual conditional expectation, and local surrogate models. It recommends choosing techniques based on the recipient and purpose of explanations.
Machine Learning Interpretability
- 1. Machine Learning Interpretability Marcel Spitzer Munich, 20.11.2018
- 2. 2 Marcel Spitzer Big Data Scientist @ inovex ● Applied Mathematics, Data Science ● SW Engineering, Data Products ● Big Data, Hadoop, Spark mspitzer@inovex.de @mspitzer243
- 3. 3
- 4. 4 Interpretation is the process of giving explanations to humans. ~ Kim B., Google Brain, Interpretable Machine Learning (ICML 2017) https://people.csail.mit.edu/beenkim/papers/BeenK_FinaleDV_ICML2017_tutorial.pdf
- 5. 5 “Interpretability is the degree to which an observer can understand the cause of a decision.” ~ Miller T., 2017, Explanation in AI: Insights from the Social Sciences ➢ humans create decision systems ➢ humans are affected by decisions ➢ humans demand for explanations https://arxiv.org/pdf/1706.07269.pdf
- 6. 6 NIPS 2016 workshop on Interpretable Machine Learning for Complex Systems ICML 2016 Workshop on Human Interpretability in Machine Learning NIPS 2017 Workshop on Interpreting, Explaining and Visualizing Deep Learning NIPS 2017 symposium and workshop: interpretable and Bayesian machine learning ICML 2017 Workshop on Human Interpretability in Machine Learning ICML 2018 Workshop on Human Interpretability in Machine Learning https://people.csail.mit.edu/beenkim/papers/BeenK_FinaleDV_ICML2017_tutorial.pdf
- 7. 7https://arxiv.org/pdf/1606.03490.pdf The additional need for interpretability
- 8. 8 safety system should provide sound decisions curiosity understand something unexpected debugging behaviour should be predictable optimality optimize for true objectives Why do we need interpretability? https://people.csail.mit.edu/beenkim/papers/BeenK_FinaleDV_ICML2017_tutorial.pdf
- 9. 9 When we may not need interpretability https://christophm.github.io/interpretable-ml-book/interpretability-importance.html low risk no significant consequences awareness problem is well-studied vulnerability prevent people from gaming the system
- 10. 10 Use models that are intrinsically interpretable and known to be easy for humans to understand. Train a black box model and apply post-hoc interpretability techniques to provide explanations. 1 2
- 11. 11 Post-hoc interpretability techniques Global Local Model-specific Model Internals, Intrinsic Feature Importance Rule Sets (Tree Structure) Model-agnostic Partial Dependence Plots, Feature Importance (perm-based), Global Surrogate Models Individual Conditional Expectation, Local Surrogate Models
- 12. 12 Post-hoc interpretability techniques Global Local Model-specific Model Internals, Intrinsic Feature Importance Rule Sets (Tree Structure) Model-agnostic Partial Dependence Plots, Feature Importance (perm-based), Global Surrogate Models Individual Conditional Expectation, Local Surrogate Models
- 13. ➢ shows dependence of the response on a feature per instance ➢ single curve results from varying a certain feature for a given instance ➢ inconsistent pattern indicates multicollinearity 13 Individual Conditional Expectation (ICE) https://christophm.github.io/interpretable-ml-book/pdp.html
- 14. 14 Partial Dependence Plots (PDP) ➢ PDP curve is the result of averaging ICE curves ➢ very intuitive, easy to understand ➢ assumption of independence is a strong drawback https://christophm.github.io/interpretable-ml-book/pdp.html
- 15. ➢ averages degradation measured by a certain loss function after repeatedly permuting single features ➢ feature is important if the error significantly increases after a shuffle 15 Feature Shuffling https://amunategui.github.io/variable-importance-shuffler/
- 16. ➢ highly compressed, global insight ➢ tied to some loss function ➢ practically infeasible in high dimensional domains (e.g. image/speech recognition, NLP) 16 Feature Shuffling https://christophm.github.io/interpretable-ml-book/feature-importance.html https://scikit-plot.readthedocs.io/en/stable/estimators.html#scikitplot.estimators.plot_feature_importances
- 18. ➢ feeds original model with small variations of instance to be explained ➢ sampled instances are weighted by proximity to the instance of interest ➢ interpretable models are fit locally on observed outcome 18 Local Surrogate Models: LIME https://christophm.github.io/interpretable-ml-book/lime.html
- 19. 19 Local Surrogate Models: LIME https://www.oreilly.com/learning/introduction-to-local-interpretable-model-agnostic-explanations-lime
- 20. 20 Recommendations for interpretability techniques ➢ Who is the recipient? ○ Lay-Men → rather intuitive, example-based local explanations ○ Analysts → global surrogates, perm-based feature importance ○ Authorities → intrinsically interpretable models ➢ What are the explanations used for? ○ Debug/Improve → PDP & ICE curves ○ Decision support → rule-based explanations ○ Auditing/Legal → intrinsically interpretable models
- 21. ➢ Molnar C., 2018, Interpretable Machine Learning - A Guide for Making Black Box Models Explainable ➢ Gill N., Hall P., 2018, An Introduction to Machine Learning Interpretability ➢ Zhao Q., Hastie T., 2017, Causal Interpretations of Black-Box Models ➢ Kim B., Doshi-Velez F., 2017, Interpretable Machine Learning: The fuss, the concrete and the questions ➢ Ribeiro, M.T., Singh, S. and Guestrin, C., 2016, August. Why should i trust you? Explaining the predictions of any classifier 21 Resources
- 22. Vielen Dank Marcel Spitzer Big Data Scientist mspitzer@inovex.de inovex GmbH Schanzenstraße 6-20 Kupferhütte 1.13 51063 Köln