Driverless AI Hands-on Focused on Machine Learning Interpretability - H2O.ai
2 likes847 views
The document features key figures in machine learning interpretability at h2o.ai, including Patrick Hall, Navdeep Gill, and Mark Chan, who discuss their contributions and thoughts on explainable AI. It outlines a framework for interpretability, addressing issues like fairness and accountability, and presents challenges faced by machine learning models. Additionally, it provides a roadmap for future product features and resources for further exploration of machine learning interpretability.
Driverless AI Hands-on Focused on Machine Learning Interpretability - H2O.ai
- 2. Patrick Hall, Navdeep Gill, Mark Chan
- 3. Machine Learning Interpretability (MLI)
- 4. MEET THE MAKERS PATRICK HALL MARK CHANNAVDEEP GILL • Patrick Hall is a senior director for data science products at H2O.ai and adjunct faculty in the Department of Decision Sciences at George Washington University. He is the lead author of a popular white paper on techniques for interpreting machine learning models and a frequent speaker on the topics of FAT/ML and explainable artificial intelligence (XAI) at conferences and on webinars. • Navdeep Gill is a software engineer and data scientist at H2O.ai. He has made important contributions to the popular open source h2o machine learning library and the newer open source h2o4gpu library. Navdeep also led a recent Silicon Valley Big Data Science Meetup about interpretable machine learning. • Mark Chan is a software engineer and customer data scientist at H2O.ai. He has contributed to the open source h2o library and to critical financial services customer products.
- 5. First-time Qwiklab Account Setup • Go to http://h2oai.qwiklab.com • Click on “JOIN” (top right) • Create a new account with a valid email address • You will receive a confirmation email • Click on the link in the confirmation email • Go back to http://h2oai.qwiklab.com and log in • Go to the Catalog on the left bar • Choose “Introduction to Driverless AI” • Wait for instructions
- 7. Big Ideas
- 8. Learning from data … Adapted from: Learning from Data. https://work.caltech.edu/textbook.html
- 9. EXPLAIN HYPOTHESIS h ≈ g, βj g(x(i) j), g(x(i) (-j)) (explain predictions with reason codes) Learning from data … transparently. Adapted from: Learning from Data. https://work.caltech.edu/textbook.html
- 10. Increasing fairness, accountability, and trust by decreasing unwanted sociological biases Source: http://money.cnn.com/, Apple Computers
- 11. A framework for interpretability Complexity of learned functions: • Linear, monotonic • Nonlinear, monotonic • Nonlinear, non-monotonic (~ Number of parameters/VC dimension) Enhancing trust and understanding: the mechanisms and results of an interpretable model should be both transparent AND dependable. Understanding ~ transparency Trust ~ fairness and accountability Scope of interpretability: Global vs. local Application domain: Model-agnostic vs. model-specific
- 12. Big Challenges
- 13. Linear Models Strong model locality Usually stable models and explanations Machine Learning Weak model locality Sometimes unstable models and explanations (a.k.a. The Multiplicity of Good Models )
- 14. Age Number of Purchases Lost profits. Wasted marketing. “For a one unit increase in age, the number of purchases increases by 0.8 on average.” Linear Models Machine Learning Exact explanations for approximate models. Approximate explanations for exact models. Age “Slope begins to decrease here. Act to optimize savings.” “Slope begins to increase here sharply. Act to optimize profits.” Number of Purchase
- 15. A Few of Our Favorite Things
- 16. Partial dependence plots Source: http://statweb.stanford.edu/~tibs/ElemStatLearn/printings/ESLII_print10.pdf HomeValue ~ MedInc + AveOccup + HouseAge + AveRooms
- 17. Surrogate models
- 18. Local interpretable model-agnostic explanations (LIME) Source: https://github.com/marcotcr/lime Weighted explanatory samples. Linear model used to explain nonlinear decision boundary in local region.
- 19. Variable importance measures Global variable importance indicates the impact of a variable on the model for the entire training data set. Local variable importance can indicate the impact of a variable for each decision a model makes – similar to reason codes.
- 20. Current product roadmap Time Frame Features Near-Term Reason Codes in MOJO (i.e. Prod), Sensitivity Analysis, Multinomial Explanations Medium-Term Table Plots, Residual Analysis, Python API, Performance Refactor (GPU), Report Export Long-Term R API, AutoMLI (Roadmap subject to change without notice.)
- 21. Resources
- 22. Machine Learning Interpretability with H2O Driverless AI http://docs.h2o.ai/driverless-ai/latest-stable/docs/booklets/MLIBooklet.pdf Ideas on Interpreting Machine Learning https://www.oreilly.com/ideas/ideas-on-interpreting-machine-learning FAT/ML http://www.fatml.org/ MLI Resources https://github.com/h2oai/mli-resources
- 23. MLI Demo
- 24. Dataset for Hands On Lab /jupyter/data/creditcard/creditcard_train_cat.csv
- 25. Questions?