GDG Community Day 2023 - Interpretable ML in production

20th May, 2023 Bengaluru
Speaker
Saradindu Sengupta
Senior ML Engineer @Nunam
Where I work on building learning systems to
forecast health and failure of Li-ion batteries.
Interpretable ML in production

Table of Contents
1. Introduction
2. When and Why model understanding
3. What is model interpretability
a. Journey of a model
b. Interpretability framework
4. Achieving model understanding
a. Inherently interpretable models
b. Post-hoc explanations
5. What-if Toolkit

Introduction - Kick-off
Predict Wolf vs Husky [6]
Only 1 mistake!

Introduction - kick-off
Predict Wolf vs Husky - A great snow detector [6]
A great snow detector…

When and Why Model Understanding?
Not all applications require model understanding
1. E.g., ad/product/friend recommendations
2. No human intervention
Model understanding not needed because:
1. Little to no consequences for incorrect predictions
2. Problem is well studied and models are extensively validated in real-world applications

When and Why Model Understanding?
High-stakes decision-making settings
1. Impact on human lives/health/finances
2. Settings relatively less well studied, models not extensively validated
Accuracy alone is no longer enough
1. Train/test data may not be representative of data encountered in practice
Auxiliary criteria are also critical:
1. Nondiscrimination
2. Right to explanation
3. Safety

What is model interpretability
Journey of a model
X1
X2
X3
.
.
xn
y,,
y*
Evaluation
Metrics
Users
But can you trust your model?
Will it work in deployment?

What is model interpretability
Interpretability Framework
1. Trust
a. A prerequisite for humans to trust models
2. Causality
a. Learned associations between variables and outcomes
3. Transferability
a. How models might fare when the test environment shifts from the training environment.
4. Fair and Ethical Decision Making
a. Algorithmic decisions must be explainable, contestable, and modifiable
Geared towards supervised learning [4]. For understanding interpretability in reinforcement learning [5], which studies the
human interpretability of robot actions.

Achieving Model Understanding
1. Build inherently interpretable predictive models

2. Explain pre-built models in a post-hoc manner
Explainer

Inherently Interpretable Models vs. Post hoc Explanations [9]
Example
In certain scenarios, accuracy-interpretability trade will may exist

Inherently Interpretable Models vs. Post hoc Explanations [9]
complex models might achieve higher accuracy
can build interpretable + accurate models

Inherently Interpretable Models
1. Rule Based Models
a. Bayesian Rule List
b. Decision sets
2. Risk Scores
a. Widely used in medicine, criminal justice system,
regulated industry such as insurance and loan
3. Linear models
a. Linear regressions
4. Generalized Additive Models
5. Prototype Based Models
6. Tree based models
a. Decision Tree
b. Tree-based ensemble Model

Post hoc Explanations
LIME (Local Interpretable Model-Agnostic Explanations) [6]
Given an example, x, LIME attempts to fit an interpretable model locally that is faithful to the output of the original model,
f(x), in a neighborhood around x
1. Sample points around xi
2. Use model to predict labels for each sample
3. Weigh samples according to distance to xi
4. Learn simple linear model on weighted samples
5. Use simple linear model to explain

Post hoc Explanations
SHAP (SHapley Additive exPlanations) [10]
It tries to add 3 attributes that we want in an interpretable model
1. Local accuracy
2. Missingness
3. Consistency
xi
P(y) = 0.9
xi
P(y) = 0.8
M(xi
, O) = 0.1
O
O/xi

What-if Toolkit [11]
Google Colab || Data Set: UCI Census Income Dataset || GitHub Repository with Tutorials
Vertex AI integration

References
1. Zachary C. Lipton, “The Mythos of Model Interpretability”, 2016, [https://arxiv.org/abs/1606.03490]
2. Finale Doshi-Velez, Been Kim, “Towards A Rigorous Science of Interpretable Machine Learning”, 2017,
[https://arxiv.org/abs/1702.08608]
3. S. Ghanta et al., "Interpretability and Reproducability in Production Machine Learning Applications," 2018 17th IEEE
International Conference on Machine Learning and Applications (ICMLA), Orlando, FL, USA, 2018, pp. 658-664, doi:
10.1109/ICMLA.2018.00105.
4. Lou,Yin, Caruana, Rich, and Gehrke, Johannes. “Intelligible models for classification and regression”.In KDD, 2012.
5. Dragan, AncaD, Lee, Kenton CT, and Srinivasa, Siddhartha S., “Legibility and predictability of robot motion. In Human-Robot
Interaction(HRI), 2013 8th ACM/IEEE International Conference on. IEEE, 2013.
6. Ribeiro et. al. 2016[https://arxiv.org/abs/1602.04938]
7. Ribeiro et al. 2018[https://homes.cs.washington.edu/~marcotcr/aaai18.pdf]
8. Lakkaraju et. al. 2019 [https://dl.acm.org/doi/10.1145/3306618.3314229]
9. Machine Learning Explainability Workshop, Stanford University
[https://youtube.com/playlist?list=PLoROMvodv4rPh6wa6PGcHH6vMG9sEIPxL]
10. Lundberg, S., & Lee, S. (2017). A Unified Approach to Interpreting Model Predictions. ArXiv. /abs/1705.07874
[https://arxiv.org/abs/1705.07874]
11. J. Wexler, M. Pushkarna, T. Bolukbasi, M. Wattenberg, F. Viégas and J. Wilson, "The What-If Tool: Interactive Probing of
Machine Learning Models," in IEEE Transactions on Visualization and Computer Graphics, vol. 26, no. 1, pp. 56-65, Jan. 2020,
doi: 10.1109/TVCG.2019.2934619.[https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=8807255]

Thank You
linkedin/saradindusengupta
@iamsaradindu

GDG Community Day 2023 - Interpretable ML in production

More Related Content

Similar to GDG Community Day 2023 - Interpretable ML in production (20)

More from SARADINDU SENGUPTA (7)

Recently uploaded (20)

GDG Community Day 2023 - Interpretable ML in production