“Explainability in Computer Vision: A Machine Learning Engineer’s Overview,” a Presentation from AltaML
0 likes166 views
The document provides an overview of explainability techniques in machine learning, particularly in the context of computer vision applications developed by Altaml, a Canadian applied machine learning firm. It outlines key concepts, such as the necessity of explainability due to deep learning's 'black box' nature, and categorizes various techniques for achieving transparency in model predictions, including perturbation based, backpropagation based, and activation based methods. Case studies demonstrate how these techniques enhance trust in machine learning models by offering insights into decision-making processes.
![© 2021 AltaML
Debugging and Building Trust
ML
Used image segmentation to give the ML model only
images of tissue and not the rest of the slide.
Decisions on an entire tissue sample. No further
information provided by the model.
Decision
18
Simulated images are used to respect the privacy agreements
Use case:
Classify high resolution microscopy
slides into normal or abnormal
ML: Image classification problem
Input Images of biopsy
slides
Labels Per tissue label;
[Normal/Abnormal]](https://image.slidesharecdn.com/ti078kottayilaltaml-210924130822/85/Explainability-in-Computer-Vision-A-Machine-Learning-Engineer-s-Overview-a-Presentation-from-AltaML-18-320.jpg)
![© 2021 AltaML
Explanations significantly helped in
building trust in the proposed
model for ML team and doctors.
Doctors identified that the heat
map pointed to the damaged cells
of interest to the doctors.
ML
+
Explanation
Decision
Localized heat map of the cells that contribute
to the decision (not directly available in labels).
19
Debugging and Building Trust
Simulated images are used to respect the privacy agreements
Input Images of biopsy
slides
Labels Per tissue label;
[Normal/Abnormal]](https://image.slidesharecdn.com/ti078kottayilaltaml-210924130822/85/Explainability-in-Computer-Vision-A-Machine-Learning-Engineer-s-Overview-a-Presentation-from-AltaML-19-320.jpg)
![© 2021 AltaML
Use case:
CV model for assessment of pet
health
Input Images from
hospital
Labels Per image label;
[happy/not happy]
Additional Insights Generated
Model predictions ‘rediscovered’ the
idea of Cat Grimace Scale.
20](https://image.slidesharecdn.com/ti078kottayilaltaml-210924130822/85/Explainability-in-Computer-Vision-A-Machine-Learning-Engineer-s-Overview-a-Presentation-from-AltaML-20-320.jpg)
“Explainability in Computer Vision: A Machine Learning Engineer’s Overview,” a Presentation from AltaML
- 1. © 2021 AltaML Computer Vision Explainability A Machine Learning Engineer's Overview Navaneeth Kamballur Kottayil AltaML
- 2. © 2021 AltaML Outline ➢ Company introduction ➢ Deep learning and trust and why explainability is needed ➢ Categories of techniques in explainability ○ Basic idea + explanation of a representative method ➢ Case studies 2
- 3. © 2021 AltaML AltaML and Computer Vision • AltaML is a Canadian applied Machine learning company that works with industry partners to augment their capabilities with AI&ML. • AltaML has had great success in generating value for its partners with use of computer vision- based ML systems. 3 LIDAR tree species detection Construction site monitoring Industrial damage detection Animal face keypoint detection Pet health Analysis Facies (rock type) classification
- 4. Deep Learning and Trust “Deep learning is a black box”
- 5. © 2021 AltaML Why Black Boxes 5 Image reference: https://medium.com/@RaghavPrabhu/cnn-architectures-lenet-alexnet-vgg-googlenet-and-resnet-7c81c017b848 Common questions: ● Clients : “Why does it make this prediction?” ● ML Dev/Data Scientist : “Why does this work?”; “Is my algorithm looking at the right things?” This is a serious problem even if performance is high. The picture is classified as a motorcycle on the basis of the following set of pixels
- 6. © 2021 AltaML Opening up Black Boxes 6 Image reference:Fei-Fei Li, Justin Johnson, Serena Yeung, CS 231n We can visualize features detected at each of the layers. • Initial layer filters detect Gabor like edges ! Deeper layer filters convey no meaningful information. Results cannot be explained with visualizations of filter coefficients or outputs.
- 7. © 2021 AltaML Filter Visualization Techniques 7 Basic idea: Synthesize inputs that can maximize a specific neuron activation. ● Input a random noise image as input. Say , to trained CNN. ● Perform a fwd pass of the image. ● Assuming that the filter that one wants to visualize is of index i, such that activation of the specific layer of interest is ● The visualization of the fitler is obtained by adding the backpropagated gradients from back to the image (usually with a scale factor to control the amount by which update is done)
- 8. © 2021 AltaML Explainability Provide either, a set of pixels or a heat map showing the pixels that were important for a classification decision. 8 Input image Explainability output
- 9. © 2021 AltaML Explainability Active field of research with very large number of research papers, tools and techniques Broad categories of research • Perturbation based methods • Backpropagation based methods • Activation based methods 9 Image reference: Das, Arun, and Paul Rad. "Opportunities and challenges in explainabl e artificial intelligence (xai): A survey." arXiv preprint arXiv:2006.11371 (2020).
- 10. © 2021 AltaML Perturbation Based Methods Basic idea: learn the behavior by perturbing the input and see how the predictions change. Image reference: Ribeiro, Marco Tulio, Sameer Singh, and Carlos Guestrin. "" Why should I trust you?" Explaining the predictions of any classifier." Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining. 2016. 10
- 11. © 2021 AltaML Perturbation Based Methods Image reference: Ribeiro, Marco Tulio, Sameer Singh, and Carlos Guestrin. "" Why should I trust you?" Explaining the predictions of any classifier." Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining. 2016. 11 LIME (Local Interpretable Model-Agnostic Explanations) ● Choose the ML model and a reference class to be explained ● Generate perturbations all over the image space (an approximation of this is done by dividing image to superpixels and randomly turn off superpixels) ● Predict the output Y, for each perturbed image, using the ML model ● Find the contribution of each of the superpixel by, training the following Linear Ridge Regression on the generated output: E(Y)= β₀ + ∑ βⱼ Xⱼ ● The β coefficients are regarded as LIME explanation. The superpixels with the largest weight is the explanation (in terms of pixels)
- 12. © 2021 AltaML Backpropagation Based Methods Basic idea: Trace the signals from classification output back to the input Image reference:http://www.heatmapping.org/ Paper reference: Binder, Alexander, et al. "Layer-wise relevance propagation for neural networks with local renormalization layers." International Conference on Artificial Neural Networks. Springer, Cham, 2016. 12
- 13. © 2021 AltaML Backpropagation Based Methods Image reference:http://www.heatmapping.org/ Paper reference: Binder, Alexander, et al. "Layer-wise relevance propagation for neural networks with local renormalization layers." International Conference on Artificial Neural Networks. Springer, Cham, 2016. 13 LRP (Layer-wise relevance propagation) ● Choose the ML model and a reference class to be explained ● Start with output neuron of class c (its probability will be considered as relevance R at output layer) and trace the result to its previous layer with formula for neuron i, at layer l. In the notation used, is output of neuron j at layer l ● This is continued till the input image
- 14. © 2021 AltaML Activation Based Methods Image reference: Chattopadhay, Aditya, et al. "Grad-cam++: Generalized gradient-based visual explanations for deep convolutional networks." 2018 IEEE Winter Conference on Applications of Computer Vision (WACV). IEEE, 2018. 14 Basic idea: Express classification results in terms of strength of feature maps* * Note that these are feature maps, ie, outputs of filters, NOT filters themselves
- 15. © 2021 AltaML Activation Based Methods Image reference: Chattopadhay, Aditya, et al. "Grad-cam++: Generalized gradient-based visual explanations for deep convolutional networks." 2018 IEEE Winter Conference on Applications of Computer Vision (WACV). IEEE, 2018. 15 GradCAM ● Choose the ML model and a reference class c to be explained ● Choose a set of k, CNN layer outputs, ie, feature maps Ak ● Compute scale factor for the feature maps as where GAP is average operation on 2D ● GradCAM is computed as for all of the k chosen feature maps * Note that these are feature maps, ie, outputs of filters, NOT filters themselves
- 16. © 2021 AltaML Comparisons 16 Perturbation based Backpropagation based Activation based Advantage ● Model agnostic ● Easy to implement ● No modification to model ● Quick to compute Fine-grained interpretation ● No modification to model ● Easy to interpret ● Reasonably fast Disadvantage ● Time consuming to run ● Need access to model weights and architecture ● Sometimes will be hard to interpret ● Only for CNN ● Different explanations based on selected feature maps ● Minimal modification of model (in some implementations) Courtesy: https://yozey.github.io/files/CNNInterpretability-1-41.pdf
- 17. Applications in Use Cases
- 18. © 2021 AltaML Debugging and Building Trust ML Used image segmentation to give the ML model only images of tissue and not the rest of the slide. Decisions on an entire tissue sample. No further information provided by the model. Decision 18 Simulated images are used to respect the privacy agreements Use case: Classify high resolution microscopy slides into normal or abnormal ML: Image classification problem Input Images of biopsy slides Labels Per tissue label; [Normal/Abnormal]
- 19. © 2021 AltaML Explanations significantly helped in building trust in the proposed model for ML team and doctors. Doctors identified that the heat map pointed to the damaged cells of interest to the doctors. ML + Explanation Decision Localized heat map of the cells that contribute to the decision (not directly available in labels). 19 Debugging and Building Trust Simulated images are used to respect the privacy agreements Input Images of biopsy slides Labels Per tissue label; [Normal/Abnormal]
- 20. © 2021 AltaML Use case: CV model for assessment of pet health Input Images from hospital Labels Per image label; [happy/not happy] Additional Insights Generated Model predictions ‘rediscovered’ the idea of Cat Grimace Scale. 20
- 21. © 2021 AltaML Conclusions • Explainability techniques are vital in computer vision-based use cases to explain the decisions of deep learning-based models. • Implementation of these techniques is not expensive. • Most basic techniques can give a lot of useful insights. Fun fact: We found out that grumpy cat is in fact, not grumpy at all! 21
- 22. © 2021 AltaML Resources 22 Perturbation based explainability: Ribeiro, Marco Tulio, Sameer Singh, and Carlos Guestrin. "" Why should I trust you?" Explaining the predictions of any classifier." Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining. 2016. Backpropagation based explainability: Binder, Alexander, et al. "Layer-wise relevance propagation for neural networks with local renormalization layers." International Conference on Artificial Neural Networks. Springer, Cham, 2016. Activation map explainability: Chattopadhay, Aditya, Anirban Sarkar, Prantik Howlader, and Vineeth N. Balasubramanian. "Grad-cam++: Generalized gradient-based visual explanations for deep convolutional networks." In 2018 IEEE Winter Conference on Applications of Computer Vision (WACV), pp. 839-847. IEEE, 2018. Visualizing Convolutional Feature maps: Zeiler, Matthew D., and Rob Fergus. "Visualizing and understanding convolutional networks." European conference on computer vision. Springer, Cham, 2014. Explainability in CV survey paper: Das, Arun, and Paul Rad. "Opportunities and challenges in explainable artificial intelligence (xai): A survey." arXiv preprint arXiv:2006.11371 (2020).
- 23. Thank You