SlideShare a Scribd company logo

Responsible AI in Industry: Practical Challenges and Lessons Learned

Download as PPTX, PDF
K
Krishnaram Kenthapadi

The document discusses several foundational principles of responsible AI, including the importance of fairness, explainability, and privacy in machine learning systems. It highlights the challenges posed by biases in AI systems, the need for transparency in model predictions, and various attribution methods used to explain AI decision-making. Additionally, it covers concepts of differential privacy and the technical approaches to preserving user data privacy while enabling useful analytics.

Internet
Related topics:
Data Mining Insights
1 of 162
Downloaded 32 times
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
© 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Krishnaram Kenthapadi Principal Scientist, Amazon AWS AI Responsible AI in Industry: Practical Challenges and Lessons Learned Invited talks @ • Responsible Use of Data series, The Academic Fringe Festival, Delft University of Technology, Jul’2021 • Flipkart Data Science Conference, Jun’2021 • Syracuse University, May’2021 • The University of Waterloo ML+Logic Colloquium, Apr’2021 • The University of British Columbia, Vancouver, Mar’2021
Massachusetts Group Insurance Commission (1997): Anonymized medical history of state employees William Weld vs Latanya Sweeney Latanya Sweeney (MIT grad student): $20 – Cambridge voter roll born July 31, 1945 resident of 02138
64% Uniquely identifiable with ZIP + birth date + gender (in the US population) Golle, “Revisiting the Uniqueness of Simple Demographics in the US Population”, WPES 2006
A History of Privacy Failures … Credit: Kobbi Nissim,Or Sheffet
Responsible AI in Industry: Practical Challenges and Lessons Learned
Responsible AI in Industry: Practical Challenges and Lessons Learned
• Ethical challenges posed by AI systems • Inherent biases present in society • Reflected in training data • AI/ML models prone to amplifying such biases Algorithmic Bias
Laws against Discrimination Immigration Reform and Control Act Citizenship Rehabilitation Act of 1973; Americans with Disabilities Act of 1990 Disability status Civil Rights Act of 1964 Race Age Discrimination in Employment Act of 1967 Age Equal Pay Act of 1963; Civil Rights Act of 1964 Sex And more...
Fairness Privacy Transparency Explainability
Motivation & Business Opportunities Regulatory. We need to understand why the ML model made a given prediction and also whether the prediction it made was free from bias, both in training and at inference. Business. Providing explanations to internal teams (loan officers, customer service rep, compliance teams) and end users/customers Data Science. Improving models through better feature engineering and training data generation, understanding failure modes of the model, debugging model predictions, etc.
© 2020, Amazon Web Services, Inc. or its Affiliates. Amazon SageMaker VISION SPEECH TEXT SEARCH CHATBOTS PERSONALIZATION FORECASTING FRAUD CONTACT CENTERS Deep Learning AMIs & Containers GPUs & CPUs Elastic Inference Trainium Inferentia FPGA AI SERVICES ML SERVICES FRAMEWORKS & INFRASTRUCTURE DeepGraphLibrary Amazon Rekognition Amazon Polly Amazon Transcribe +Medical Amazon Lex Amazon Personalize Amazon Forecast Amazon Comprehend +Medical Amazon Textract Amazon Kendra Amazon CodeGuru Amazon Fraud Detector Amazon Translate INDUSTRIAL AI CODE AND DEVOPS NEW Amazon DevOps Guru Voice ID For AmazonConnect Contact Lens NEW Amazon Monitron NEW AWS Panorama + Appliance NEW Amazon Lookout for Vision NEW Amazon Lookout for Equipment Scaling Fairness, Explainability & Privacy across the AWS ML Stack NEW Amazon HealthLake HEALTHCARE AI NEW Amazon Lookout for Metrics ANOMALY DETECTION Amazon Transcribe for Medical Amazon Comprehend for Medical Label data NEW Aggregate & prepare data NEW Store & share features Auto ML Spark/R NEW Detect bias Visualize in notebooks Pick algorithm Train models Tune parameters NEW Debug & profile Deploy in production Manage & monitor NEW CI/CD Human review review NEW: Model management for edge devices NEW: SageMaker JumpStart SAGEMAKER STUDIO IDE
LinkedIn operates the largest professional network on the Internet Tell your story 740M members 55M+ companies are represented on LinkedIn 90K schools listed (high school & college) 36K skills listed 14M+ open jobs on LinkedIn Jobs 280B Feed updates
Talk Outline* • Fairness-aware ML: An overview • Explainable AI: An overview • Privacy-preserving ML: An overview • Responsible AI tools • Case studies • Key takeaways * For a longer version, please see our tutorial titled, “Responsible AI in Industry”: https://sites.google.com/view/ResponsibleAITutorial
Explainable AI: An Overview
What is Explainable AI? Data Opaque AI AI product Confusion with Today’s Opaque AI ● Why did you do that? ● Why did you not do that? ● When do you succeed or fail? ● How do I correct an error? Opaque AI Prediction, Recommendation Clear & Transparent Predictions ● I understand why ● I understand why not ● I know why you succeed or fail ● I understand, so I trust you Explainable AI Data Explainable AI Explainable AI Product Prediction Explanation Feedback
- Humans may have follow-up questions - Explanations cannot answer all users’ concerns Weld, D., and Gagan Bansal. "The challenge of crafting intelligible intelligence." Communications of the ACM (2018). Example of an End-to-End XAI System
Neural Net CNN GAN RNN Ensemble Method Random Forest XGB Statistical Model AOG SVM Graphical Model Bayesian Belief Net SLR CRF HBN MLN Markov Model Decision Tree Linear Model Non-Linear functions Polynomial functions Quasi-Linear functions Accuracy Explainability Interpretability Learning • Challenges: • Supervised • Unsupervised learning • Approach: • Representation Learning • Stochastic selection • Output: • Correlation • No causation How to Explain? Accuracy vs. Explainability
Text Tabular Images On the Role of Data in XAI
Explainable AI Methods Approach 1: Post-hoc explain a given AI model ● Individual prediction explanations in terms of input features, influential examples, concepts, local decision rules ● Global prediction explanations in terms of entire model in terms of partial dependence plots, global feature importance, global decision rules Approach 2: Build an interpretable model ● Logistic regression, Decision trees, Decision lists and sets, Generalized Additive Models (GAMs) 19
Credit: Slide based on https://twitter.com/chandan_singh96/status/1138811752769101825 Integrated Gradients Opaque, complex model
Explaining Model Predictions: Credit Lending 21 Fair lending laws [ECOA, FCRA] require credit decisions to be explainable
Feature Attribution Problem Attribute a model’s prediction on an input to features of the input Examples ● Attribute a lending model’s prediction to its features ● Attribute an object recognition model’s prediction to its pixels ● Attribute a text sentiment model’s prediction to individual words Several attribution methods: ● Ablations ● Gradient based methods (specific to differentiable models) ● Score back-propagation based methods (specific to neural networks) ● Game theory based methods (Shapley values) 22
Application of Attributions ● Debugging model predictions E.g., Attribution an image misclassification to the pixels responsible for it ● Generating an explanation for the end-user E.g., Expose attributions for a lending prediction to the end-user ● Analyzing model robustness E.g., Craft adversarial examples using weaknesses surfaced by attributions ● Extract rules from the model E.g., Combine attribution to craft rules (pharmacophores) capturing prediction logic of a drug screening network 23
Next few slides We will cover the following attribution methods** ● Ablations ● Gradient based methods (specific to differentiable models) ● Score Backpropagation based methods (specific to NNs) We will also discuss game theory (Shapley value) in attributions **Not a complete list! See Ancona et al. [ICML 2019], Guidotti et al. [arxiv 2018] for a comprehensive survey 24
Ablations Drop each feature and attribute the change in prediction to that feature Pros: ● Simple and intuitive to interpret Cons: ● Unrealistic inputs ● Improper accounting of interactive features ● Can be computationally expensive 25
Feature*Gradient Attribution to a feature is feature value times gradient, i.e., xi* 𝜕y/𝜕xi ● Gradient captures sensitivity of output w.r.t. feature ● Equivalent to Feature*Coefficient for linear models ○ First-order Taylor approximation of non-linear models ● Popularized by SaliencyMaps [NeurIPS 2013], Baehrens et al. [JMLR 2010] 26 Gradients in the vicinity of the input seem like noise?
Local linear approximations can be too local 27 score “fireboat-ness” of image Interesting gradients uninteresting gradients (saturation) 1.0 0.0
Score Back-Propagation based Methods Re-distribute the prediction score through the neurons in the network ● LRP [JMLR 2017], DeepLift [ICML 2017], Guided BackProp [ICLR 2014] Easy case: Output of a neuron is a linear function of previous neurons (i.e., ni = ⅀ wij * nj) e.g., the logit neuron ● Re-distribute the contribution in proportion to the coefficients wij 28 Image credit heatmapping.org
Score Back-Propagation based Methods Re-distribute the prediction score through the neurons in the network ● LRP [JMLR 2017], DeepLift [ICML 2017], Guided BackProp [ICLR 2014] Tricky case: Output of a neuron is a non-linear function, e.g., ReLU, Sigmoid, etc. ● Guided BackProp: Only consider ReLUs that are on (linear regime), and which contribute positively ● LRP: Use first-order Taylor decomposition to linearize activation function ● DeepLift: Distribute activation difference relative a reference point in proportion to edge weights 29 Image credit heatmapping.org
Score Back-Propagation based Methods Re-distribute the prediction score through the neurons in the network ● LRP [JMLR 2017], DeepLift [ICML 2017], Guided BackProp [ICLR 2014] Pros: ● Conceptually simple ● Methods have been empirically validated to yield sensible result Cons: ● Hard to implement, requires instrumenting the model ● Often breaks implementation invariance Think: F(x, y, z) = x * y *z and G(x, y, z) = x * (y * z) Image credit heatmapping.org
Baselines and additivity ● When we decompose the score via backpropagation, we imply a normative alternative called a baseline ○ “Why Pr(fireboat) = 0.91 [instead of 0.00]” ● Common choice is an informationless input for the model ○ E.g., All black or all white image for image models ○ E.g., Empty text or zero embedding vector for text models ● Additive attributions explain F(input) - F(baseline) in terms of input features
score intensity Interesting gradients uninteresting gradients (saturation) 1.0 0.0 Baseline … scaled inputs ... … gradients of scaled inputs …. Input Another approach: gradients at many points
IG(input, base) ::= (input - base) * ∫0 -1▽F(𝛂*input + (1-𝛂)*base) d𝛂 Original image Integrated Gradients Integrated Gradients [ICML 2017] Integrate the gradients along a straight-line path from baseline to input
34 © 2020 Amazon Web Services, Inc. or its affiliates. All rights reserved | Feature Attributions using Shapley Values Coalition Game ● Players collaborating to generate some payoff ● Shapley value: Fair way to quantify each player’s contribution to the game’s payoff 34
35 © 2020 Amazon Web Services, Inc. or its affiliates. All rights reserved | Shapley Value: Intuition 35
36 © 2020 Amazon Web Services, Inc. or its affiliates. All rights reserved | Shapley Value [Annals of Mathematical studies,1953] = Average over all player orderings & subsets SC M / {i} Marginal contribution Coalition contribution with and without this player f – payoff of the game M – all players S – subset of players i – specific player Fair way to quantify each player’s contribution to the game’s payoff
37 © 2020 Amazon Web Services, Inc. or its affiliates. All rights reserved | Shapley Value Justification Shapley values are unique under four simple axioms ● Dummy: If a player never contributes to the game then it must receive zero attribution ● Efficiency: Attributions must add to the total payoff ● Symmetry: Symmetric players must receive equal attribution ● Linearity: Attribution for the (weighted) sum of two games must be the same as the (weighted) sum of the attributions for each of the games 37
38 © 2020 Amazon Web Services, Inc. or its affiliates. All rights reserved | Explaining Model Predictions using Shapley Values f Game payoff Game theory All players Subset of players Specific player Instance being explained Model prediction All features Subset of features Specific feature Instance being explained M S i x Machine learning https://github.com/slundberg/shap Lundberg et al: NeurIPS (2017)
39 © 2020 Amazon Web Services, Inc. or its affiliates. All rights reserved | Explaining Model Predictions: SHAP Algorithm How it works = Average over all feature orderings & subsets SC M / {i} f – model prediction M – all features S – subset of features i – specific feature x – instance being explained https://github.com/slundberg/shap Lundberg et al: NeurIPS (2017)
40 © 2020 Amazon Web Services, Inc. or its affiliates. All rights reserved | Explaining Model Predictions: SHAP Algorithm Different solutions for runtime and algorithmic challenges = Average over all feature orderings & subsets SC M / {i} f – model prediction M – all features S – subset of features i – specific feature x – instance being explained https://github.com/slundberg/shap Lundberg et al: NeurIPS (2017) Exponential time How to run a model with missing features?
41 © 2020 Amazon Web Services, Inc. or its affiliates. All rights reserved | Explaining Model Predictions: SHAP Algorithm Different solutions for runtime and algorithmic challenges = Average over all feature orderings & subsets SC M / {i} f – model prediction M – all features S – subset of features i – specific feature x – instance being explained https://github.com/slundberg/shap Lundberg et al: NeurIPS (2017) Exponential time How to run a model with missing features? Shap.KernelExplainer • All models • Explanations Sample subsets, defaults to 2|M| + 2048 Fill missing values from a background dataset • Training set • Median of the dataset • K-means representatives • All-black image Kernel Open-source SHAP implementation
Baselines (or Norms) are essential to explanations [Kahneman-Miller 86] ● E.g., A man suffers from indigestion. Doctor blames it to a stomach ulcer. Wife blames it on eating turnips. Both are correct relative to their baselines. ● The baseline may also be an important analysis knob. Attributions are contrastive, whether we think about it or not. Lesson learned: Baselines are important
Some limitations and caveats for attributions
Some things that are missing: ● Feature interactions (ignored or averaged out) ● What training examples influenced the prediction (training agnostic) ● Global properties of the model (prediction-specific) An instance where attributions are useless: ● A model that predicts TRUE when there are even number of black pixels and FALSE otherwise Attributions don’t explain everything
Attributions are for human consumption Naive scaling of attributions from 0 to 255 Attributions have a large range and long tail across pixels After clipping attributions at 99% to reduce range ● Humans interpret attributions and generate insights ○ Doctor maps attributions for x-rays to pathologies ● Visualization matters as much as the attribution technique
Privacy-preserving ML: An Overview
What is Privacy? • Right of/to privacy • “Right to be let alone” [L. Brandeis & S. Warren, 1890] • “No one shall be subjected to arbitrary interference with [their] privacy, family, home or correspondence, nor to attacks upon [their] honor and reputation.” [The United Nations Universal Declaration of Human Rights] • “The right of a person to be free from intrusion into or publicity concerning matters of a personal nature” [Merriam-Webster] • “The right not to have one's personal matters disclosed or publicized; the right to be left alone” [Nolo’s Plain-English Law Dictionary]
Data Privacy (or Information Privacy) • “The right to have some control over how your personal information is collected and used” [IAPP] • “Privacy has fast-emerged as perhaps the most significant consumer protection issue—if not citizen protection issue—in the global information economy” [IAPP]
Data Privacy vs. Security • Data privacy: use & governance of personal data • Data security: protecting data from malicious attacks & the exploitation of stolen data for profit • Security is necessary, but not sufficient for addressing privacy.
Data Privacy:Technical Problem Given a dataset with sensitive personal information, how can we compute and release functions of the dataset while protecting individual privacy? Credit: Kobbi Nissim
A History of Privacy Failures … Credit: Kobbi Nissim,Or Sheffet
Lessons Learned … • Attacker’s advantage: Auxiliary information; high dimensionality; enough to succeed on a small fraction of inputs; active; observant … • Unanticipated privacy failures from new attack methods • Need for rigorous privacy notions & techniques
Threat Models
© 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Threat Models
© 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Threat Models
Threat Models User Access Only • Users store their data • Noisy data or analytics transmitted Trusted Curator • Stored by organization • Managed only by a trusted curator/admin • Access only to noisy analytics or synthetic data External Threat • Stored by organization • Organization has access • Only privacy enabled models deployed
Differential Privacy
Curator Defining Privacy
Defining Privacy 59 Curator Curator + your data - your data
Differential Privacy 60 Databases D and D′ are neighbors if they differ in one person’s data. Differential Privacy: The distribution of the curator’s output M(D) on database D is (nearly) the same as M(D′). Curator + your data - your data Dwork, McSherry, Nissim, Smith [TCC 2006] Curator
(ε, 𝛿)-Differential Privacy: The distribution of the curator’s output M(D) on database D is (nearly) the same as M(D′). Differential Privacy 61 Curator Parameter ε quantifies information leakage ∀S: Pr[M(D)∊S] ≤ exp(ε) ∙ Pr[M(D′)∊S]＋𝛿. Curator Parameter 𝛿 gives some slack Dwork, Kenthapadi, McSherry, Mironov, Naor [EUROCRYPT 2006] + your data - your data Dwork, McSherry, Nissim, Smith [TCC 2006]
Differential Privacy: Random Noise Addition If ℓ1-sensitivity of f : D → ℝn: maxD,D′ ||f(D) − f(D′)||1 = s, then adding Laplacian noise to true output f(D) + Laplacen(s/ε) offers (ε,0)-differential privacy. Dwork, McSherry, Nissim, Smith [TCC 2006]
Examples of Attacks • Membership inference attacks • Reconstruction attacks • Model inversion attacks • Different attacks correspond to different threat models
Privacy-preserving MLTechniques /Tools • Differentially private model training • Differentially private stochastic gradient descent • PATE • Differentially private synthetic data generation • Federated learning • Trusted execution environments (secure enclaves) • Secure multiparty computation • Homomorphic encryption • Different techniques applicable under different threat models • May need a hybrid approach
Case Studies
Fairness in AI @ LinkedIn Fairness-aware Talent Search Ranking* *Work done while at LinkedIn
Guiding Principle: “Diversity by Design”
Insights to Identify Diverse Talent Pools Representative Talent Search Results Diversity Learning Curriculum “Diversity by Design” in LinkedIn’s Talent Solutions
Plan for Diversity
Representative Ranking for Talent Search S. C. Geyik, S. Ambler, K. Kenthapadi, Fairness- Aware Ranking in Search & Recommendation Systems with Application to LinkedIn Talent Search, KDD’19. [Microsoft’s AI/ML conference (MLADS’18). Distinguished Contribution Award] Building Representative Talent Search at LinkedIn (LinkedIn engineering blog)
Intuition for Measuring and Achieving Representativeness Ideal: Top ranked results should follow a desired distribution on gender/age/… E.g., same distribution as the underlying talent pool Inspired by “Equal Opportunity” definition [Hardt et al, NIPS’16] Defined measures (skew, divergence) based on this intuition
Measuring (Lack of) Representativeness Skew@k (Logarithmic) ratio of the proportion of candidates having a given attribute value among the top k ranked results to the corresponding desired proportion Variants: MinSkew: Minimum over all attribute values MaxSkew: Maximum over all attribute values Normalized Discounted Cumulative Skew Normalized Discounted Cumulative KL-divergence
Fairness-aware Reranking Algorithm (Simplified) Partition the set of potential candidates into different buckets for each attribute value Rank the candidates in each bucket according to the scores assigned by the machine-learned model Merge the ranked lists, balancing the representation requirements and the selection of highest scored candidates Representation requirement: Desired distribution on gender/age/… Algorithmic variants based on how we choose the next attribute
Architecture
Validating Our Approach Gender Representativeness Over 95% of all searches are representative compared to the qualified population of the search Business Metrics A/B test over LinkedIn Recruiter users for two weeks No significant change in business metrics (e.g., # InMails sent or accepted) Ramped to 100% of LinkedIn Recruiter users worldwide
Lessons learned • Post-processing approach desirable • Model agnostic • Scalable across different model choices for our application • Acts as a “fail-safe” • Robust to application-specific business logic • Easier to incorporate as part of existing systems • Build a stand-alone service or component for post-processing • No significant modifications to the existing components • Complementary to efforts to reduce bias from training data & during model training • Collaboration/consensus across key
Lessons learned • Post-processing approach desirable • Model agnostic • Scalable across different model choices for our application • Acts as a “fail-safe” • Robust to application-specific business logic • Easier to incorporate as part of existing systems • Build a stand-alone service or component for post-processing • No significant modifications to the existing components • Complementary to efforts to reduce bias from training data & during model training • Collaboration/consensus across key
Evaluating Fairness Using Permutations Tests [DiCiccio, Vasudevan, Basu, Kenthapadi, Agarwal, KDD’20] • Is the measured discrepancy across different groups statistically significant? • Use statistical hypothesis tests! • Can we perform hypothesis tests in a metric-agnostic manner? • Non-parametric tests can help! • Permutation testing framework
Brief Review of Permutation Tests Observe data from two populations: and Are the populations the same? A reasonable test statistic might be
Brief Review of Permutation Tests (Continued) A p-value is the chance of observing a test statistic at least as “extreme” as the value we actually observed Permutation test approach: ●Randomly shuffle the population designations of the observations ●Recompute the test statistic T ●Repeat many times Permutation p-value: the proportion of permuted datasets resulting in a larger test statistic than the original value This test is exact!
A Fairness Example Consider testing whether the true positive rate of a classifier is equal between two groups Test Statistic: difference in proportion of negative labeled observations that are classified as positive between the two groups Permutation test: Randomly reshuffle group labels, recompute test statistic
Permutations Tests for Evaluating Fairness in ML Models • Issues with classical permutation test • Want to check: just equality of the fairness metric (e.g., false positive rate) across groups, and not if the two groups have identical distribution • Exact for the strong null hypothesis … • … but may not be valid (even asymptotically) for the weak null hypothesis • Our paper: A fix for this issue • Choose a pivotal statistic (asymptotically distribution-free; does not depend on the observed data’s distribution) • E.g., Studentize the test statistic
Permutations Tests for Evaluating Fairness in ML Models • Issues with classical permutation test • Want to check: just equality of the fairness metric (e.g., false positive rate) across groups, and not if the two groups have identical distribution • Exact for the strong null hypothesis … • … but may not be valid (even asymptotically) for the weak null hypothesis • Our paper: A fix for this issue • Choose a pivotal statistic (asymptotically distribution-free; does not depend on the observed data’s distribution) • E.g., Studentize the test statistic
Engineering for Fairness in AI Lifecycle S.Vasudevan, K. Kenthapadi, LiFT: A Scalable Framework for Measuring Fairness in ML Applications, CIKM’20 https://github.com/linkedin/LiFT
LiFT System Architecture [Vasudevan & Kenthapadi, CIKM’20] •Flexibility of Use (Platform agnostic) •Ad-hoc exploratory analyses •Deployment in offline workflows •Integration with ML Frameworks •Scalability •Diverse fairness metrics •Conventional fairness metrics •Benefit metrics •Statistical tests
LiFT System Architecture [Vasudevan & Kenthapadi, CIKM’20] •Flexibility of Use (Platform agnostic) •Ad-hoc exploratory analyses •Deployment in offline workflows •Integration with ML Frameworks •Scalability •Diverse fairness metrics •Conventional fairness metrics •Benefit metrics •Statistical tests
Acknowledgements LinkedIn Talent Solutions Diversity team, Hire & Careers AI team, Anti-abuse AI team, Data Science Applied Research team Special thanks to Deepak Agarwal, Parvez Ahammad, Stuart Ambler, Kinjal Basu, Jenelle Bray, Erik Buchanan, Bee-Chung Chen, Fei Chen, Patrick Cheung, Gil Cottle, Cyrus DiCiccio, Patrick Driscoll, Carlos Faham, Nadia Fawaz, Priyanka Gariba, Meg Garlinghouse, Sahin Cem Geyik, Gurwinder Gulati, Rob Hallman, Sara Harrington, Joshua Hartman, Daniel Hewlett, Nicolas Kim, Rachel Kumar, Monica Lewis, Nicole Li, Heloise Logan, Stephen Lynch, Divyakumar Menghani, Varun Mithal, Arashpreet Singh Mor, Tanvi Motwani, Preetam Nandy, Lei Ni, Nitin Panjwani, Igor Perisic, Hema Raghavan, Romer Rosales, Guillaume Saint-Jacques, Badrul Sarwar, Amir Sepehri, Arun Swami, Ram Swaminathan, Grace Tang, Ketan Thakkar, Sriram Vasudevan, Janardhanan Vembunarayanan, James Verbus, Xin Wang, Hinkmond Wong, Ya Xu, Lin Yang, Yang Yang, Chenhui Zhai, Liang Zhang, Yani Zhang
88 © 2020 Amazon Web Services, Inc. or its affiliates. All rights reserved | 88 © 2020 Amazon Web Services, Inc. or its affiliates. All rights reserved | Detect bias in ML models and understand model predictions Amazon SageMaker Clarify
89 © 2020 Amazon Web Services, Inc. or its affiliates. All rights reserved | Predictive Maintenance Manufacturing, Automotive, IoT Demand Forecasting Retail, Consumer Goods, Manufacturing Fraud Detection Financial Services, Online Retail Credit Risk Prediction Financial Services, Retail Extract and Analyze Data from Documents Healthcare, Legal, Media/Ent, Education Computer Vision Healthcare, Pharma, Manufacturing Autonomous Driving Automotive, Transportation Personalized Recommendations Media & Entertainment, Retail, Education Churn Prediction Retail, Education, Software & Internet https://aws.amazon.c om/sagemaker/gettin g-started Amazon SageMaker Customer ML Use cases
90 © 2020 Amazon Web Services, Inc. or its affiliates. All rights reserved | Bias and Explainability: Challenges 1 Without detection, it is hard to know if bias has entered an ML model: • Imbalances may be present in the initial dataset • Bias may develop during training • Bias may develop over time after model deployment 2 Machine learning models are often complex & opaque, making explainability critical: • Regulations may require companies to be able to explain model predictions • Internal stakeholders and customers may need explanations for model behavior • Data science teams can improve models if they understand model behavior
91 © 2020 Amazon Web Services, Inc. or its affiliates. All rights reserved | Amazon SageMaker Clarify Detect bias in ML models and understand model predictions Detect bias during data preparation Identify imbalances in data Evaluate the degree to which various types of bias are present in your model Check your trained model for bias Understand the relative importance of each feature to your model’s behavior Explain overall model behavior Understand the relative importance of each feature for individual inferences Explain individual predictions Provide alerts and detect drift over time due to changing real-world conditions Detect drift in bias and model behavior over time Generate automated reports Produce reports on bias and explanations to support internal presentations
92 © 2020 Amazon Web Services, Inc. or its affiliates. All rights reserved | SageMaker Clarify works across the ML lifecycle Collect and prepare training data Train and tune model Evaluate and qualify model Deploy model in production Monitor model in production Measure Bias Metrics Measure and Tune Bias Metrics Measure Explainability Metrics Catalog Model Metrics Measure Bias Metrics Measure Explainability Metrics Monitor Bias Metric Drift Monitor Explainability Drift SageMaker Data Wrangler SageMaker Training Autopilot Hyperparameter Tuning SageMaker Processing SageMaker Hosting SageMaker Model Monitor
93 © 2020 Amazon Web Services, Inc. or its affiliates. All rights reserved | How SageMaker Clarify works Amazon SageMaker Clarify
94 © 2020 Amazon Web Services, Inc. or its affiliates. All rights reserved | SageMaker Clarify – Detect Bias During Data Preparation Bias report in SageMaker Data Wrangler
95 © 2020 Amazon Web Services, Inc. or its affiliates. All rights reserved | SageMaker Clarify – Check Your Trained Model for Bias Bias report in SageMaker Experiments
96 © 2020 Amazon Web Services, Inc. or its affiliates. All rights reserved | SageMaker Clarify – Monitor Your Model for Bias Drift Bias Drift in SageMaker Model Monitor
97 © 2020 Amazon Web Services, Inc. or its affiliates. All rights reserved | SageMaker Clarify – Understand Your Model Model Explanation in SageMaker Experiments
98 © 2020 Amazon Web Services, Inc. or its affiliates. All rights reserved | SageMaker Clarify – Monitor Your Model for Drift in Behavior Explainability Drift in SageMaker Model Monitor
99 © 2020 Amazon Web Services, Inc. or its affiliates. All rights reserved | Case Study & Demo: https://youtu.be/cQo2ew0DQw0
100 © 2020 Amazon Web Services, Inc. or its affiliates. All rights reserved | SageMaker Clarify Use Cases Regulatory Compliance Internal Reporting Operational Excellence Customer Service
Lessons learned • Fairness as a Process • Notions of bias & fairness are highly application dependent • Choice of the attribute(s) for which bias is to be measured & the choice of the bias metrics to be guided by social, legal, and other non-technical considerations • Collaboration/consensus across key stakeholders • Wide spectrum of customers with different levels of technical background • Managed service vs. open source packages • Monitoring of the deployed model • Fairness & explainability considerations across the ML lifecycle
102 © 2020 Amazon Web Services, Inc. or its affiliates. All rights reserved | Best Practices • Fairness as a Process: • The notions of bias and fairness are highly application dependent and the choice of the attribute(s) for which bias is to be measured, as well as the choice of the bias metrics, may need to be guided by social, legal, and other non- technical considerations. • Building consensus and achieving collaboration across key stakeholders (such as product, policy, legal, engineering, and AI/ML teams, as well as end users and communities) is a prerequisite for the successful adoption of fairness-aware ML approaches in practice. • Fairness and explainability considerations may be applicable during each stage of the ML lifecycle.
103 © 2020 Amazon Web Services, Inc. or its affiliates. All rights reserved | Fairness and Explainability by Design in the ML Lifecycle
104 © 2020 Amazon Web Services, Inc. or its affiliates. All rights reserved | Additional Pointers For more information on Amazon SageMaker Clarify, please refer: • https://aws.amazon.com/sagemaker/clarify • Amazon Science / AWS Articles • https://aws.amazon.com/blogs/aws/new-amazon-sagemaker-clarify-detects- bias-and-increases-the-transparency-of-machine-learning-models • https://www.amazon.science/latest-news/how-clarify-helps-machine-learning- developers-detect-unintended-bias • Technical papers: (1) Amazon SageMaker Clarify [KDD’21] (2) Fairness Measures for Machine Learning in Finance • https://github.com/aws/amazon-sagemaker-clarify Acknowledgments: Amazon SageMaker Clarify core team, Amazon AWS AI team, and partners across Amazon
105 © 2020 Amazon Web Services, Inc. or its affiliates. All rights reserved | Additional Pointers For more information on Amazon SageMaker Clarify, please refer: • https://aws.amazon.com/sagemaker/clarify • Amazon Science / AWS Articles • https://aws.amazon.com/blogs/aws/new-amazon-sagemaker-clarify-detects- bias-and-increases-the-transparency-of-machine-learning-models • https://www.amazon.science/latest-news/how-clarify-helps-machine-learning- developers-detect-unintended-bias • Technical papers: (1) Amazon SageMaker Clarify [KDD’21] (2) Fairness Measures for Machine Learning in Finance • https://github.com/aws/amazon-sagemaker-clarify Acknowledgments: Amazon SageMaker Clarify core team, Amazon AWS AI team, and partners across Amazon
© 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Trademark Amazon SageMaker Debugger Debug and profile ML model training and get real-time insights
© 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Trademark Why debugging and profiling Training bugs Large compute instances Long training times Training ML models is difficult and compute intensive
© 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Trademark SageMaker Debugger Real-time monitoring Relevant data capture Automatic error detection SageMaker Studio integration Debug data while training is ongoing Zero code change Persistent in your S3 bucket Built-in and custom rules Early termination Alerts about rule status Save time and cost Find issues early Accelerate prototyping Detect performance bottlenecks View suggestions on resolving bottlenecks, Interactive visualizations Monitor utilization Profile by step or time duration Right size instance Improve utilization Reduce cost System resource usage Time spent by training operations
© 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Trademark https://www.amazon.science/publications/amazon-sagemaker-debugger-a-system-for-real-time-insights-into-machine-learning-model- training
110 © 2020 Amazon Web Services, Inc. or its affiliates. All rights reserved | Additional Pointers For more information on Amazon SageMaker Debugger, please refer: • https://aws.amazon.com/sagemaker/debugger • AWS Articles • https://aws.amazon.com/blogs/aws/amazon-sagemaker-debugger-debug-your- machine-learning-models • https://aws.amazon.com/blogs/machine-learning/detecting-hidden-but-non-trivial- problems-in-transfer-learning-models-using-amazon-sagemaker-debugger • Technical paper: Amazon SageMaker Debugger: A System for Real-Time Insights into Machine Learning Model Training (MLSys 2021) • https://pypi.org/project/smdebug Acknowledgments: Amazon SageMaker Debugger core team, Amazon AWS AI team, and partners across Amazon
Fairness for Opaque Models via Model Tuning (Hyperparameter Optimization) • Can we tune the hyperparameters of a model to achieve both accuracy and fairness? • Can we support both opaque models and opaque fairness constraints? • Use Bayesian optimization for HPO with fairness constraints! • Explore hyperparameter configurations where fairness constraints are satisfied V. Perrone, M. Donini, M. B. Zafar, R. Schmucker, K. Kenthapadi, C. Archambeau, Fair Bayesian Optimization, AIES 2021 (Best paper award @ ICML 2020 AutoML workshop)
Defining Fairness
Fairness Definitions - Examples
ϵ-Fairness
Global Optimization for Opaque Functions
Bayesian Optimization
Fair Bayesian Optimization
Results: Model-specific vs. Model-agnostic Methods
Human-in-the-loop frameworks Desirable to augment ML model predictions with expert inputs Popular examples – Healthcare models Content moderation tasks Useful for improving accuracy, incorporating additional information, and auditing models. Child maltreatment hotline screening
Errors and biases in human-in-the-loop frameworks ML tasks often suffer from group-specific bias, induced due to misrepresentative data or models. Human-in-the-loop frameworks can reflect biases or inaccuracies of the human experts. Concerns include: • Racial bias in human-in-the-loop framework for recidivism risk assessment (Green, Chen – FAccT 2019) • Ethical concerns regarding audits of commercial facial processing technologies (Raji et al. – AIES 2020) • Automation bias in time critical decision support systems (Cummings – ISTC 2004) Can we design human-in-the-loop frameworks that take into account the expertise and biases of the human experts?
Model 𝑋 − non-protected attributes; 𝑌 − class label; 𝑍 − protected attribute/group membership Number of experts available = 𝑚 − 1 Input −𝑋 Classifier 𝐹: 𝑋 → 𝑌 Deferrer 𝐷: 𝑋 → 0,1 𝑚 . . If 𝐷1 = 1 If 𝐷2 = 1 If 𝐷3 = 1 If 𝐷4 = 1 If 𝐷𝑚−1 = 1 If 𝐷𝑚 = 1 Deferrer 𝐷 choose a committee of experts. The majority decision of committee is the final prediction Final output of selected committee Experts might have access to additional information, including group membership 𝑍. There might be a cost/penalty associated with each expert review.
Fairness in human-in-the-loop settings • Joint learning framework to learn a classifier and a deferral system for multiple experts simultaneously • Synthetic and real-world experiments on the efficacy of our method V. Keswani, M. Lease, K. Kenthapadi, Towards Unbiased and Accurate Deferral to Multiple Experts, AIES 2021.
Minimax Group Fairness: Algorithms and Experiments • Equality of error rates: an intuitive & well-studied group fairness notion • May require artificially inflating error on easier-to-predict groups  • Undesirable when most/all of the targeted population is disadvantaged • Goal: minimize maximum group error [Martinez et al, 2020] • “Ensure that the worst-off group is as well-off as possible” • Our work: algorithms based on a zero-sum game between a Learner and a Regulator • Theoretical results and experimental evaluation E. Diana, W. Gill, M. Kearns, K. Kenthapadi, A. Roth, Minimax Group Fairness: Algorithms and Experiments, AIES 2021.
Robust Interpretability of Neural Text Classifiers • Feature attribution methods used for understanding model predictions • How robust are feature attributions for neural text classifiers? • Are they identical under different random initializations of the same model? • Do they differ between a model with trained parameters and a model with random parameters? • Common feature attribution methods fail both tests! M. B. Zafar, M. Donini, D. Slack, C. Archambeau, S. Das, K. Kenthapadi, On the Lack of Robust Interpretability of Neural Text Classifiers, Findings in ACL 2021.
Privacy in AI @ LinkedIn PriPeARL: Framework to compute robust, privacy-preserving analytics
Analytics & Reporting Products at LinkedIn Profile View Analytics 126 Content Analytics Ad Campaign Analytics All showing demographics of members engaging with the product
Admit only a small # of predetermined query types Querying for the number of member actions, for a specified time period, together with the top demographic breakdowns Analytics & Reporting Products at LinkedIn
Admit only a small # of predetermined query types Querying for the number of member actions, for a specified time period, together with the top demographic breakdowns Analytics & Reporting Products at LinkedIn E.g., Title = “Senior Director” E.g., Clicks on a given ad
Privacy Requirements Attacker cannot infer whether a member performed an action E.g., click on an article or an ad Attacker may use auxiliary knowledge E.g., knowledge of attributes associated with the target member (say, obtained from this member’s LinkedIn profile) E.g., knowledge of all other members that performed similar action (say, by creating fake accounts)
Possible Privacy Attacks 130 Targeting: Senior directors in US, who studied at Cornell Matches ~16k LinkedIn members → over minimum targeting threshold Demographic breakdown: Company = X May match exactly one person → can determine whether the person clicks on the ad or not Require minimum reporting threshold Attacker could create fake profiles! E.g. if threshold is 10, create 9 fake profiles that all click. Rounding mechanism E.g., report incremental of 10 Still amenable to attacks E.g. using incremental counts over time to infer individuals’ actions Need rigorous techniques to preserve member privacy (not reveal exact aggregate counts)
Problem Statement Compute robust, reliable analytics in a privacy- preserving manner, while addressing the product needs.
PriPeARL: A Framework for Privacy-Preserving Analytics K. Kenthapadi, T. T. L. Tran, ACM CIKM 2018 132 Pseudo-random noise generation, inspired by differential privacy ● Entity id (e.g., ad creative/campaign/account) ● Demographic dimension ● Stat type (impressions, clicks) ● Time range ● Fixed secret seed Uniformly Random Fraction ● Cryptographic hash ● Normalize to (0,1) Random Noise Laplace Noise ● Fixed ε True Count Noisy Count To satisfy consistency requirements ● Pseudo-random noise → same query has same result over time, avoid averaging attack. ● For non-canonical queries (e.g., time ranges, aggregate multiple entities) ○ Use the hierarchy and partition into canonical queries ○ Compute noise for each canonical queries and sum up the noisy counts
PriPeARL System Architecture
Lessons Learned from Deployment (> 1 year) Semantic consistency vs. unbiased, unrounded noise Suppression of small counts Online computation and performance requirements Scaling across analytics applications Tools for ease of adoption (code/API library, hands-on how-to tutorial) help! Having a few entry points (all analytics apps built over Pinot)  wider adoption
Summary Framework to compute robust, privacy-preserving analytics Addressing challenges such as preserving member privacy, product coverage, utility, and data consistency Future Utility maximization problem given constraints on the ‘privacy loss budget’ per user E.g., noise with larger variance to impressions but less noise to clicks (or conversions) E.g., more noise to broader time range sub-queries and less noise to granular time range sub-queries Reference: K. Kenthapadi, T. Tran, PriPeARL: A Framework for Privacy- Preserving Analytics and Reporting at LinkedIn, ACM CIKM 2018.
Acknowledgements Team: AI/ML: Krishnaram Kenthapadi, Thanh T. L. Tran Ad Analytics Product & Engineering: Mark Dietz, Taylor Greason, Ian Koeppe Legal / Security: Sara Harrington, Sharon Lee, Rohit Pitke Acknowledgements Deepak Agarwal, Igor Perisic, Arun Swami
LinkedIn Salary
LinkedIn Salary (launched in Nov, 2016)
Data Privacy Challenges Minimize the risk of inferring any one individual’s compensation data Protection against data breach No single point of failure
Problem Statement How do we design LinkedIn Salary system taking into account the unique privacy and security challenges, while addressing the product requirements? K. Kenthapadi, A. Chudhary, and S. Ambler, LinkedIn Salary: A System for Secure Collection and Presentation of Structured Compensation Insights to Job Seekers, IEEE PAC 2017 (arxiv.org/abs/1705.06976)
Title Region $$ User Exp Designer SF Bay Area 100K User Exp Designer SF Bay Area 115K ... ... ... Title Region $$ User Exp Designer SF Bay Area 100K De-identification Example Title Region Company Industry Years of exp Degree FoS Skills $$ User Exp Designer SF Bay Area Google Internet 12 BS Interactive Media UX, Graphics, ... 100K Title Region Industry $$ User Exp Designer SF Bay Area Internet 100K Title Region Years of exp $$ User Exp Designer SF Bay Area 10+ 100K Title Region Company Years of exp $$ User Exp Designer SF Bay Area Google 10+ 100K #data points > threshold? Yes ⇒ Copy to Hadoop (HDFS) Note: Original submission stored as encrypted objects.
System Architecture
Acknowledgements Team: AI/ML: Krishnaram Kenthapadi, Stuart Ambler, Xi Chen, Yiqun Liu, Parul Jain, Liang Zhang, Ganesh Venkataraman, Tim Converse, Deepak Agarwal Application Engineering: Ahsan Chudhary, Alan Yang, Alex Navasardyan, Brandyn Bennett, Hrishikesh S, Jim Tao, Juan Pablo Lomeli Diaz, Patrick Schutz, Ricky Yan, Lu Zheng, Stephanie Chou, Joseph Florencio, Santosh Kumar Kancha, Anthony Duerr Product: Ryan Sandler, Keren Baruch Other teams (UED, Marketing, BizOps, Analytics, Testing, Voice of Members, Security, …): Julie Kuang, Phil Bunge, Prateek Janardhan, Fiona Li, Bharath Shetty, Sunil Mahadeshwar, Cory Scott, Tushar Dalvi, and team Acknowledgements David Freeman, Ashish Gupta, David Hardtke, Rong Rong, Ram
Privacy Research @ Amazon - Sampler
Differentially Private Query Release • Problem: answering marginal queries privately with high accuracy • Marginal queries are a special class of linear queries that count slices of a dataset. • “How many authors have visited Charlotte, graduated in the last two year and work in the Bay Area?” – A 3-way marginal query on the below dataset. • Privacy: Marginals computed against our dataset should protect against inferences on an individual’s membership (using Differential Privacy)
Differentially Private Query Release • Projection Mechanism: Evaluate noisy answers to all queries in a query class Q and find the synthetic dataset (D′) in the space of feasible datasets that minimizes error with respect to some norm. Q is a class of queries. Nikolov, Talwar, Zhang, The geometry of differential privacy: the sparse and approximate cases, STOC 2013
Differentially Private Query Release: Key Ideas 1. Relax the data domain: one-hot encode the non-continuous data and expand the domain to real numbers. Extend the differentiable queries to the new domain. 2. Adaptively select queries: repeatedly choose the k worst performing queries privately and optimize D′ to answer those well. S. Aydore, W. Brown, M. Kearns, K. Kenthapadi, L. Melis, A. Roth, A. Siva, Differentially Private Query Release Through Adaptive Projection, ICML 2021.
Simple but effective, privacy-preserving mechanism Task: subsample from dataset using additional information in privacy- preserving way. Building on existing exponential analysis of k-anonymity, amplified by sampling… Mechanism M is (β, ε, δ)-differentially private Model uncertainty via Bayesian NN ”Privacy-preserving Active Learning on Sensitive Data for User Intent Classification” [Feyisetan, Balle, Diethe, Drake; PAL 2019]
Analysis of DP sampling Avg impact privacy budget budget applied to downstream classification task. Relative trade-off on accuracy for privacy gains (from β & k) in downstream classification task
Differentially-private text redaction Task: automatically redact sensitive text for privatizing various ML models.  Perturb sentences but maintain meaning e.g. “goalie wore a hockey helmet”  “keeper wear the nhl hat” Apply metric DP and analysis of word embeddings to scramble sentences Mechanism M is d χ – differentially private Establish plausible deniability statistics: Nw := Pr[M(w ) = w ] Sw := Expected number of words output by M(w) “Privacy- and Utility-Preserving Textual Analysis via Calibrated Multivariate Perturbations” [Feyisetan, Drake, Diethe, Balle; WSDM 2020]
Analysis of DP redaction Show plausible deniability via dist. of Nw & Sw for ε: ε  0 : Nw decreases, Sw increases ε  inf : Nw increases, Sw decreases. Impact of accuracy given ε (epsilon) on multi-class classification and question answering tasks, respectively:
Improving data utility of DP text redaction Task: redact text, but use additional structured information to better preserve utility. Can we improve redaction for models that fail for extraneous words? ~Recall-sensitive Extend d χ privacy to hyperbolic embeddings [Tifrea 2018] via Hyperbolic: utilize high-dimensional geometry to infuse embeddings with graph structure E.g. uni- or bi-directional syllogisms from WebIsADb New privacy analysis of Poincaré model and sampling procedure Mechanism takes advantage of density in data to apply perturbations more precisely. “Leveraging Hierarchical Representations for Preserving Privacy and Utility in Text” Feyisetan, Drake, Diethe; ICDM 2019 Tiling in Poincaré disk Hyperbolic Glove emb. projected into B2 Poincaré disk
Analysis of Hyperbolic redaction New method improves over privacy and utility because of ability to encode meaningful structure in embeddings. Accuracy scores on classification tasks. * indicates results better than 1 baseline, ** better than 2 baselines Plausible deniability stat Nw (Pr[M(w ) = w) improved.
Beyond Accuracy Performance and Cost Fairness and Bias Transparency and Explainability Privacy Security Safety Robustness
Fairness, Explainability & Privacy: Opportunities
Fairness in ML Application specific challenges Conversational AI systems: Unique bias/fairness/ethics considerations E.g., Hate speech, Complex failure modes Beyond protected categories, e.g., accent, dialect Entire ecosystem (e.g., including apps such as Alexa skills) Two-sided markets: e.g., fairness to buyers and to sellers, or to content consumers and producers Fairness in advertising (externalities) Tools for ensuring fairness (measuring & mitigating bias) in AI lifecycle Pre-processing (representative datasets; modifying features/labels) ML model training with fairness constraints Post-processing Experimentation & Post-deployment
Explainability in ML Actionable explanations Balance between explanations & model secrecy Robustness of explanations to failure modes (Interaction between ML components) Application-specific challenges Conversational AI systems: contextual explanations Gradation of explanations Tools for explanations across AI lifecycle Pre & post-deployment for ML models Model developer vs. End user focused
Privacy in ML Privacy for highly sensitive data: model training & analytics using secure enclaves, homomorphic encryption, federated learning / on- device learning, or a hybrid Privacy-preserving model training, robust against adversarial membership inference attacks (Dynamic settings + Complex data / model pipelines) Privacy-preserving mechanisms for data marketplaces
Reflections “Fairness, Explainability, and Privacy by Design” when building AI products Collaboration/consensus across key stakeholders NYT / WSJ / ProPublica test :)
Related Tutorials / Resources • ACM Conference on Fairness, Accountability, and Transparency (ACM FAccT) • AAAI/ACM Conference on Artificial Intelligence, Ethics, and Society (AIES) • Sara Hajian, Francesco Bonchi, and Carlos Castillo, Algorithmic bias: From discrimination discovery to fairness-aware data mining, KDD Tutorial, 2016. • Solon Barocas and Moritz Hardt, Fairness in machine learning, NeurIPS Tutorial, 2017. • Kate Crawford, The Trouble with Bias, NeurIPS Keynote, 2017. • Arvind Narayanan, 21 fairness definitions and their politics, FAccT Tutorial, 2018. • Sam Corbett-Davies and Sharad Goel, Defining and Designing Fair Algorithms, Tutorials at EC 2018 and ICML 2018. • Ben Hutchinson and Margaret Mitchell, Translation Tutorial: A History of Quantitative Fairness in Testing, FAccT Tutorial, 2019. • Henriette Cramer, Kenneth Holstein, Jennifer Wortman Vaughan, Hal Daumé III, Miroslav Dudík, Hanna Wallach, Sravana Reddy, and Jean Garcia-Gathright, Translation Tutorial: Challenges of incorporating algorithmic fairness into industry practice, FAccT Tutorial, 2019.
Related Tutorials / Resources • Sarah Bird, Ben Hutchinson, Krishnaram Kenthapadi, Emre Kiciman, Margaret Mitchell, Fairness-Aware Machine Learning: Practical Challenges and Lessons Learned, Tutorials at WSDM 2019, WWW 2019, KDD 2019. • Krishna Gade, Sahin Cem Geyik, Krishnaram Kenthapadi, Varun Mithal, Ankur Taly, Explainable AI in Industry, Tutorials at KDD 2019, FAccT 2020, WWW 2020. • Himabindu Lakkaraju, Julius Adebayo, Sameer Singh, Explaining Machine Learning Predictions: State-of-the-art, Challenges, and Opportunities, NeurIPS 2020 Tutorial. • Kamalika Chaudhuri, Anand D. Sarwate, Differentially Private Machine Learning: Theory, Algorithms, and Applications, NeurIPS 2017 Tutorial. • Krishnaram Kenthapadi, Ilya Mironov, Abhradeep Guha Thakurta, Privacy-preserving Data Mining in Industry, Tutorials at KDD 2018, WSDM 2019, WWW 2019. • Krishnaram Kenthapadi, Ben Packer, Mehrnoosh Sameki, Nashlie Sephus, Responsible AI in Industry, Tutorials at AAAI 2021, FAccT 2021, WWW 2021, ICML 2021.
© 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark ThankYou

More Related Content

PDF
Explainability and bias in AI
Bill Liu
 
PDF
Responsible AI
Neo4j
 
PPTX
Responsible AI in Industry (ICML 2021 Tutorial)
Krishnaram Kenthapadi
 
PPTX
Explainable AI in Industry (AAAI 2020 Tutorial)
Krishnaram Kenthapadi
 
PPTX
Responsible AI in Industry (Tutorials at AAAI 2021, FAccT 2021, and WWW 2021)
Krishnaram Kenthapadi
 
PPTX
Explainable AI in Industry (KDD 2019 Tutorial)
Krishnaram Kenthapadi
 
PDF
Responsible AI
Data Con LA
 
PDF
How Does Generative AI Actually Work? (a quick semi-technical introduction to...
ssuser4edc93
 
Explainability and bias in AI
Bill Liu
 
Responsible AI
Neo4j
 
Responsible AI in Industry (ICML 2021 Tutorial)
Krishnaram Kenthapadi
 
Explainable AI in Industry (AAAI 2020 Tutorial)
Krishnaram Kenthapadi
 
Responsible AI in Industry (Tutorials at AAAI 2021, FAccT 2021, and WWW 2021)
Krishnaram Kenthapadi
 
Explainable AI in Industry (KDD 2019 Tutorial)
Krishnaram Kenthapadi
 
Responsible AI
Data Con LA
 
How Does Generative AI Actually Work? (a quick semi-technical introduction to...
ssuser4edc93
 

What's hot (20)

PPTX
Generative models
Birger Moell
 
PPTX
Generative AI Use cases for Enterprise - Second Session
Gene Leybzon
 
PDF
Explainable AI (XAI)
Manojkumar Parmar
 
PDF
The current state of generative AI
Benjaminlapid1
 
PDF
Unlocking the Power of Generative AI An Executive's Guide.pdf
PremNaraindas1
 
PDF
Using the power of Generative AI at scale
Maxim Salnikov
 
PDF
Microsoft Build 2023 Updates – Copilot Stack and Azure OpenAI Service (Machin...
Naoki (Neo) SATO
 
PDF
GENERATIVE AI, THE FUTURE OF PRODUCTIVITY
Andre Muscat
 
PDF
Generative AI
lutzsuarnaba1
 
PDF
𝐆𝐞𝐧𝐞𝐫𝐚𝐭𝐢𝐯𝐞 𝐀𝐈: 𝐂𝐡𝐚𝐧𝐠𝐢𝐧𝐠 𝐇𝐨𝐰 𝐁𝐮𝐬𝐢𝐧𝐞𝐬𝐬 𝐈𝐧𝐧𝐨𝐯𝐚𝐭𝐞𝐬 𝐚𝐧𝐝 𝐎𝐩𝐞𝐫𝐚𝐭𝐞𝐬
VINCI Digital - Industrial IoT (IIoT) Strategic Advisory
 
PDF
Bias in AI-systems: A multi-step approach
Eirini Ntoutsi
 
PPTX
Ethics of Analytics and Machine Learning
Mark Underwood
 
PDF
Automated Machine Learning
Yuriy Guts
 
PPTX
Explainable AI in Industry (WWW 2020 Tutorial)
Krishnaram Kenthapadi
 
PDF
Explainable AI
Dinesh V
 
PPTX
Implementing Ethics in AI
Pekka Abrahamsson / Tampere University
 
PDF
Fairness and Bias in Machine Learning
Surya Dutta
 
PDF
ChatGPT OpenAI Primer for Business
Dion Hinchcliffe
 
PPTX
Explainable AI in Industry (FAT* 2020 Tutorial)
Krishnaram Kenthapadi
 
PPTX
An Introduction to XAI! Towards Trusting Your ML Models!
Mansour Saffar
 
Generative models
Birger Moell
 
Generative AI Use cases for Enterprise - Second Session
Gene Leybzon
 
Explainable AI (XAI)
Manojkumar Parmar
 
The current state of generative AI
Benjaminlapid1
 
Unlocking the Power of Generative AI An Executive's Guide.pdf
PremNaraindas1
 
Using the power of Generative AI at scale
Maxim Salnikov
 
Microsoft Build 2023 Updates – Copilot Stack and Azure OpenAI Service (Machin...
Naoki (Neo) SATO
 
GENERATIVE AI, THE FUTURE OF PRODUCTIVITY
Andre Muscat
 
Generative AI
lutzsuarnaba1
 
𝐆𝐞𝐧𝐞𝐫𝐚𝐭𝐢𝐯𝐞 𝐀𝐈: 𝐂𝐡𝐚𝐧𝐠𝐢𝐧𝐠 𝐇𝐨𝐰 𝐁𝐮𝐬𝐢𝐧𝐞𝐬𝐬 𝐈𝐧𝐧𝐨𝐯𝐚𝐭𝐞𝐬 𝐚𝐧𝐝 𝐎𝐩𝐞𝐫𝐚𝐭𝐞𝐬
VINCI Digital - Industrial IoT (IIoT) Strategic Advisory
 
Bias in AI-systems: A multi-step approach
Eirini Ntoutsi
 
Ethics of Analytics and Machine Learning
Mark Underwood
 
Automated Machine Learning
Yuriy Guts
 
Explainable AI in Industry (WWW 2020 Tutorial)
Krishnaram Kenthapadi
 
Explainable AI
Dinesh V
 
Implementing Ethics in AI
Pekka Abrahamsson / Tampere University
 
Fairness and Bias in Machine Learning
Surya Dutta
 
ChatGPT OpenAI Primer for Business
Dion Hinchcliffe
 
Explainable AI in Industry (FAT* 2020 Tutorial)
Krishnaram Kenthapadi
 
An Introduction to XAI! Towards Trusting Your ML Models!
Mansour Saffar
 
Ad

Similar to Responsible AI in Industry: Practical Challenges and Lessons Learned (20)

PDF
C3 w5
Ajay Taneja
 
PPTX
algorithmic-decisions, fairness, machine learning, provenance, transparency
Paolo Missier
 
PDF
Model Evaluation in the land of Deep Learning
Pramit Choudhary
 
PDF
Explainable AI - making ML and DL models more interpretable
Aditya Bhattacharya
 
PDF
Machine Learning Model Validation (Aijun Zhang 2024).pdf
Aijun Zhang
 
PDF
Keepler Data Tech | Entendiendo tus propios modelos predictivos
Keepler Data Tech
 
PDF
ExplainingMLModels.pdf
LHong526661
 
PPTX
Towards Responsible AI - NY.pptx
Luis775803
 
PDF
Can Machine Learning Models be Trusted? Explaining Decisions of ML Models
Darek Smyk
 
PDF
Practical Explainable AI: How to build trustworthy, transparent and unbiased ...
Raheel Ahmad
 
PDF
DSDT meetup July 2021
DSDT_MTL
 
PDF
Deep Credit Risk Ranking with LSTM with Kyle Grove
Databricks
 
PDF
Quant university MRM and machine learning
QuantUniversity
 
PDF
Reds interpretability report
Raouf KESKES
 
PDF
Towards Human-Centered Machine Learning
Sri Ambati
 
PPTX
Machine Learning Interpretability - Mateusz Dymczyk - H2O AI World London 2018
Sri Ambati
 
PPTX
Get hands-on with Explainable AI at Machine Learning Interpretability(MLI) Gym!
Sri Ambati
 
PPTX
Transparency in ML and AI (humble views from a concerned academic)
Paolo Missier
 
PPTX
Deciphering AI - Unlocking the Black Box of AIML with State-of-the-Art Techno...
Analytics India Magazine
 
PDF
Interpretable Machine Learning
inovex GmbH
 
C3 w5
Ajay Taneja
 
algorithmic-decisions, fairness, machine learning, provenance, transparency
Paolo Missier
 
Model Evaluation in the land of Deep Learning
Pramit Choudhary
 
Explainable AI - making ML and DL models more interpretable
Aditya Bhattacharya
 
Machine Learning Model Validation (Aijun Zhang 2024).pdf
Aijun Zhang
 
Keepler Data Tech | Entendiendo tus propios modelos predictivos
Keepler Data Tech
 
ExplainingMLModels.pdf
LHong526661
 
Towards Responsible AI - NY.pptx
Luis775803
 
Can Machine Learning Models be Trusted? Explaining Decisions of ML Models
Darek Smyk
 
Practical Explainable AI: How to build trustworthy, transparent and unbiased ...
Raheel Ahmad
 
DSDT meetup July 2021
DSDT_MTL
 
Deep Credit Risk Ranking with LSTM with Kyle Grove
Databricks
 
Quant university MRM and machine learning
QuantUniversity
 
Reds interpretability report
Raouf KESKES
 
Towards Human-Centered Machine Learning
Sri Ambati
 
Machine Learning Interpretability - Mateusz Dymczyk - H2O AI World London 2018
Sri Ambati
 
Get hands-on with Explainable AI at Machine Learning Interpretability(MLI) Gym!
Sri Ambati
 
Transparency in ML and AI (humble views from a concerned academic)
Paolo Missier
 
Deciphering AI - Unlocking the Black Box of AIML with State-of-the-Art Techno...
Analytics India Magazine
 
Interpretable Machine Learning
inovex GmbH
 
Ad

More from Krishnaram Kenthapadi (13)

PPTX
Responsible AI in Industry: Practical Challenges and Lessons Learned
Krishnaram Kenthapadi
 
PPTX
Amazon SageMaker Clarify
Krishnaram Kenthapadi
 
PPTX
Privacy in AI/ML Systems: Practical Challenges and Lessons Learned
Krishnaram Kenthapadi
 
PPTX
Fairness and Privacy in AI/ML Systems
Krishnaram Kenthapadi
 
PPTX
Fairness and Privacy in AI/ML Systems
Krishnaram Kenthapadi
 
PDF
Fairness-aware Machine Learning: Practical Challenges and Lessons Learned (KD...
Krishnaram Kenthapadi
 
PPTX
Fairness-aware Machine Learning: Practical Challenges and Lessons Learned (WW...
Krishnaram Kenthapadi
 
PPTX
Privacy-preserving Data Mining in Industry (WWW 2019 Tutorial)
Krishnaram Kenthapadi
 
PPTX
Fairness-aware Machine Learning: Practical Challenges and Lessons Learned (WS...
Krishnaram Kenthapadi
 
PPTX
Privacy-preserving Data Mining in Industry (WSDM 2019 Tutorial)
Krishnaram Kenthapadi
 
PPTX
Fairness, Transparency, and Privacy in AI @ LinkedIn
Krishnaram Kenthapadi
 
PPTX
Privacy-preserving Analytics and Data Mining at LinkedIn
Krishnaram Kenthapadi
 
PPTX
Privacy-preserving Data Mining in Industry: Practical Challenges and Lessons ...
Krishnaram Kenthapadi
 
Responsible AI in Industry: Practical Challenges and Lessons Learned
Krishnaram Kenthapadi
 
Amazon SageMaker Clarify
Krishnaram Kenthapadi
 
Privacy in AI/ML Systems: Practical Challenges and Lessons Learned
Krishnaram Kenthapadi
 
Fairness and Privacy in AI/ML Systems
Krishnaram Kenthapadi
 
Fairness and Privacy in AI/ML Systems
Krishnaram Kenthapadi
 
Fairness-aware Machine Learning: Practical Challenges and Lessons Learned (KD...
Krishnaram Kenthapadi
 
Fairness-aware Machine Learning: Practical Challenges and Lessons Learned (WW...
Krishnaram Kenthapadi
 
Privacy-preserving Data Mining in Industry (WWW 2019 Tutorial)
Krishnaram Kenthapadi
 
Fairness-aware Machine Learning: Practical Challenges and Lessons Learned (WS...
Krishnaram Kenthapadi
 
Privacy-preserving Data Mining in Industry (WSDM 2019 Tutorial)
Krishnaram Kenthapadi
 
Fairness, Transparency, and Privacy in AI @ LinkedIn
Krishnaram Kenthapadi
 
Privacy-preserving Analytics and Data Mining at LinkedIn
Krishnaram Kenthapadi
 
Privacy-preserving Data Mining in Industry: Practical Challenges and Lessons ...
Krishnaram Kenthapadi
 

Recently uploaded (20)

PPTX
June-4-Sermon-Powerpoint.pptx USE THIS FOR YOUR MOTIVATION
KuyaRogie
 
PPTX
Internet___Basics___Styled_ presentation
poonam62133
 
PDF
Triggering QUIC, presented by Geoff Huston at IETF 123
APNIC
 
PDF
Vigrab.top – Online Tool for Downloading and Converting Social Media Videos a...
Vigrab Top
 
PPTX
international classification of diseases ICD-10 review PPT.pptx
naveenithkrishnan
 
PDF
The New Creative Director: How AI Tools for Social Media Content Creation Are...
SageTitans
 
PPTX
Funds Management Learning Material for Beg
NKKumar16
 
PPTX
introduction about ICD -10 & ICD-11 ppt.pptx
naveenithkrishnan
 
PPTX
presentation_pfe-universite-molay-seltan.pptx
yahyamohibme
 
PDF
Decoding a Decade: 10 Years of Applied CTI Discipline
Andreas Sfakianakis
 
PDF
APNIC Update, presented at PHNOG 2025 by Shane Hermoso
APNIC
 
PPTX
Introuction about WHO-FIC in ICD-10.pptx
naveenithkrishnan
 
PPTX
Introduction about ICD -10 and ICD11 on 5.8.25.pptx
naveenithkrishnan
 
PPTX
Digital Literacy And Online Safety on internet
riksa rifqi
 
PDF
💰 𝐔𝐊𝐓𝐈 𝐊𝐄𝐌𝐄𝐍𝐀𝐍𝐆𝐀𝐍 𝐊𝐈𝐏𝐄𝐑𝟒𝐃 𝐇𝐀𝐑𝐈 𝐈𝐍𝐈 𝟐𝟎𝟐𝟓 💰
GRAB
 
DOCX
Unit-3 cyber security network security of internet system
shivangisharma636228
 
PDF
RPKI Status Update, presented by Makito Lay at IDNOG 10
APNIC
 
PPT
tcp ip networks nd ip layering assotred slides
pakadamfdp
 
PPTX
Introuction about ICD -10 and ICD-11 PPT.pptx
naveenithkrishnan
 
PDF
Sims 4 Historia para lo sims 4 para jugar
lolpopy34
 
June-4-Sermon-Powerpoint.pptx USE THIS FOR YOUR MOTIVATION
KuyaRogie
 
Internet___Basics___Styled_ presentation
poonam62133
 
Triggering QUIC, presented by Geoff Huston at IETF 123
APNIC
 
Vigrab.top – Online Tool for Downloading and Converting Social Media Videos a...
Vigrab Top
 
international classification of diseases ICD-10 review PPT.pptx
naveenithkrishnan
 
The New Creative Director: How AI Tools for Social Media Content Creation Are...
SageTitans
 
Funds Management Learning Material for Beg
NKKumar16
 
introduction about ICD -10 & ICD-11 ppt.pptx
naveenithkrishnan
 
presentation_pfe-universite-molay-seltan.pptx
yahyamohibme
 
Decoding a Decade: 10 Years of Applied CTI Discipline
Andreas Sfakianakis
 
APNIC Update, presented at PHNOG 2025 by Shane Hermoso
APNIC
 
Introuction about WHO-FIC in ICD-10.pptx
naveenithkrishnan
 
Introduction about ICD -10 and ICD11 on 5.8.25.pptx
naveenithkrishnan
 
Digital Literacy And Online Safety on internet
riksa rifqi
 
💰 𝐔𝐊𝐓𝐈 𝐊𝐄𝐌𝐄𝐍𝐀𝐍𝐆𝐀𝐍 𝐊𝐈𝐏𝐄𝐑𝟒𝐃 𝐇𝐀𝐑𝐈 𝐈𝐍𝐈 𝟐𝟎𝟐𝟓 💰
GRAB
 
Unit-3 cyber security network security of internet system
shivangisharma636228
 
RPKI Status Update, presented by Makito Lay at IDNOG 10
APNIC
 
tcp ip networks nd ip layering assotred slides
pakadamfdp
 
Introuction about ICD -10 and ICD-11 PPT.pptx
naveenithkrishnan
 
Sims 4 Historia para lo sims 4 para jugar
lolpopy34
 

Responsible AI in Industry: Practical Challenges and Lessons Learned

  • 1. © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Krishnaram Kenthapadi Principal Scientist, Amazon AWS AI Responsible AI in Industry: Practical Challenges and Lessons Learned Invited talks @ • Responsible Use of Data series, The Academic Fringe Festival, Delft University of Technology, Jul’2021 • Flipkart Data Science Conference, Jun’2021 • Syracuse University, May’2021 • The University of Waterloo ML+Logic Colloquium, Apr’2021 • The University of British Columbia, Vancouver, Mar’2021
  • 2. Massachusetts Group Insurance Commission (1997): Anonymized medical history of state employees William Weld vs Latanya Sweeney Latanya Sweeney (MIT grad student): $20 – Cambridge voter roll born July 31, 1945 resident of 02138
  • 3. 64% Uniquely identifiable with ZIP + birth date + gender (in the US population) Golle, “Revisiting the Uniqueness of Simple Demographics in the US Population”, WPES 2006
  • 4. A History of Privacy Failures … Credit: Kobbi Nissim,Or Sheffet
  • 7. • Ethical challenges posed by AI systems • Inherent biases present in society • Reflected in training data • AI/ML models prone to amplifying such biases Algorithmic Bias
  • 8. Laws against Discrimination Immigration Reform and Control Act Citizenship Rehabilitation Act of 1973; Americans with Disabilities Act of 1990 Disability status Civil Rights Act of 1964 Race Age Discrimination in Employment Act of 1967 Age Equal Pay Act of 1963; Civil Rights Act of 1964 Sex And more...
  • 10. Motivation & Business Opportunities Regulatory. We need to understand why the ML model made a given prediction and also whether the prediction it made was free from bias, both in training and at inference. Business. Providing explanations to internal teams (loan officers, customer service rep, compliance teams) and end users/customers Data Science. Improving models through better feature engineering and training data generation, understanding failure modes of the model, debugging model predictions, etc.
  • 11. © 2020, Amazon Web Services, Inc. or its Affiliates. Amazon SageMaker VISION SPEECH TEXT SEARCH CHATBOTS PERSONALIZATION FORECASTING FRAUD CONTACT CENTERS Deep Learning AMIs & Containers GPUs & CPUs Elastic Inference Trainium Inferentia FPGA AI SERVICES ML SERVICES FRAMEWORKS & INFRASTRUCTURE DeepGraphLibrary Amazon Rekognition Amazon Polly Amazon Transcribe +Medical Amazon Lex Amazon Personalize Amazon Forecast Amazon Comprehend +Medical Amazon Textract Amazon Kendra Amazon CodeGuru Amazon Fraud Detector Amazon Translate INDUSTRIAL AI CODE AND DEVOPS NEW Amazon DevOps Guru Voice ID For AmazonConnect Contact Lens NEW Amazon Monitron NEW AWS Panorama + Appliance NEW Amazon Lookout for Vision NEW Amazon Lookout for Equipment Scaling Fairness, Explainability & Privacy across the AWS ML Stack NEW Amazon HealthLake HEALTHCARE AI NEW Amazon Lookout for Metrics ANOMALY DETECTION Amazon Transcribe for Medical Amazon Comprehend for Medical Label data NEW Aggregate & prepare data NEW Store & share features Auto ML Spark/R NEW Detect bias Visualize in notebooks Pick algorithm Train models Tune parameters NEW Debug & profile Deploy in production Manage & monitor NEW CI/CD Human review review NEW: Model management for edge devices NEW: SageMaker JumpStart SAGEMAKER STUDIO IDE
  • 12. LinkedIn operates the largest professional network on the Internet Tell your story 740M members 55M+ companies are represented on LinkedIn 90K schools listed (high school & college) 36K skills listed 14M+ open jobs on LinkedIn Jobs 280B Feed updates
  • 13. Talk Outline* • Fairness-aware ML: An overview • Explainable AI: An overview • Privacy-preserving ML: An overview • Responsible AI tools • Case studies • Key takeaways * For a longer version, please see our tutorial titled, “Responsible AI in Industry”: https://sites.google.com/view/ResponsibleAITutorial
  • 14. Explainable AI: An Overview
  • 15. What is Explainable AI? Data Opaque AI AI product Confusion with Today’s Opaque AI ● Why did you do that? ● Why did you not do that? ● When do you succeed or fail? ● How do I correct an error? Opaque AI Prediction, Recommendation Clear & Transparent Predictions ● I understand why ● I understand why not ● I know why you succeed or fail ● I understand, so I trust you Explainable AI Data Explainable AI Explainable AI Product Prediction Explanation Feedback
  • 16. - Humans may have follow-up questions - Explanations cannot answer all users’ concerns Weld, D., and Gagan Bansal. "The challenge of crafting intelligible intelligence." Communications of the ACM (2018). Example of an End-to-End XAI System
  • 17. Neural Net CNN GAN RNN Ensemble Method Random Forest XGB Statistical Model AOG SVM Graphical Model Bayesian Belief Net SLR CRF HBN MLN Markov Model Decision Tree Linear Model Non-Linear functions Polynomial functions Quasi-Linear functions Accuracy Explainability Interpretability Learning • Challenges: • Supervised • Unsupervised learning • Approach: • Representation Learning • Stochastic selection • Output: • Correlation • No causation How to Explain? Accuracy vs. Explainability
  • 19. Explainable AI Methods Approach 1: Post-hoc explain a given AI model ● Individual prediction explanations in terms of input features, influential examples, concepts, local decision rules ● Global prediction explanations in terms of entire model in terms of partial dependence plots, global feature importance, global decision rules Approach 2: Build an interpretable model ● Logistic regression, Decision trees, Decision lists and sets, Generalized Additive Models (GAMs) 19
  • 20. Credit: Slide based on https://twitter.com/chandan_singh96/status/1138811752769101825 Integrated Gradients Opaque, complex model
  • 21. Explaining Model Predictions: Credit Lending 21 Fair lending laws [ECOA, FCRA] require credit decisions to be explainable
  • 22. Feature Attribution Problem Attribute a model’s prediction on an input to features of the input Examples ● Attribute a lending model’s prediction to its features ● Attribute an object recognition model’s prediction to its pixels ● Attribute a text sentiment model’s prediction to individual words Several attribution methods: ● Ablations ● Gradient based methods (specific to differentiable models) ● Score back-propagation based methods (specific to neural networks) ● Game theory based methods (Shapley values) 22
  • 23. Application of Attributions ● Debugging model predictions E.g., Attribution an image misclassification to the pixels responsible for it ● Generating an explanation for the end-user E.g., Expose attributions for a lending prediction to the end-user ● Analyzing model robustness E.g., Craft adversarial examples using weaknesses surfaced by attributions ● Extract rules from the model E.g., Combine attribution to craft rules (pharmacophores) capturing prediction logic of a drug screening network 23
  • 24. Next few slides We will cover the following attribution methods** ● Ablations ● Gradient based methods (specific to differentiable models) ● Score Backpropagation based methods (specific to NNs) We will also discuss game theory (Shapley value) in attributions **Not a complete list! See Ancona et al. [ICML 2019], Guidotti et al. [arxiv 2018] for a comprehensive survey 24
  • 25. Ablations Drop each feature and attribute the change in prediction to that feature Pros: ● Simple and intuitive to interpret Cons: ● Unrealistic inputs ● Improper accounting of interactive features ● Can be computationally expensive 25
  • 26. Feature*Gradient Attribution to a feature is feature value times gradient, i.e., xi* 𝜕y/𝜕xi ● Gradient captures sensitivity of output w.r.t. feature ● Equivalent to Feature*Coefficient for linear models ○ First-order Taylor approximation of non-linear models ● Popularized by SaliencyMaps [NeurIPS 2013], Baehrens et al. [JMLR 2010] 26 Gradients in the vicinity of the input seem like noise?
  • 27. Local linear approximations can be too local 27 score “fireboat-ness” of image Interesting gradients uninteresting gradients (saturation) 1.0 0.0
  • 28. Score Back-Propagation based Methods Re-distribute the prediction score through the neurons in the network ● LRP [JMLR 2017], DeepLift [ICML 2017], Guided BackProp [ICLR 2014] Easy case: Output of a neuron is a linear function of previous neurons (i.e., ni = ⅀ wij * nj) e.g., the logit neuron ● Re-distribute the contribution in proportion to the coefficients wij 28 Image credit heatmapping.org
  • 29. Score Back-Propagation based Methods Re-distribute the prediction score through the neurons in the network ● LRP [JMLR 2017], DeepLift [ICML 2017], Guided BackProp [ICLR 2014] Tricky case: Output of a neuron is a non-linear function, e.g., ReLU, Sigmoid, etc. ● Guided BackProp: Only consider ReLUs that are on (linear regime), and which contribute positively ● LRP: Use first-order Taylor decomposition to linearize activation function ● DeepLift: Distribute activation difference relative a reference point in proportion to edge weights 29 Image credit heatmapping.org
  • 30. Score Back-Propagation based Methods Re-distribute the prediction score through the neurons in the network ● LRP [JMLR 2017], DeepLift [ICML 2017], Guided BackProp [ICLR 2014] Pros: ● Conceptually simple ● Methods have been empirically validated to yield sensible result Cons: ● Hard to implement, requires instrumenting the model ● Often breaks implementation invariance Think: F(x, y, z) = x * y *z and G(x, y, z) = x * (y * z) Image credit heatmapping.org
  • 31. Baselines and additivity ● When we decompose the score via backpropagation, we imply a normative alternative called a baseline ○ “Why Pr(fireboat) = 0.91 [instead of 0.00]” ● Common choice is an informationless input for the model ○ E.g., All black or all white image for image models ○ E.g., Empty text or zero embedding vector for text models ● Additive attributions explain F(input) - F(baseline) in terms of input features
  • 32. score intensity Interesting gradients uninteresting gradients (saturation) 1.0 0.0 Baseline … scaled inputs ... … gradients of scaled inputs …. Input Another approach: gradients at many points
  • 33. IG(input, base) ::= (input - base) * ∫0 -1▽F(𝛂*input + (1-𝛂)*base) d𝛂 Original image Integrated Gradients Integrated Gradients [ICML 2017] Integrate the gradients along a straight-line path from baseline to input
  • 34. 34 © 2020 Amazon Web Services, Inc. or its affiliates. All rights reserved | Feature Attributions using Shapley Values Coalition Game ● Players collaborating to generate some payoff ● Shapley value: Fair way to quantify each player’s contribution to the game’s payoff 34
  • 35. 35 © 2020 Amazon Web Services, Inc. or its affiliates. All rights reserved | Shapley Value: Intuition 35
  • 36. 36 © 2020 Amazon Web Services, Inc. or its affiliates. All rights reserved | Shapley Value [Annals of Mathematical studies,1953] = Average over all player orderings & subsets SC M / {i} Marginal contribution Coalition contribution with and without this player f – payoff of the game M – all players S – subset of players i – specific player Fair way to quantify each player’s contribution to the game’s payoff
  • 37. 37 © 2020 Amazon Web Services, Inc. or its affiliates. All rights reserved | Shapley Value Justification Shapley values are unique under four simple axioms ● Dummy: If a player never contributes to the game then it must receive zero attribution ● Efficiency: Attributions must add to the total payoff ● Symmetry: Symmetric players must receive equal attribution ● Linearity: Attribution for the (weighted) sum of two games must be the same as the (weighted) sum of the attributions for each of the games 37
  • 38. 38 © 2020 Amazon Web Services, Inc. or its affiliates. All rights reserved | Explaining Model Predictions using Shapley Values f Game payoff Game theory All players Subset of players Specific player Instance being explained Model prediction All features Subset of features Specific feature Instance being explained M S i x Machine learning https://github.com/slundberg/shap Lundberg et al: NeurIPS (2017)
  • 39. 39 © 2020 Amazon Web Services, Inc. or its affiliates. All rights reserved | Explaining Model Predictions: SHAP Algorithm How it works = Average over all feature orderings & subsets SC M / {i} f – model prediction M – all features S – subset of features i – specific feature x – instance being explained https://github.com/slundberg/shap Lundberg et al: NeurIPS (2017)
  • 40. 40 © 2020 Amazon Web Services, Inc. or its affiliates. All rights reserved | Explaining Model Predictions: SHAP Algorithm Different solutions for runtime and algorithmic challenges = Average over all feature orderings & subsets SC M / {i} f – model prediction M – all features S – subset of features i – specific feature x – instance being explained https://github.com/slundberg/shap Lundberg et al: NeurIPS (2017) Exponential time How to run a model with missing features?
  • 41. 41 © 2020 Amazon Web Services, Inc. or its affiliates. All rights reserved | Explaining Model Predictions: SHAP Algorithm Different solutions for runtime and algorithmic challenges = Average over all feature orderings & subsets SC M / {i} f – model prediction M – all features S – subset of features i – specific feature x – instance being explained https://github.com/slundberg/shap Lundberg et al: NeurIPS (2017) Exponential time How to run a model with missing features? Shap.KernelExplainer • All models • Explanations Sample subsets, defaults to 2|M| + 2048 Fill missing values from a background dataset • Training set • Median of the dataset • K-means representatives • All-black image Kernel Open-source SHAP implementation
  • 42. Baselines (or Norms) are essential to explanations [Kahneman-Miller 86] ● E.g., A man suffers from indigestion. Doctor blames it to a stomach ulcer. Wife blames it on eating turnips. Both are correct relative to their baselines. ● The baseline may also be an important analysis knob. Attributions are contrastive, whether we think about it or not. Lesson learned: Baselines are important
  • 43. Some limitations and caveats for attributions
  • 44. Some things that are missing: ● Feature interactions (ignored or averaged out) ● What training examples influenced the prediction (training agnostic) ● Global properties of the model (prediction-specific) An instance where attributions are useless: ● A model that predicts TRUE when there are even number of black pixels and FALSE otherwise Attributions don’t explain everything
  • 45. Attributions are for human consumption Naive scaling of attributions from 0 to 255 Attributions have a large range and long tail across pixels After clipping attributions at 99% to reduce range ● Humans interpret attributions and generate insights ○ Doctor maps attributions for x-rays to pathologies ● Visualization matters as much as the attribution technique
  • 47. What is Privacy? • Right of/to privacy • “Right to be let alone” [L. Brandeis & S. Warren, 1890] • “No one shall be subjected to arbitrary interference with [their] privacy, family, home or correspondence, nor to attacks upon [their] honor and reputation.” [The United Nations Universal Declaration of Human Rights] • “The right of a person to be free from intrusion into or publicity concerning matters of a personal nature” [Merriam-Webster] • “The right not to have one's personal matters disclosed or publicized; the right to be left alone” [Nolo’s Plain-English Law Dictionary]
  • 48. Data Privacy (or Information Privacy) • “The right to have some control over how your personal information is collected and used” [IAPP] • “Privacy has fast-emerged as perhaps the most significant consumer protection issue—if not citizen protection issue—in the global information economy” [IAPP]
  • 49. Data Privacy vs. Security • Data privacy: use & governance of personal data • Data security: protecting data from malicious attacks & the exploitation of stolen data for profit • Security is necessary, but not sufficient for addressing privacy.
  • 50. Data Privacy:Technical Problem Given a dataset with sensitive personal information, how can we compute and release functions of the dataset while protecting individual privacy? Credit: Kobbi Nissim
  • 51. A History of Privacy Failures … Credit: Kobbi Nissim,Or Sheffet
  • 52. Lessons Learned … • Attacker’s advantage: Auxiliary information; high dimensionality; enough to succeed on a small fraction of inputs; active; observant … • Unanticipated privacy failures from new attack methods • Need for rigorous privacy notions & techniques
  • 54. © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Threat Models
  • 55. © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Threat Models
  • 56. Threat Models User Access Only • Users store their data • Noisy data or analytics transmitted Trusted Curator • Stored by organization • Managed only by a trusted curator/admin • Access only to noisy analytics or synthetic data External Threat • Stored by organization • Organization has access • Only privacy enabled models deployed
  • 60. Differential Privacy 60 Databases D and D′ are neighbors if they differ in one person’s data. Differential Privacy: The distribution of the curator’s output M(D) on database D is (nearly) the same as M(D′). Curator + your data - your data Dwork, McSherry, Nissim, Smith [TCC 2006] Curator
  • 61. (ε, 𝛿)-Differential Privacy: The distribution of the curator’s output M(D) on database D is (nearly) the same as M(D′). Differential Privacy 61 Curator Parameter ε quantifies information leakage ∀S: Pr[M(D)∊S] ≤ exp(ε) ∙ Pr[M(D′)∊S]＋𝛿. Curator Parameter 𝛿 gives some slack Dwork, Kenthapadi, McSherry, Mironov, Naor [EUROCRYPT 2006] + your data - your data Dwork, McSherry, Nissim, Smith [TCC 2006]
  • 62. Differential Privacy: Random Noise Addition If ℓ1-sensitivity of f : D → ℝn: maxD,D′ ||f(D) − f(D′)||1 = s, then adding Laplacian noise to true output f(D) + Laplacen(s/ε) offers (ε,0)-differential privacy. Dwork, McSherry, Nissim, Smith [TCC 2006]
  • 63. Examples of Attacks • Membership inference attacks • Reconstruction attacks • Model inversion attacks • Different attacks correspond to different threat models
  • 64. Privacy-preserving MLTechniques /Tools • Differentially private model training • Differentially private stochastic gradient descent • PATE • Differentially private synthetic data generation • Federated learning • Trusted execution environments (secure enclaves) • Secure multiparty computation • Homomorphic encryption • Different techniques applicable under different threat models • May need a hybrid approach
  • 66. Fairness in AI @ LinkedIn Fairness-aware Talent Search Ranking* *Work done while at LinkedIn
  • 68. Insights to Identify Diverse Talent Pools Representative Talent Search Results Diversity Learning Curriculum “Diversity by Design” in LinkedIn’s Talent Solutions
  • 70. Representative Ranking for Talent Search S. C. Geyik, S. Ambler, K. Kenthapadi, Fairness- Aware Ranking in Search & Recommendation Systems with Application to LinkedIn Talent Search, KDD’19. [Microsoft’s AI/ML conference (MLADS’18). Distinguished Contribution Award] Building Representative Talent Search at LinkedIn (LinkedIn engineering blog)
  • 71. Intuition for Measuring and Achieving Representativeness Ideal: Top ranked results should follow a desired distribution on gender/age/… E.g., same distribution as the underlying talent pool Inspired by “Equal Opportunity” definition [Hardt et al, NIPS’16] Defined measures (skew, divergence) based on this intuition
  • 72. Measuring (Lack of) Representativeness Skew@k (Logarithmic) ratio of the proportion of candidates having a given attribute value among the top k ranked results to the corresponding desired proportion Variants: MinSkew: Minimum over all attribute values MaxSkew: Maximum over all attribute values Normalized Discounted Cumulative Skew Normalized Discounted Cumulative KL-divergence
  • 73. Fairness-aware Reranking Algorithm (Simplified) Partition the set of potential candidates into different buckets for each attribute value Rank the candidates in each bucket according to the scores assigned by the machine-learned model Merge the ranked lists, balancing the representation requirements and the selection of highest scored candidates Representation requirement: Desired distribution on gender/age/… Algorithmic variants based on how we choose the next attribute
  • 75. Validating Our Approach Gender Representativeness Over 95% of all searches are representative compared to the qualified population of the search Business Metrics A/B test over LinkedIn Recruiter users for two weeks No significant change in business metrics (e.g., # InMails sent or accepted) Ramped to 100% of LinkedIn Recruiter users worldwide
  • 76. Lessons learned • Post-processing approach desirable • Model agnostic • Scalable across different model choices for our application • Acts as a “fail-safe” • Robust to application-specific business logic • Easier to incorporate as part of existing systems • Build a stand-alone service or component for post-processing • No significant modifications to the existing components • Complementary to efforts to reduce bias from training data & during model training • Collaboration/consensus across key
  • 77. Lessons learned • Post-processing approach desirable • Model agnostic • Scalable across different model choices for our application • Acts as a “fail-safe” • Robust to application-specific business logic • Easier to incorporate as part of existing systems • Build a stand-alone service or component for post-processing • No significant modifications to the existing components • Complementary to efforts to reduce bias from training data & during model training • Collaboration/consensus across key
  • 78. Evaluating Fairness Using Permutations Tests [DiCiccio, Vasudevan, Basu, Kenthapadi, Agarwal, KDD’20] • Is the measured discrepancy across different groups statistically significant? • Use statistical hypothesis tests! • Can we perform hypothesis tests in a metric-agnostic manner? • Non-parametric tests can help! • Permutation testing framework
  • 79. Brief Review of Permutation Tests Observe data from two populations: and Are the populations the same? A reasonable test statistic might be
  • 80. Brief Review of Permutation Tests (Continued) A p-value is the chance of observing a test statistic at least as “extreme” as the value we actually observed Permutation test approach: ●Randomly shuffle the population designations of the observations ●Recompute the test statistic T ●Repeat many times Permutation p-value: the proportion of permuted datasets resulting in a larger test statistic than the original value This test is exact!
  • 81. A Fairness Example Consider testing whether the true positive rate of a classifier is equal between two groups Test Statistic: difference in proportion of negative labeled observations that are classified as positive between the two groups Permutation test: Randomly reshuffle group labels, recompute test statistic
  • 82. Permutations Tests for Evaluating Fairness in ML Models • Issues with classical permutation test • Want to check: just equality of the fairness metric (e.g., false positive rate) across groups, and not if the two groups have identical distribution • Exact for the strong null hypothesis … • … but may not be valid (even asymptotically) for the weak null hypothesis • Our paper: A fix for this issue • Choose a pivotal statistic (asymptotically distribution-free; does not depend on the observed data’s distribution) • E.g., Studentize the test statistic
  • 83. Permutations Tests for Evaluating Fairness in ML Models • Issues with classical permutation test • Want to check: just equality of the fairness metric (e.g., false positive rate) across groups, and not if the two groups have identical distribution • Exact for the strong null hypothesis … • … but may not be valid (even asymptotically) for the weak null hypothesis • Our paper: A fix for this issue • Choose a pivotal statistic (asymptotically distribution-free; does not depend on the observed data’s distribution) • E.g., Studentize the test statistic
  • 84. Engineering for Fairness in AI Lifecycle S.Vasudevan, K. Kenthapadi, LiFT: A Scalable Framework for Measuring Fairness in ML Applications, CIKM’20 https://github.com/linkedin/LiFT
  • 85. LiFT System Architecture [Vasudevan & Kenthapadi, CIKM’20] •Flexibility of Use (Platform agnostic) •Ad-hoc exploratory analyses •Deployment in offline workflows •Integration with ML Frameworks •Scalability •Diverse fairness metrics •Conventional fairness metrics •Benefit metrics •Statistical tests
  • 86. LiFT System Architecture [Vasudevan & Kenthapadi, CIKM’20] •Flexibility of Use (Platform agnostic) •Ad-hoc exploratory analyses •Deployment in offline workflows •Integration with ML Frameworks •Scalability •Diverse fairness metrics •Conventional fairness metrics •Benefit metrics •Statistical tests
  • 87. Acknowledgements LinkedIn Talent Solutions Diversity team, Hire & Careers AI team, Anti-abuse AI team, Data Science Applied Research team Special thanks to Deepak Agarwal, Parvez Ahammad, Stuart Ambler, Kinjal Basu, Jenelle Bray, Erik Buchanan, Bee-Chung Chen, Fei Chen, Patrick Cheung, Gil Cottle, Cyrus DiCiccio, Patrick Driscoll, Carlos Faham, Nadia Fawaz, Priyanka Gariba, Meg Garlinghouse, Sahin Cem Geyik, Gurwinder Gulati, Rob Hallman, Sara Harrington, Joshua Hartman, Daniel Hewlett, Nicolas Kim, Rachel Kumar, Monica Lewis, Nicole Li, Heloise Logan, Stephen Lynch, Divyakumar Menghani, Varun Mithal, Arashpreet Singh Mor, Tanvi Motwani, Preetam Nandy, Lei Ni, Nitin Panjwani, Igor Perisic, Hema Raghavan, Romer Rosales, Guillaume Saint-Jacques, Badrul Sarwar, Amir Sepehri, Arun Swami, Ram Swaminathan, Grace Tang, Ketan Thakkar, Sriram Vasudevan, Janardhanan Vembunarayanan, James Verbus, Xin Wang, Hinkmond Wong, Ya Xu, Lin Yang, Yang Yang, Chenhui Zhai, Liang Zhang, Yani Zhang
  • 88. 88 © 2020 Amazon Web Services, Inc. or its affiliates. All rights reserved | 88 © 2020 Amazon Web Services, Inc. or its affiliates. All rights reserved | Detect bias in ML models and understand model predictions Amazon SageMaker Clarify
  • 89. 89 © 2020 Amazon Web Services, Inc. or its affiliates. All rights reserved | Predictive Maintenance Manufacturing, Automotive, IoT Demand Forecasting Retail, Consumer Goods, Manufacturing Fraud Detection Financial Services, Online Retail Credit Risk Prediction Financial Services, Retail Extract and Analyze Data from Documents Healthcare, Legal, Media/Ent, Education Computer Vision Healthcare, Pharma, Manufacturing Autonomous Driving Automotive, Transportation Personalized Recommendations Media & Entertainment, Retail, Education Churn Prediction Retail, Education, Software & Internet https://aws.amazon.c om/sagemaker/gettin g-started Amazon SageMaker Customer ML Use cases
  • 90. 90 © 2020 Amazon Web Services, Inc. or its affiliates. All rights reserved | Bias and Explainability: Challenges 1 Without detection, it is hard to know if bias has entered an ML model: • Imbalances may be present in the initial dataset • Bias may develop during training • Bias may develop over time after model deployment 2 Machine learning models are often complex & opaque, making explainability critical: • Regulations may require companies to be able to explain model predictions • Internal stakeholders and customers may need explanations for model behavior • Data science teams can improve models if they understand model behavior
  • 91. 91 © 2020 Amazon Web Services, Inc. or its affiliates. All rights reserved | Amazon SageMaker Clarify Detect bias in ML models and understand model predictions Detect bias during data preparation Identify imbalances in data Evaluate the degree to which various types of bias are present in your model Check your trained model for bias Understand the relative importance of each feature to your model’s behavior Explain overall model behavior Understand the relative importance of each feature for individual inferences Explain individual predictions Provide alerts and detect drift over time due to changing real-world conditions Detect drift in bias and model behavior over time Generate automated reports Produce reports on bias and explanations to support internal presentations
  • 92. 92 © 2020 Amazon Web Services, Inc. or its affiliates. All rights reserved | SageMaker Clarify works across the ML lifecycle Collect and prepare training data Train and tune model Evaluate and qualify model Deploy model in production Monitor model in production Measure Bias Metrics Measure and Tune Bias Metrics Measure Explainability Metrics Catalog Model Metrics Measure Bias Metrics Measure Explainability Metrics Monitor Bias Metric Drift Monitor Explainability Drift SageMaker Data Wrangler SageMaker Training Autopilot Hyperparameter Tuning SageMaker Processing SageMaker Hosting SageMaker Model Monitor
  • 93. 93 © 2020 Amazon Web Services, Inc. or its affiliates. All rights reserved | How SageMaker Clarify works Amazon SageMaker Clarify
  • 94. 94 © 2020 Amazon Web Services, Inc. or its affiliates. All rights reserved | SageMaker Clarify – Detect Bias During Data Preparation Bias report in SageMaker Data Wrangler
  • 95. 95 © 2020 Amazon Web Services, Inc. or its affiliates. All rights reserved | SageMaker Clarify – Check Your Trained Model for Bias Bias report in SageMaker Experiments
  • 96. 96 © 2020 Amazon Web Services, Inc. or its affiliates. All rights reserved | SageMaker Clarify – Monitor Your Model for Bias Drift Bias Drift in SageMaker Model Monitor
  • 97. 97 © 2020 Amazon Web Services, Inc. or its affiliates. All rights reserved | SageMaker Clarify – Understand Your Model Model Explanation in SageMaker Experiments
  • 98. 98 © 2020 Amazon Web Services, Inc. or its affiliates. All rights reserved | SageMaker Clarify – Monitor Your Model for Drift in Behavior Explainability Drift in SageMaker Model Monitor
  • 99. 99 © 2020 Amazon Web Services, Inc. or its affiliates. All rights reserved | Case Study & Demo: https://youtu.be/cQo2ew0DQw0
  • 100. 100 © 2020 Amazon Web Services, Inc. or its affiliates. All rights reserved | SageMaker Clarify Use Cases Regulatory Compliance Internal Reporting Operational Excellence Customer Service
  • 101. Lessons learned • Fairness as a Process • Notions of bias & fairness are highly application dependent • Choice of the attribute(s) for which bias is to be measured & the choice of the bias metrics to be guided by social, legal, and other non-technical considerations • Collaboration/consensus across key stakeholders • Wide spectrum of customers with different levels of technical background • Managed service vs. open source packages • Monitoring of the deployed model • Fairness & explainability considerations across the ML lifecycle
  • 102. 102 © 2020 Amazon Web Services, Inc. or its affiliates. All rights reserved | Best Practices • Fairness as a Process: • The notions of bias and fairness are highly application dependent and the choice of the attribute(s) for which bias is to be measured, as well as the choice of the bias metrics, may need to be guided by social, legal, and other non- technical considerations. • Building consensus and achieving collaboration across key stakeholders (such as product, policy, legal, engineering, and AI/ML teams, as well as end users and communities) is a prerequisite for the successful adoption of fairness-aware ML approaches in practice. • Fairness and explainability considerations may be applicable during each stage of the ML lifecycle.
  • 103. 103 © 2020 Amazon Web Services, Inc. or its affiliates. All rights reserved | Fairness and Explainability by Design in the ML Lifecycle
  • 104. 104 © 2020 Amazon Web Services, Inc. or its affiliates. All rights reserved | Additional Pointers For more information on Amazon SageMaker Clarify, please refer: • https://aws.amazon.com/sagemaker/clarify • Amazon Science / AWS Articles • https://aws.amazon.com/blogs/aws/new-amazon-sagemaker-clarify-detects- bias-and-increases-the-transparency-of-machine-learning-models • https://www.amazon.science/latest-news/how-clarify-helps-machine-learning- developers-detect-unintended-bias • Technical papers: (1) Amazon SageMaker Clarify [KDD’21] (2) Fairness Measures for Machine Learning in Finance • https://github.com/aws/amazon-sagemaker-clarify Acknowledgments: Amazon SageMaker Clarify core team, Amazon AWS AI team, and partners across Amazon
  • 105. 105 © 2020 Amazon Web Services, Inc. or its affiliates. All rights reserved | Additional Pointers For more information on Amazon SageMaker Clarify, please refer: • https://aws.amazon.com/sagemaker/clarify • Amazon Science / AWS Articles • https://aws.amazon.com/blogs/aws/new-amazon-sagemaker-clarify-detects- bias-and-increases-the-transparency-of-machine-learning-models • https://www.amazon.science/latest-news/how-clarify-helps-machine-learning- developers-detect-unintended-bias • Technical papers: (1) Amazon SageMaker Clarify [KDD’21] (2) Fairness Measures for Machine Learning in Finance • https://github.com/aws/amazon-sagemaker-clarify Acknowledgments: Amazon SageMaker Clarify core team, Amazon AWS AI team, and partners across Amazon
  • 106. © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Trademark Amazon SageMaker Debugger Debug and profile ML model training and get real-time insights
  • 107. © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Trademark Why debugging and profiling Training bugs Large compute instances Long training times Training ML models is difficult and compute intensive
  • 108. © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Trademark SageMaker Debugger Real-time monitoring Relevant data capture Automatic error detection SageMaker Studio integration Debug data while training is ongoing Zero code change Persistent in your S3 bucket Built-in and custom rules Early termination Alerts about rule status Save time and cost Find issues early Accelerate prototyping Detect performance bottlenecks View suggestions on resolving bottlenecks, Interactive visualizations Monitor utilization Profile by step or time duration Right size instance Improve utilization Reduce cost System resource usage Time spent by training operations
  • 109. © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Trademark https://www.amazon.science/publications/amazon-sagemaker-debugger-a-system-for-real-time-insights-into-machine-learning-model- training
  • 110. 110 © 2020 Amazon Web Services, Inc. or its affiliates. All rights reserved | Additional Pointers For more information on Amazon SageMaker Debugger, please refer: • https://aws.amazon.com/sagemaker/debugger • AWS Articles • https://aws.amazon.com/blogs/aws/amazon-sagemaker-debugger-debug-your- machine-learning-models • https://aws.amazon.com/blogs/machine-learning/detecting-hidden-but-non-trivial- problems-in-transfer-learning-models-using-amazon-sagemaker-debugger • Technical paper: Amazon SageMaker Debugger: A System for Real-Time Insights into Machine Learning Model Training (MLSys 2021) • https://pypi.org/project/smdebug Acknowledgments: Amazon SageMaker Debugger core team, Amazon AWS AI team, and partners across Amazon
  • 111. Fairness for Opaque Models via Model Tuning (Hyperparameter Optimization) • Can we tune the hyperparameters of a model to achieve both accuracy and fairness? • Can we support both opaque models and opaque fairness constraints? • Use Bayesian optimization for HPO with fairness constraints! • Explore hyperparameter configurations where fairness constraints are satisfied V. Perrone, M. Donini, M. B. Zafar, R. Schmucker, K. Kenthapadi, C. Archambeau, Fair Bayesian Optimization, AIES 2021 (Best paper award @ ICML 2020 AutoML workshop)
  • 115. Global Optimization for Opaque Functions
  • 118. Results: Model-specific vs. Model-agnostic Methods
  • 119. Human-in-the-loop frameworks Desirable to augment ML model predictions with expert inputs Popular examples – Healthcare models Content moderation tasks Useful for improving accuracy, incorporating additional information, and auditing models. Child maltreatment hotline screening
  • 120. Errors and biases in human-in-the-loop frameworks ML tasks often suffer from group-specific bias, induced due to misrepresentative data or models. Human-in-the-loop frameworks can reflect biases or inaccuracies of the human experts. Concerns include: • Racial bias in human-in-the-loop framework for recidivism risk assessment (Green, Chen – FAccT 2019) • Ethical concerns regarding audits of commercial facial processing technologies (Raji et al. – AIES 2020) • Automation bias in time critical decision support systems (Cummings – ISTC 2004) Can we design human-in-the-loop frameworks that take into account the expertise and biases of the human experts?
  • 121. Model 𝑋 − non-protected attributes; 𝑌 − class label; 𝑍 − protected attribute/group membership Number of experts available = 𝑚 − 1 Input −𝑋 Classifier 𝐹: 𝑋 → 𝑌 Deferrer 𝐷: 𝑋 → 0,1 𝑚 . . If 𝐷1 = 1 If 𝐷2 = 1 If 𝐷3 = 1 If 𝐷4 = 1 If 𝐷𝑚−1 = 1 If 𝐷𝑚 = 1 Deferrer 𝐷 choose a committee of experts. The majority decision of committee is the final prediction Final output of selected committee Experts might have access to additional information, including group membership 𝑍. There might be a cost/penalty associated with each expert review.
  • 122. Fairness in human-in-the-loop settings • Joint learning framework to learn a classifier and a deferral system for multiple experts simultaneously • Synthetic and real-world experiments on the efficacy of our method V. Keswani, M. Lease, K. Kenthapadi, Towards Unbiased and Accurate Deferral to Multiple Experts, AIES 2021.
  • 123. Minimax Group Fairness: Algorithms and Experiments • Equality of error rates: an intuitive & well-studied group fairness notion • May require artificially inflating error on easier-to-predict groups  • Undesirable when most/all of the targeted population is disadvantaged • Goal: minimize maximum group error [Martinez et al, 2020] • “Ensure that the worst-off group is as well-off as possible” • Our work: algorithms based on a zero-sum game between a Learner and a Regulator • Theoretical results and experimental evaluation E. Diana, W. Gill, M. Kearns, K. Kenthapadi, A. Roth, Minimax Group Fairness: Algorithms and Experiments, AIES 2021.
  • 124. Robust Interpretability of Neural Text Classifiers • Feature attribution methods used for understanding model predictions • How robust are feature attributions for neural text classifiers? • Are they identical under different random initializations of the same model? • Do they differ between a model with trained parameters and a model with random parameters? • Common feature attribution methods fail both tests! M. B. Zafar, M. Donini, D. Slack, C. Archambeau, S. Das, K. Kenthapadi, On the Lack of Robust Interpretability of Neural Text Classifiers, Findings in ACL 2021.
  • 125. Privacy in AI @ LinkedIn PriPeARL: Framework to compute robust, privacy-preserving analytics
  • 126. Analytics & Reporting Products at LinkedIn Profile View Analytics 126 Content Analytics Ad Campaign Analytics All showing demographics of members engaging with the product
  • 127. Admit only a small # of predetermined query types Querying for the number of member actions, for a specified time period, together with the top demographic breakdowns Analytics & Reporting Products at LinkedIn
  • 128. Admit only a small # of predetermined query types Querying for the number of member actions, for a specified time period, together with the top demographic breakdowns Analytics & Reporting Products at LinkedIn E.g., Title = “Senior Director” E.g., Clicks on a given ad
  • 129. Privacy Requirements Attacker cannot infer whether a member performed an action E.g., click on an article or an ad Attacker may use auxiliary knowledge E.g., knowledge of attributes associated with the target member (say, obtained from this member’s LinkedIn profile) E.g., knowledge of all other members that performed similar action (say, by creating fake accounts)
  • 130. Possible Privacy Attacks 130 Targeting: Senior directors in US, who studied at Cornell Matches ~16k LinkedIn members → over minimum targeting threshold Demographic breakdown: Company = X May match exactly one person → can determine whether the person clicks on the ad or not Require minimum reporting threshold Attacker could create fake profiles! E.g. if threshold is 10, create 9 fake profiles that all click. Rounding mechanism E.g., report incremental of 10 Still amenable to attacks E.g. using incremental counts over time to infer individuals’ actions Need rigorous techniques to preserve member privacy (not reveal exact aggregate counts)
  • 131. Problem Statement Compute robust, reliable analytics in a privacy- preserving manner, while addressing the product needs.
  • 132. PriPeARL: A Framework for Privacy-Preserving Analytics K. Kenthapadi, T. T. L. Tran, ACM CIKM 2018 132 Pseudo-random noise generation, inspired by differential privacy ● Entity id (e.g., ad creative/campaign/account) ● Demographic dimension ● Stat type (impressions, clicks) ● Time range ● Fixed secret seed Uniformly Random Fraction ● Cryptographic hash ● Normalize to (0,1) Random Noise Laplace Noise ● Fixed ε True Count Noisy Count To satisfy consistency requirements ● Pseudo-random noise → same query has same result over time, avoid averaging attack. ● For non-canonical queries (e.g., time ranges, aggregate multiple entities) ○ Use the hierarchy and partition into canonical queries ○ Compute noise for each canonical queries and sum up the noisy counts
  • 134. Lessons Learned from Deployment (> 1 year) Semantic consistency vs. unbiased, unrounded noise Suppression of small counts Online computation and performance requirements Scaling across analytics applications Tools for ease of adoption (code/API library, hands-on how-to tutorial) help! Having a few entry points (all analytics apps built over Pinot)  wider adoption
  • 135. Summary Framework to compute robust, privacy-preserving analytics Addressing challenges such as preserving member privacy, product coverage, utility, and data consistency Future Utility maximization problem given constraints on the ‘privacy loss budget’ per user E.g., noise with larger variance to impressions but less noise to clicks (or conversions) E.g., more noise to broader time range sub-queries and less noise to granular time range sub-queries Reference: K. Kenthapadi, T. Tran, PriPeARL: A Framework for Privacy- Preserving Analytics and Reporting at LinkedIn, ACM CIKM 2018.
  • 136. Acknowledgements Team: AI/ML: Krishnaram Kenthapadi, Thanh T. L. Tran Ad Analytics Product & Engineering: Mark Dietz, Taylor Greason, Ian Koeppe Legal / Security: Sara Harrington, Sharon Lee, Rohit Pitke Acknowledgements Deepak Agarwal, Igor Perisic, Arun Swami
  • 138. LinkedIn Salary (launched in Nov, 2016)
  • 139. Data Privacy Challenges Minimize the risk of inferring any one individual’s compensation data Protection against data breach No single point of failure
  • 140. Problem Statement How do we design LinkedIn Salary system taking into account the unique privacy and security challenges, while addressing the product requirements? K. Kenthapadi, A. Chudhary, and S. Ambler, LinkedIn Salary: A System for Secure Collection and Presentation of Structured Compensation Insights to Job Seekers, IEEE PAC 2017 (arxiv.org/abs/1705.06976)
  • 141. Title Region $$ User Exp Designer SF Bay Area 100K User Exp Designer SF Bay Area 115K ... ... ... Title Region $$ User Exp Designer SF Bay Area 100K De-identification Example Title Region Company Industry Years of exp Degree FoS Skills $$ User Exp Designer SF Bay Area Google Internet 12 BS Interactive Media UX, Graphics, ... 100K Title Region Industry $$ User Exp Designer SF Bay Area Internet 100K Title Region Years of exp $$ User Exp Designer SF Bay Area 10+ 100K Title Region Company Years of exp $$ User Exp Designer SF Bay Area Google 10+ 100K #data points > threshold? Yes ⇒ Copy to Hadoop (HDFS) Note: Original submission stored as encrypted objects.
  • 143. Acknowledgements Team: AI/ML: Krishnaram Kenthapadi, Stuart Ambler, Xi Chen, Yiqun Liu, Parul Jain, Liang Zhang, Ganesh Venkataraman, Tim Converse, Deepak Agarwal Application Engineering: Ahsan Chudhary, Alan Yang, Alex Navasardyan, Brandyn Bennett, Hrishikesh S, Jim Tao, Juan Pablo Lomeli Diaz, Patrick Schutz, Ricky Yan, Lu Zheng, Stephanie Chou, Joseph Florencio, Santosh Kumar Kancha, Anthony Duerr Product: Ryan Sandler, Keren Baruch Other teams (UED, Marketing, BizOps, Analytics, Testing, Voice of Members, Security, …): Julie Kuang, Phil Bunge, Prateek Janardhan, Fiona Li, Bharath Shetty, Sunil Mahadeshwar, Cory Scott, Tushar Dalvi, and team Acknowledgements David Freeman, Ashish Gupta, David Hardtke, Rong Rong, Ram
  • 144. Privacy Research @ Amazon - Sampler
  • 145. Differentially Private Query Release • Problem: answering marginal queries privately with high accuracy • Marginal queries are a special class of linear queries that count slices of a dataset. • “How many authors have visited Charlotte, graduated in the last two year and work in the Bay Area?” – A 3-way marginal query on the below dataset. • Privacy: Marginals computed against our dataset should protect against inferences on an individual’s membership (using Differential Privacy)
  • 146. Differentially Private Query Release • Projection Mechanism: Evaluate noisy answers to all queries in a query class Q and find the synthetic dataset (D′) in the space of feasible datasets that minimizes error with respect to some norm. Q is a class of queries. Nikolov, Talwar, Zhang, The geometry of differential privacy: the sparse and approximate cases, STOC 2013
  • 147. Differentially Private Query Release: Key Ideas 1. Relax the data domain: one-hot encode the non-continuous data and expand the domain to real numbers. Extend the differentiable queries to the new domain. 2. Adaptively select queries: repeatedly choose the k worst performing queries privately and optimize D′ to answer those well. S. Aydore, W. Brown, M. Kearns, K. Kenthapadi, L. Melis, A. Roth, A. Siva, Differentially Private Query Release Through Adaptive Projection, ICML 2021.
  • 148. Simple but effective, privacy-preserving mechanism Task: subsample from dataset using additional information in privacy- preserving way. Building on existing exponential analysis of k-anonymity, amplified by sampling… Mechanism M is (β, ε, δ)-differentially private Model uncertainty via Bayesian NN ”Privacy-preserving Active Learning on Sensitive Data for User Intent Classification” [Feyisetan, Balle, Diethe, Drake; PAL 2019]
  • 149. Analysis of DP sampling Avg impact privacy budget budget applied to downstream classification task. Relative trade-off on accuracy for privacy gains (from β & k) in downstream classification task
  • 150. Differentially-private text redaction Task: automatically redact sensitive text for privatizing various ML models.  Perturb sentences but maintain meaning e.g. “goalie wore a hockey helmet”  “keeper wear the nhl hat” Apply metric DP and analysis of word embeddings to scramble sentences Mechanism M is d χ – differentially private Establish plausible deniability statistics: Nw := Pr[M(w ) = w ] Sw := Expected number of words output by M(w) “Privacy- and Utility-Preserving Textual Analysis via Calibrated Multivariate Perturbations” [Feyisetan, Drake, Diethe, Balle; WSDM 2020]
  • 151. Analysis of DP redaction Show plausible deniability via dist. of Nw & Sw for ε: ε  0 : Nw decreases, Sw increases ε  inf : Nw increases, Sw decreases. Impact of accuracy given ε (epsilon) on multi-class classification and question answering tasks, respectively:
  • 152. Improving data utility of DP text redaction Task: redact text, but use additional structured information to better preserve utility. Can we improve redaction for models that fail for extraneous words? ~Recall-sensitive Extend d χ privacy to hyperbolic embeddings [Tifrea 2018] via Hyperbolic: utilize high-dimensional geometry to infuse embeddings with graph structure E.g. uni- or bi-directional syllogisms from WebIsADb New privacy analysis of Poincaré model and sampling procedure Mechanism takes advantage of density in data to apply perturbations more precisely. “Leveraging Hierarchical Representations for Preserving Privacy and Utility in Text” Feyisetan, Drake, Diethe; ICDM 2019 Tiling in Poincaré disk Hyperbolic Glove emb. projected into B2 Poincaré disk
  • 153. Analysis of Hyperbolic redaction New method improves over privacy and utility because of ability to encode meaningful structure in embeddings. Accuracy scores on classification tasks. * indicates results better than 1 baseline, ** better than 2 baselines Plausible deniability stat Nw (Pr[M(w ) = w) improved.
  • 154. Beyond Accuracy Performance and Cost Fairness and Bias Transparency and Explainability Privacy Security Safety Robustness
  • 156. Fairness in ML Application specific challenges Conversational AI systems: Unique bias/fairness/ethics considerations E.g., Hate speech, Complex failure modes Beyond protected categories, e.g., accent, dialect Entire ecosystem (e.g., including apps such as Alexa skills) Two-sided markets: e.g., fairness to buyers and to sellers, or to content consumers and producers Fairness in advertising (externalities) Tools for ensuring fairness (measuring & mitigating bias) in AI lifecycle Pre-processing (representative datasets; modifying features/labels) ML model training with fairness constraints Post-processing Experimentation & Post-deployment
  • 157. Explainability in ML Actionable explanations Balance between explanations & model secrecy Robustness of explanations to failure modes (Interaction between ML components) Application-specific challenges Conversational AI systems: contextual explanations Gradation of explanations Tools for explanations across AI lifecycle Pre & post-deployment for ML models Model developer vs. End user focused
  • 158. Privacy in ML Privacy for highly sensitive data: model training & analytics using secure enclaves, homomorphic encryption, federated learning / on- device learning, or a hybrid Privacy-preserving model training, robust against adversarial membership inference attacks (Dynamic settings + Complex data / model pipelines) Privacy-preserving mechanisms for data marketplaces
  • 159. Reflections “Fairness, Explainability, and Privacy by Design” when building AI products Collaboration/consensus across key stakeholders NYT / WSJ / ProPublica test :)
  • 160. Related Tutorials / Resources • ACM Conference on Fairness, Accountability, and Transparency (ACM FAccT) • AAAI/ACM Conference on Artificial Intelligence, Ethics, and Society (AIES) • Sara Hajian, Francesco Bonchi, and Carlos Castillo, Algorithmic bias: From discrimination discovery to fairness-aware data mining, KDD Tutorial, 2016. • Solon Barocas and Moritz Hardt, Fairness in machine learning, NeurIPS Tutorial, 2017. • Kate Crawford, The Trouble with Bias, NeurIPS Keynote, 2017. • Arvind Narayanan, 21 fairness definitions and their politics, FAccT Tutorial, 2018. • Sam Corbett-Davies and Sharad Goel, Defining and Designing Fair Algorithms, Tutorials at EC 2018 and ICML 2018. • Ben Hutchinson and Margaret Mitchell, Translation Tutorial: A History of Quantitative Fairness in Testing, FAccT Tutorial, 2019. • Henriette Cramer, Kenneth Holstein, Jennifer Wortman Vaughan, Hal Daumé III, Miroslav Dudík, Hanna Wallach, Sravana Reddy, and Jean Garcia-Gathright, Translation Tutorial: Challenges of incorporating algorithmic fairness into industry practice, FAccT Tutorial, 2019.
  • 161. Related Tutorials / Resources • Sarah Bird, Ben Hutchinson, Krishnaram Kenthapadi, Emre Kiciman, Margaret Mitchell, Fairness-Aware Machine Learning: Practical Challenges and Lessons Learned, Tutorials at WSDM 2019, WWW 2019, KDD 2019. • Krishna Gade, Sahin Cem Geyik, Krishnaram Kenthapadi, Varun Mithal, Ankur Taly, Explainable AI in Industry, Tutorials at KDD 2019, FAccT 2020, WWW 2020. • Himabindu Lakkaraju, Julius Adebayo, Sameer Singh, Explaining Machine Learning Predictions: State-of-the-art, Challenges, and Opportunities, NeurIPS 2020 Tutorial. • Kamalika Chaudhuri, Anand D. Sarwate, Differentially Private Machine Learning: Theory, Algorithms, and Applications, NeurIPS 2017 Tutorial. • Krishnaram Kenthapadi, Ilya Mironov, Abhradeep Guha Thakurta, Privacy-preserving Data Mining in Industry, Tutorials at KDD 2018, WSDM 2019, WWW 2019. • Krishnaram Kenthapadi, Ben Packer, Mehrnoosh Sameki, Nashlie Sephus, Responsible AI in Industry, Tutorials at AAAI 2021, FAccT 2021, WWW 2021, ICML 2021.
  • 162. © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark ThankYou