Combatting Bias in Machine Learning
- 1. #FIGHTBIAS
- 2. SOCIETY Prejudice in favor of or against one thing, person, or group compared with another, usually in a way considered to be unfair. The bias error is an error from erroneous assumptions in the learning algorithm. High bias can cause an algorithm to miss the relevant relations between features and target outputs (underfitting). MACHINE LEARNING
- 5. AWARENESS
- 6. STANDARDIZING HOW DATASETS ARE DOCUMENTED Many projects are based on well known public datasets, but there is currently no standard for how these datasets are compiled and documented.
- 7. IGNORANCE IS BIAS It shouldn't need be explained how it's ethically wrong to attempt to predict a person's sexuality based on appearance.
- 8. FAILING PEOPLE OF COLOR Many widely used facial reconigtion algorithms by Microsoft Amazon, and Face++ performed 35% worse on dark skinned women. In 2017 mulitple Chinese users reported being able to log into another's iPhone. using FaceID.
- 9. INNACURACIES AND INCONVENIENCE There is bias in travel for people who may be considered immigrants to some, that will be exemplified with automated boarding.
- 10. OF FACIAL REGOGNITION RESEARCH IGNORES NON-BINAR Y PEOPLE
- 12. Algorithms with little feedback have little opportunity to "learn" Algorithms can reflect xenophobic bias and use limited examples to misidentify people with dark skin as criminal. With few examples of successful minority candidates predection based on these imbalanced sets will amplify bias. Online ads that determine a user's race to be black display ads for high interest credit cards at a higher rate than others.
- 13. With great power comes great responsibility
- 14. BIASED DATA = BIASED OUTCOMES Data used for training models is hardly ever actually representative of people it will be used on. HUMANS ARE IMPERFECT This doesn't mean algorithms unbiased, it means we have to assume bias will persist until we take steps to remove it. MASTERS OF OUR DEMISE We are in one of few fields where we get to pick the metric we're measured against.
- 15. COMPAS Consistently ranks black and brown incarcerated people as more likely to offend and higher risk than white prisoners. FAILED SENSORS Milimeter wave sensors used by the TSA consistently have trouble with black women's hair causing travel delays. FLAWED HARDWARE Camera hardware has been tuned and developed to highlight lighter skin tones.
- 17. ACTION
- 18. ASSESING THE NEED Not all problems are best solved with machine learning. TEST WITH EDGE CASES Hardware should be tested on dark skinned first. EXTREME SKEPTICISM Assesing models with harsh criticism towards performance on edge cases.
- 20. HOW TO START A ML PROJECT (ETHICALLY) QUESTION IF THE SOLUTION FITS THE PROBLEM EXAMINE FOR AND REMOVE BIASED PROXIES GIVE YOUR MODEL FEEDBACK AND TEST FOR BIAS
- 21. Will the product be better? If so, how?
- 22. How will this impact my users?
- 23. How often will this model get feedback?
- 25. ZIP CODE Applying for a credit card with a 90210 zip code shouldn't improve your chances of getting approved. HOMOGEN OUS EXAMPLES If there is an extreme imbalance between classes any model can attribute an occurence to subtle proxies the model learns. WORD EMBEDDINGS Tools used to measure the distance of the meaning of words can embed our sexist cultural norms and exclude nonbinary people.
- 26. MINDSET CHANGE Assume there will be some aspect of bias in your models and asses potential consequences. ACCOUNTABIL IT Y Build user trust and be open to algorithmic criticism by making models open source. Transparency leads to accountability, WORD EMBEDDINGS Tools used to measure the distance of the meaning of words can embed our sexist cultural norms and exclude nonbinary people.