Understanding Algorithmic Decisions
1 like438 views
This document summarizes work being done by researchers at the University of Oxford on understanding algorithmic decisions and machine learning models. It discusses how machine learning models are built using labeled training data to make predictions and decisions. The researchers are studying issues of accountability, transparency, and fairness in these models and how biases in training data can affect model outcomes. They also explore how algorithmic decisions can be explained to increase understanding and perceptions of justice.
Understanding Algorithmic Decisions
- 1. UNDERSTANDING ALGORITHMIC DECISIONS Updates on work in progress from the SOCIAM team at Oxford CS… Dr. Reuben Binns, Dr Jun Zhao, Dr Max Van Kleek, Prof. Sir. Nigel Shadbolt reuben.binns@cs.ox.ac.uk Dept. Computer Science, University of Oxford
- 2. QUESTION: WHAT DO THEY DO WITH THE DATA? ▸ Transparency over data collection is important, but then what happens to it? ▸ How will they use it? Will they treat me differently? UNDERSTANDING ALGORITHMIC DECISIONS
- 3. UNDERSTANDING ALGORITHMIC DECISIONS …BUILD MODELS! ▸ ML systems: build a model which can predict or classify things ▸ Examples: ▸ What products will this person buy? ▸ will they pay back their loan? ▸ Is this email spam?
- 4. MACHINE LEARNING AND SOCIAL MACHINES ▸ People label data (‘spam’ / ‘not spam’, ‘good credit risk’ / ‘bad credit risk’), machines build models from it ▸ Models used to decide things: ▸ what adverts are seen ▸ who gets a loan ▸ what goes in the spam box UNDERSTANDING ALGORITHMIC DECISIONS
- 5. ACCOUNTABILITY, TRANSPARENCY, FAIRNESS ▸ How do the biases of humans in training data find their way into machine models? ▸ How should machines explain the outputs of their models to humans? Can explanations help people assess the fairness of those outputs? UNDERSTANDING ALGORITHMIC DECISIONS
- 6. AUTOMATED CONTENT MODERATION ▸ Manual, community-driven flagging ▸ Paid moderators ▸ Blacklisted words
- 10. ALGORITHMIC MODERATION AND BIAS ▸ 100k Wikipedia talk page comments, each annotated by 10 different people for `toxicity’. ▸ Do different demographic sub-groups have different norms of offence? ▸ Yes: men and women often disagreed. ▸ Women had more diverse norms of offence. y n♀ ♀ y y ♂ ♂
- 11. CREATING BIASED TRAINING DATA ▸ Created 30 training data sets, sampling men / women / mixed genders from original Detox dataset ▸ Trained new offensive text classifiers based on these biased samples ♀ ♀ ♀ ♀ ♀ ♀ ♀♀ ♀ ♀ ♂ ♂ ♂ ♂ ♂ ♂♂ ♂ ♂ ⚥ ⚥ ♂ ⚥ ⚥ ⚥⚥ ⚥ ⚥ ⚥ ⚥ ⚥ ♂ ♀ test upon
- 12. TESTING BIASED OFFENCE DETECTORS ▸ Test on unseen examples, labelled by each group (male / female / balanced) ▸ All classifiers performed worse on female-labelled test data ▸ Different coefficients between m / f. Female Male Balanced 0.96 0.97 0.98 0.96 0.97 0.98 0.96 0.97 0.98 0.44 0.48 0.52 Specificity (true negative rate) Sensitivity(true positiverate) Training set Female Balanced Male Test
- 13. EXPLAINING ALGORITHMIC DECISIONS ▸ ML systems used to decide: ▸ Who gets a loan ▸ Who to invite to an interview ▸ Insurance premiums ▸ How should these decisions be explained?
- 15. WHY DOES COMPUTER SAY NO? ▸ Data protection laws require organisations to provide `meaningful information about the logic’ behind automated decisions ▸ US laws require credit scoring companies to provide `statements of reasons’
- 16. DECISION TREES?
- 17. LOCAL, INTERPRETABLE, MODEL-AGNOSTIC EXPLANATIONS ▸ E.g. Ribeiro, Marco Tulio, Sameer Singh, and Carlos Guestrin. "Why should i trust you?: Explaining the predictions of any classifier." Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. ACM, 2016.
- 18. SENSITIVITY ▸ What would I have to change in order to get a different result?
- 19. CASE BASED ▸ Marian is like Vivian, and Vivian paid back her loan, so Marian will pay back her loan Nugent, Conor, and Pádraig Cunningham. "A case-based explanation system for black-box systems." Artificial Intelligence Review 24.2 (2005): 163-178.
- 20. DEMOGRAPHIC ▸ What are the characteristics of people who received this outcome? ▸ What outcomes did other people in my demographic categories get? Ardissono, Liliana, et al. "Intrigue: personalized recommendation of tourist attractions for desktop and hand held devices." Applied Artificial Intelligence 17.8-9 (2003): 687-714.
- 21. DO EXPLANATIONS AFFECT PERCEPTIONS OF JUSTICE? ▸ Tested people’s perceptions of justice in response to various hypothetical cases using different explanation styles…
- 22. DO EXPLANATIONS AFFECT PERCEPTIONS OF JUSTICE? “She’s been a victim of this computer system that has to generalise based on, like, somebody else” “If we were in a court of law, I would argue we don’t know his circumstances, but given this computer model and the way it works it’s deserved” “This is just simply reducing a human being to a percentage”