Simone Scardapane - The dark side of deep learning - Codemotion Milan 2017
4 likes983 views
The document discusses the limitations and concerns associated with deep learning (DL), highlighting issues such as bias and discrimination, adversarial attacks, privacy threats, security risks, and hidden technical debt. It emphasizes that DL is not a magical solution but a powerful tool whose effectiveness is largely dependent on the quality of data and objectives it is given. Additionally, the need for a thorough social-systems analysis of AI technologies is stressed to address their broad societal implications.
![The deep learning craze
[Business Insider, 2017]
IBM speech recognition is on the verge of super-human accuracy
[Time, 2017]
Are Computers Already Smarter Than Humans?
[LiveScience, 2016]
Artificial Intelligence Beats 'Most Complex Game Devised by
Humans'
[RECODE, 2017]
Intel is paying more than $400 million to buy deep-learning
startup Nervana Systems
3](https://image.slidesharecdn.com/pd9bwtltqyoilrwwabmi-signature-302d17ecd3dc867d4ee022fe9a8d9a60e6ac5da52c28eb36c18228c236c1a6df-poli-171121170044/85/Simone-Scardapane-The-dark-side-of-deep-learning-Codemotion-Milan-2017-3-320.jpg)
![Racism is definitely bad PR!
[New Statesman, 2016]The rise of the racist robots
13](https://image.slidesharecdn.com/pd9bwtltqyoilrwwabmi-signature-302d17ecd3dc867d4ee022fe9a8d9a60e6ac5da52c28eb36c18228c236c1a6df-poli-171121170044/85/Simone-Scardapane-The-dark-side-of-deep-learning-Codemotion-Milan-2017-13-320.jpg)
![Not just an economic problem
[an investigation] found that the proprietary algorithms widely
used by judges to help determine the risk of reoffending are
almost twice as likely to mistakenly flag black defendants than
white defendants [There is a blind spot in AI research]
14](https://image.slidesharecdn.com/pd9bwtltqyoilrwwabmi-signature-302d17ecd3dc867d4ee022fe9a8d9a60e6ac5da52c28eb36c18228c236c1a6df-poli-171121170044/85/Simone-Scardapane-The-dark-side-of-deep-learning-Codemotion-Milan-2017-14-320.jpg)
![Discrimination and fairness
[Google Research Blog]Attacking discrimination with smarter machine learning
15](https://image.slidesharecdn.com/pd9bwtltqyoilrwwabmi-signature-302d17ecd3dc867d4ee022fe9a8d9a60e6ac5da52c28eb36c18228c236c1a6df-poli-171121170044/85/Simone-Scardapane-The-dark-side-of-deep-learning-Codemotion-Milan-2017-15-320.jpg)
![(NIPS 2015)
Machine learning offers a fantastically powerful toolkit for
building useful complex prediction systems quickly. ... it is
dangerous to think of these quick wins as coming for free. ... it is
common to incur massive ongoing maintenance costs in real-
world ML systems. [Risk factors include] boundary erosion,
entanglement, hidden feedback loops, undeclared consumers,
data dependencies, configuration issues, changes in the external
world, and a variety of system-level anti-patterns.
Hidden Technical Debt in Machine Learning Systems
29](https://image.slidesharecdn.com/pd9bwtltqyoilrwwabmi-signature-302d17ecd3dc867d4ee022fe9a8d9a60e6ac5da52c28eb36c18228c236c1a6df-poli-171121170044/85/Simone-Scardapane-The-dark-side-of-deep-learning-Codemotion-Milan-2017-29-320.jpg)
Simone Scardapane - The dark side of deep learning - Codemotion Milan 2017
- 1. The dark side of deep learning Milan, 11 November 2017 1
- 2. Just about any DL presentation... 2
- 3. The deep learning craze [Business Insider, 2017] IBM speech recognition is on the verge of super-human accuracy [Time, 2017] Are Computers Already Smarter Than Humans? [LiveScience, 2016] Artificial Intelligence Beats 'Most Complex Game Devised by Humans' [RECODE, 2017] Intel is paying more than $400 million to buy deep-learning startup Nervana Systems 3
- 4. 4
- 5. 5
- 6. I know there's a proverb which that says 'To err is human,' but a human error is nothing to what a computer can do if it tries. --- Agatha Christie People worry that computers will get too smart and take over the world, but the real problem is that they're too stupid and they've already taken over the world. --- Pedro Domingos 6
- 7. What about the limitations of DL? DL is not magic - it is an incredibly powerful tool for extracting regularities from data according a given objective. Corollary #1: A DL program will be just as smart as the data it gets. Corollary #2: A DL program will be just as smart as the objective it optimizes. 7
- 8. Something to worry about #1 Bias and discrimination 8
- 9. Word embeddings Can convert words to vectors of numbers - at the hearth of most NLP applications with deep learning 9
- 10. Embeddings are highly sexists! Bolukbasi, T., Chang, K.W., Zou, J., Saligrama, V. and Kalai, A., 2016. Quantifying and reducing stereotypes in word embeddings. arXiv preprint 10
- 11. Hundreds of papers were published before this was openly discussed! Bolukbasi, T., Chang, K.W., Zou, J.Y., Saligrama, V. and Kalai, A.T., 2016. . In Advances in Neural Information Processing Systems (pp. 4349-4357). This is because gender biases probably account for an increase in testing accuracy. Man is to computer programmer as woman is to homemaker? Debiasing word embeddings 11
- 12. Recent years have brought extraordinary advances in the technical domains of AI. Alongside such efforts, designers and researchers from a range of disciplines need to conduct what we call social-systems analyses of AI. They need to assess the impact of technologies on their social, cultural and political settings --- There is a blind spot in AI research, Nature, 2016 12
- 13. Racism is definitely bad PR! [New Statesman, 2016]The rise of the racist robots 13
- 14. Not just an economic problem [an investigation] found that the proprietary algorithms widely used by judges to help determine the risk of reoffending are almost twice as likely to mistakenly flag black defendants than white defendants [There is a blind spot in AI research] 14
- 15. Discrimination and fairness [Google Research Blog]Attacking discrimination with smarter machine learning 15
- 16. Something to worry about #2 Adversarial attacks 16
- 17. Can we break neural networks? 17
- 18. Fooling neural networks (Andrej Karpathy blog)Breaking linear classifiers on Imagenet 18
- 19. Universal perturbations! Moosavi-Dezfooli, S.M., Fawzi, A., Fawzi, O. and Frossard, P., 2016. Universal adversarial perturbations. arXiv preprint arXiv:1610.08401. 19
- 20. 20
- 21. Something to worry about #3 Privacy 21
- 22. Anonymous data? De Montjoye, Y.A., Radaelli, L. and Singh, V.K., 2015. Unique in the shopping mall: On the reidentifiability of credit card metadata. Science, 347(6221), pp.536-539. 22
- 23. Given access to a black-box classifier, can we infer whether a specific example was part of the training dataset? We can with shadow training: Shokri, R., Stronati, M., Song, C. and Shmatikov, V., 2017, May. Membership inference attacks against machine learning models. In 2017 IEEE Symposium on Security and Privacy (SP), Â (pp. 3-18). IEEE. 23
- 24. Privacy in distributed environments Hitaj, B., Ateniese, G. and Perez-Cruz, F., 2017. Deep Models Under the GAN: Information Leakage from Collaborative Deep Learning. arXiv preprint arXiv:1702.07464. 24
- 25. Something to worry about #4 Security threats 25
- 27. Something to worry about #5 Hidden technical debt 27
- 28. DL is just a tiny component! (NIPS 2015) Hidden Technical Debt in Machine Learning Systems 28
- 29. (NIPS 2015) Machine learning offers a fantastically powerful toolkit for building useful complex prediction systems quickly. ... it is dangerous to think of these quick wins as coming for free. ... it is common to incur massive ongoing maintenance costs in real- world ML systems. [Risk factors include] boundary erosion, entanglement, hidden feedback loops, undeclared consumers, data dependencies, configuration issues, changes in the external world, and a variety of system-level anti-patterns. Hidden Technical Debt in Machine Learning Systems 29
- 30. If you are in Rome, check out our Meetup: And our new association: Italian Association for Machine Learning 30