Security and Privacy of Machine Learning
1 like2,500 views
The document presents insights into machine learning's intersection with security, highlighting malware detection, privacy of training data, and various attack vectors such as adversarial examples and model theft. It outlines the significance of differential privacy, training set poisoning, and the challenges faced by current defenses based on unrealistic threat models. The presentation also emphasizes the need for robust methodologies when deploying machine learning models, particularly those that handle sensitive data.
Security and Privacy of Machine Learning
- 1. SESSION ID: #RSAC Ian Goodfellow SECURITY AND PRIVACY OF MACHINE LEARNING Staff Research Scientist Google Brain @goodfellow_ian
- 2. (Goodfellow 2018) #RSAC Machine Learning and Security 2 Machine Learning for Security Malware detection Intrusion detection … Security against Attacks that use Machine Learning Password guessing Fake reviews …
- 3. (Goodfellow 2018) #RSAC Security of Machine Learning 3
- 4. (Goodfellow 2018) #RSAC An overview of a field 4 This presentation summarizes the work of many people, not just my own / my collaborators Download the slides for this link to extensive references The presentation focuses on the concepts, not the history or the inventors
- 5. (Goodfellow 2018) #RSAC Machine Learning Pipeline 5 Training data Learning algorithm Learned parameters Test input Test output
- 6. (Goodfellow 2018) #RSAC Privacy of Training Data 6
- 7. (Goodfellow 2018) #RSAC Defining (ε, δ)-Differential Privacy 7 (Abadi 2017)
- 8. (Goodfellow 2018) #RSAC Private Aggregation of Teacher Ensembles 8 (Papernot et al 2016)
- 9. (Goodfellow 2018) #RSAC Training Set Poisoning 9
- 10. (Goodfellow 2018) #RSAC ImageNet Poisoning 10 (Koh and Liang 2017)
- 14. (Goodfellow 2018) #RSAC Deep Dive on Adversarial Examples 14 ...solving CAPTCHAS and reading addresses... ...recognizing objects and faces…. (Szegedy et al, 2014) (Goodfellow et al, 2013) (Taigmen et al, 2013) (Goodfellow et al, 2013) and other tasks... Since 2013, deep neural networks have matched human performance at...
- 16. (Goodfellow 2018) #RSAC Turning objects into airplanes 16
- 17. (Goodfellow 2018) #RSAC Attacking a linear model 17
- 18. (Goodfellow 2018) #RSAC Wrong almost everywhere 18
- 19. (Goodfellow 2018) #RSAC Cross-model, cross-dataset transfer 19
- 20. (Goodfellow 2018) #RSAC Transfer across learning algorithms 20 (Papernot 2016)
- 21. (Goodfellow 2018) #RSAC Transfer attack 21 Train your own model Target model with unknown weights, machine learning algorithm, training set; maybe non- differentiable Substitute model mimicking target model with known, differentiable function Adversarial examples Adversarial crafting against substitute Deploy adversarial examples against the target; transferability property results in them succeeding
- 22. (Goodfellow 2018) #RSAC Enhancing Transfer with Ensembles 22 (Liu et al, 2016)
- 23. (Goodfellow 2018) #RSAC Transfer to the Human Brain 23 (Elsayed et al, 2018)
- 24. (Goodfellow 2018) #RSAC Transfer to the Physical World 24 (Kurakin et al, 2016)
- 26. (Goodfellow 2018) #RSAC Adversarial Training vs Certified Defenses 26 Adversarial Training: Train on adversarial examples This minimizes a lower bound on the true worst-case error Achieves a high amount of (empirically tested) robustness on small to medium datasets Certified defenses Minimize an upper bound on true worst-case error Robustness is guaranteed, but amount of robustness is small Verification of models that weren’t trained to be easy to verify is hard
- 27. (Goodfellow 2018) #RSAC Limitations of defenses 27 Even certified defenses so far assume unrealistic threat model Typical model: attacker can change input within some norm ball Real attacks will be stranger, hard to characterize ahead of time (Brown et al., 2017)
- 28. (Goodfellow 2018) #RSAC Clever Hans 28 (“Clever Hans, Clever Algorithms,” Bob Sturm)
- 29. (Goodfellow 2018) #RSAC Get involved! 29 https://github.com/tensorflow/cleverhans
- 30. (Goodfellow 2018) #RSAC Apply What You Have Learned 30 Publishing an ML model or a prediction API? Is the training data sensitive? -> train with differential privacy Consider how an attacker could cause damage by fooling your model Current defenses are not practical Rely on situations with no incentive to cause harm / limited amount of potential harm