[DL輪読会]Understanding Measures of Uncertainty for Adversarial Example Detection
7 likes947 views
This document summarizes research on measuring uncertainty in neural networks to detect adversarial examples. It discusses common methods for generating adversarial examples like basic iterative method, fast gradient method, and momentum iterative method. It also explores measures of uncertainty like softmax variance and Bayesian neural networks with Monte Carlo dropout. The researchers tested these uncertainty metrics on MNIST and Kaggle ASSIRA datasets and found dropout-based measures had higher AUC for detecting adversarial examples compared to non-probabilistic neural networks.
![DEEP LEARNING JP
[DL Papers]
Understanding Measures of Uncertainty for
Adversarial Example Detection
Makoto Kawano, Keio Univ.
http://deeplearning.jp/](https://image.slidesharecdn.com/dlreadingpaper20180406-180406002929/85/DL-Understanding-Measures-of-Uncertainty-for-Adversarial-Example-Detection-1-320.jpg)
![● [Kurakin, 2016]
● [Sharif, 2016]](https://image.slidesharecdn.com/dlreadingpaper20180406-180406002929/85/DL-Understanding-Measures-of-Uncertainty-for-Adversarial-Example-Detection-6-320.jpg)
![[Szegedy et al., 2013]
●
● A B
●
ckx − ˜xk2
2 + Loss(˜x, y)
˜x = x + ⌘](https://image.slidesharecdn.com/dlreadingpaper20180406-180406002929/85/DL-Understanding-Measures-of-Uncertainty-for-Adversarial-Example-Detection-7-320.jpg)
![Fast Gradient Method [Goodfellow et al., 2014]
●NN
●
●
● 1
˜x = x + ✏sign(rxLoss(x, y))
wT
˜x = wT
x + wT
⌘
˜x = x + ⌘ k⌘k1 < ✏](https://image.slidesharecdn.com/dlreadingpaper20180406-180406002929/85/DL-Understanding-Measures-of-Uncertainty-for-Adversarial-Example-Detection-8-320.jpg)
![Basic Iterative Method [Kurakin et al., 2016]
●FGM
●
• JPEG
•
˜x0 = x, ˜xN+1 = Clipx,✏{˜xN + ↵sign(rxJ(˜xN , ytrue))}](https://image.slidesharecdn.com/dlreadingpaper20180406-180406002929/85/DL-Understanding-Measures-of-Uncertainty-for-Adversarial-Example-Detection-9-320.jpg)
![[DL輪読会]Understanding Measures of Uncertainty for Adversarial Example Detection](https://image.slidesharecdn.com/dlreadingpaper20180406-180406002929/85/DL-Understanding-Measures-of-Uncertainty-for-Adversarial-Example-Detection-10-320.jpg)
![[DL輪読会]Understanding Measures of Uncertainty for Adversarial Example Detection](https://image.slidesharecdn.com/dlreadingpaper20180406-180406002929/85/DL-Understanding-Measures-of-Uncertainty-for-Adversarial-Example-Detection-11-320.jpg)
![Momentum Iterative Method [Dong et al., 2017]
●FGM/BIM
●NIPS
●Carlini
•](https://image.slidesharecdn.com/dlreadingpaper20180406-180406002929/85/DL-Understanding-Measures-of-Uncertainty-for-Adversarial-Example-Detection-12-320.jpg)
![● x y
H[P(y|x)] = −
X
y2Y
P(y|x) log P(y|x)
I(X, Y ) = H[P(X)] − EP (y)H[P(X|Y )]
= H[P(Y )] − EP (x)H[P(Y |X)]
I(w, y|D, x) = H[p(y|x, D)] − Ep(w|D)H[p(y|x, w)]
x y
w y
●](https://image.slidesharecdn.com/dlreadingpaper20180406-180406002929/85/DL-Understanding-Measures-of-Uncertainty-for-Adversarial-Example-Detection-17-320.jpg)
![●
•
• MNIST 1 7 →
●
•
•
•
•
I(w, y|D, x) = H[p(y|x, D)] − Ep(w|D)H[p(y|x, w)]](https://image.slidesharecdn.com/dlreadingpaper20180406-180406002929/85/DL-Understanding-Measures-of-Uncertainty-for-Adversarial-Example-Detection-18-320.jpg)
![● q p
•
●L
●q
Wl = Ml · diag([zl,j]Kl
j=1)
where zl,j ⇠ Bernoulli(pl), l = 1..L, j = 1..Kl−1
Ki ⇥ Ki−1
✓ = {Ml, pl|l = [1..L]}
※ pl
LV I :=
Z
q✓(w) log p(D|w)dw − DKL(q✓kp(w))](https://image.slidesharecdn.com/dlreadingpaper20180406-180406002929/85/DL-Understanding-Measures-of-Uncertainty-for-Adversarial-Example-Detection-22-320.jpg)
![MC
●
•
●
p(y|D, x) '
1
T
TX
i=1
p(y|wi, x)
:= pMC (y|D, x)
H[p(y|D, x)] ' H[pMC(y|D, x)]
I(w, y|D, x) ' H[pMC(y|D, x)] −
1
T
TX
i=1
H[p(y|wi, x)] wi ⇠ q(w|D)
Ep(w|D)[fw
(x)] =
Z
p(w|D)fw
(x)dw
'
Z
q✓(w)fw
(x)dw
'
1
T
TX
i=1
fw
(x), w1..T ⇠ q✓(w)](https://image.slidesharecdn.com/dlreadingpaper20180406-180406002929/85/DL-Understanding-Measures-of-Uncertainty-for-Adversarial-Example-Detection-23-320.jpg)
![[DL輪読会]Understanding Measures of Uncertainty for Adversarial Example Detection](https://image.slidesharecdn.com/dlreadingpaper20180406-180406002929/85/DL-Understanding-Measures-of-Uncertainty-for-Adversarial-Example-Detection-25-320.jpg)
![[DL輪読会]Understanding Measures of Uncertainty for Adversarial Example Detection](https://image.slidesharecdn.com/dlreadingpaper20180406-180406002929/85/DL-Understanding-Measures-of-Uncertainty-for-Adversarial-Example-Detection-32-320.jpg)
![[DL輪読会]Understanding Measures of Uncertainty for Adversarial Example Detection](https://image.slidesharecdn.com/dlreadingpaper20180406-180406002929/85/DL-Understanding-Measures-of-Uncertainty-for-Adversarial-Example-Detection-34-320.jpg)
[DL輪読会]Understanding Measures of Uncertainty for Adversarial Example Detection
- 1. DEEP LEARNING JP [DL Papers] Understanding Measures of Uncertainty for Adversarial Example Detection Makoto Kawano, Keio Univ. http://deeplearning.jp/
- 2. ● Understanding measures of Uncertainty for Adversarial Example Detection ● Lewis Smith, Yarin Gal • Department of Engineering Science, University of Oxford ● 2018 3 22 ● twitter • Adversarial examples / • Gal •
- 3. ●Gal MC • • • • MC • MNIST Kaggle •
- 4. ●Adversarial examples • Basic Iterative Method • Fast Gradient Method • Momentum Iterative Method ●Measures of uncertainty • / • Softmax Variance ●Bayesian Neural Networks/MC Dropout • Bayesian Neural Network ●Experiments • • •
- 6. ● [Kurakin, 2016] ● [Sharif, 2016]
- 7. [Szegedy et al., 2013] ● ● A B ● ckx − ˜xk2 2 + Loss(˜x, y) ˜x = x + ⌘
- 8. Fast Gradient Method [Goodfellow et al., 2014] ●NN ● ● ● 1 ˜x = x + ✏sign(rxLoss(x, y)) wT ˜x = wT x + wT ⌘ ˜x = x + ⌘ k⌘k1 < ✏
- 9. Basic Iterative Method [Kurakin et al., 2016] ●FGM ● • JPEG • ˜x0 = x, ˜xN+1 = Clipx,✏{˜xN + ↵sign(rxJ(˜xN , ytrue))}
- 12. Momentum Iterative Method [Dong et al., 2017] ●FGM/BIM ●NIPS ●Carlini •
- 13. ●Adversarial examples • Basic Iterative Method • Fast Gradient Method • Momentum Iterative Method ●Measures of uncertainty • / • Softmax Variance ●Bayesian Neural Networks/MC Dropout • Bayesian Neural Network ●Experiments • • •
- 14. ●
- 15. ● • • • • ● •
- 16. ● aleatoric uncertainty • ≒ • • p( )=p( )=0.5 ● epistemic uncertainty • • •
- 17. ● x y H[P(y|x)] = − X y2Y P(y|x) log P(y|x) I(X, Y ) = H[P(X)] − EP (y)H[P(X|Y )] = H[P(Y )] − EP (x)H[P(Y |X)] I(w, y|D, x) = H[p(y|x, D)] − Ep(w|D)H[p(y|x, w)] x y w y ●
- 18. ● • • MNIST 1 7 → ● • • • • I(w, y|D, x) = H[p(y|x, D)] − Ep(w|D)H[p(y|x, w)]
- 19. = X j 1 T X i pij(pij − 1) ! − ˆpj(ˆpj − 1) + . . .= 1 C 0 @ CX j=1 1 T TX i=1 p2 ij ! − ˆp2 j 1 A ˆσ2 = 1 C CX j=1 1 T TX i=1 (pij − ˆpi)2 ˆI = H(ˆp) − 1 T X i H(pi) = X j 1 T X i pij log pij ! − ˆpj log ˆpj = X j 1 T X i p2 ij ! − ˆp2 j − 1 T X i pij ! + ˆpj + . . . = CX j 1 T TX i p2 ij ! − ˆp2 j + . . . Softmax
- 20. ●Adversarial examples • Basic Iterative Method • Fast Gradient Method • Momentum Iterative Method ●Measures of uncertainty • / • Softmax Variance ●Bayesian Neural Networks/MC Dropout • Bayesian Neural Network ●Experiments • • •
- 21. ● • • L2 ˆw = arg min w X i E(f(xi; w), y) + λ X l kWlk2 ● • w p(w) p(w) w p(y|x, D) = Z p(y|x, w)p(w|D)dw
- 22. ● q p • ●L ●q Wl = Ml · diag([zl,j]Kl j=1) where zl,j ⇠ Bernoulli(pl), l = 1..L, j = 1..Kl−1 Ki ⇥ Ki−1 ✓ = {Ml, pl|l = [1..L]} ※ pl LV I := Z q✓(w) log p(D|w)dw − DKL(q✓kp(w))
- 23. MC ● • ● p(y|D, x) ' 1 T TX i=1 p(y|wi, x) := pMC (y|D, x) H[p(y|D, x)] ' H[pMC(y|D, x)] I(w, y|D, x) ' H[pMC(y|D, x)] − 1 T TX i=1 H[p(y|wi, x)] wi ⇠ q(w|D) Ep(w|D)[fw (x)] = Z p(w|D)fw (x)dw ' Z q✓(w)fw (x)dw ' 1 T TX i=1 fw (x), w1..T ⇠ q✓(w)
- 24. ●Adversarial examples • Basic Iterative Method • Fast Gradient Method • Momentum Iterative Method ●Measures of uncertainty • / • Softmax Variance ●Bayesian Neural Networks/MC Dropout • Bayesian Neural Network ●Experiments • • •
- 26. ● ●
- 27. ● ●
- 28. ● ●
- 29. ● ●
- 30. ●VAE
- 31. ●VAE
- 33. ● • ● • • KL • !
- 35. ●MNIST ●Kaggle ASSIRA • • ●ResNet50 Dropout FC ● ROC • NN PE/dropout PE/dropout MI ● BIM/FGM/MIM
- 37. ● ( ) • • • • ( Dropout )