Introduction to Deep Learning for Neuroimaging
- 1. Intro to Deep Learning for NeuroImaging Andrew Doyle @crocodoyle McGill Centre for Integrative Neuroscience
- 2. Outline 1. GET EXCITED 2. Artificial Neural Networks 3. Backpropagation 4. Convolutional Neural Networks
- 3. ImageNet-1000 Results Image courtesy Aaron Courville, 2016
- 4. Generative Models Deep Blood by Team BloodArtBrainBrush Gatys, Leon A., Alexander S. Ecker, and Matthias Bethge. "Image style transfer using convolutional neural networks." Computer Vision and Pattern Recognition (CVPR), 2016 IEEE Conference on. IEEE, 2016.
- 5. Generative Models Zhang, Han, et al. "StackGAN: Text to photo-realistic image synthesis with stacked generative adversarial networks." arXiv preprint arXiv:1612.03242 (2016). StackGAN
- 6. Generative Models Zhu, Jun-Yan, et al. "Unpaired image-to-image translation using cycle- consistent adversarial networks." arXiv preprint arXiv:1703.10593 (2017). CycleGAN
- 7. Generative Models Wolterink, Jelmer M., et al. "Deep MR to CT synthesis using unpaired data." International Workshop on Simulation and Synthesis in Medical Imaging. Springer, Cham, 2017.
- 8. Introduction For Deep Learning, you need: 1. Artificial Neural Network 2. Loss 3. Optimizer 4. Data
- 10. Artificial Neurons 𝑜 = 𝑓 𝑥 = 𝑓 𝒘 𝑻 𝒊 + 𝒃 i1 i2 i3 o w1i1 w2i2 w3i3 b x
- 12. Artificial Neurons i1 i2 i3 o 𝑜 = 𝜎 𝑥 = 𝜎 𝒘 𝑻 𝒊 + 𝒃 w1i1 w2i2 w3i3 b x Logistic Regression Hartl, Florian. “Logistic Regression – Geometric Intuition” https://florianhartl.com/logistic-regression-geometric-intuition.html i o
- 14. Neural Networks x1 x2 h2 h1 y h5 h4 h1 h2 h3 h4 h5 h6 h7 h3 h6 h7 x1 x2 y Sethi, Ishwar Krishnan. "Entropy nets: From decision trees to neural networks." Proceedings of the IEEE 78.10 (1990): 1605-1613
- 15. Neural Networks x1 x2 h2 h1 y h5 h4 h1 h2 h3 h4 h5 h6 h7 h3 h6 h7 x1 x2 y Sethi, Ishwar Krishnan. "Entropy nets: From decision trees to neural networks." Proceedings of the IEEE 78.10 (1990): 1605-1613 x1 x2 h2 h1 y h9 h8 h3 h10 h11 h5 h4 h6 h13 h12 h14 h15 h7 h1 h2 h3 h4 h5 h6 h7 x1 x2 h8 h9 h11h10 h13h12 h14 h15 y
- 16. Neural Networks x1 x2 h2 h1 y 𝑓 𝑥2 = 𝜎(𝑖2 𝑤 𝑥2,𝑖2 + 𝑏 𝑥2 ) 𝑓 ℎ2 = 𝜎(𝑤ℎ2,𝑥1 𝑓 𝑥1 + 𝑤ℎ2,𝑥2 𝑓 𝑥2 + 𝑏ℎ2 ) = 𝜎(𝑤ℎ2,𝑥1 𝜎 𝑖1 𝑤 𝑥1,𝑖1 + 𝑏 𝑥1 + 𝑤ℎ2,𝑥2 𝜎 𝑖2 𝑤 𝑥2,𝑖2 + 𝑏 𝑥2 + 𝑏ℎ2) 𝑓 𝑦 = 𝜎(𝑤 𝑦,ℎ1 𝑓 ℎ1 + 𝑤 𝑦,ℎ2 𝑓 ℎ2 + 𝑏 𝑦) = 𝜎(𝑤 𝑦,ℎ1 𝜎(𝑤ℎ1,𝑥1 𝜎 𝑖1 𝑤 𝑥1 + 𝑏 𝑥1 + 𝑤1,𝑥2 𝜎 𝑖2 𝑤 𝑥2,𝑖2 + 𝑏 𝑥2 + 𝑏ℎ1) + 𝑤 𝑦,ℎ2 𝜎(𝑤ℎ2,𝑥1 𝜎 𝑖1 𝑤 𝑥1,𝑖1 + 𝑏 𝑥1 + 𝑤ℎ2,𝑥2 𝜎 𝑖2 𝑤 𝑥2,𝑖2 + 𝑏 𝑥2 + 𝑏ℎ2) + 𝑏 𝑦) i1 i2 o 17 parameters θ = {w, b}
- 17. Backpropagation 1. Random θ initialization Iterate: 1. Forward - compute loss 2. Backward - update parameters forward pass backward pass
- 18. Backpropagation x1 x2 h2 h1 y i1 i2 i1 i2 o 0 0 0 0 1 1 1 0 1 1 1 0 𝑦 ≈ 𝑃(𝑜) XOR forward pass backward pass 𝐽 𝑜, 𝑦 = 1 2 (𝑜 − 𝑦)2 𝛻 𝜃 𝐽 𝑜, 𝑦 = 𝜕𝐽 𝜕𝑤 𝑥1,𝑖1 , 𝜕𝐽 𝜕𝑏 𝑥1 , 𝜕𝐽 𝜕𝑤 𝑥2,𝑖2 , 𝜕𝐽 𝜕𝑏 𝑥2 , … , 𝜕𝐽 𝜕𝑤 𝑦,ℎ2 𝑇
- 19. Backpropagation J w 𝜕𝐽 𝜕𝑤 forward pass backward pass 𝑤′ = 𝑤 − 𝛼 𝜕𝐽 𝜕 𝑤 learning rate Initialize w
- 20. Backpropagation x1 x2 h2 h1 y i1 i2 𝑦 ≈ 𝑜 𝜕𝐽 𝜕𝑤 𝑦,ℎ1 = 𝜕𝐽 𝜕 𝑦 ∗ 𝜕 𝑦 𝜕𝑤 𝑦,ℎ1 = −𝜎 𝑦 1 − 𝜎 𝑦 𝑓 ℎ1 …
- 21. Backpropagation x1 x2 h2 h1 y i1 i2 𝑦 ≈ 𝑜 𝜕𝐽 𝜕𝑤ℎ1,𝑥1 = 𝜕𝐽 𝜕𝑦 ∗ 𝜕𝑦 𝜕ℎ1 ∗ 𝜕ℎ1 𝜕𝑤ℎ1,𝑥1 𝜕𝐽 𝜕𝑤ℎ2,𝑥2 = 𝜕𝐽 𝜕𝑦 ∗ 𝜕𝑦 𝜕ℎ2 ∗ 𝜕ℎ2 𝜕𝑤ℎ2,𝑥2
- 22. Backpropagation x1 x2 h2 h1 y i1 i2 𝑦 ≈ 𝑜 𝜕𝐽 𝜕𝑤 𝑥1,𝑖1 = 𝜕𝐽 𝜕𝑦 ∗ 𝜕𝑦 𝜕ℎ1 ∗ 𝜕ℎ1 𝜕𝑥1 ∗ 𝜕𝑥1 𝜕𝑤 𝑥1,𝑖1 + 𝜕𝐽 𝜕𝑦 ∗ 𝜕𝑦 𝜕ℎ2 ∗ 𝜕ℎ2 𝜕𝑥1 ∗ 𝜕𝑥1 𝜕𝑤 𝑥1,𝑖1
- 23. Image courtesy Chris Olah, 2014 Data Manifold Data distribution: • Class 1 • Class 2 X-Y grid: • Param (θ) space
- 24. Image courtesy Chris Olah, 2014 Data Manifold Data distribution: • Class 1 • Class 2 X-Y grid: • Param (θ) space
- 25. Convolutional Neural Networks CNN/convnet neurons: 1. Have receptive field 2. Share weights
- 35. Convolutional Neural Networks 90% parametersAlexNet trained using: 1. Dropout 2. Batch Normalization Krizhevsky, Alex, Ilya Sutskever, and Geoffrey E. Hinton. "ImageNet classification with deep convolutional neural networks." Advances in neural information processing systems. 2012.
- 36. Convolutional Neural Networks Krizhevsky, Alex, Ilya Sutskever, and Geoffrey E. Hinton. "ImageNet classification with deep convolutional neural networks." Advances in neural information processing systems. 2012.
- 37. Challenges 1. Data quantity 2. Data size 3. Data quality 4. Data variability 5. Unexpected pathology