PR-252: Making Convolutional Networks Shift-Invariant Again
- 1. Making Convolutional Networks Shift-Invariant Again Hyeongmin Lee Image and Video Pattern Recognition LAB Electrical and Electronic Engineering Dept, Yonsei University 5th Semester PR-252
- 4. What is shift-invariancy?? Shift-variant?? Change in performance??
- 5. What is shift-invariancy?? Shift-variant??
- 6. Aliasing
- 7. Aliasing Fourier Transform & Frequency Domain 𝑡𝑡 Ω 𝑋𝑋 Ω = � −∞ ∞ 𝑥𝑥 𝑡𝑡 𝑒𝑒−𝑗𝑗Ω𝑡𝑡 𝑑𝑑𝑑𝑑
- 8. Aliasing Down-sampling in frequency domain 𝑇𝑇 𝑥𝑥 𝑡𝑡 = � −∞ ∞ 𝑥𝑥 𝑡𝑡 𝑒𝑒−𝑗𝑗Ω𝑡𝑡 𝑑𝑑𝑑𝑑 = 𝑋𝑋(Ω) 𝑇𝑇 𝑥𝑥 𝑎𝑎𝑡𝑡 = � −∞ ∞ 𝑥𝑥 𝑎𝑎𝑡𝑡 𝑒𝑒−𝑗𝑗Ω𝑡𝑡 𝑑𝑑𝑑𝑑 = 1 |𝑎𝑎| � −∞ ∞ 𝑥𝑥 𝑡𝑡′ 𝑒𝑒−𝑗𝑗 Ω 𝑎𝑎 𝑡𝑡′ 𝑑𝑑𝑑𝑑′ 𝑇𝑇 𝑥𝑥 𝑎𝑎𝑡𝑡 = 1 |𝑎𝑎| 𝑋𝑋(Ω/𝑎𝑎)
- 9. Aliasing Down-sampling in frequency domain 𝑡𝑡 Ω 𝑡𝑡 Ω
- 10. Aliasing Discrete Signal in Frequency Domain (Discrete Time Fourier Transform) 𝜔𝜔𝑛𝑛 2𝜋𝜋−2𝜋𝜋 𝑋𝑋 𝜔𝜔 = � 𝑛𝑛=−∞ ∞ 𝑥𝑥 𝑛𝑛 𝑒𝑒−𝑗𝑗𝑗𝑗𝑗𝑗 𝑋𝑋 𝜔𝜔 + 2𝜋𝜋 = � 𝑛𝑛=−∞ ∞ 𝑥𝑥 𝑛𝑛 𝑒𝑒−𝑗𝑗𝑗𝑗𝑗𝑗 𝑒𝑒−𝑗𝑗𝑗𝑗𝑗𝑗𝑗 = � 𝑛𝑛=−∞ ∞ 𝑥𝑥 𝑛𝑛 𝑒𝑒−𝑗𝑗𝑗𝑗𝑗𝑗 = 𝑋𝑋(𝜔𝜔)
- 11. Aliasing Discrete Down-Sampling in Frequency Domain 𝑓𝑓 𝑓𝑓 𝑛𝑛 2𝜋𝜋−2𝜋𝜋 𝑛𝑛 2𝜋𝜋−2𝜋𝜋 Aliasing Aliasing
- 13. Aliasing in CNN Max pooling Average pooling Strided convolution 0 0 1 1 0 0 1 1 0 1 0 1 0 1 1 0 0 1 1 0 1 1 1 1 Max-pooling Max-pooling
- 14. Anti-aliasing
- 15. Anti-Aliasing Shift Invariancy & Shift Equivariance • Shift Equivariance • Shift Invariancy Shift-Equivariant Shift-Invariant
- 16. Anti-Aliasing Anti-aliasing 𝑓𝑓2𝜋𝜋−2𝜋𝜋 Low Pass Filtering (Blurring) 𝑓𝑓2𝜋𝜋−2𝜋𝜋 𝑓𝑓2𝜋𝜋−2𝜋𝜋 Sampling
- 17. Anti-Aliasing Anti-aliasing for max pooling 0 0 1 1 0 0 1 1 0 1 0 10 1 1 1 0 1 1 1 Max Sampling 0 1 1 0 0 1 1 0 1 1 1 11 1 1 0 1 1 1 0 Shift-Equivariant!!
- 18. Anti-Aliasing Anti-aliasing for max pooling 0 1 1 1 0 1 1 1 0.5 1 0.5 10.5 1 1 0.5 0.5 1 1 0.5 Blurring Subsampling 1 1 1 0 1 1 1 0 1 0.5 1 0.51 1 0.5 0.5 1 1 0.5 0.5 Max
- 19. Anti-Aliasing Anti-aliasing for various sampling operations in CNN • MaxPool • AveragePool • StrideConv
- 21. Results
- 22. Results Improvement in Consistency
- 23. Results Improvement in Accuracy
- 24. Results Improvement in Accuracy
- 25. Results Improvement in Image Translation
- 26. Thank You!