![ESRGAN: Enhanced Super-Resolution
Generative Adversarial Networks
[ECCV 2018]](https://image.slidesharecdn.com/adafm-200419150756/85/PR-240-Modulating-Image-Restoration-with-Continual-Levels-via-Adaptive-Feature-Modification-Layers-2-320.jpg)
![ESRGAN
◆ SRGAN [CVPR 2017, PR-030]
Ledig, Christian, et al. "Photo-realistic single image super-resolution using a generative adversarial network." CVPR 2017.](https://image.slidesharecdn.com/adafm-200419150756/85/PR-240-Modulating-Image-Restoration-with-Continual-Levels-via-Adaptive-Feature-Modification-Layers-3-320.jpg)
![ESRGAN
◆ Perception-Distortion Tradeoff [CVPR 2018, PR-113]
Blau, Yochai, and Tomer Michaeli. "The perception-distortion tradeoff." CVPR 2018.](https://image.slidesharecdn.com/adafm-200419150756/85/PR-240-Modulating-Image-Restoration-with-Continual-Levels-via-Adaptive-Feature-Modification-Layers-4-320.jpg)
![ESRGAN
◆ ESRGAN [ECCV 2018]
Wang, Xintao, et al. "Esrgan: Enhanced super-resolution generative adversarial networks." ECCV 2018.
SRGAN의 Upgraded Version](https://image.slidesharecdn.com/adafm-200419150756/85/PR-240-Modulating-Image-Restoration-with-Continual-Levels-via-Adaptive-Feature-Modification-Layers-5-320.jpg)
![ESRGAN
◆ ESRGAN [ECCV 2018]
Network Interpolation](https://image.slidesharecdn.com/adafm-200419150756/85/PR-240-Modulating-Image-Restoration-with-Continual-Levels-via-Adaptive-Feature-Modification-Layers-6-320.jpg)
![Deep Network Interpolation
For Continuous Imagery Effect Transition
[CVPR 2019]](https://image.slidesharecdn.com/adafm-200419150756/85/PR-240-Modulating-Image-Restoration-with-Continual-Levels-via-Adaptive-Feature-Modification-Layers-7-320.jpg)
![DNI
◆ DNI [CVPR 2019]
Wang, Xintao, et al. "Deep network interpolation for continuous imagery effect transition." CVPR 2019.](https://image.slidesharecdn.com/adafm-200419150756/85/PR-240-Modulating-Image-Restoration-with-Continual-Levels-via-Adaptive-Feature-Modification-Layers-8-320.jpg)
![Modulating Image Restoration with
Continual Levels via
Adaptive Feature Modification Layers
[CVPR 2019]](https://image.slidesharecdn.com/adafm-200419150756/85/PR-240-Modulating-Image-Restoration-with-Continual-Levels-via-Adaptive-Feature-Modification-Layers-16-320.jpg)
![AdaFM
◆ AdaFM [CVPR 2019]](https://image.slidesharecdn.com/adafm-200419150756/85/PR-240-Modulating-Image-Restoration-with-Continual-Levels-via-Adaptive-Feature-Modification-Layers-17-320.jpg)
PR-240: Modulating Image Restoration with Continual Levels via Adaptive Feature Modification Layers
- 1. Modulating Image Restoration with Continual Levels via Adaptive Feature Modification Layers Hyeongmin Lee Image and Video Pattern Recognition LAB Electrical and Electronic Engineering Dept, Yonsei University PR-240 2020.4.19
- 2. ESRGAN: Enhanced Super-Resolution Generative Adversarial Networks [ECCV 2018]
- 3. ESRGAN ◆ SRGAN [CVPR 2017, PR-030] Ledig, Christian, et al. "Photo-realistic single image super-resolution using a generative adversarial network." CVPR 2017.
- 4. ESRGAN ◆ Perception-Distortion Tradeoff [CVPR 2018, PR-113] Blau, Yochai, and Tomer Michaeli. "The perception-distortion tradeoff." CVPR 2018.
- 5. ESRGAN ◆ ESRGAN [ECCV 2018] Wang, Xintao, et al. "Esrgan: Enhanced super-resolution generative adversarial networks." ECCV 2018. SRGAN의 Upgraded Version
- 6. ESRGAN ◆ ESRGAN [ECCV 2018] Network Interpolation
- 7. Deep Network Interpolation For Continuous Imagery Effect Transition [CVPR 2019]
- 8. DNI ◆ DNI [CVPR 2019] Wang, Xintao, et al. "Deep network interpolation for continuous imagery effect transition." CVPR 2019.
- 9. DNI ◆ Retraining vs Fine-tuning Retraining을 하게 되면 Filter가 바뀌어 버리지만, Fine-tuning은 Filter의 correlation이 높게 유지된다.
- 10. DNI ◆ Correlation Fine-tune 결과와 Interpolation 결과의 Correlation 양상은 비슷하다.
- 11. DNI ◆ Correlation 어느 Layer의 Filter냐에 관계 없이 Correlation 변화 양상은 비슷하다.
- 16. Modulating Image Restoration with Continual Levels via Adaptive Feature Modification Layers [CVPR 2019]
- 17. AdaFM ◆ AdaFM [CVPR 2019]
- 18. AdaFM ◆ Observation 1 Noise Level만 바꿔서 Fine-tuning을 한 결과는 Filter의 correlation이 높게 유지된다.
- 19. AdaFM ◆ Observation 2 앞에서 구한 𝑔를 이용하여 𝑓 𝑚𝑖𝑑를 구하면 Continuous한 결과를 얻을 수 있다. 𝑓 𝑚𝑖𝑑 = 𝑓15 + 𝜆 𝑔 − 𝐼 ∗ 𝑓15 = 𝜆𝑔 ∗ 𝑓15 + (1 − 𝜆)𝑓15
- 20. AdaFM ◆ AdaFM Layer Filter가 Input Feature Map이 Input AdaFM Layer
- 21. AdaFM ◆ Network Architecture Train Phase 1: Baseline Network 학습 Train Phase 2: Baseline Network parameter freeze, AdaFM parameter 학습 Test Phase: AdaFM parameter Modulation
- 22. AdaFM ◆ Experiment Baseline Model Performance Ablation on the locations of AdaFM Layers
- 23. AdaFM ◆ Experiment Ablation on AdaFM Filter Size
- 24. AdaFM ◆ Experiment Ablation on various levels
- 25. AdaFM ◆ Experiment Ablation on various levels
- 26. AdaFM ◆ Experiment Interpolation Performance 𝝈𝟏𝟓 → 𝝈𝟕𝟓 𝒒𝟖𝟎 → 𝒒𝟏𝟎 × 𝟑 →× 𝟒
- 28. Thank You!