Tensor flow03 generativemodels part2 8x super resolution
2 likes397 views
This document discusses using convolutional neural networks (CNNs) for generative models beyond classification, such as super resolution. It describes a CNN architecture using perceptual loss that takes a low resolution input image and outputs a higher resolution version. The network uses residual blocks for upsampling and two CNNs to calculate perceptual loss by comparing features of the low and high resolution images. In summary, this discusses using CNNs for super resolution tasks, describes a network architecture that uses perceptual loss, and explains how CNNs can be applied to many generative image processing problems beyond classification.
Tensor flow03 generativemodels part2 8x super resolution
- 1. Generative Models Part 2: Super Resolution By Sam Witteveen
- 2. Neural Network Hidden Layers Loss Function Desired Result Optimize Input Often Labels Output
- 3. Neural Network Inference Hidden Layers Raw Input Output No More Loss or Optimize No More Labels
- 4. Classification CNN Conv Blocks(Conv+Pooling) Relu Reduction of HxW increased number of filters(K) FCN layers Prediction
- 5. The Magic of CNNs • CNNs are not just for classification • Don’t fall into the trap of thinking CNNs are just a precursor to a set of dense layers, logits and classes • CNNs give us rich feature representations of what we put into them, if we cut off the dense layers and we use convolution blocks as our outputs
- 6. Old Style CNN Reduction of HxW increase of Filters Output All these reduce theConv Block 3x3 Dense Layer Logits Dense Layer Dense Layers Input Max PoolingConv Block Conv Block Conv Block Softmax Layer
- 7. Modified CNN Reduction of HxW increased number of filters Output Conv Block 3x3 Input Conv Block Conv Block Conv Block very deep (k) output eg. lots of filters
- 8. The paper "Perceptual Losses for Real-Time Style Transfer and Super-Resolution" by Johnson et.al http://arxiv.org/abs/1603.08155
- 9. The SR Network VGG2 Loss UpSampling 4x-8x VGG1 LowRes input Optimize NonTrainable NormalRes input NonTrainable SR Net Perceptual Loss
- 10. The SR Network VGG2 Loss UpSampling 4x-8x VGG1 LowRes input Optimize NonTrainable NormalRes input NonTrainable SR Net Perceptual Loss The Loss Part
- 11. Perceptual Loss MSE ConvNet for original Image Output Features not prediction Perceptual Loss ConvNet for Up Sampled Image Output Features not prediction
- 12. Up Sampling CNN Reduction of HxW increase of Filters Output Increase Receptive fieldConv Block 1x1 UpSample Block ResNet Block ResNet Block ResNet Block ResNet Block Conv with tanh activation UpSample Block Increasing Pixels LowRes Input What are these pixels? Done at smaller image size
- 13. Receptive Field 3x3 filter on 3x3 input give 5x5 receptive field
- 16. The SR Network VGG2 Loss UpSampling 4x-8x VGG1 LowRes input Optimize NonTrainable NormalRes input NonTrainable SR Net Perceptual Loss The Loss Part
- 19. Colorization
- 20. Segmentation
- 21. Others • Depth Perception • De-noising • Visual Filters • Audio Clarity • Audio Filters
- 22. Summary • Generative models go far beyond just artist models • The power of CNN beyond classification • Perceptual Loss from comparing 2 CNNs • Generative = image in -> image out • Try putting a CNN between some data to manipulate it to get what you want
- 23. The End Contact email: s@maclea.ai twitter: sam_witteveen https://github.com/samwit/TensorFlowTalks