Gan intro
3 likes778 views
The document discusses different types of generative models including auto-regressive models, variational auto-encoders, and generative adversarial networks. It provides examples of each type of model and highlights some of their features and issues during training. Specific models discussed in more detail include PixelRNNs, DCGANs, WGANs, BEGANs, Pix2Pix, and CycleGANs. The document aims to introduce deep generative models and their applications.
Gan intro
- 6. Let’s Think
- 7. A CAT ? 😺
- 8. A CAT ? 😺
- 9. A CAT ! 😺
- 13. Ideal Generative Model Model CAT Short Hair Big Ear Tabby Savannah Cat
- 14. Why Generative • Use high-dimensional, complicated probability distributions • Combining with Reinforcement learning • Missing data • Semi-supervised learning • Multi-modal outputs • Code • Can make data with realistic generation
- 15. –Richard Feynman “What I cannot create, I do not understand.”
- 17. What is Generative Model https://blog.openai.com/generative-models/
- 18. Toy Example
- 19. Generative model
- 21. Result … 😢 Maybe we need more conditions…
- 22. Deep Generative Models • Auto-Regressive Models • Variational Auto-Encoder • Generative Adversarial Networks
- 24. Auto-Regressive Models http://slazebni.cs.illinois.edu/spring17/lec13_advanced.pdf
- 25. Multi-Dimensional RNNs <Graves et al, Multi-Dimensional Recurrent Neural Networks, 2013> 2D RNN Forward and Backward passes Sequence ordering (not fixed) Multi-directional MDRNNs
- 26. Spatial LSTM <Theis et al, Generative Image Modeling Using Spatial LSTMs, 2015>
- 27. Pixel RNN <Aaron et al, Pixel Recurrent Neural Networks, 2016>
- 28. Sampling • Feed the 2D vector of zeros( I ) to the generator ➔ The output vector : O • O is softmax activations for each pixel ➔ Probability of first pixel value : p(0, 0) • Random sampling from p(0, 0) ➔ Set the very first pixel of I with sampled value • Feed I to the generator • Iteration…
- 29. Results <Aaron et al, Pixel Recurrent Neural Networks, 2016>
- 30. Features • Simple and stable training process • Best log likelihoods so far • Inefficient during sampling • Don’t easily provide simple low-dimensional codes for images
- 35. Latent variables Model CAT Short Hair Big Ear Tabby Savannah Cat Latent Space Data Space
- 36. Latent variables Model Latent Space Data Space Low Dimension High Dimension Mapping Function
- 37. Variational Inference Well Known Distribution : Multivariate Gaussian
- 38. Variational Inference Well Known Distribution : Multivariate Gaussian Sampling
- 39. Variational Inference Well Known Distribution : Multivariate Gaussian Sampling Make Distribution : Pairs of Mean, Variance
- 43. Variational Inference Well Known Distribution : Multivariate Gaussian Sampling Make Distribution : Pairs of Mean, Variance Optimize
- 45. Objective (Evidence Lower BOund)
- 46. Objective (Evidence Lower BOund) Maximize Log-likelihood Minimize the distance of p and q
- 47. Auto-Encoder
- 49. Results
- 50. Features • Simple and stable training process • Can check log likelihood • Latent variable • Low quality
- 52. GANs
- 53. GANs
- 54. Vanilla GANs
- 55. Vanilla GANs
- 56. Vanilla GANs <Goodfellow et al, Generative Adversarial Networks, 2014>
- 57. Results <Goodfellow et al, Generative Adversarial Networks, 2014>
- 58. Features • Advanced quality • Unstable training • Mode collapsing • Cannot check log likelihood
- 60. DCGAN
- 61. Deep Convolutional GAN <Radford et al, Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks, 2015>
- 62. Tips !!
- 63. Issues during Training • Mode Collapsing / Oscillating • Intractable Training Loss • Balance btw Generator & Discriminator • Manipulation • Not Enough Quality
- 64. Mode Collapsing / Oscillating The generator rotates through the modes of the data distribution. <Metz et al, Unrolled Generative Adversarial Networks, 2016>
- 65. Mode Collapsing / Oscillating < https://www.slideshare.net/HyungjooCho2/deep-generative-modelpdf >
- 66. Mode Collapsing / Oscillating GAN uses Jenen-Shannon Divergence
- 67. Mode Collapsing / Oscillating Target MLE JS R-KL
- 68. Mode Collapsing / Oscillating Target MLE JS R-KL JS and Reverse KL Divergence tend to favor under-generalization. It never converges to a fixed distribution, and only ever assigns significant probability mass to a single data mode at once.
- 69. Intractable Loss
- 71. Intractable Loss
- 72. Intractable Loss
- 74. Intractable Loss The Wasserstein distance(left plot) is continuous and provides a usable gradient everywhere. The JS plot(right) is not continuous and does not provide a usable gradient. <Arjovsky et al, Wasserstein Generative Adversarial Networks, 2017>
- 75. Intractable Loss The WGAN’s loss decreases consistently as training progresses and sample quality increases.
- 76. Balance Boundary Equilibrium GAN <Berthelot et al, BEGAN, 2017>
- 77. Manipulation Conditional GAN <Mirza et al, Conditional Generative Adversarial Networks, 2014>
- 78. Quality Progressive Growing of GAN <Karras et al, Progressive Growing of GANs For Improved Quality, Stability, and Variation, 2017>
- 79. Quality
- 81. Info GAN InfoGAN successfully disentangles writing styles <Chen et al, InfoGAN, 2017>
- 82. Info GAN Real fake Classifying time series data through unsupervised way Clustering < https://github.com/buriburisuri/timeseries_gan >
- 83. Pix2Pix Most successful GAN architecture !! <Isola et al, Image-to-image translation with conditional GAN, 2016>
- 85. Stain Style Transfer SST achieves the highest performance on original images on tumor classification
- 86. Domain Cross GAN Unsupervised version of Pix2Pix <Taigman et al, Unsupervised Cross-Domain Image Generation, 2016>
- 87. DiscoGAN / CycleGAN <Kim et al, Learning to Discover Cross Domain Relations with Generative Adversarial Networks, 2017>
- 88. DiscoGAN / CycleGAN <Zhu et al, Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks, 2017>
- 89. Simulated + Unsupervised Learning <Shrivastava et al, Learning from Simulated and Unsupervised Images through Adversarial Training, 2016>
- 90. AnoGAN <Schlegl et al,Unsupervised Anomaly Detection with Generative Adversarial Networks to Guide Marker discovery, 2017>
- 91. AmbientGAN
- 92. Generative Replay <Shin et al,Continual Learning with Deep Generative Replay, 2017>
- 93. Thanks ☺