Stable Diffusion path
- 1. Stable Diffusion path presented by Vitaly Bondar Generative DL enthusiast johngull @ gmail
- 2. Autoencoder Illustration: Lilian Weng LeCun, Y. Modeles connexionistes de l’apprentissage ` . Ph.D. thesis, 1987
- 3. Variational autoencoder (VAE) Illustrations: Lilian Weng, D. Kingma Kingma, Welling. Auto-Encoding Variational Bayes. 2013
- 4. Variational autoencoder (VAE) Kingma, Welling. Auto-Encoding Variational Bayes. 2013
- 5. Vector Quantized Variational Autoencoder (VQ-VAE) Oord et al. Neural Discrete Representation Learning, 2017
- 6. Vector Quantized Variational Autoencoder (VQ-VAE) Oord et al. Neural Discrete Representation Learning, 2017
- 7. Vector Quantized Variational Autoencoder (VQ-VAE) Oord et al. Neural Discrete Representation Learning, 2017
- 8. VQ-GAN Esser, Rombach et al. Taming Transformers for High-Resolution Image Synthesis, 2020
- 9. VQ-GAN Esser, Rombach et al. Taming Transformers for High-Resolution Image Synthesis, 2020
- 10. VQ-GAN Esser, Rombach et al. Taming Transformers for High-Resolution Image Synthesis, 2020
- 11. Recap: diffusion models Sohl-Dickstein et al. Deep Unsupervised Learning using Nonequilibrium Thermodynamics, 2015; Yang & Ermon, 2019; DDPM; Ho et al. 2020; … ?
- 12. Latent diffussions Rombach, Blattmann et al. High-Resolution Image Synthesis with Latent Diffusion Models, 2021
- 13. Latent diffussions Training phases: ● Autoencoder ○ Loss: Patch-based GAN loss + Perceptual loss ○ Regularization: KL-loss (close to VAE) OR quantization in Decoder (like VQ-GAN) ● Various generative tasks ○ Loss: classical diffusion L2 restoration loss ○ All trainings done on single A100 Rombach, Blattmann et al. High-Resolution Image Synthesis with Latent Diffusion Models, 2021
- 14. Latent diffussions Downsampling for 4-16x: speedup of generative training without sampling quality loss Rombach, Blattmann et al. High-Resolution Image Synthesis with Latent Diffusion Models, 2021
- 15. Latent diffussions Rombach, Blattmann et al. High-Resolution Image Synthesis with Latent Diffusion Models, 2021
- 20. Imagen (important influencer) Saharia, Chan. Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding, 2022
- 21. Imagen (important influencer) Saharia, Chan. Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding, 2022
- 23. Stable diffusion ● No paper (core team is from Latent diffusion authors) ● Open source: https://github.com/CompVis/stable-diffusion ● Well improved and well trained latent diffusion model
- 24. Stable diffusion Key components: ● High quality decoder from VQ-GAN ● Diffusion in latent space ● Frozen language model (CLIP ViT-L/14 embeddings) ● Classifier-free guidance ● A lot of data (LAION-5B and its subsets) ● A lot of compute power
- 25. Stable diffusion