LSGAN - SIMPle(Simple Idea Meaningful Performance Level up)
2 likes602 views
The document discusses the concept and performance of Least Squares Generative Adversarial Networks (LSGAN) as detailed in the proceedings of the IEEE International Conference on Computer Vision 2017. It reviews the research trends in Generative Adversarial Networks (GAN), introduces various models including vanilla GAN, DCGAN, and InfoGAN, and provides insights into their objective functions and optimality. Additionally, experimental results and model performance evaluations are presented, particularly highlighting their capability in generating realistic data.
]([log),(maxmin )(~)(~ zGDExDEGDV zpzxpx
DG zdata
Smart D
)))]((1[log()]([log )(~)(~ zGDExDE zpzxpx zdata
Real case
Fake case
1
)))]((1[log()]([log )(~)(~ zGDExDE zpzxpx zdata
0
should be 0
should be 0
1
Log(x)
cf.
Stupid D
)))]((1[log()]([log )(~)(~ zGDExDE zpzxpx zdata
Real case
Fake case
0
)))]((1[log()]([log )(~)(~ zGDExDE zpzxpx zdata
1
should be negative infinity
should be negative infinity
D perspective,
it should be maximum.](https://image.slidesharecdn.com/lsgan-190521075423/85/LSGAN-SIMPle-Simple-Idea-Meaningful-Performance-Level-up-9-320.jpg)
]([log),(maxmin )(~)(~ zGDExDEGDV zpzxpx
DG zdata
Generator
)))]((1[log()]([log )(~)(~ zGDExDE zpzxpx zdata
1
should be negative infinity
1
Log(x)
cf.
G perspective,
it should be minimum.
Smart G
Stupid G )))]((1[log()]([log )(~)(~ zGDExDE zpzxpx zdata
0
should be 0](https://image.slidesharecdn.com/lsgan-190521075423/85/LSGAN-SIMPle-Simple-Idea-Meaningful-Performance-Level-up-10-320.jpg)
![Introduction
2019-04-09
15
• InfoGAN - Latent Code
GNoise
Latent code
0.001
0.008
1.000
0.007
…
0.005
? : 실제 latent code의 구조는 복잡하여
해석이 어려움(entangled).
Let's make the latent code simple.
The proper generation is difficult.
[0.001, 0.008, …, 0.005] c
Latent code
0.001
0.008
1.000
0.007
…
0.005
0
0
0
0
…
1
Z C
Z C : Condition
How about adding latent code?
해석이 가능한 Condition을 제공.
Idea](https://image.slidesharecdn.com/lsgan-190521075423/85/LSGAN-SIMPle-Simple-Idea-Meaningful-Performance-Level-up-15-320.jpg)
![Introduction
2019-04-09
16
• InfoGAN - Latent Code
G
Latent code
Z C
“뭐야? 그러면 C를 Z 옆에 바로
붙이면 되는 거야?”
[0.001, 0.008, …, 005 | 0, 0, … 1]
z c
[0.001, 0.008, …, 005 | 1, 0, … 0]
z c
[0.001, 0.008, …, 005 ]
z
[0.001, 0.008, …, 005 ]
z
Ignore the additional latent code c
Cost function을 수정하여 c의 영향을 만듦.),(maxmin GDV
DG
(Mutual Information)](https://image.slidesharecdn.com/lsgan-190521075423/85/LSGAN-SIMPle-Simple-Idea-Meaningful-Performance-Level-up-16-320.jpg)
LSGAN - SIMPle(Simple Idea Meaningful Performance Level up)
- 1. SIMPLe : Simple Idea Meaningful Performance Level up* ISL Lab Seminar Hansol Kang * Mao, Xudong, et al. "Least squares generative adversarial networks." Proceedings of the IEEE International Conference on Computer Vision. 2017.
- 2. Contents 2019-04-09 2 Introduction Research Trend Review LSGAN Concept Objective Function Global Optimality Parameter Selection Results Experiment Source Code Celeb-A(DCGAN, LSGAN) Summary
- 3. I. Introduction Introduction Research Trend, Review(Concept, Vanilla GAN, DCGAN, InfoGAN)
- 5. Introduction • Research Trend 2019-04-09 5 • 70,000 high-quality PNG images at 1024×1024 resolution • Considerable variation in terms of age, ethnicity and image background • Good coverage of accessories such as eyeglasses, sunglasses, hats, etc. Flickr-Faces-HQ (FFHQ)
- 7. Introduction • Concept of GAN 2019-04-09 7 VsD GF1 F1 F1 F1 FakeR1
- 8. Introduction • Concept of GAN 2019-04-09 8 VsD G Fake?R1 @ F1
- 9. Introduction • Vanilla GAN : Adversarial Nets 2019-04-09 9 )))]((1[log()]([log),(maxmin )(~)(~ zGDExDEGDV zpzxpx DG zdata Smart D )))]((1[log()]([log )(~)(~ zGDExDE zpzxpx zdata Real case Fake case 1 )))]((1[log()]([log )(~)(~ zGDExDE zpzxpx zdata 0 should be 0 should be 0 1 Log(x) cf. Stupid D )))]((1[log()]([log )(~)(~ zGDExDE zpzxpx zdata Real case Fake case 0 )))]((1[log()]([log )(~)(~ zGDExDE zpzxpx zdata 1 should be negative infinity should be negative infinity D perspective, it should be maximum.
- 10. Introduction • Vanilla GAN : Adversarial Nets 2019-04-09 10 )))]((1[log()]([log),(maxmin )(~)(~ zGDExDEGDV zpzxpx DG zdata Generator )))]((1[log()]([log )(~)(~ zGDExDE zpzxpx zdata 1 should be negative infinity 1 Log(x) cf. G perspective, it should be minimum. Smart G Stupid G )))]((1[log()]([log )(~)(~ zGDExDE zpzxpx zdata 0 should be 0
- 11. Introduction 2019-04-09 11 • Vanilla GAN : Mathematical Proof 1) Global Optimality of datag pp 2) Convergence of Algorithm D GVs x )(xpdata “Generative Adversarial Networks” Goal Method
- 12. Introduction 2019-04-09 12 • DCGAN : Network D G “쟤들 뭐하냐?” “CNN이 MLP보다 훨씬 낫지롱” D “우리가 짱이야” G Vanilla GAN DCGAN “VAE 죽어요 ㅠㅠ”
- 13. Introduction 2019-04-09 13 • DCGAN : Latent Space 0 1 0.1 0.15 0.18 0.143 0.5 0.45 0.47 0.473 0.9 0.95 0.96 0.937 0.607± 고차원(Image)에서 의미 x 저차원(Latent code)에서 의미 o
- 14. Introduction 2019-04-09 14 • InfoGAN - Network D “우리가 짱이야” G D DCGAN InfoGAN Gz“나도 신경 써줘…” zc z “우리는 Z(Latent code)를 더 세분화 해서 조작이 가능해!”
- 15. Introduction 2019-04-09 15 • InfoGAN - Latent Code GNoise Latent code 0.001 0.008 1.000 0.007 … 0.005 ? : 실제 latent code의 구조는 복잡하여 해석이 어려움(entangled). Let's make the latent code simple. The proper generation is difficult. [0.001, 0.008, …, 0.005] c Latent code 0.001 0.008 1.000 0.007 … 0.005 0 0 0 0 … 1 Z C Z C : Condition How about adding latent code? 해석이 가능한 Condition을 제공. Idea
- 16. Introduction 2019-04-09 16 • InfoGAN - Latent Code G Latent code Z C “뭐야? 그러면 C를 Z 옆에 바로 붙이면 되는 거야?” [0.001, 0.008, …, 005 | 0, 0, … 1] z c [0.001, 0.008, …, 005 | 1, 0, … 0] z c [0.001, 0.008, …, 005 ] z [0.001, 0.008, …, 005 ] z Ignore the additional latent code c Cost function을 수정하여 c의 영향을 만듦.),(maxmin GDV DG (Mutual Information)
- 17. Introduction 2019-04-09 17 • InfoGAN - Latent Code : Generator와 c 사이의 연관성을 cost로 정의 ),(;),(),(maxmin czGcIGDVGDVI DG Maximize Hard to maximize directly as it requires access to the posterior )|( xcP )(||)|()(|log),,( )|( zpxzqKLzgxExL xzq ),,(min xL Reconstruction Error Regularization VAE Seminar (18.07.23)
- 18. Introduction 2019-04-09 18 • InfoGAN - Results
- 19. Introduction 2019-04-09 19 GAN Concept Performance Manipulability Stability Concept • GAN Performance • DCGAN, LSGAN Stability • DCGAN, LSGAN Manipulability • DCGAN, InfoGAN
- 20. II. LSGAN LSGAN Concept, Objective Function, Global Optimality, Parameter Selection, Results
- 21. LSGAN 2019-04-09 21 • Concept D학습이 잘되었다 (=50:50) 학습이 잘되었다 (=Good representation) G Real Fake F5 R1 R2 R5 F1 F5 R1 F1 여전히 너무나도 가짜 같은 데이터가 존재 If 60:40 then stupid G If 40:60 then stupid D
- 22. LSGAN 2019-04-09 22 • Concept G Real Fake F5 R1 R2 R5 F1 F5 R1 F1 경계 근처 ≈ Real? Fake? Fake -> 경계 근처 F5 F1 훨씬 헷갈리는(Real에 가까운) 데이터 생성
- 24. LSGAN • Objective function 2019-04-09 24 2 )(~ 2 ~ ))(( 2 1 )( 2 1 )(min )( azGDEbxDEDV zpxpxLSGAN D zxdata Smart D 2 )(~ 2 ~ ))(( 2 1 )( 2 1 )( azGDEbxDE zpxpx zxdata 2 )(~ 2 ~ ))(( 2 1 )( 2 1 )( azGDEbxDE zpxpx zxdata Real case Fake case 1 0 (b=1), should be 0 (a=0), should be 0 Stupid D D perspective, it should be minimum. 2 )(~ 2 ~ ))(( 2 1 )( 2 1 )( azGDEbxDE zpxpx zxdata 2 )(~ 2 ~ ))(( 2 1 )( 2 1 )( azGDEbxDE zpxpx zxdata Real case Fake case 0 1 (b=1), should be 1 (a=0), should be 1 a : fake label. b : real label. c : G wants to make D believe for fake data 2 ~ ))(( 2 1 )(min )( czGDEGV zzpzLSGAN G
- 25. LSGAN • Objective function 2019-04-09 25 2 )(~ 2 ~ ))(( 2 1 )( 2 1 )(min )( azGDEbxDEDV zpxpxLSGAN D zxdata Generator 2 ~ ))(( 2 1 )( czGDE zzpz Smart G Stupid G 1 (c=1), should be 0 (c=1), should be 1 2 ~ ))(( 2 1 )( czGDE zzpz 0 G perspective, it should be minimum. 2 ~ ))(( 2 1 )(min )( czGDEGV zzpzLSGAN G a : fake label. b : real label. c : G wants to make D believe for fake data
- 26. LSGAN • Objective function 2019-04-09 26 a : fake label. b : real label. c : G wants to make D believe for fake data 조금 더 직관적으로 생각해보면, D = Classifier 2 )(~ 2 ~ ))(( 2 1 )( 2 1 )(min )( azGDEbxDEDV zpxpxLSGAN D zxdata 2 ~ ))(( 2 1 )(min )( czGDEGV zzpzLSGAN G Prediction - Label
- 27. LSGAN 2019-04-09 27 • Global Optimality – Vanilla GAN LSGAN : 𝝌 𝑷𝒆𝒂𝒓𝒔𝒐𝒏 𝟐 를 통한 증명 Vanilla GAN : JSD를 통한 증명
- 28. LSGAN 2019-04-09 28 • Global Optimality – LSGAN 2* ~ 2* ~ )()()(2 cxDEcxDEGC gd pxpx 2 )(~ 2 ~ ))(( 2 1 )( 2 1 )(min )( azGDEbxDEDV zpxpxLSGAN D zxdata 2 ~ ))(( 2 1 )(min )( czGDEGV zzpzLSGAN G )()( )()( )(* xpxp xapxbp xD gdata gdata 2 ~ 2 ~ )()( )()( )()( )()( c xpxp xapxbp Ec xpxp xapxbp E gdata gdata px gdata gdata px gd 2 ~ )( 2 1 )( cxDE xdatapx This term does not contain parameters of G
- 29. LSGAN 2019-04-09 29 • Global Optimality – LSGAN 22 )()( )()()()( )( )()( )()()()( )( xpxp xpcaxpcb xpdx xpxp xpcaxpcb xp gdata gdata x g gdata gdata x data 2 ~ 2 ~ )()( )()( )()( )()( c xpxp xapxbp Ec xpxp xapxbp E gdata gdata px gdata gdata px gd gdata gdata gdata gdata gdata gdata pp xpcapcb xpxp xcpxcp xpxp xapxbp )()()( )()( )()( )()( )()( x gdata gdata dx xpxp xpcaxpcb )()( )()()()( 2 x gdata gdata gdata dx xpxp xpcaxpcb xpxp 2 )()( )()()()( )()( x gdata ggdata dx xpxp xpabxpxpcb )()( )()()()()( 2
- 30. LSGAN 2019-04-09 30 • Global Optimality – LSGAN x gdata ggdata dx xpxp xpabxpxpcb )()( )()()()()( 2 dx xpxp xpxpxp GC x gdata gdatag )()( ))()(()(2 )(2 2 )2||(2 ggdataPearson ppp datag pp )( )()( 2 2 xp xpxq Pearson 21 abandcbIf we set If minimum
- 31. LSGAN 2019-04-09 31 • Parameters Selection 21 abandcb 01,1 candba 2 )(~ 2 ~ ))(( 2 1 )( 2 1 )(min )( azGDEbxDEDV zpxpxLSGAN D zxdata 2 ~ ))(( 2 1 )(min )( czGDEGV zzpzLSGAN G 2 )(~ 2 ~ 1))(( 2 1 1)( 2 1 )(min )( zGDExDEDV zpxpxLSGAN D zxdata i) 2 ~ ))(( 2 1 )(min )( zGDEGV zzpzLSGAN G bcandba 1,0 2 )(~ 2 ~ 1))(( 2 1 1)( 2 1 )(min )( zGDExDEDV zpxpxLSGAN D zxdata ii) 2 ~ ))(( 2 1 )(min )( zGDEGV zzpzLSGAN G ->조건을 따르지 않는 경우 성능은 비슷하며, 큰 차이가 없음!
- 34. LSGAN 2019-04-09 34 • Results Example of mode collapse
- 36. III. Experiment Experiment Source Code, Celeb-A, Korean Idol
- 37. Experiment • Source Code 2019-04-09 37 https://github.com/messy-snail/GAN_PyTorch
- 39. Experiment • Celeb-A(without BN) 2019-04-09 39 DCGAN LSGAN ep=1 ep=2 ep=3 ep=5
- 40. Experiment 2019-04-09 40 • Korean Idol(DCGAN) ep 1 ep 6 ep 51ep 21 ep 201ep 101
- 41. Experiment 2019-04-09 41 • Korean Idol(LSGAN) ep 1 ep 6 ep 51ep 21 ep 201ep 101
- 42. Experiment 2019-04-09 42 • Gueess#1 Optimizer problem?
- 43. Experiment 2019-04-09 43 • Gueess#2 Domain problem?
- 44. Experiment • Celeb-A 2019-04-09 44 https://www.slideshare.net/NaverEngineering/1-gangenerative-adversarial-network “저자들이 source code를 공개했으면 한다.” “같은 구조라면, LSGAN이 훨씬 잘 동작한다.”
- 45. IV. Summary Summary Summary, Future Work
- 46. Summary 2019-04-09 46 • 기존의 GAN보다 Real에 가까운 데이터를 생성하고, 안정성도 확보함. • Pearson Chi square divergence으로 global optimality를 증명함. (기존 GAN은 JSD로 증명) • 클래스가 많은 데이터에 대해서도 정상적으로 데이터를 생성함. • 기존의 코드에서 단순히 loss만을 변경하기에 손쉽게 적용이 가능함.
- 47. GAN Research Vanilla GAN DCGAN InfoGAN LSGAN BEGAN Cycle GAN Style GAN SRGAN Tools Document Programming PyTorch Python executable & UI I Know What You Did Last Faculty C++ Coding Standard Mathematical theory LSM applications Other Research Level Processor Ice Propagation Future work 2019-04-09 47
- 48. &