![Goals
Generate pop music
。of multiple tracks
。in piano-roll format
。using GAN with CNNs
[Source Code]
https://github.com/
salu133445/musegan
[Demo Page]
https://salu133445.
github.io/musegan/
3](https://image.slidesharecdn.com/musegan-aaai2018-slides-180217131818/85/MuseGAN-Multi-track-Sequential-Generative-Adversarial-Networks-for-Symbolic-Music-Generation-and-Accompaniment-AAAI-2018-3-320.jpg)
![Data
LPD (Lakh Pianoroll Dataset)
。>170,000 multi-track piano-rolls
。Derived from Lakh MIDI Dataset
。Mainly pop songs
Pypianoroll (Python package)
。Manipulation & Visualization
。Efficient Save/Load
。Parse/Write MIDI files
。On PYPI (pip installable)
[Dataset]
https://salu133445.gith
ub.io/musegan/dataset
[Pypianoroll]
https://salu133445.
github.io/pypianoroll/
12](https://image.slidesharecdn.com/musegan-aaai2018-slides-180217131818/85/MuseGAN-Multi-track-Sequential-Generative-Adversarial-Networks-for-Symbolic-Music-Generation-and-Accompaniment-AAAI-2018-12-320.jpg)
MuseGAN: Multi-track Sequential Generative Adversarial Networks for Symbolic Music Generation and Accompaniment (AAAI 2018)
- 1. MuseGAN: Multi-track Sequential Generative Adversarial Networks for Symbolic Music Generation and Accompaniment Hao-Wen Dong*, Wen-Yi Hsiao*, Li-Chia Yang, Yi-Hsuan Yang Research Center of IT Innovation, Academia Sinica Demo Page https://salu133445.github.io/musegan/ *these authors contributed equally to this work
- 2. Outline 。Goals & Challenges 。Data 。Proposed Model 。Results & Evaluation 。Future Works Source Code https://github.com/salu133445/musegan Demo Page https://salu133445.github.io/musegan/ 2
- 3. Goals Generate pop music 。of multiple tracks 。in piano-roll format 。using GAN with CNNs [Source Code] https://github.com/ salu133445/musegan [Demo Page] https://salu133445. github.io/musegan/ 3
- 4. Challenge I Multitrack Interdependency vocal piano bass drums strings music & clip by phycause Multi-track GAN 4
- 6. Challenge III Temporal Structure paragraph 1 paragraph 2 paragraph 3 phrase 1 phrase 2 phrase 3 phrase 4 bar 1 bar 2 bar 3 bar 4 beat 1 beat 2 beat 3 beat 4 step 1 step 2 ··· step 24 song phrase 2 4/4 time 6
- 7. Challenge III Temporal Structure bar 1 bar 2 bar 3 bar 4 beat 1 beat 2 beat 3 beat 4 step 1 step 2 ··· step 24 phrase 2 Fixed Structure Convolutional Neural Networks 4/4 time 7
- 8. Data Representation pitch time Bar 1 Bar 2 Bar 3 Bar 4 time step 8 Piano-roll polyphonic multi-track (with symbolic timing)
- 9. Data Representation pitch time Piano-roll Bar 1 Bar 2 Bar 3 Bar 4 polyphonic multi-track (with symbolic timing) 9 A3 t0 t1
- 10. Data Representation Multi-track Piano-roll pitch time tracks polyphonic multi-track (with symbolic timing) 10
- 11. Data Representation 11 96 time steps 84 pitches 5 tracks 4 bars a 4×96×84×5 tensor Drums GuitarPiano Strings Bass
- 12. Data LPD (Lakh Pianoroll Dataset) 。>170,000 multi-track piano-rolls 。Derived from Lakh MIDI Dataset 。Mainly pop songs Pypianoroll (Python package) 。Manipulation & Visualization 。Efficient Save/Load 。Parse/Write MIDI files 。On PYPI (pip installable) [Dataset] https://salu133445.gith ub.io/musegan/dataset [Pypianoroll] https://salu133445. github.io/pypianoroll/ 12
- 13. Generative Adversarial Networks X real data Gz~p(z) G(z) random noise fake data Generator D real/fake Discriminator 4-bar phrases of 5 tracks critic (wgan-gp) 13
- 14. MuseGAN – An Overview Gtemp 4 latent variables1 random noise temporal generator bar generator 4 piano-roll matrices Gbar 14
- 19. Time Dependent Independent Track Dependent Melody Groove Independent Chords Style zzzzz MuseGAN z Bar Generator Gz GGGGG zzzz zzzz zzzzz z z z z 19 GGGGG Chords Style Chords Groove
- 20. Results More Samples on Demo Page https://salu133445.github.io/musegan/ Sample 1 Sample 2 20 Bass Drums Guitar Strings Piano Step 0 Step 700 Step 2500 Step 6000 Step 7900 Drum pattern Chords Bass Line
- 21. Objective Metrics UPC step QN step UPC number of used pitch classes per bar QN ratio of qualified notes Monitor the Training 21 step 2000 4000 6000 8000 104 106 108 1010 1012 0 Negative Critic Loss
- 22. User Study H: harmonious R: rhythmic MS: musically structured C: coherent OR: overall rating composer jamming hybrid 22
- 23. Summary 。MuseGAN ◦ a novel GAN for multi-track sequence generation ◦ multi-track, polyphonic music ◦ human-AI cooperative scenario (see the paper) 。Lakh Pianoroll Dataset (LPD) (new dataset!!) 。Pypianoroll (new package!!) 23
- 24. Future Works Full Song Generation bar 1 bar 2 bar 3 bar 4 beat 1 beat 2 beat 3 beat 4 step 1 step 2 ··· step 24 phrase 2 paragraph 1 paragraph 2 paragraph 3 phrase 1 phrase 2 phrase 3 phrase 4 song Hierarchical Temporal Structure 24
- 25. Future Works Cross-modal Generation 。Music + Video 。Music + Lyrics 。Video + Text 25
- 26. Q&A MuseGAN: Multi-track Sequential Generative Adversarial Networks for Symbolic Music Generation and Accompaniment Source Code https://github.com/salu133445/musegan Demo Page https://salu133445.github.io/musegan/