Project Linne 徵音梅林 -- Virtual Singer Sound-bank Processed with Python
2 likes869 views
The document discusses the development of a virtual singer sound bank called Project Linne, utilizing Python for sound processing and machine learning techniques for vowel detection. It covers the algorithms for sound synthesis, sound bank parameters for various languages, particularly Japanese and Chinese, and the tools used for analysis and visualization. The project aims to create a free and open platform for virtual singing voice synthesis, featuring contributions from multiple collaborators.
Project Linne 徵音梅林 -- Virtual Singer Sound-bank Processed with Python
- 1. Project Linne 徵音梅林 –– Virtual Singer Sound-bankVirtual Singer Sound-bank Processing with PythonProcessing with Python Yuan CHAO ( 趙元 ) PyCon TW 2016/06/03-05 ㄓ ˇ
- 2. Who am I ？ Yuan CHAO (John) YChao ...
- 3. A researcher working on HEP using OSS...
- 4. No physics today Let's talk about Virtual Singer (VS)...
- 5. Familiar with TTS? (Text-to-Speech synth.) Siri, Ok Google
- 6. Virtual Singer – Note-to-Vocal synth.
- 9. Sound bank From commercial company or volunteers Editor Notes and Lyrics Vocal Synth. Synthesized song Block diagram for a VS system Songs
- 10. 飴屋 P - UTAU の基本的アルゴリズムと開発経緯 http://udn.utau-synth.com/documents/kouen/20120325/ Step 1: cut the recored sounds through into sound elements (phonons)
- 11. 飴屋 P - UTAU の基本的アルゴリズムと開発経緯 http://udn.utau-synth.com/documents/kouen/20120325/ Step 2: connect the elements following the lyrics
- 12. 飴屋 P - UTAU の基本的アルゴリズムと開発経緯 http://udn.utau-synth.com/documents/kouen/20120325/ Step 3: Adjust the pitches and lengths of the lyrics
- 14. Consonants / Vowel 子音 / 母音 聲母 / 韻母 (fixed / variable length)
- 15. Start Beat matching position Beginning of Vowel Consonants End Next sound Sound Parameters
- 16. Sound bank for Japanese
- 18. Japanese 50 sounds x 2 (voiced/unvoiced) ( 清濁 )
- 19. Japanese Fifty sounds x 2 (voiced/unvoiced) + Half-voiced, palatalized... 拗音、半濁音及其他輔音
- 20. Japanese Fifty sounds x 2 + others ~150 basic sounds
- 21. Japanese with connected vowels ~150 sounds x 6
- 22. Japanese With connected vowels ~150 sounds x 6 Total ~ 1000 sounds
- 23. Japanese With connected vowels ~150 sounds x 6 Total ~ 1000 sounds (~10 samples/hr. for well trained people)
- 24. Chinese
- 25. Chinese If counted with bopomofo ㄅㄆㄇㄈ
- 26. Chinese 21 consonants, 16 vowels 聲母 21 個、韻母 16 個
- 28. Chinese with connected vowels ~450 sounds x 9 一ㄨㄩㄚㄛㄜㄝㄦ n ( ㄣㄥㄢㄤ ) ( ㄞㄟㄠㄡ )
- 29. Chinese With connected vowels ~450 sounds x 9 ~4000 sounds
- 30. https://github.com/benlau/linne-analyzer http://www.gnu.org/software/octave/ https://github.com/jsawruk/pymir Analysis framework by Ben Lau PyMIR lib .wav I/O Feature extraction GNU Octave visualization
- 33. Threshold method doesn't give good results
- 34. Try in frequency domain
- 37. Time domain vs. Frequency domain
- 38. Thousands of samples to be processed...
- 39. Try with ML tools – SciKit Learn
- 41. Using SVM to determine the vowel positions http://www.cmlab.csie.ntu.edu.tw/~cyy/learning/tutorials/SVM3.pdf https://en.wikipedia.org/wiki/Support_vector_machine http://www.csie.ntu.edu.tw/~cjlin/libsvm/index.html
- 42. Using SVM to set vowels あいうえおん
- 43. Using SVM to set vowels ㄚㄛㄜㄝ一ㄨㄩㄦ ㄢㄣㄤㄥ (n)
- 44. Supervised learning - Training sample? https://github.com/yuanchao/linne-analyzer/blob/vowel_det/src/linne/analyzer/cmd/linne-train2.py
- 45. Take part of the data as the training sample – Data-driven Analysis https://github.com/yuanchao/linne-analyzer/blob/vowel_det/src/linne/analyzer/cmd/linne-spect2.py
- 47. A-A-I-A-U N U E O I A https://github.com/yuanchao/linne-analyzer/blob/vowel_det/src/linne/analyzer/cmd/linne-test2.py Detecting connected vowels fen-fia-fou-a-fe ㄈㄣ - ㄈ一ㄚ - ㄈㄡ - ㄚ - ㄈㄜ N ㄩ ㄨ 一 ㄝ ㄜ ㄛ ㄚ Still some room for improvements
- 48. Fork Me on GitHub! https://github.com/yuanchao/linne-analyzer/tree/vowel_det/src/linne/analyzer/cmd
- 49. ㄓ ˇ ㄧㄣㄇㄟ ˊ ㄌㄧㄣ ˊ 徵音梅林開發計畫 https://github.com/ProjectMeilin
- 50. ち おんメイリン 徵音梅林開發計畫 Free and open VS platform 痴音
- 51. ㄓ ˇ 徵音梅林開發計畫 Software: Paul Liu, MGDesigner, Ben Lau, Atsushieno, Yuan Chao
- 52. ㄓ ˇ 徵音梅林開發計畫 Vocal: 羅竺 License: CC-BY https://www.youtube.com/watch?v=OZNrVq50wEY
- 53. Welcome
- 55. 以上 謝謝