Vladyslav Hamolia "How to choose ASR (automatic speech recognition) system"
Download as PPTX, PDF0 likes119 views
This document provides an overview of automatic speech recognition (ASR) technologies. It discusses the history and challenges of ASR, the general processing flow including speech representation techniques like MFCCs and acoustic and language modeling using HMMs and DNNs. It also introduces some open source ASR tools like CMU Sphinx and Kaldi and provides some ASR demonstration results on the TIMIT dataset using a BiLSTM model. The document concludes with questions for discussion.
Vladyslav Hamolia "How to choose ASR (automatic speech recognition) system"
- 1. How to choose ASR AI & BIG DATA DAY Hamolia Vladyslav skype: vhamolya ELEKS
- 2. Agenda ● History of ASR ● ASR challenges ● General overview of ASR processing ● Speech representation ● Implement ASR using HMM and DNN ● Open Source tools ● Q&A 2
- 3. History Isolated words Continuous speech Connected digits Connected words Spoken dialog Filter-bank analysis Pattern recognition HHM, Stochastic language model Statistical learning, Multi-layer perceptron Concatenative synthesis; Machine Learning, LSTM 1962 1967 1972 1977 1982 1987 1998 2005 + 3
- 4. ● Large vocabulary ● Background Noise ● Regional and Social Dialects ● Spoken Language vs Written Language ● Spontaneous vs read speech ● Ambiguity ASR Сhallenges 4
- 6. Speech Representation Fast fourier transform: - Time domain to frequency domain - Shows energy in different frequency band - Complex spectra: all information is preserved - Supporting sound source separation (sources overlap less in time frequency than in time domain) invertibility 6
- 7. Spectrum Estimation Spectrum of audio signals is typically estimated in short consecutive segments, frames - Real audio signals are not stationary but vary through time - Framewise processing assumes the signal is time-invariant - Frame length for audio application in between 10ms - 100ms - Frame length for ASR 25ms 7
- 8. Bhiksha Raj, Rita Singh: Techniques for Noise Robust Automatic Speech Recognition Common path: - Take the FFT of a signal - Map the powers of the spectrum obtained above onto the mel scale - Cosine transform of mel log powers - MFCCs are amplitudes of the resulting spectrum MFCC 8
- 9. Acoustic Model - Phonemes are fundamental units - “cat” -> /k/, /at/, /t/ - Split each phoneme in 3 states - ~ 10% advantage using phonemes in contrast to words - Training model for each word requires a lot of data 9
- 10. p r aa b iy p r ay b i p r aw i uh p r aa i iy p r aa b uw p ow ih p aa iy p aa b uh b l iy p aa ah iy s eh n t s s ih t s eh v r ax b ax d iy eh v er ax d iy eh ux b ax iy eh r uw ay eh b ah iy Phonemes 10 do ow n d ow d ow n t d ow t d ah n ow n ax d ax n ax n uw probably sense everybody don’t
- 11. 11 Formant space of vowels
- 12. HMM-based Recogniser Optimization problem: O Overvation (features) P(O | S) Acoustic mode P(S | W) Pronunciation model P(W) Language model 12
- 13. HMM-based Recogniser - For each example, use current HMM models to assign feature vectors to HMM states - Viterbi algorithm, find the most likely path through the composite HMM model - Group the feature vectors assigned to each HMM - GMM for computing P(O|S) (acoustic model) 13
- 15. Language Model - Word sequence - Bigram approximation - N-gram approximation 15 - …. and LSTMs
- 16. DNN ● Two ways of using DNN for ASR task: ○ Extracting nonlinear features (and modeling in GMM) ○ Estimate phonetic probabilities ● Train the network as a classifier with a softmax across the phonetics units ● Will converge to posterior across phonetic state ● Architectures ○ Fully connected ○ Convolutional networks (CNNs) ○ Recurrent (LSTMs, GRUs) ● Dependencies not long at speech frame rates (100Hz) DNN LSTM LSTM LSTM Conv Log Mel 16
- 17. 17 Open Source ASR ● Offline tools ○ Cmusphinx ○ Kaldi ○ Julius ● Libs ○ Time tools ○ Automatic Speech Recognition ○ KerasDeepSpeech
- 18. 18 Results Implementation details: Lib: Automatic_Speech_Recognition Dataset: TIMIT Architecture: BiLSTM Speakers: 2 Target: Elderly people are often excluded. Predicted: Early people are often excluded. Target: Drop five forms in the box before you go out. Predicted: Drop wave forms in the box before you got it. Target: one who writes of such an era labours under a troublesome disadvantage Predicted: one how rights of such an er a labours onder a troubles hom disadvantage Target: Don't ask me to carry an oily rag like that. Predicted: Don't ask me to carefully rog like that. Target: Calcium makes bones and teeth strong. Predicted: Calcium makes bones and tea strong.
- 19. 19 Q&A
- 20. 1. TIMIT dataset 2. http://www.speech.sri.com/projects/srilm/ 3. Human parity in speech recognition 4. Cloud solution vs open source comparing 20 Links