Emotion Speech Recognition - Convolutional Neural Network Capstone Project
0 likes4,499 views
The document discusses a capstone project focused on speech emotion recognition using a CNN model to detect emotions from audio files, which can help route calls to humans when users are upset. It describes the data collection process involving 1,170 audio files representing 8 emotions and explains the feature extraction using mel-frequency cepstral coefficients (MFCCs). The model showed improved accuracy of 75% using a 1D CNN over random classification, although issues like data imbalance and misclassification of certain emotions were noted.
Emotion Speech Recognition - Convolutional Neural Network Capstone Project
- 1. Speech Emotion Recognition with CNN Capstone Project Diego Rios 11
- 2. Can we detect emotion from audio files? 2
- 3. How can would this be applicable? Recognizing when users are upset over phone calls with bots and then speeding call to a human or agent. 3
- 4. Data ◦ 2 data sets ◦ 8 emotions ◦ 24 actors 4
- 5. You can think of sound as vibrations that propagate as an acoustic wave 5
- 6. 6 Data Description 1. Sample the audio file at a specific rate; resulting in an array of numbers. 2. Converts audio into a time series analysis 3. Plot the amplitude of time 4. Measure Mel-frequency cepstral coefficients (MFCCs) a. Assimilate to human hearing b. Transforms linear frequencies to quasi-logarithmic frequencies scaled Image source: https://manual.audacityteam.org/man/digital_audio.html
- 7. Complete Clean-Up & Modelling ◦ MFCC become our input features ◦ Started with 1170 audio files and got 1170 rows of data ◦ There are 8 target values: ◦ Angry ◦ Calm ◦ Disgust ◦ Fear ◦ Happy ◦ Neutral ◦ Sad ◦ Surprised 40 features => Averages of frequency of audio target feature
- 8. How accurate is the model? 1 dimensional CNN
- 9. Random: 12.5% 8 emotions (1 divided by 8) 1D CNN: 75% Much improved result! Random Forest: 46% Better but still low... 9
- 10. 10 Confusion Matrix & Next Steps 1. Fear is commonly misclassified 2. Surprised is also misclassified 3. Data imbalance with some emotions having more audio files than others. 4. Record audio files and used trained CNN to validate results and model.
- 11. 11 Thanks! Any questions? You can find me at: ◦ https://medium.com/@Markeko/speech-emoti on-recognition-with-convolutional-neural-netw ork-ae5406a1c0f7 ◦ diegoerios@gmail.com ◦ https://github.com/diegoerios