IRJET- Facial Emotion Detection using Convolutional Neural Network
1 like99 views
This document describes a system for facial emotion detection using convolutional neural networks. The system uses Haar cascade classifiers to detect faces in images and then applies a convolutional neural network to recognize seven basic emotions (happiness, sadness, anger, fear, disgust, surprise, contempt) from facial expressions. The convolutional neural network architecture includes convolutional layers to extract features, ReLU layers for non-linearity, pooling layers for dimensionality reduction, and fully connected layers for emotion classification. The system is described as having potential applications in security systems, driver monitoring systems, and other real-time emotion detection use cases.
![International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395-0056
Volume: 06 Issue: 04 | Apr 2019 www.irjet.net p-ISSN: 2395-0072
© 2019, IRJET | Impact Factor value: 7.211 | ISO 9001:2008 Certified Journal | Page 318
Where x is the input to a neuron. Rectifying activation
functions were used to separate specific excitation and
unspecific inhibition in the Neural Abstraction pyramid,
which was trained in a supervised way to learn several
computer vision tasks.
Pooling layer helps to reduce the number of parameters
when the images are too large. The main of this layer is to
reduce the dimensionality of each map by retaining the
important information. It operates on each feature map
independently. The most common approach used in pooling
is max pooling. Max pooling takes the largest element from
the rectified feature map and reduces it.
Fully Connected Layers are used to detect specific global
configurations of the features detected bythelowerlayersin
the Network. Neurons in a Fully Connected Layer have full
connections to all activations in the previous layer. Matrix
gets reduced into a vector and then fed into a Fully
Connected Layer. Here features are combined to create a
model. After feature extraction we need to classify the data
into various classes, this can be doneusinga FullyConnected
Neural Network.
FUTURE WORK
The automated framework can be improved
efficiently by including more emotions to the system. The
accuracy can be improved by considering the individual
portions of the face comparing with the individual
components of the input image. The system can be
implemented in a particular application by considering only
one emotion. It can be implemented to driver management
system by alerting the driver by monitoring his emotions. It
can be included in a security system in order to alert about
different crimes by observing the emotion of a particular
criminal.
CONCLUTION
In present era, since the humans tend to depend
more on machines, there is an urge necessity to include
machines in the everyday life. A good emotional classifier
should be able to classify the emotions independent of other
factors like age, gender, styles, etc. Though there are several
applications to classify emotions, Convolutional neural
network has been implemented in our system asitimproves
the efficiency and simplify the classification process and it
uses relatively littlepre-processingcomparedtootherimage
classification algorithms. The Haar classifier has been used
in our system for facial detection as it has the at most speed
calculation compared to other detection frameworks. Our
system can be implemented in many applications which
could be used in real time. The emotion can also be detected
in a video sequence and also in live video streaming
applications. The objective of the research paper is to give
brief overview of the project, the various technologies used,
and their implementation.
ACKNOWLEDGEMENT
We would like to thank ourinstitutionmanagement,
Principal Dr Ravi Shankar M, HoD Dr Madhu B K, Project
Coordinator Madhusudhan G K, Project Guide Mrs.Ramya M
and all the teaching and non-teaching staffs of our
department for their continuous support and
encouragement throughout in completion of the project.
REFERENCES
The various references helped our system to explore
different concepts used in facial emotion detection and
choose appropriate technique to minimize the errors and
limitations.
[1] Rituparna Halder, Sushmit Sengupta, ArnabPal,Sudipta
Ghosh, Debashish Kundu, “Real Time Facial Emotion
Recognision based on Image Processing and Machine](https://image.slidesharecdn.com/irjet-v6i469-190614090528/85/IRJET-Facial-Emotion-Detection-using-Convolutional-Neural-Network-3-320.jpg)
![International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395-0056
Volume: 06 Issue: 04 | Apr 2019 www.irjet.net p-ISSN: 2395-0072
© 2019, IRJET | Impact Factor value: 7.211 | ISO 9001:2008 Certified Journal | Page 319
Learning”, International Journal of Computer
applications, April 2016.
[2] Renuka Deshmukh,Vandana Jagtap “A Comprehensive
Survey on Techniques for Facial Emotion Recognition”
International Journal of Computer Science and
Information Security, March 2017.
[3] Kudiri M. Krishna, Said Abas Md, Nayan M Yunus,
"Emotion Detection Using Sub-image Based Features
Through Human Facial Expressions" in International on
Computer & Information Science (ICCIS), IEEE.
[4] D. Anurag, S. Ashim, "A Comparative Study on different
approaches of Real Time Human Emotion Recognition
based on Facial Expression Detection", International
Conference on Advances in Computer Engineering and
Applications, 2015.
[5] Pal Pritam, N. Iyer Ananth,E.YantornoRobert,"Emotion
Detection From Infant Facial Expression", Internationa
Conference on Acoustics Speech and Signal Processing,
2016.
[6] S L Happy, Routray Aurobinda, "Autamatic facial
Expression Recognition Using Features Salient Facial
Patterns" in IEEE Tranjactions on Affective Computing,
IEEE, 2014.](https://image.slidesharecdn.com/irjet-v6i469-190614090528/85/IRJET-Facial-Emotion-Detection-using-Convolutional-Neural-Network-4-320.jpg)
IRJET- Facial Emotion Detection using Convolutional Neural Network
- 1. International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395-0056 Volume: 06 Issue: 04 | Apr 2019 www.irjet.net p-ISSN: 2395-0072 © 2019, IRJET | Impact Factor value: 7.211 | ISO 9001:2008 Certified Journal | Page 316 Facial Emotion Detection using Convolutional Neural Network Sushmitha S1, Anand HN2, R Chetan Kumar3, Nidhi U4 1234Dept. of CSE, VVIET, Mysore, India ---------------------------------------------------------------------***--------------------------------------------------------------------- Abstract - In today’s world, there is a tremendous increase in usage of machinery. The world needs machines to understand humans and interpretaccordingly. Theperception of machines helps them to understandtheirsurroundings, thus facial emotion identification helps in finding this perception. Since emotion contains an abundant amount of information about our state of mind, a system is developed to detect facial emotion using neural network and image processing techniques. Key Words: Neural Network, Convolution neural network, Tensor Flow, Haarcascadeclassifier,FER2013 image database. 1. INTRODUCTION In any human communication, Facial expression plays an important role. As now a day, the worldseemstodependmostlyonmachines,itbecomes important that a machineshouldbeabletoanalyzeand understand a human emotion. Though there are several ways to detect emotion, the more practicaland easier method is via facial expression. There are seven universally recognized facial emotions: (1) Happiness (2) Sadness (3) Anger (4)Fear (5) Disgust (6) Surprise (7) Contempt.Themaindifficultyisclassifyingemotion depending on whether the input image is static or in a transition frame. Also, since the emotions change frequently the real challenge exists in detecting the emotions in dynamic cases. Since convolutional neural network can achieve greater performance on visual recognition tasks, we have implementedconvolutional neural network for face detection. In the past two decades, the research on emotion detection has increased significantly the areas contributing include psychology, sociology, business etc. It’s always better to have a safer world if we have an auto-scanning system for signs of terrorism in terms of facialemotion detection.Thefacialexpressionrecognitionsystemhad four important steps: Input image given to the system, Face Detection, Emotion Recognition, Output image indicating emotion of the human. 2. RELATED WORK A. A comprehensive study on techniques for facial emotion recognition. The various emotion classification problems are discussed and to overcome certain emotion recognition methods are proposed. Haar function has beendiscussedfor feature extraction where in bazier curve is applied in order to approximate extracting regions. The distance of each feature has been calculated and the relationships between these features are found. A two layered feed forward neural network has been used as classifying tool. Various Techniques are discussed with their predicted accuracy of result for facial emotion recognition. B. Real Time Facial Emotion Recognition based on Image Processing and Machine Learning. A neural network based prototype system has been proposed in this paper for different human emotions. Image processing techniques are used to process the input image given by the user. The universal emotions are considered
- 2. International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395-0056 Volume: 06 Issue: 04 | Apr 2019 www.irjet.net p-ISSN: 2395-0072 © 2019, IRJET | Impact Factor value: 7.211 | ISO 9001:2008 Certified Journal | Page 317 and image processing techniques combined with machine learning algorithms help classify these emotions to find different features. After the feature extraction, the features are given as an input to the neural network in order to classify these emotions to the six universal emotions. 3. METHODOLOGY The identification of human facial expressions is determined using facial muscles movements. There are various methods used in recognition of facial expressions. Before recognizing facial expression, it is important todetect faces. Since there are numerous variations in faces, detecting the face is a challenging task. Even in facial detection there are various methods, one suchmethodusedisHaarclassifier. Haar function can be used for face, eyes andmouthdetection. Edge detection method is the basic approach used for sharpening and detecting boundaries of the image. 1. Face Detection Basically, forfacial detectionthe Haarcascadeclassifieris used. The Haar Cascade is a classifier which is used to detect the object for which it has been trained using the source. It is an approach used in machine learning where both positive and negative images are considered to train the cascade function. The algorithm is fed withbothpositiveandnegative images. Negative images are those images without a face. Later features are extracted to train the classifier. There are basically three Haar features: 1. Edge feature 2. Line feature 3. Four rectangle features. Firstly, find the sum of pixels and later find the difference withrespect to specific sum of pixels in the given image. Integral imagesareintroducedtosimplify the calculation by reducing the number of pixels if it is too large. The irrelevant features which are calculated will be removed using Adaboost methodology. A threshold will be set foreach featurewhich will classify the inputimagefaceto either positive or negative.Alsofeatureswithminimumerror rate will be given more priority. A strong classifier will be considered as the final classifier as it is the sum of all classifiers. This classifier is specially introduced to reduce number of steps in classification process by dividing it into several stages and eliminating in each stage. 2. Emotion Recognition After the recognition of face next comes the recognition of emotion. Here we have used the Convolution Neural Network to recognize the emotion. Convolution Neural Network: Convolution Neural Network is a type of artificial Neural Network used in image processing that is specifically designed to process Pixel data. The layers ofa CNN consistof an Input Layer, an Output Layer and a Hidden Layer that includes 1. Multiple Convolutional Layers, 2. ReLU Layer, 3. Pooling Layer and 4. Fully Connected Layer. Convolutional Layer is an initial layer which uses mathematical operations by taking two input matrices. i.e., image matrix and filter matrix. Input matrix is multiplied with the filter matrix to produce feature map as Output. Mainly it helps in performing operations such as edge detection and sharpening by applying filters. ReLu (Rectified Linear Unit) layer introduces non - linearity in the Convolution Network. The rectifier is an activation function defined as the positive part of its arguments:
- 3. International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395-0056 Volume: 06 Issue: 04 | Apr 2019 www.irjet.net p-ISSN: 2395-0072 © 2019, IRJET | Impact Factor value: 7.211 | ISO 9001:2008 Certified Journal | Page 318 Where x is the input to a neuron. Rectifying activation functions were used to separate specific excitation and unspecific inhibition in the Neural Abstraction pyramid, which was trained in a supervised way to learn several computer vision tasks. Pooling layer helps to reduce the number of parameters when the images are too large. The main of this layer is to reduce the dimensionality of each map by retaining the important information. It operates on each feature map independently. The most common approach used in pooling is max pooling. Max pooling takes the largest element from the rectified feature map and reduces it. Fully Connected Layers are used to detect specific global configurations of the features detected bythelowerlayersin the Network. Neurons in a Fully Connected Layer have full connections to all activations in the previous layer. Matrix gets reduced into a vector and then fed into a Fully Connected Layer. Here features are combined to create a model. After feature extraction we need to classify the data into various classes, this can be doneusinga FullyConnected Neural Network. FUTURE WORK The automated framework can be improved efficiently by including more emotions to the system. The accuracy can be improved by considering the individual portions of the face comparing with the individual components of the input image. The system can be implemented in a particular application by considering only one emotion. It can be implemented to driver management system by alerting the driver by monitoring his emotions. It can be included in a security system in order to alert about different crimes by observing the emotion of a particular criminal. CONCLUTION In present era, since the humans tend to depend more on machines, there is an urge necessity to include machines in the everyday life. A good emotional classifier should be able to classify the emotions independent of other factors like age, gender, styles, etc. Though there are several applications to classify emotions, Convolutional neural network has been implemented in our system asitimproves the efficiency and simplify the classification process and it uses relatively littlepre-processingcomparedtootherimage classification algorithms. The Haar classifier has been used in our system for facial detection as it has the at most speed calculation compared to other detection frameworks. Our system can be implemented in many applications which could be used in real time. The emotion can also be detected in a video sequence and also in live video streaming applications. The objective of the research paper is to give brief overview of the project, the various technologies used, and their implementation. ACKNOWLEDGEMENT We would like to thank ourinstitutionmanagement, Principal Dr Ravi Shankar M, HoD Dr Madhu B K, Project Coordinator Madhusudhan G K, Project Guide Mrs.Ramya M and all the teaching and non-teaching staffs of our department for their continuous support and encouragement throughout in completion of the project. REFERENCES The various references helped our system to explore different concepts used in facial emotion detection and choose appropriate technique to minimize the errors and limitations. [1] Rituparna Halder, Sushmit Sengupta, ArnabPal,Sudipta Ghosh, Debashish Kundu, “Real Time Facial Emotion Recognision based on Image Processing and Machine
- 4. International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395-0056 Volume: 06 Issue: 04 | Apr 2019 www.irjet.net p-ISSN: 2395-0072 © 2019, IRJET | Impact Factor value: 7.211 | ISO 9001:2008 Certified Journal | Page 319 Learning”, International Journal of Computer applications, April 2016. [2] Renuka Deshmukh,Vandana Jagtap “A Comprehensive Survey on Techniques for Facial Emotion Recognition” International Journal of Computer Science and Information Security, March 2017. [3] Kudiri M. Krishna, Said Abas Md, Nayan M Yunus, "Emotion Detection Using Sub-image Based Features Through Human Facial Expressions" in International on Computer & Information Science (ICCIS), IEEE. [4] D. Anurag, S. Ashim, "A Comparative Study on different approaches of Real Time Human Emotion Recognition based on Facial Expression Detection", International Conference on Advances in Computer Engineering and Applications, 2015. [5] Pal Pritam, N. Iyer Ananth,E.YantornoRobert,"Emotion Detection From Infant Facial Expression", Internationa Conference on Acoustics Speech and Signal Processing, 2016. [6] S L Happy, Routray Aurobinda, "Autamatic facial Expression Recognition Using Features Salient Facial Patterns" in IEEE Tranjactions on Affective Computing, IEEE, 2014.