IRJET- Facial Expression Recognition
0 likes21 views
This document proposes a methodology for recognizing facial expressions with increased speed and accuracy. It involves detecting faces, preprocessing images through greyscaling and contrast adjustment, extracting features through local binary patterns, and classifying expressions. The proposed system includes face detection using Haar classifiers, image processing steps, feature detection by analyzing distances between action points on facial characteristics, and using a classifier to output the expression. Future work involves testing this approach against other methods using specialized datasets and hardware to develop a faster and more reliable facial expression recognition system.
![International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395-0056
Volume: 06 Issue: 11 | Nov 2019 www.irjet.net p-ISSN: 2395-0072
© 2019, IRJET | Impact Factor value: 7.34 | ISO 9001:2008 Certified Journal | Page 1856
Facial Expression Recognition
Hrishikesh Kulkarni1, Vaibhav Jajoo2, Sachin Devangan3, Smita Kulkarni4, Kranti Avhad5, Prof.
Nital Adikane6
1,2,3,5Student, MIT College of Engineering, Pune
4Student, Sinhgad College of Engineering, Pune
6Professor, Dept. of Information Technology, MIT College of Engineering, Pune, Maharashtra, India
---------------------------------------------------------------------***----------------------------------------------------------------------
Abstract - Human face is considered to be the mirror of the heart & the emotions and expressionsplayaveryprominentroleinthe
delivery of one’s feelings. It helps in non-verbal communication by the motion of specific musclesunderneaththeskin. Onaregular
basis, humans recognize and respond to situations on thebasisofcharacteristicfeaturesandactionpointsofthehuman facewhich
are a major part of facial emotions.
This paper proposes a survey and a methodology to understand andrecognizethefacialexpressionsalong withan abstractviewof
the system which we are going to implement for increasing the speed and accuracy in FER (facial expressionrecognition)tousein
feedback systems, investigation operations, human computer interactions, computer vision systems etc.
Key Words: Face recognition, Convolution Neural Network (CNN), Local Binary Patterns Histograms and Classifier.
1. INTRODUCTION
Technology has become a crucial part of everyone’s daily life now-a-days. With improvementsintechnology,theadvancement
in online systems has become exponential. A few years back it was a dream to achieve what has been already done. To further
aid these systems and products, computing systems are used. An efficient facial expression recognition system can further
improve these systems and the overall user experience by using in human-computerinteractions,feedback systems,computer
vision systems etc. We have proposed a method to increase the speed and accuracy of traditional FER systems using effective
and suitable databases along with advanced classifiers. Our proposed system has the following components:
i) Image processing for grey-scaling and contrast stretching.
ii) Datasets and CNN for face recognition and localization.
iii) Local Binary Pattern Histograms and Classifiers for FER.
Improvement in this concept could result in betterment of various sectors such as the FMCG (fast moving consumer goods),
medicine, education, business, agriculture when used in association with computing technology. Furthermore, they could be
useful in obtaining subtle expressions of people with psychological disabilities such as autism or cerebral palsy.
2. LITERATURE SURVEY
Zhang et al. [1] researched over two types of features, one being geometry based features and other Gabor wavelet based
features, for facial expression recognition. They identified that knowledge and template based methods are face detection
strategies while local binary pattern phase correlation, haar classifier and AdaBoost are expression detection techniques.
Chen, J., Chen, Z., Chi, Z., & Fu, H. [2] they suggested that face reader is the premier for automatic Facial expressionanalysisand
Affectiva, Emotiet etc are APIs for emotion detection. Automatic FER includes two aspects: facial feature representation and
classifier problem.
Dalal and Triggs et al. [3] in 2005 studied Histogram of Gradient (HoG), SIFT, Local binary pattern (LBP) for facial expression
representation. LBP is a simple texture operator that labels the pixels by using thresholding techniques. HoG provides
numerical values to the appearance of gradient orientation in an image.
Happy, S. L., George, A., and Routray, A. et al [5] in 2012 suggested that use of LBP results in higher accuracy algorithms.
Various tests conducted by standardized testing agencies found out that algorithms produce more than 93.3% accuracy if it
implements Local Binary Patterns (LBP). It is also compatible with a significant number of classifiers, filters etc.](https://image.slidesharecdn.com/irjet-v6i11193-200122093744/85/IRJET-Facial-Expression-Recognition-1-320.jpg)
![International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395-0056
Volume: 06 Issue: 11 | Nov 2019 www.irjet.net p-ISSN: 2395-0072
© 2019, IRJET | Impact Factor value: 7.34 | ISO 9001:2008 Certified Journal | Page 1858
4. FUTURE SCOPE
An efficient, fast and accurate Expression recognition systemismostimportantaspect ofthecomputervisionsystemandcould
be a major part of the Human-Computer System.
FER combined with AI (Artificial Intelligence) couldresult inworldchangingtechnological advancementintermsof innovation
and invention.
5. CONCLUSION
The proposed survey provides an approach and methodology for classification and recognition of facial expression presented
by humans. The provided method could be improved by using specifically designed datasets andadvancedhardwaresystems.
The proposed method is based on the learnings from the previously implemented approaches mentioned in the references. It
provides a simple solution to a complex problem and focuses on feasibility in terms of real-life situations. Ourfuture objective
would be to test and compare various other approaches along with improved hardware to put forthanevenfasterandreliable
system.
6. REFERENCES
[1] Zhang, S., Zhao, X. and Lei, B. (2012). Facial expression recognition based on local binary patterns and local fisher
discriminant analysis. WSEAS Trans. Signal Process, 8(1), 21-31.
[2] Chen, J., Chen, Z., Chi, Z., & Fu, H. (2014, August). Facial expression recognition based on facial components detection and
hog features. In International Workshops on Electrical and Computer Engineering Subfields (pp. 884-888).
[3] Ahmed, F., Bari, H. and Hossain, E. (2014). Person-independent facial expression recognition based on compound local
binary pattern (CLBP). Int. Arab J. Inf. Technol., 11(2), 195-203.
[4] Happy, S. L., George, A. and Routray, A. (2012, December). A real time facial expression classification system using Local
Binary Patterns. In Intelligent Human Computer Interaction (IHCI), 2012 4th International Conference on (pp. 1-5). IEEE.
[5] Shan, C., Gong, S., & McOwan, P. W. (2005, September). Robust facial expression recognition using local binary patterns. In
Image Processing, 2005. ICIP 2005. IEEE International Conference on (Vol. 2, pp. II-370). IEEE.
[6] Sokolova, M., Japkowicz, N., & Szpakowicz, S. (2006, December).Beyondaccuracy,F-scoreandROC:a familyofdiscriminant
measures for performance evaluation. In Australasian Joint Conference on Artificial Intelligence (pp. 10151021). Springer
Berlin Heidelberg.](https://image.slidesharecdn.com/irjet-v6i11193-200122093744/85/IRJET-Facial-Expression-Recognition-3-320.jpg)
IRJET- Facial Expression Recognition
- 1. International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395-0056 Volume: 06 Issue: 11 | Nov 2019 www.irjet.net p-ISSN: 2395-0072 © 2019, IRJET | Impact Factor value: 7.34 | ISO 9001:2008 Certified Journal | Page 1856 Facial Expression Recognition Hrishikesh Kulkarni1, Vaibhav Jajoo2, Sachin Devangan3, Smita Kulkarni4, Kranti Avhad5, Prof. Nital Adikane6 1,2,3,5Student, MIT College of Engineering, Pune 4Student, Sinhgad College of Engineering, Pune 6Professor, Dept. of Information Technology, MIT College of Engineering, Pune, Maharashtra, India ---------------------------------------------------------------------***---------------------------------------------------------------------- Abstract - Human face is considered to be the mirror of the heart & the emotions and expressionsplayaveryprominentroleinthe delivery of one’s feelings. It helps in non-verbal communication by the motion of specific musclesunderneaththeskin. Onaregular basis, humans recognize and respond to situations on thebasisofcharacteristicfeaturesandactionpointsofthehuman facewhich are a major part of facial emotions. This paper proposes a survey and a methodology to understand andrecognizethefacialexpressionsalong withan abstractviewof the system which we are going to implement for increasing the speed and accuracy in FER (facial expressionrecognition)tousein feedback systems, investigation operations, human computer interactions, computer vision systems etc. Key Words: Face recognition, Convolution Neural Network (CNN), Local Binary Patterns Histograms and Classifier. 1. INTRODUCTION Technology has become a crucial part of everyone’s daily life now-a-days. With improvementsintechnology,theadvancement in online systems has become exponential. A few years back it was a dream to achieve what has been already done. To further aid these systems and products, computing systems are used. An efficient facial expression recognition system can further improve these systems and the overall user experience by using in human-computerinteractions,feedback systems,computer vision systems etc. We have proposed a method to increase the speed and accuracy of traditional FER systems using effective and suitable databases along with advanced classifiers. Our proposed system has the following components: i) Image processing for grey-scaling and contrast stretching. ii) Datasets and CNN for face recognition and localization. iii) Local Binary Pattern Histograms and Classifiers for FER. Improvement in this concept could result in betterment of various sectors such as the FMCG (fast moving consumer goods), medicine, education, business, agriculture when used in association with computing technology. Furthermore, they could be useful in obtaining subtle expressions of people with psychological disabilities such as autism or cerebral palsy. 2. LITERATURE SURVEY Zhang et al. [1] researched over two types of features, one being geometry based features and other Gabor wavelet based features, for facial expression recognition. They identified that knowledge and template based methods are face detection strategies while local binary pattern phase correlation, haar classifier and AdaBoost are expression detection techniques. Chen, J., Chen, Z., Chi, Z., & Fu, H. [2] they suggested that face reader is the premier for automatic Facial expressionanalysisand Affectiva, Emotiet etc are APIs for emotion detection. Automatic FER includes two aspects: facial feature representation and classifier problem. Dalal and Triggs et al. [3] in 2005 studied Histogram of Gradient (HoG), SIFT, Local binary pattern (LBP) for facial expression representation. LBP is a simple texture operator that labels the pixels by using thresholding techniques. HoG provides numerical values to the appearance of gradient orientation in an image. Happy, S. L., George, A., and Routray, A. et al [5] in 2012 suggested that use of LBP results in higher accuracy algorithms. Various tests conducted by standardized testing agencies found out that algorithms produce more than 93.3% accuracy if it implements Local Binary Patterns (LBP). It is also compatible with a significant number of classifiers, filters etc.
- 2. International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395-0056 Volume: 06 Issue: 11 | Nov 2019 www.irjet.net p-ISSN: 2395-0072 © 2019, IRJET | Impact Factor value: 7.34 | ISO 9001:2008 Certified Journal | Page 1857 3. PROPOSED SYSEM Fig 1.Block diagram of proposed system 3.1 Input Image First, an image is taken as input by the use of a decent web-cam or chosen from a dataset. 3.2 Face Detection Our proposed system will use the Haar classifier algorithm. The algorithm will scan the image and each section of the image is represented as the candidate for identification of a face. A candidate is rectangular window, which is cross checked for a characteristic feature of a normal human face. Characteristic features include eyes, nose, ears, moustaches, beard, hair styles, teeth, cheek bone elevations etc. A cascade is used to eliminatecandidatesthatlack characteristicfeatures.Ifoneof thefeatures is found, adjacent candidates are examined and combined. At the end of this step, the output is compared to a thresholdresult. If all the stages are passed, then the image is said to have a face. 3.3 Image Processing This step requires implementation of three things: Stretching contrast: Contrast is the difference in color, brightness and luminance. This makes it distinguishable from other images. In the process of contrast stretching, the minimum contrast section is darkened and the maximum contrast is brightened up, making the edge detection easier. Connected Region Identification: Once a particular characteristic feature is identified, its connected regionsareidentified and labeled with the same value as that of the characteristic feature point. Each of these regions is detected usingmethodsofLocal binary patterns. Grey Scaling and Anti-aliasing: The face localized in the foreground of the image, divided into several regions is converted to various shades of black, white and grey colors. The background, which is usually the blurred portion of the image is darkened to a black shade. By using Gaussian filters the entire image is anti-aliased pixel-by-pixel to produce a smooth and sharp input image. 3.4 Feature Detection The resulting image obtained after previous steps gives us grey-scaledfacewithlocalizedcharacteristicsfeatures.Eachfeature is now divided into 8 or more action points situated on the edge of the characteristic feature. The Euclidian distance between these action points is calculated. A mean of these distances is plot on the Bezier curve. The classifier analyses this Beziercurve to recognize the expression or the emotion which is presentedastheoutput. Theoutputcanbeused byvarioussectorsinorder to generate their required and suitable product.
- 3. International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395-0056 Volume: 06 Issue: 11 | Nov 2019 www.irjet.net p-ISSN: 2395-0072 © 2019, IRJET | Impact Factor value: 7.34 | ISO 9001:2008 Certified Journal | Page 1858 4. FUTURE SCOPE An efficient, fast and accurate Expression recognition systemismostimportantaspect ofthecomputervisionsystemandcould be a major part of the Human-Computer System. FER combined with AI (Artificial Intelligence) couldresult inworldchangingtechnological advancementintermsof innovation and invention. 5. CONCLUSION The proposed survey provides an approach and methodology for classification and recognition of facial expression presented by humans. The provided method could be improved by using specifically designed datasets andadvancedhardwaresystems. The proposed method is based on the learnings from the previously implemented approaches mentioned in the references. It provides a simple solution to a complex problem and focuses on feasibility in terms of real-life situations. Ourfuture objective would be to test and compare various other approaches along with improved hardware to put forthanevenfasterandreliable system. 6. REFERENCES [1] Zhang, S., Zhao, X. and Lei, B. (2012). Facial expression recognition based on local binary patterns and local fisher discriminant analysis. WSEAS Trans. Signal Process, 8(1), 21-31. [2] Chen, J., Chen, Z., Chi, Z., & Fu, H. (2014, August). Facial expression recognition based on facial components detection and hog features. In International Workshops on Electrical and Computer Engineering Subfields (pp. 884-888). [3] Ahmed, F., Bari, H. and Hossain, E. (2014). Person-independent facial expression recognition based on compound local binary pattern (CLBP). Int. Arab J. Inf. Technol., 11(2), 195-203. [4] Happy, S. L., George, A. and Routray, A. (2012, December). A real time facial expression classification system using Local Binary Patterns. In Intelligent Human Computer Interaction (IHCI), 2012 4th International Conference on (pp. 1-5). IEEE. [5] Shan, C., Gong, S., & McOwan, P. W. (2005, September). Robust facial expression recognition using local binary patterns. In Image Processing, 2005. ICIP 2005. IEEE International Conference on (Vol. 2, pp. II-370). IEEE. [6] Sokolova, M., Japkowicz, N., & Szpakowicz, S. (2006, December).Beyondaccuracy,F-scoreandROC:a familyofdiscriminant measures for performance evaluation. In Australasian Joint Conference on Artificial Intelligence (pp. 10151021). Springer Berlin Heidelberg.