Efficient Face Expression Recognition Methods FER A Literature Review

International Journal of Trend in Scientific Research and Development (IJTSRD)
Volume 3 Issue 5, August 2019 Available Online: www.ijtsrd.com e-ISSN: 2456 – 6470
@ IJTSRD | Unique Paper ID – IJTSRD28026 | Volume – 3 | Issue – 5 | July - August 2019 Page 2416
Efficient Face Expression Recognition Methods (FER):
A Literature Review
Sheena Gaur1, Shashi Kant Sharma2, Lovendra Solanki2, Firdos Alam Sheikh1, Ahsan Z Rizvi1
1Mewar University, Chittorgarh, Rajasthan, India
2B K Birla Institute of Technology, Pilani, Rajasthan, India
How to cite this paper: Sheena Gaur |
Shashi Kant Sharma | Lovendra Solanki |
Firdos Alam Sheikh | Ahsan Z Rizvi
"Efficient Face Expression Recognition
Methods (FER): A Literature Review"
Published in
International
Journal of Trend in
Scientific Research
and Development
(ijtsrd), ISSN: 2456-
6470, Volume-3 |
Issue-5, August
2019, pp.2416-2420,
https://doi.org/10.31142/ijtsrd28026
Copyright © 2019 by author(s) and
International Journal ofTrend inScientific
Research and Development Journal. This
is an Open Access article distributed
under the terms of
the Creative
Commons Attribution
License (CC BY 4.0)
(http://creativecommons.org/licenses/by
/4.0)
ABSTRACT
Recognition of artificial faces is an intriguing and testing problem and affects
important applications in various regions, such as cooperation between
human computers and data-oriented activity. Facial expression is the fastest
correspondence methods for transmitting data.
It is straightforward the outward appearance of anindividualbylookingathis
/ her face yet somehow or other with regard to machines it ends updifficultto
pass judgment on the outward appearance while using PC devices yet it is not
incomprehensible in any way. This not only revealed any individual's
affectability or sentiments, but it can also be used to make a judgement on the
psychological views, yet again it could not fully understand the perception of
human behaviour, the discovery of mental problems and fabricated human
expressions. Expressions such as SAD, HAPPY, DISGUST, FEAR, ANGER,
NEUTRAL and SURPRISE have been suggested in a broad range of processes
This paper includes implementing face recognition along with facial
expression recognition, analyzing recent and pastresearchtoextract effective
and efficient methods for recognition of facial expression.
KEYWORDS: Face Expression Recognition, GaborFilter, Active AppearanceModel
(AAM), Hidden Markov Model (HMM)
1. INTRODUCTION
Face denotes a significant role in interaction, and it is also imperativetoappear
and perceive how an individual feels at a particular minute. Recognition of
appearance externally is a strategy for perceiving expression.
The face of a person has so many emotions that are
recognized and understood by looking at the face. Happy,
Fear, Sad, Disgust, Angry, Neutral, and Surprise might be
these emotions and expressions. People have misconstrued
once in a while that there is a stark difference between face
recognition and facial expression. The important thing is as
follows:
Face Recognition: This application identifies or verifies an
individual from a digital image or video. It includes
information acquisition; processing of inputs, classification
of face images and decision making. It is commonly used in
voting verification, ATM banking, mobile password, and so
on.
Facial Expression Recognition:
This recognizes any person's facial expressions using either
an image or a video clip or the person himself. It includes
face detection, extraction of features, and clincludes
classification of speech. It is commonly used in the
healthcare, games and e-learning sectors.
Woody Bledsoe[1], Helen Chan Wolf and Charles Bission[2]
mainly used facial expression recognition technology.
Together with Helen Chan and Charles Bission, Bledsoe
chipped away during 1964 and 1965 using the PC to
perceive human faces.
Applications based on "Biometric Artificial Intelligence" can
particularly differentiate a person by dissecting instances
depending on the face surface and shape of the
individual[3][4].
Facial recognition can be delegated recognition or holistic
recognition where, together with a mixing unit, Principal
Recognition involves outfit of highlight extractors or
classifiers.
Principal recognition orholistic recognitioncanbedelegated
where Principal Recognition involves a collection of
highlighted extractors or classifiers together with a mixing
unit.
Holistic Recognition-This provides the whole face a solo
contribution to the structure of recognition.
Recognition of facial expression is comprised as follows in a
few significant steps:-
1. Face detection and processing of image also known as
Image Acquisition
2. Pre-Processing
3. Feature Extraction
4. Expression Classification
IJTSRD28026

International Journal of Trend in Scientific Research and Development (IJTSRD) @ www.ijtsrd.com eISSN: 2456-6470
Figure1. FER System Flow Chart
Below is a brief introduction of the following steps involved
in Facial Expression Recognition:
(1) Image AcquisitionorFaceDetection: Theimagecan be
static image above all else, or successions of images must
contain more information than a still static image.
For Artificial ExpressionRecognition,2Dmonochrome(dark
scale) facial image successions are the most common typeof
images used. There are four methods that can efficiently
distinguish a face, such as LearningBased,FeatureInvariant,
Template Matching, and Appearance Based.
Figure2. Methods of face detection
Figure3. Various Feature Extraction Techniques
(2) Pre-Processing: It includes Signal Conditioning{suchas
elimination of noise, standardization against the variety of
pixel position or splendor, etc.} The standardization of the
image depends on eye or nostril references.
(3) Feature Extraction: It shifts to a higher-level depiction
over Pixel Data. It decreases the input space dimensionality.
Using the feature extraction techniques, it tends to be
completed. This includes a Discrete Cosine Transform[DCT]
Gabor Filter, Main Component Analysis[ PCA], Independent
Component Analysis[ LDA].
(4) Expression Classification: As examined before an
person has so many expressions at certaintimeframes anda
broad range of methods has been suggested to define those
expressions. Recognizing expressions such as happy, sad,
fear, disgust, angry, neutral, surprise. Facial expression
recognition systems do not recognize either six expressions
or the AUs more frequently than anything it requires. Wide-
ranging study on facial expression analyses has been
conducted over the past decades. The most commonly used
facial expression analysis is performed as far as the action
units suggested in the Facial Action Coding System are
concerned and as far as all inclusiveemotions areconcerned:
joy, sadness, anger, surprise, disgust and fear. The two main
categories used in facial expression recognition are activity
units (AUs)[10] and Ekman's prototypical facial
expressions[11].
2. FACIAL EXPRESSION RECOGNITION APPROACHES
Facial Action Coding System (FACS)
In 1978, the framework forestimating facial expressions was
provided by Ekman et al.[12] called the FACS–Facial Action
Coding System. FACS was developed by investigating the
relationships between contraction of muscle(s)andchanges
in their face appearance. The Face can be divided into Upper
Face and Lower Face Action units[13] and the resulting
appearances are recognized as well. The figures show a part
of the activity units that are joined. Muscle constraints
responsible for a comparable activityaredistinguishedas an
Action Unit (AU).Theundertaking ofexpression examination
using FACS is dependent on disintegrating observed
expression into the action unit structure. There are 46 AUs
that talk to outward appearance modifications and 12 AUs
connected with the direction and direction of the eye stare.
Activity units are deeply engaging as far as facial
developments are concerned; in any event, they do not
provide any information on the message to which they are
speaking.
Prototypical Facial Expression
A usually small subset of seven important categories of
expressions, observed through FER frameworks to be
noticeable cross-sectionallyovercultureforuse.Asstatedby
the hypothesis of the Ekman[14], there are six vital
expressions of emotions that are all inclusive to peoplefrom
various nations and societies. They are anger, neutral,
disgust, fear, happy, sad and surprise..
Rather than depicting the detailed facial features most facial
expression recognition system attempts to perceive a small
arrangement of prototypical passionate expressions. Some
facial expressions are a mixture of morethanone expression
as for example of fear, sorrow and disgust, they state a
combination that occurs. A few methodologies were used to
overcome the above problem. There are two main classes of
feature classification strategy: forinstance,statisticalnon-AI
approach, Euclidean and direct segregation research[15].
Machine learning approaches, for instance, Feed Forward
Neural Network [16], Hidden Markov Model[17], Multilayer
Perception, Support Vector Machine[18], and so on.
There are two categories that can be divided into current
methods: image-based strategies and model-based
strategies.
Picture-based methodologies that focus on perceiving facial
operations by observing changes in the facial appearance of

the officer more frequently than doing whatever it takes not
to autonomously and statically organize behavior or AUs
More often than not, this kind ofapproachincludes twomain
phases. To begin with, different facial highlights, e.g., optical
stream [20][21], unambiguous element estimation (e.g.,
wrinkle length and educational level)[22], Haar
highlights[23], Local Binary Patterns (LBP)
highlights[24][25] autonomous partinvestigation(ICA)[26],
include focuses[27], Gabor wavelets[28] and so on. The
expressions/ AUs are acknowledgedby recognitionsystems,
such as Neural Networks, Support Vector Machines (SVM),
rule-based methodology, AdaBoost classifiers, Sparse
Representation (SR) classifiers,andsoon,giventheseparate
facial highlights. The ordinary shortcoming of picture-based
approaches for AU recognition is that they will generally
legitimately view each AU or certain AUmixindividually and
statically from the image data, regardless of the semantic
and dynamic links between AUs, although some of them
explore the transient characteristics of facial highlights.
By using the links between AUs, model-based approaches
overcome this deficiency and perceive the AUs at the same
moment. Lien et al.[29] used many Hidden Markov Models
(HMMs) to talk in time about the growth of facialoperations.
Classification is accomplished by selecting the AU or AU
blend that amplifies the likelihood of the separate facial
features generated by the HMM. Valstar et al.[30] used a
mixture of SVMs and HMMs and flanked the SVM methodfor
almost every AU by showingthetemporaryprogressoffacial
activity. The two techniques misuse AU's worldly
circumstances. As it may be, they suffer failure to exploit
AU's spatial circumstances. The solution for this problem is:
Tong and Ji used a Dynamic Bayesian scheme to show the
spatiotemporal associations between AUs effectively and to
achieve remarkable improvements over the picture-based
method. In this paper, apart from showing the spatial and
worldly connections between AUs, we also use the attitude
and facial element focus information and,morecritically,the
coupling and associations between them. Expression
Classification by classifiers should be feasible. It includes
hidden MarkovModel [HMM],NeuralNetwork[NN],Support
Vector Machine SVM, AdaBoost,SpareRepresentation[SRC].
3. COMPARATIVE ANALYSIS
Similar investigation of the above mentioned facial
expression recognition methodologies is shown in the table
in this section. These methodologies are evaluated for
standard facial expression databases such as Japanese
woman outward appearance (JAFFE), FERET as far as the
particular system's recognition rate, advantages and faults
are concerned.
Table1. Facial Expression Recognition approaches
Comparison
Recognition
Approach
Database
Recognition
Rate
LBP JAFFE 80%
ICA FERET 89%
PCA AR-Faces 70%
PCA+ Gabor JAFFE 85%
PCA JAFFE 70%
LDP JAFFE 89%
LTP JAFFE 89%
Figure4. Graphical comparison of various facial expression
techniques applied on standard face databases such as
JAFFE, FERET etc.
Figure5. Comparison of facial expression recognition
methods
4. FUTURE DIRECTIONS
Facial expression recognition these days achieves a
important place in various areas as it operates effectively
under the circumstances that are required. A great deal of
studies has been achieved and is going on in facial
expression recognition, yet there is a need for progress,
enhancement and development at the same moment.
Considering the above examined outward appearance
recognition techniques, which show better results in static
conditions but failures under shifting conditions, e.g.change
in modernity, variety present, maturing element and
expressions. These are major causes thataffectthedisplayof
almost all usual facial expression recognition techniques.. In
this manner, future work should be feasible to overcome
these problems and interpret the emotions of the individual
"Environment in real-time under minimumconstraints." For
instance, enhancement and high objectives, there are two
promising points for expression identification in the future.
Both should be able to construct the recognition rate in
recognition of facial expression.
5. CONCLUSION
Recognition of facial expression is an exceedingly best
assignment in the field of PC vision, which has achieved
significance as a result of its various applications in the last
few years. Many specialists have worked thoroughly to
demonstrate that accurate recognition system for facial
expression is mandatory. This paper offers guidance for
different applications. For the growth and advancement of
the new methodology, numerous experts have worked
carefully to demonstrate accurate facial expression
identification analysis. Key focuses, demerits and
applications of the few techniques have been substantially
evaluated in this paper. In addition, a closeinvestigation was
carried out to delineate the exhibition and accuracy of

various methodologies. This paper finally concludes by
recommending to the scientist the possible instructions for
enhancing the facial expression identification system
exhibition.
REFERENCES
[1] Banu, Danciu, Boboc, Moga, Balan; “A novel approach
for face expression recognition”, IEEE 10th Jubilee
International Symposium on Intelligent Systems and
Informatics 2012.
[2] Wang Zhen, Ying Zilu; “Facial expression recognition
based on adaptive local binary pattern and sparse
representation”, 2012 IEEE.
[3] Shashi Kant Sharma, Maitreyee Dutta and Kota
Solomon Raju, “Comparative Analysis of Face
Recognition using Extended LBP and PCA”,
International Journal of Engineering Science &
Technology Vol. 8, Issue 10, ISSN: 0975-5462, 2017.
[4] Shashi Kant Sharma, Maitreyee Dutta and Kota
Solomon Raju, “Comparative Study of Efficient Face
Recognition Methods: A Literature Survey”, in
Proceedings of International Conference on
Communication, Computing and Networking (ICCCN),
Vol. 2, ISBN: 978-8-193-38970-6, 2017.
[5] Jizheng, Xia, Lijang, Yuli, Angelo; “Facial expression
recognition considering differences in facial structure
and texture”, IET Computer Vision 2013.
[6] Jiawei, Congting, Hongyun, Zilu; “Facial expression
recognition based on completed local binary pattern
and sparse representation”, Ninth International
Conference on Natural Computation (ICNC) 2013.
[7] Jizheng, Xia, Yuli, Angolo; “Facial expression
recognition based on t-SNE and adaboost M2”, IEEE
International Conference on Green Computing and
Communications and IEEE Internet of Thingsand IEEE
Cyber, Physical and Social Computing 2013.
[8] J.J. Lee, Md. Zia Uddin,T.S. Kim; “Spatiotemporalhuman
facial expression recognition using fisher independent
component analysis and hidden markov model”, 30th
Annual International IEEE EMBS Conference 2008.
[9] Weifeng, Caifeng, Yanjiang; “facial expression
recognition based on discriminativedistancelearning”,
21st International Conference on Pattern Recognition
(ICPR 2012).
[10] Ying, Zhang; “facial expression recognition based on
NMF and SVM”, International Forum on Information
Technology and Applications 2009.
[11] Anagha, Dr. Kulkarnki; “Facial detection and facial
expression recognition system”, International
Conference on Electronics and CommunicationSystem
(ICECS -2014).
[12] G. Donato, M. S. Bartlett, J. C. Hager, P. Ekman, T. J.
Sejnowski, "Classifying Facial Actions", IEEE Trans.
Pattern Analysis and Machine Intelligence, Vol. 21, No.
10, pp. 974-989, 1999
[13] Shashi Kant Sharma, Kota Solomon Raju, “Application
of Gaussian Filter with Principal Component Analysis
Algorithm for the Efficient Face Recognition”,
International Journal of Electronics and
Communication Engineering & Technology (IJECET),
ISSN 0976 – 6464(Print), ISSN 0976 – 6472(Online),
Volume 4, Issue 7 (2013) © IAEME, 2013.
[14] Solomon Raju Kota, J. L. Raheja, Archana Rathi, Shashi
Kant Sharma, “Principal Component Analysis for
Gesture Recognition using SystemC”, International
Conference on Advances in Recent Technologies in
Communication and Computing© 2009 IEEE DOI
10.1109/ARTCom.2009.177, 2009.
[15] Mahesh Kumar Sharma, ShashikantSharma,Nopbhorn
Leeprechanon, and Aashish Ranjan, “Wavelet
decomposition based principal component analysisfor
face recognition using MATLAB”, Advancement in
Science and Technology AIP Conf. Proc. 1715, 020055-
1–020055-10; doi: 10.1063/1.4942737, 2015.
[16] Wiskott, L., Fellous, J. M., Kuiger, N. and Von M. Pantic,
L. J. M. Rothkrantz, "Automatic Analysis of Facial
Expressions: the State of the Art", IEEE Trans. Pattern
Analysis and Machine Intelligence, Vol. 22, No. 12, pp.
1424-1445, 2000
[17] C. P. Sumathi, T. Santhanam and M. Mahadevi,”
Automatic Facial Expression Analysis A survey”,
International Journal of Computer Science &
Engineering Survey (IJCSES) Vol.3, No.6,
December2012
[18] P. Ekman, Emotion in the Human Face. Cambridge
Univ. Press,1982
[19] S. M. Lajevardi, Z. M. Hussain, Novel higher-order local
autocorrelationlikefeatureextraction methodology for
facial expression recognition, IET Image Processing,
2009.
[20] Kuan-Chieh Huang, Sheng-Yu Huang and Yau-Hwang
Kuo, Emotion Recognition Based on a Novel Triangular
Facial Feature Extraction Method, IEEE 2010.
[21] Ruicong Zhi, Markus Flierl, Qiuqi RuanandW.Bastiaan
Kleijn, Graph-Preserving Sparse Nonnegative Marix
Factor with Application to Facial Expression
Recognition, IEEE Transactions on systems, man and
cybernetics partB, Vol. 41, NO. 1 2011
[22] I. Kotsia, I. Buciu and I. Pitas, “An Analysis of Facial
Expression Recognition under Partial Facial Image
Occlusion,” Image and Vision Computing, vol. 26, no. 7,
pp. 1052-1067, July 2008.
[23] Petar S. Aleksic, Member IEEE. Aggelos K. Katsaggelos,
Fellow Member, IEEE,” Animation Parameters and
Multistream HMM’s, IEEE Transactions onInformation
Forensics and Security”, Vol.1 No.1 March 2006.
[24] J. J. Lien, T. Kanade, J. F. Cohn, and C. Li, “Detection,
tracking, and classification of action units in facial
expression,” J. Robot. Auto. Syst., vol. 31, no.3,pp.131–
146, 2000.
[25] G. Donato, M. S. Bartlett, J. C. Hager, P. Ekman, and T. J.
Sejnowski, “Classifying facial actions,” IEEE Trans.
Pattern Anal. Mach. Intell., vol. 21, no. 10, pp. 974–989,
Oct. 1999.
[26] Y. Tian, T. Kanade, and J. F. Cohn, “Recognizing action
units for facial expression analysis,” IEEE Trans.
Pattern Anal. Mach. Intell., vol. 23, no. 2, pp. 97–115,
Feb. 2001.

[27] R. Lienhart and J. Maydt, “An extended set of Haar-like
features for rapid object detection,” in Proc. IEEE Int.
Conf. Image Process., vol. 1.Sep. 2002, pp. 900–903.
[28] C. Shan, S. Gong, and P. W. McOwan, “Facial expression
recognition based on local binary patterns: A
comprehensive study,” Image Vis. Comput., vol. 27, no.
6, pp. 803–816, 2009.
[29] G. Zhao and M. Pietikainen, “Dynamic texture
recognition using local binary patterns with an
application to facial expressions,” IEEE Trans. Pattern
Anal. Mach. Intell., vol. 29, no. 6, pp. 915–928, Jun.
2007.
[30] B. A. Draper, K. Baek, M. S. Bartlett, and J. R. Beveridge,
“Recognizing faces with PCA and ICA,” Comput. Vis.
Image Understand., vol. 91, nos. 1–2, pp. 115–137,
2003.
[31] M. Valstar and M. Pantic, “Fully automatic recognition
of the temporal phases of facial actions,” IEEE Trans.
Syst., Man, Cybern. B, Cybern., vol. 42, no. 1, pp. 28–43,
Feb. 2012.
[32] M. S. Bartlett, G. Littlewort, M. G. Frank, C. Lainscsek, I.
Fasel, and J. R. Movellan, “Recognizing facial
expression: Machine learning and application to
spontaneous behavior,” in Proc. IEEE Conf. Comput.
Vis. Pattern Recognit., vol. 2. Jun. 2005, pp. 568– 573.
[33] J. J. Lien, T. Kanade, J. F. Cohn, and C. Li, “Detection,
tracking, and classification of action units in facial
expression,” J. Robot. Auto. Syst., vol. 31, no.3,pp.131–
146, 2000.
[34] M. Valstar and M. Pantic, “Combined support vector
machines and hidden Markov models for modeling
facial action temporal dynamics,” in Proc. IEEE Int.
Conf. Human-Comput. Interact., vol. 4796. Oct. 2007,
pp. 118–127.

Efficient Face Expression Recognition Methods FER A Literature Review

More Related Content

Similar to Efficient Face Expression Recognition Methods FER A Literature Review (20)

More from ijtsrd (20)

Recently uploaded (20)

Efficient Face Expression Recognition Methods FER A Literature Review