Different Viewpoints of Recognizing Fleeting Facial Expressions with DWT

International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395-0056
Volume: 04 Issue: 10 | Oct -2017 www.irjet.net p-ISSN: 2395-0072
© 2017, IRJET | Impact Factor value: 6.171 | ISO 9001:2008 Certified Journal | Page 1199
Different Viewpoints of Recognizing Fleeting Facial Expressions with
DWT
VAIBHAV SHUBHAM1, MR. SANJEEV SHRIVASTAVA2, DR. MOHIT GANGWAR3
---------------------------------------------------------------------***---------------------------------------------------------------------
ABSTRACT: Most research of facial expression recognition
used static, front view and long-lasting stimuli of expressions.
A paucity of research exists concerning recognition of the
fleeting expressions from different viewpoints. To investigate
how duration and viewpoints together influence the
expression recognition, we employed expressions with two
different viewpoints (three-quarters and profile views) and
showed them to the participants transiently. The duration of
expressions was one of the following: 20, 40, 80, 120, 160, 200,
240, or 280 ms. In experiment 1, we used static facial
expressions; In experiment 2 we added dynamic information
by adding two neutral expressions before and after the
emotional expressions. The results showed an interaction
effect between viewpoint and duration on expression
recognition. Furthermore, we found that happiness is the
easiest expression to recognize even under the conditions of
fleeting presentation and side-view. This study informed the
automatic expression recognition of human data under
conditions of short duration and different viewpoints.
Keywords: Facial Expression, CurveletTransform,MATLAB
1. Introduction
From the evolutionary perspective, facial expressions can
convey much information about feelings (angry or happy)
and intentions (hostile or friendly) of strangers we
encounter. The accurate interpretation of emotional
expression, therefore, can promote both survival and
reproduction. Today, facial expressionsstill playanessential
role in human communication. Some of the most influential
authors in the field of expressions recognitionindicatedlong
ago that recognizingfacial expressionswasofimportance for
many practical applications, such as emotion analysis[1],
deception detection,human-computerinteraction,andsoon.
There is considerable research in psychology and computer
science about recognizing facial expressions. Dynamic
surface acknowledgment can be seen as a speculation of
appearance based methodologies [2]. At the end of the day,
notwithstanding the above exhibited approach dynamic
composition based methodology can be respected for
outward appearanceexamination.Outwardappearancescan
be considered as a dynamic composition as a result of face
muscles action is powerful.Alongtheselines,theappearance
and movements of dynamic composition can be considered
in two headings, it implies that data of spatial and fleeting
areas is joined together. Saha et al. [3] were perceived
outward appearances by mix of curvelet change and
neighborhood parallel examples. Additionally, they utilized
curvelet entropy for characterizing facial expressions.
Though, they just probed the still picture and performed on
JAFFE databases and last picture of picture groupings of
Cohn-Kanade databases. Juxiang et al. [4]
An efficient emotion recognition system is essential need in
the area of HCI (Human Computer Interaction).Ifcomputers
are able to recognize (perceive and respond) human
emotions efficiently then we can make more interactive,
trustworthy and easy to use systems. It means that we can
promote multimedia and interactive systems more
efficiently if computer sympathetic of human emotion is
robust and efficient. Computer vision systemcannot obey all
the facts of the human vision system it means computer
vision system may not work as human visionsystem,butitis
vital need to make out and investigate the real complication
behind its supremacy, reliability and elasticity. Computer
vision can be explained as the acquirement and analysis of
visual information to get desired information for
understanding a picture or controlling and responding an
action (activity).Moderncomputervisionresearchconcerns,
not only understanding and analysing the course of vision,
but also designing efficient vision systems for a range of
existent world purpose. Human expression recognition
system is an example of machine vision system. Outward
appearances is super class of feelings,itimpliesfeelingsgoes
under the classification of outward appearances.
Figure: 1: Six Universal Emotions and a neutral
Facial expression is broad class and emotion is the subclass
of facial expression, because facial expressionincludessome
cognitive process like thinking, boredom, drowsinessandso
on, but Psychological researches have shown that only six
central emotions are collectively coupled with different

facial expressions: sadness, anger, surprise, fear, happiness,
and disgust.
This thesis work has mainly addressed to recognition of
these 6 central emotions but it can be customized to some
added facial expression when needed. These six central
emotions are shown in the figure 1.
2. Literature Review
The Automatic Face Recognition (AFR) system is a complex
system, because in most of the cases only a few face images
are available for training the system and different problems
arise when training and test images are obtained under
different conditions. The researchers have been facing
problems with the sensibility of theclassifiertoillumination,
pose, facial expression, occlusion, and low resolution face
recognition. Therefore, recent works on face recognitionare
classified based on their main contribution in finding
solution to some of the above mentioned problems.
However, only little work has been done in literature on low
resolution face recognitionandpartial facerecognition.After
going through literature related to different face recognition
techniques and challenges .In this work, a novel face
recognition model is proposed to work well with both low
and high resolution face images, using hybrid approach and
multi-scaling of facial components concept for facial feature
extraction, PCA or LDA for dimensional reduction and
artificial neural network as classifier. In this chapter
literature survey of different face recognition methods
(Challenges in Face Recognition, Holistic Face Recognition
Methods, Component based Face Recognition Methods,
Hybrid Face Recognition Methods, Low Resolution Face
Recognition and Partial Face Recognition) and formulation
of problem for the proposed work is explained.
NorhaizaBtYa Abdullah et al., “Folder Lock by using
Multimodal Biometric: Fingerprint and Signature
Authentication”, Fourth International Conference on
Cyber Security, Cyber Warfare, and Digital Forensic
2015, lock folder is one of method that used to ensure
nobody intentionally gets access to your private and
confidential information. Presently used password based
systems have a number of associated inconveniences and
problems such as user needs to remember passwords,
passwords can be guessed or broken down via brute force
and also there is problem of non-repudiation.. Besides,
password authentication method as a keyword permission
to access something is breakable. Hence, it can be leakedout
and cracked by using any methods such as dictionaryattack,
or social engineering. Due to the drawback, this method is
lack of universality of some characteristics and the
recognition performance of the systems is upper limit and it
is unacceptable error rates for the single modal
authentication system. Multimodal biometric can be at least
combination of two types of any physical or behavioral
biometric as it applies in the system that has been
developed. Therefore, a system is proposed to overcomethe
aforementioned problems by adding multimodal biometric
authentication will provide another layer of security. Those
problems encountered have beingovercomeanditisproven
that by adding another layer ofsecurityastheauthentication
is more secure. It has been proved and has been tested that
using combination of two biometric methods; fingerprint
and signature as an authentication method is more secure
and reliable.
NakisaAbounasr et al., “Facial Expression Recognition
Based on Combination of Spatial-temporal and Spectral
Features in Local Facial Regions”, 2013 8th Iranian
Conference on Machine Vision and Image Processing
(MVIP),this paper presents two new methodologies for
outward appearance acknowledgment in light of
computerized curvelet change and neighborhood double
examples from three orthogonal planes (LBP-TOP) for both
still picture and picture groupings. The components are
separated by utilizing the computerized curvelet change on
facial locales in still picture. In this approach, some sub-
groups compare to edge of facial localeisutilized.Thesesub-
groups comprise of more recurrencedata.Thecomputerized
curvelet coefficients and LBPTOP are spoken to consolidate
spatio-worldly and unearthly components for picture
arrangements. The got results by our proposed approaches
on the Cohn-Kanade outward appearance database have
adequate acknowledgment rates of 91.90% and 88.38% for
still picture and picture arrangements, individually.
3. METHOD/ APPROACHES FOR FACE DETECTION
In general, FD can be implemented by four methods:
knowledge based methods, template matching, invariant
feature methods and learning based methods. These
methods are as follows [8]:
Knowledge based methods: The models use human
knowledge to find face patterns from the testing images.
Based on the nature of human faces, algorithms scan the
image from top-to-bottom and left-to-right in order to find
facial feature. For instance, faceshouldbeincludingtwoeyes
and mouth etc.
o Pros: Easily applicable in simple rules
o Cons: Difficult to detect in invariantbackground,such
as different pose, uncontrolled illumination etc. Well
results based on well-defined rules. This algorithm
does not work on the pose.
Template Marching: The method uses several templatesto
find out the face class and extract facial features. Rules are
pre-defined and decide whether there is face in the image.

Figure: 3.1: Sample of template marching
o Pros: Simple to apply.
o Cons: Similar to knowledge based method, hard to
detect face in different poses. Algorithms are
sensitive to scale size, face shape and pose.
Invariant feature methods: The model is bottom-up
approach and is used to find a facial feature (eyebrows,
nose), even in the presence of composition,perspectivevary,
and so it is difficult to find a face in real time using this
method. Statistical models are developed to determine the
faces. Facial features of humanfacesare:shape,texture,skin.
o Pros: Unlike knowledge-based method, it is
invariant to pose and expression.
o Cons:1. Not suitable to detect facial features from
uncontrolled background
2. Time consuming algorithms
3. Detection rate is not accuracy, because of need to
combine
differentfeature and processing it.
Learning based methods: The models are trained from a
set of training set before detection. For the large amount of
training data, high accuracy recognition rate to resist
variation, expression and pose of faces images can be
achieved. For instance, many of “non-face”and“face”images
import into the system. Machine learning techniques are
employed to train the system based on the statistical
properties and probability distribution function. Principle
Component Analysis (PCA), Support Vector Machine (SVM),
Naïve Bayes Classifier, Hidden Markov model, Neural
Network and Adaboost are well-known classifiers to use for
face detection.
o Pros: Fast to detect face. Can detect different pose
and orientation if have enough training set. Show
good empirical results.
o Cons: Need more and more “non-face” and “face”
sample for training, need to scan different scale.
4. APPROACH FOR BIOMETRIC FACE AND
FINGERPRINT RECOGNITION
Anbiometric face and fingerprint recognition system has
four components, namely Face-detection, Features
extraction from face, Classification and morphological. Flow
chart for proposed system is shown in the figure given
below.
Figure 4.1 (a): Proposed Biometric Face and Fingerprint
Recognition System
Figure 4.1 (b): Flow Chart of Biometric Face and
Fingerprint Recognition System
Face
Detection
Component
Features
Extraction
Componen
t
Classificati
on
Componen
t
Facial Image
The Voila-Jones Face
Detection Method
Discrete Wavelet
Transform
PCA (Principle
Component Analysis)
Classification using KNN
Emotion (Happy,
Disgust, Sad, Fear,
Surprise, Anger,
Neutral)
Morphological
Operation for
Fingerprint

We have concentrated our implementation on multimodal
biometric face and fingerprint recognition. Multimodal
biometrics system for identity verification using two traits:
face and fingerprint System based on adaptive principal
component analysis and multilayer perception A proposed
scheme of multimodal biometric face and fingerprint
recognition is parallel the multimodal biometric takes the
individual scores of two traits (face and fingerprint) which
generate range approximate value for training that is in
discrete interval form than system will produce good
accurate result with high efficiency. Currentwork dealswith
an efficient face and fingerprint recognition algorithm
combining ridge based and Eigen face approach for parallel
execution. Here I am proposing a method to overcome the
drawback of earlier problem, which based on combination
on neural network an efficient Face and Fingerprint
recognition algorithm combining ridge based andEigenface
approach. The main purpose of the proposed system is to
reduce the error rate as low as possible and improve the
performance of the systembyachievinggoodacceptable rate
during identification and authentication.
Database Browser: We combine the biometric traits taken
from different sensors to form a composite biometric trait
and process. Here an image of an object or a scene is
captured by a digital camera or is scanned for use as the
input to the system.
Feature level: Signal coming from different biometric
channels are first pre-processed, and feature vectors are
extracted separately, using specific algorithm and we
combine these vectors to form a composite feature vector.
This is useful in classification. These are a series of steps
which should be taken for making an image suitable for
manipulation and interpretation by subsequent stages. The
steps include removal of noise and variation of intensity
recorded, sharpening, improving the contrast and stringing
the texture of the image. Another important aspect is image
restoration which extracts image information from a
degraded form to make it suitableforsubsequentprocessing
and interpretation.
5. Simulation Tools
In the below figure we show the starting screen of MATLAB.
We use MATLAB because it is user friendly and it contains a
number of libraries which is used in the work.
Figure 3: MATLAB Starting Window
This is the screen shot of our application; in this GUI we
show the testing and training module.
Figure 4: GUI of The Application
6. Simulation
In this section we expose and discuss the results of FER
system which is based on proposed methodology for FER.
We perform two experiments on proposed FER system with
different number of training and testing images of JAFFE
dataset. In first experiment out of 213 images, 143 images
(on average of 2 images per expression per subject) for
training purpose and rest 70 images (on average of 1 image

per expression per subject)fortestingpurpose.Resultof this
experiment for 7 expressions is shown in the table below:
Table: 4.1 Result of Experiment FER System
Anger
Disg
ust
Fe
ar
Hap
py
Neut
ral
Sa
d
Surpris
e
Ange
r
17 2 0 0 0 1 0
Disgu
st
2 17 1 0 0 0 0
Fear 0 1 16 0 0 1 3
Happ
y
0 0 0 18 2 1 0
Neut
ral
0 0 0 1 19 0 0
Sad
0 0 3 1 2 1
5
0
Surpr
ise
0 0 0 0 0 0 20
Figure 5: Accuracy of each expression for experiment
7. Conclusion
Our proposed FER system performed better than existing
FER systems on JAFFE database. But limitation of our
method is that when we test our system on any subject
which is not the part of training than it gives very poor
results. So we can say that our method for facial expression
recognition is person dependent. Itmeansbeforetestingany
subject (person) we must have to train our system on that
subject. This is due to we take mean of discrete coefficients
of all training images for performing PCA (principle
component analysis) for feature selection and same mean is
used by testing images.
8. REFERENCES
[1] B. Fasel and J. Luettin, “Automatic facial expression
analysis: A survey,” Pattern Recognition, vol.36,pp.
259–275, 1999.
[2] C. Shan, S. Gong, and P.W. McOwan, “Facial
expression recognition based on local binary
patterns: A comprehensivestudy,”ImageandVision
Computing, vol. 27(4), pp. 803–816, 2008.
[3] Y. Tian, T. Kanade, and J. Cohn, Handbook of Face
Recognition, chapter 11, Springer, 2005.
[4] M.S. Bartlett, G. Littlewort, M. Frank, C. Lainscsek, I.
Fasel, and J. Movellan, “Recognizing facial
expression: Machine learning and application to
spotaneous behavior,” in IEEE Computer Society
Conference on Computer Vision and Pattern
Recognition, 2005, vol. 2, pp. 568–573.
[5] M.J. Lyons, J. Budynek, and S. Akamatsu, “Automatic
classification of single facial images,” IEEE
Transactions on Pattern Analysis and Machine
Intelligence, vol. 21(12), pp. 1357–1362, 1999.
[6] T. Mandal, A. Majumdar, and Q.M.J. Wu, “Face
recognition by curvelet basedfeatureextraction,”in
International Conference on Image Analysis and
Recognition, LNCS, 2007, vol. 4633, pp. 806–817.
[7] A.A. Mohammed, R. Minhas, Q.M.J. Wu, andM.A.Sid-
Ahmed, “A novel technique for human face
recognition using nonlinear curvelet feature
subspace,” in International Conference on Image
Analysis and Recognition, LNCS,2009,vol.5627,pp.
512–521.
[8] T. Ojala, M. Pietikainen, and D. Harwood, “A
comparative study of tex- ¨ ture measures with
classification based on featured distribution,”
Pattern Recognition, vol. 29 (1), pp. 51–59, 1996.
[9] S. Liao, W. Fan, C.S. Chung, and D.Y. Yeung, “Facial
expression recognition using advanced local binary
patterns, tsallis entropies and global appearance
features,” in IEEE International Conference on
Image Processing, 2006, pp. 665–668.
[10] M.J. Lyons, S. Akamatsu, M. Kamachi, and J.
Goba, “Coding facial expressions with gabor
wavelets,” in IEEE International Conference on
Automatic Face and Gesture Recognition, 1998, pp.
200–205.

Different Viewpoints of Recognizing Fleeting Facial Expressions with DWT

More Related Content

What's hot (18)

Similar to Different Viewpoints of Recognizing Fleeting Facial Expressions with DWT (20)

More from IRJET Journal (20)

Recently uploaded (20)

Different Viewpoints of Recognizing Fleeting Facial Expressions with DWT