Facial image classification and searching –a survey

International Journal of Information Technology, Modeling and Computing (IJITMC) Vol. 2, No.2, May 2014
DOI : 10.5121/ijitmc.2014.2203 19
FACIAL IMAGE CLASSIFICATION AND
SEARCHING –A SURVEY
Dr.S.Vijayarani1
and Mrs.M.Vinupriya2
1 Assistant Professor, Department of Computer Science, School of Computer
Science and Engineering, Bharathiar University, Coimbatore -641 046, Tamil
Nadu
2 Research Scholar, Department of Computer Science, School of Computer
Science and Engineering, Bharathiar University, Coimbatore - 641 046,
Tamil Nadu
ABSTRACT
Recent developments in the area of image mining have shown the way for incredible growth in
extensively large and detailed image databases. The images which are available in these
databases, if checked, can endow with valuable information to the human users. As one of the
most successful applications of image analysis and understanding, face recognition has
recently gained important attention particularly throughout the past many years. Though
tracking and recognizing face objects is a routine task, building such a system is still an active
research. Among several proposed face recognition schemes, shape based approaches are
possibly the most promising ones. This paper provides an overview of various
classification and retrieval methods that were proposed earlier in literature. Also, this paper
provides a marginal summary for future research and enhancements in face detection.
KEYWORDS
Image mining, face detection, classification, retrieval, SVM technique, searching.
1. INTRODUCTION
Data mining is a process which uses a mixture of data analysis tools to discover
patterns and relationships in data that may be used to make valid predictions. It permits
users to analyze data from many different dimensions or angles, classify it, and
summarize the relationships identified. Technically, data mining is the method of
identifying correlations or patterns among dozens of fields in massive relative
databases. Some of the research areas in data mining are web mining, text mining, data
streams, image mining, sequence mining and multimedia mining. Image mining
facilitates the abstraction of hidden information which is not clearly accrued in the
image. It is used to detect unfamiliar patterns and abstract inherent and useful data
from images stored in the large databases. Therefore image mining deals with making
relationships between different images from large image databases. A large amount of
characteristics used for image category are: shape, texture, color and spatial features.

20
Image mining is not only a limb of data mining and knowledge discovery, but also an
highly developed research area which includes digital image processing, image
indexing and retrieval, image understanding, database, artificial intelligence, pattern
discovery and so on[7].
Research in image mining can be broadly classified in two main directions [6] (1)
Domain- specific applications where the focus is in the process of extracting the most
applicable image features into a form appropriate for data mining. (2) General
applications where the focus is to create the image patterns that may be helpful in the
understanding of the communication
between high-level human perceptions of image and low level image features. Both are
used to extract most relevant image feature and later to generate image patterns. A
number of image mining systems have been developed for different applications which
include remote sensing, natural scene recognition, weather forecasting, egeria
detection, criminal investigation, image segmentation, medical diagnosis, space
research, biology, image classification and retrieval etc.
Face Recognition System is a computer application for automatically identifying or
verifying a person from a digital image or a single frame from a video source.
This can be done by comparing selected facial characteristics of the likeness and a
facial database. It executes that by comparing the face of the accessing user with a
database of faces previously stored in memory. Face recognition presents a challenging
problem in the area of image analysis and computer vision, and it has received a
great deal of interest over the last few years because of its applications in
different domains. There are many problems that exist due to many factors that can
affect the images. When processing images one must take into account the variations
in lightweight, image quality, the individuals pose and facial expressions along with
others.
The rest of the paper is organized as follows. Section 2 describes the different
approaches of image based face recognition and ITS applications. Section 3 discusses
about the importance of classification and searching in facial images. Section 4
provides a literature survey. Section 5 shows various research directions involved in the
face recognition and conclusion is given in Section 6.
2. DIFFERENT APPROACHES AND APPLICATIONS IN
FACE RECOGNITION
Different types of facial images are taken for face detection. The images that are
collected in a semi-controlled environment are used as input. Images were taken in
uncontrolled indoor environment and different facial expressions (such as open / closed
eyes, smiling / not smiling) or configurations (such as w/glasses, center-light, happy,
left-light, normal, right-light, w/no glasses, sad, sleepy, surprised, and wink). The
images are taken by varying the lighting and facial details (glasses / no glasses) at
different times and are used as input.

21
2.1. Different approaches
There are numerous ways to detect a face in a scene and it is easier and harder ones.
Here is a list of the most familiar approaches in face detection:
 Identifying faces in images with prohibited background
 Identifying faces by motion
 Using a combination of the above
 Identifying faces in unconstrained scenes
Figure 1: Taxonomy of Iamge based Face Recognition Approaches
The existing techniques are reviewed to detect faces from a single intensity or color
image. Single image detection methods are classified into four categories [11],
1. Knowledge-based methods. These rule-based procedures encode human
information of what constitutes a distinctive face. Usually, the rules confine the
relationships between facial features. These procedures are designed mainly for face
localization. Knowledge based methods can be studied by two different approaches;
which are top-down methods and bottom-up methods.
2. Feature invariant approaches. These algorithms intend to find structural
features that live even when the pose, viewpoint, or lighting conditions differ, and then
use these to find faces. These methods are designed mainly for face localization.
3. Template matching methods. There are number of standard patterns of a face
are stored which is used to describe the face as a whole or the facial features separately.
The associations between an input image and the stored patterns are worked out for
detection. These methods have been applied for both face localization and detection.
Template matching researches can be divided into two subcategories which are,
researches using predefined templates and the other is researches using deformable
templates.
4. Appearance-based methods. In difference to template matching, the
models (or templates) are first learned from a set of training images which should
capture the representative variability of facial appearance. These learned models are

22
then utilised for detection. These methods are designed largely for face detection.
There are number of approaches using the appearance based methods are,
eigenfaces, distribution-based methods, neural networks, support vector machines,
sparse network of winnows, Naïve Bayes classifiers, hidden Markov model ,
information theoretical approach and inductive learning.
2.2APPLICATIONS
Face recognition is utilised for two primary jobs:
1. Verification (one-to-one matching): When offered with a face image of an
unidentified individual along with a claim of individuality, ascertaining whether the
individual is who he/she claims to be.
2. Identification (one-to-many matching): When a system offered with an image
of an unidentified individual, determining that person’s identity by matching
(possibly after encoding) that image with a database of (possibly encoded) images of
identified individuals.
There are several application areas in which face recognition can be exploited for these
two reasons, a few of which are outlined below.
 Security: It is mostly utilised for accessing controls to airports/seaports,
buildings, airports/seaports, ATM machines and border checkpoints [8];
computer/ network security [4]; email authentication on multimedia
workstations.
 Surveillance: It can be seen with a large number of CCTVs can be examined to
look for identified criminals,drug offenders, etc. and authorities can be notified
when one is to be found; for instance, this procedure was used at the Super
Bowl 2001 game at Tampa, Florida [3];
 General identity verification is done for electoral registration, electronic
commerce, banking, identifying newborns, passports, national IDs, drivers’
licenses, workers IDs, etc.
 Criminal justice systems such as mug-shot/booking systems, post-event
investigation, forensics.
 Image database investigations: used for searching image databases of licensed
drivers, promote recipients, immigrants, missing children and police bookings.
 “Smart Card” applications: In the maintenance of facial images database, the
face-print
can be stored in a bar code, smart card or magnetic stripe, authentication of which is
achieved by matching the live image and the retained template [18].

23
 Multi-media environments with adaptive individual computer interfaces
a part of ubiquitous or context aware systems, behavior examining at
childcare or old people’s centers, identifying a customer and assessing his
needs [16, 8].
3. FACIAL IMAGE CLASSIFICATION AND SEARCHING
Image classification intends to find a description that can best describe the images in
one class and to distinguish these images from all the other classes. It is one of the most
often used methods for extracting information from images. Classification involves verdict
rules that partition the data into disjoint groups. Training data set is the input for the
classification data set, whose class labels are already known. In Classification, usually
several features are used for a set of pixels i.e., many images of a particular object are
needed. Image search engines are currently dependent on textual metadata. This
information can be in the form of filenames, images, manual annotations, or surrounding
text. However, for the huge bulk of images on the internet and in peoples’ private
collections, this data is often confusing, erroneous, or simply not present. This
presents a great opportunity to use attribute classiﬁers on images with faces, thereby
making them searchable. An image retrieval system is a computer system which is used for
searching, browsing and retrieving images from large set of databases. Many image retrieval
systems both business and research have been built. A large amount of image retrieval
systems support one or more of the following choices:
 Random browsing
 Searching through example
 Searching through sketch
 Searching through text (including key word or speech)
 Steering with modified image categories.
4. RELATED WORKS
Several researches have been carried on this face detection. This section presents a study on
various classification and retrieval methods that were proposed earlier.
S.Ravi, S.Wilson [15] proposed a color model conversion algorithm based on
chrominance color, this information is used for detecting face region. The facial features
such as eyes, nose, and mouth are used to find the region of the feature pixel by applying
threshold measurements. The algorithm provides a better accuracy for classifying the gender
using SVM classification algorithm. The main issue involved is choosing the correct
threshold parameter for facial feature detection and gender classification, which clearly
might be inadequate for the data.
Lin-Lin Huanga, AkinobuShimizu, Yoshihoro Hagihara, Hidefumi Kobatake [9], proposed
a classification-based approach for locating frontal and nearly frontal faces in cluttered
images. The directional decomposition of gradient provides a better discrimination ability

24
than the image intensity and the gradient map, and it was shown that the combination of
gradient
direction and intensity further improves the detection performance. To improve the
detection performance, gradient direction features from local window images are extracted
as input for underlying two-class classifier. Simple and complex background images are
taken to check the performance and it provides a better performance.
Yongmin Li, Shaogang Gong and Heather Liddell [19], discussed about the SVM
approach used for multi-view face detection and recognition. It involves the following
methods
1.SVM construct pose estimators which gives fast estimation of head pose. 2.
Symmetrical property of the face images are used to classify the half possible views with the
trained set. 3. The improvement of detection accuracy and easy computation, pose
information is used to guide the selection of face detectors by using pose change smoothing
technique.
Saman cooray, Noel O’ Connor [14], has discussed to detect frontal faces; the author had
used a technique that combines feature extraction and statistical face classification. The use
of eye facial feature points enables to derive a normalized search space by eliminating
the requirement of analyzing the image at multiple scales for detecting different sized faces.
Since the search space is normalized, the need for analyzing the image at each image pixel
location is eliminated. This in turn provides a promising approach for facial classification.
Bo WU, Haizhou AI and Chang HUANG [2], described a LUT weak classifier based
boosting method for face retrieval by demographic classification. A Haar feature based
2D LUT-type weak classifier was developed for multi-class problems and had used a
variation of boosting algorithm for multi-class multilabel problems, Adaboost.MH to learn
the demographic classifiers. A model of automatic demographic face retrieval system is
presented and the experimental results show its potentials in the management of a large
facial image database for online retrieval applications.
Ahmed Abdu Alattab, Sameem Abdul Kareem [1]: has discussed about the
integration of verbal description of human face and eigen face feature achieved excellent
results in the retrieval of face image when compared to retrieval by image content that
reduces the semantic gap between high level query requirement represented by user verbal
description and low level facial features represented by image content features. Combining
the two methods of query by description and query by image example, the accuracy of
the retrieval process is improved automatically and the needed time to find the desired
faces is also minimized. The idea was based on matching the user verbal description of the
query face with the annotated description of the faces in the database. The system then uses
eigen faces features for further searches on narrowed down search space of the pruned
set facial images, achieving more accurate results.
Yogesh R. Tayade1, Prof. S.M. Bansode2 [18], developed an efficient method for face
retrieval by merging three different algorithms SIFT, LBP and IDSC. An image is given as
an input, it is filtered and it is represented in a sparse matrix that derives SIFT and LBP

25
features. Through this work, the face images are successfully retrieved from the
trained dataset of Labeled Faces in Wild (LFW) images which competently achieves better
retrieval rate.
Yannis Avrithis, Nicolas Tsapatsoulis and Stefanos Kollias [17], portrays that to extract
objects from images color segmentation has proved to be a powerful tool, especially in the
case of human faces that are usually characterized by uniform color. The M-RSST algorithm
eliminates facial details and provides a single object for each face. Moreover,
chrominance components provided with a probabilistic model are used in an efficient way
for retrieving facial images from image databases. The interactive form of the proposed
system adapts the model to the needs of the user and consequently leads to much more
meaningful retrieval results.
Neeraj Kumar, et al., [12], discussed about how to automatically train classiﬁers for
describable aspects of visual appearance – attributes and similes. These classiﬁers are
learned using huge collections of labeled images obtained from the internet. They
demonstrated the use of these describable attributes for performing face verification and
image search. It was performing better in aspects such as, attribute classification, face
verification, and search (qualitatively).
5. RESEARCH DIRECTIONS
The challenges combined with face detection can be attributed to the following
factors
[8, 4, 10]:
1. Pose: The images of a face vary due to the relative camera-face pose (frontal, 45 degree,
profile, upside down), and some facial characteristics such as an eye or the nose may become
partly or wholly occluded.
2. Presence or absence of structural elements: Facial characteristics such as beards,
mustaches, and glasses may or may not be present and there is a great deal of variability
among these components including color, shape, and size.
3. Facial expression. The appearances of faces are straightly influenced by a person’s
facial expression.
4. Occlusion. Faces may be partly occluded by other objects. An image with a group of
people, some faces may partly occlude other faces.
5. Image orientation: Face images straightly differ for different rotations about the
camera’s optical axis.
6. Imaging conditions: When the image is produced, features such as lighting (spectra,
source distribution and intensity) and camera characteristics (sensor response, lenses)
affects the outer shell of a face.
7. The age: As the face matures, it modifies some of its most enduring properties (e.g.,
shape of cranium) and acquires new attributes (e.g., wrinkles,spot). These changes gives the
basic information about the aging of the face.
8. Changes in illumination: Person face was slower and less accurate at matching and
naming faces when there was a change in illumination direction.

26
9. Forensic Sketch Recognition: Sketches can be drawn from human memory when no
image is available. For example when most terrible of crimes committed such as murder,
sexual assualt, etc..
10. Component-based face recognition: Perform face matching and image retrieval per
facial component such as eyes, nose, mouth, eye brows, chin. It promotes partial face
matching and individuality models.
6. CONCLUSION
Face recognition technologies have been connected generally with very costly top
secure applications. Beside with all the biometric organisms, face recognition approach
possesses one vast advantage, which is its user-friendliness. This paper presents a survey on
various face detection techniques and algorithms that was proposed earlier by
researchers for the better development in the field of classification and content based image
retrieval. It also provides an overview of some of the ongoing researches in face detection.
Face detection in entirely unrestrained settings remains a very challenging task, particularly
due to the major pose and lighting variations.
REFERENCES
[1] Ahmed Abdu Alattab, Sameem Abdul Kareem Facial Image Retrieval Based on Eigen faces
and Semantic Features Proc. of the International Conference on Advances in Computer and
Information Technology - ACIT 2012
[2] Bo WU, Haizhou AI and Chang HUANG Facial Image Retrieval Based on Demographic
Classification
[3] D. McCullagh, "Call It Super Bowl Face Scan 1," in Wired Magazine, 2001.
[4] J. N. K. Liu, M. Wang, and B. Feng, "iBotGuard: “an Internet-based intelligent robot security system
using invariant face recognition against intruder," IEEE Transactions on Systems Man
and Cybernetics Part C-Applications and Reviews, Vol.35, pp.97-105, 2005.
[5] Jawad Nagi, Syed Khaleel Ahmed Farrukh Nagi A MATLAB based Face Recognition System using
Image Processing and Neural Networks 4th International Colloquium on Signal Processing and its
Applications, March 7-9, 2008, Kuala Lumpur, Malaysia. ISBN: 978-983-42747-9-5
[6] Ji Zhang Wynne Hsu Mong Li Lee “Image Mining: Trends and Developments”.
[7] Ji Zhang Wynne Hsu Mong Li Lee “Image Mining: Issues, Frameworks and Techniques”
Proceedings of the Second International Workshop on Multimedia Data Mining (MDM/KDD 2001)
Proceedings of the Second International Workshop on Multimedia Data Mining
(MDM/KDD'2001), in conjunction with ACM SIGKDD conference. San Francisco, USA, August
26, 2001.
[8] Kwangbaek Kim Intelligent Immigration Control System by Using Passport Recognition and
Face Verification ISNN 2005, LNCS 3497, pp. 147–156, 2005. © Springer-Verlag Berlin
Heidelberg 2005.
[9] Lin-Lin Huanga; AkinobuShimizu a, Yoshihoro Hagiharab, Hidefumi Kobatakea Gradient
feature extraction for classi"cation-based face Detection Pattern Recognition 36 (2003) 2501 – 2511
Pattern Recognition Society. Published by Elsevier Ltd.
[10] Manisha Deorao Urkude Dr. S.B. Kishor Dr. Sunil Gulabrao Naranje “Study of Open Challenges in
Face Recognition and Expected Result “Indian Journal of Applied Research Volume: 3 | Issue: 6 |
June 2013 | ISSN - 2249-555X
[11] Ming-Hsuan Yang, Member, IEEE, David J. Kriegman, Senior Member, IEEE, and Narendra Ahuja,
Fellow, IEEE “Detecting Faces in Images: A Survey “IEEE Transaction On Pattern Analysis And
Machine Intelligence, vol. 24, no. 1, January 2002

27
[12] Neeraj Kumar, Student Member, IEEE, Alexander C. Berg, Member, IEEE, Peter N. Belhumeur, and
Shree K. Nayar, Member, IEEE “Describable Visual Attributes for Face Veriﬁcation and
Image Search” IEEE Transactions On Pattern Analysis And Machine Intelligence.
[13] Rabia Jafri* and Hamid R. Arabnia* “A Survey of Face Recognition Techniques” Journal of
Information Processing Systems, Vol.5, No.2, June 2009
[14] Saman cooray, Noel O’ Connor Facial features and appearance-based classification for face
detection in color images.
[15] S.Ravi , S.Wilson Face Detection with Facial Features and Gender Classification Based On Support
Vector Machine 2010 Special Issue - International Journal of Imaging Science and Engineering
[16] T. Choudhry, B. Clarkson, T. Jebara, and A. Pentland, "Multimodal person recognition using
unconstrained audio and video," in Proceedings, International Conference on Audio and Video-Based
Person Authentication, 1999, pp.176-181
[17] Yannis Avrithis, Nicolas Tsapatsoulis and Stefanos Kollias Color-Based Retrieval of Facial Images.
[18] Yogesh R. Tayade1, Prof. S.M. Bansode2 An Efficient Face Recognition and Retrieval Using
LBP and SIFT International Journal of Advanced Research in Computer and Communication
Engineering Vol. 2, Issue 4, April 2013
[19 ]Yongmin Li, Shaogang Gong and Heather Liddell Support Vector Regression and
Classification Based Multi-view Face Detection and Recognition.
[20] Zeynep Orman, Abdulkadir Battal and Erdem Kemer A Study on Face, Eye Detection and Gaze
Estimation International Journal of Computer Science & Engineering Survey (IJCSES) Vol.2, No.3,
August 2011.
Authors
Dr. S.Vijayarani has completed MCA, M.Phil and Ph.D in Computer
Science. She is working as Assistant Professor in the School of Computer
Science and Engineering, Bharathiar University, Coimbatore. Her fields of
research interest are data mining, privacy, security, bioinformatics and data
streams. She has published papers in the international journals and
presented research papers in international and national conferences.
Mrs. M.Vinupriya has completed MCA. She is currently pursuing her
M.Phil in Computer Science in the School of Computer Science and
Engineering,Bharathiar University, Coimbatore. Her fields of interest are
Image Mining in data mining and privacy preserving in Data mining.

Facial image classification and searching –a survey

More Related Content

What's hot (18)

Viewers also liked (17)

Similar to Facial image classification and searching –a survey (20)

Recently uploaded (20)

Facial image classification and searching –a survey