SlideShare a Scribd company logo
Bulletin of Electrical Engineering and Informatics
Vol. 7, No. 4, December 2018, pp. 552~560
ISSN: 2302-9285, DOI: 10.11591/eei.v7i4.843  552
Journal homepage: http://journal.portalgaruda.org/index.php/EEI/index
Features for Cross Spectral Image Matching: A Survey
Maulisa Oktiana1
, Fitri Arnia2
, Yuwaldi Away3
, Khairul Munadi4
1,2,3,4
Post Graduate Program in Engineering, Universitas Syiah Kuala, Indonesia
2,3,4
Department of Electrical and Computer Engineering Universitas Syiah Kuala, Indonesia
Article Info ABSTRACT
Article history:
Received Nov 11, 2017
Revised May 20, 2018
Accepted Oct 08, 2018
In recent years, cross spectral matching has been gaining attention in various
biometric systems for identification and verification purposes. Cross spectral
matching allows images taken under different electromagnetic spectrums to
match each other. In cross spectral matching, one of the keys for successful
matching is determined by the features used for representing an image.
Therefore, the feature extraction step becomes an essential task. Researchers
have improved matching accuracy by developing robust features. This paper
presents most commonly selected features used in cross spectral matching.
This survey covers basic concepts of cross spectral matching, visual and
thermal features extraction, and state of the art descriptors. In the end, this
paper provides a description of better feature selection methods in cross
spectral matching.
Keywords:
Cross spectral matching
Feature
Robust features
Thermal images
Visible light images Copyright © 2018 Institute of Advanced Engineering and Science.
All rights reserved.
Corresponding Author:
Khairul Munadi,
Department of Electrical and Computer Engineering,
Universitas Syiah Kuala,
Jalan Teuku Nyak Arief, Darussalam, Banda Aceh 23111, Indonesia
Email: khairul.munadi@unsyiah.ac.id
1. INTRODUCTION
Cross-spectral matching also known as cross-spectrum matching is a matching process between two
images taken on different electromagnetic spectrum [1]. Typically, we cross-matched the thermal spectrum
images with visible light (VL) images. The thermal spectrum consists of four sub-bands, i.e., Near Infrared
(SWIR), Short Wave Infrared (SWIR), Medium Wave Infrared (MWIR), and Longwave IR (LWIR) [2]. The
study in the cross-spectral domain increases rapidly in line with applications of biometric systems [3]. Cross-
spectral matching is widely used for security, national ID programs, also for personal identification and
authentication using iris and face recognition [4-7]. By using cross-spectral image matching scheme, the
identification and authentication process becomes more accurate and efficient because it utilises the
additional information contained in both different spectrum and wavelength [8].
The performance of cross-spectral matching is dependent on features that can represent information
from VL and thermal images. VL and thermal images represent information from the same subject even
though the visual appearance and structure of the two images are different. Therefore the most challenging
task in cross-spectral image matching is how to choose a representative feature for both VL and thermal
images [9]. Various features are employed in cross-spectral image matching with high
recognition performance.
Trokielewicz [10] developed cross spectral mobile based verification using the Discrete Cosine
Transform (DCT) and Gabor Wavelet features. DCT is used on Monro Iris Recognition Library (MIRLIN)
software, whereas Gabor Wavelet is used on Open Source for IRIS (OSIRIS) software. DCT and Gabor
wavelet features are suitable for cross spectral iris recognition with low Equal Error Rate (EER). Abdullah et
al. [11] explored the Binarized Statistical Image Features (BSIF) to extract the statistical features from NIR
and VL iris images. BSIF features able to represent the statistical properties of NIR and VL iris images
appropriately with high iris recognition performance. Another interesting work was conducted by klare and
BEEI ISSN: 2302-9285 
Features for Cross Spectral Image Matching: A Survey (Maulisa Oktiana)
553
Jain [12] exploiting Histogram of Oriented Gradient (HOG) integrated with Local Binary Pattern (LBP) to
describe the structure of the face in NIR and VL images. Experimental result showed that HOG and LBP
have better performance in representing NIR and VIS images.
A survey on state of the art feature in cross-spectral image matching is carried out in this paper. We
describe several features based on the representation of VL and thermal image features. The main
contribution of this paper is to provide a brief description about features in cross-spectral image matching so
that it can help in selecting the most suitable feature used in the face and iris recognition. This paper is
organized as follows. Section 2 describes basic background concepts in cross-spectral matching and the
feature extraction, such as the definition of cross-spectral image matching, and block diagram matching
process in the cross-spectral domain. Section 3 reviews the properties and characteristics of the VL and the
thermal image features. Also, how to extracts features from the VL image and the NIR images for cross
spectral matching are described in section 3. Section 4 reviews the robust feature used in cross-spectral
matching including the feature extraction process. Conclusions are summarized in section 5.
2. CROSS SPECTRAL MATCHING
Cross spectral matching represents the ability to recognize the objects presented in two different
modalities. Cross spectral matching is illustrated in Figure 1. First, pre-processing was applied to prepare an
image for further processing. The pre-processing step comprises image cropping, photometric and geometric
normalization, and restoration. Next step is feature extraction which is carried out by taking unique features
of thermal and VL image by using certain descriptors. The results of this process are used as inputs to the
matching step using matching or classification algorithms.
Figure 1. Cross spectral matching
3. VISIBLE AND THERMAL IMAGES FEATURES
Image features are unique attributes of both numerical and alphanumerical that represent
information about the content of a digital image. Numerical representation means any physical information of
an image, such as heat, color, pressure, range, non visible wavelength, etc. replaced by numerical values that
make it easier to be analyzed in various applications. In VL and thermal images, a feature can be either an
image visual characteristics or interpretative response to the spatial, symbolic, semantic, or emotional image
characteristics [13].
Feature extraction is an initial stage for all applications in image analysis. Feature extraction is
known as the process of getting the unique characteristics of an image that distinguish one image with
another. Feature extraction is used to indicate the relevant information of an image in completing the
computational tasks associated with a particular application [14]. Feature extraction process for VL and
thermal images are described in Figure 2.
In cross spectral matching, performance and matching accuracy depends on proper feature
extraction. VL and thermal images are retrieved based on the value of the vector feature, therefore, feature
selection and feature extraction process are important to be considered. Feature selection is defined as a
process of selecting the best features that suitable for a particular application [15].
The feature selection based on the visual properties of the image is shown in Table 1. Low level is
defined as the image features can be extracted directly without considering the spatial relationship. While
high-level concerns the spatial information. The feature selection process must consider [16]:
Pre-
processing
Feature
Extraction
Probe
Pre-
processing
Feature
Extraction
Matching Result
Gallery
 ISSN: 2302-9285
BEEI, Vol. 7, No. 4, December 2018 : 552 – 560
554
a. Similarities between two matched images, if the images similar the feature distance between this image
is small. We can confirm that if VL and thermal images are similar the distance of feature vector of
those two image is small.
b. Computational task is not complex.
c. Small feature dimensions do not affect the matching efficiency.
Figure 2. Representation of image features
Table 1. Visual properties of visible and thermal images
Low level Middle level High level
Color
Shape
Texture
Regions
Spatial relationship
Localization
Image categorization
Objects identification
d. The size of the dataset does not affect matching performance.
e. Robust against variation illuminations and geometric transformation.
Figure 3 presents several methods of feature extraction classification for cross spectral matching.
Feature extraction methods divided into:
a. Structural methods : this method identifies structural features of an image. Feature calculated based on
topological and geometric properties [17].
b. Statistical methods : identifies statistical features of an image based on statistical distributions of
pixels [17].
Visible
Images
Visual properties
of VL images
Numerical
representation
Feature
extraction
Feature
vector
Thermal
Images
Properties of
thermal images
Numerical
representation
Feature
Extraction
Feature
vector
Matching
Indexing/Classify
Similarity rate
BEEI ISSN: 2302-9285 
Features for Cross Spectral Image Matching: A Survey (Maulisa Oktiana)
555
c. Spectrum or transform based : feature calculated over spectral composition, low and high frequencies of
an image. They can further be devided into : spatial, frequencies, and combined transform [14].
d. Block based : based on images block. This method combines the important feature of image into one
block, consequently size and number of blocks affect the resulting features [14].
e. Combined methods : feature calculated by concatenated several individual methods to provide greater
ability [14].
Figure 3. Classification of features extraction methods for cross spectral image
4. STATE OF THE ART FEATURE DESCRIPTORS
In this section, we present some popular descriptors used in cross spectral matching. These
descriptors frequently use because robust in describing VL and thermal image. The descriptors commonly
used in the cross spectral matching are presented in Table 2.
4.1. Difference of Gaussian Filter
The Difference of Gaussian (DoG) is a bandpass filter that used Gaussians filter to produce a
normalized image [2]. DoG can suppress variations of noise and aliasing on the cross-spectral image caused
by the frequency difference between VL and thermal images. Also, the DOG has a low computational
complexity and popularly used in publications [18], [19].
DOG can accommodate difficult lighting condition by setting inner dan outer Gaussian values. For
strong lighting variations datasets, the recognition gives the best result at outer Gaussian ≈ 2 pixels and up to
about 4 for datasets less extreme lighting variations while the inner Gaussian is quite narrow at 1 pixel [20].
Band-pass filter is obtained by subtracting two Gaussians with different σ that eliminating all
frequencies between frequencies cut-off of the two Gaussian. The image feature is located between this
frequency band was extracted.
The DOG feature extractions consists of three stages [20]:
a. The input image is convolving with two Gaussian kernels having differing standard deviations as shown
in (1) to produce a blurred image.
b. Next two blurred images version is subtracted each other to obtain a normalized image.
Fourier
Transform
Statistical
Methods
Local Binary
Pattern
(LBP)
Structural /
Geometric
Histogram of
Oriented
Gradient
(HOG)
Combined
Methods
Log
Gabor
Wavelet
Scale
Invariant
Feature
Transform
(SIFT)
Gabor
Local
Binary
Pattern
(GLBP)
Spatial Frequencies
Spectrum Based
Cumm
ulative
Sum
based
Fractal
and Chaos
Theory
Color
Texture
Feature
Gabor
Transform
Wavelet
Transform
Discrete
Cosine
Transform
Cross Spectral Features Extraction Methods
Binarized
Statistical
Image
Features
(BSIF)
Block
Based
Additive
Fusion
Combined
Transform
Shift
Invariant
(SIFE)
Fourier
Mellin
Radon
Transform
Local Ternary
Pattern (LTP)
Pyramid
Histogram
of
Oriented
Gradient
(PHOG)
 ISSN: 2302-9285
BEEI, Vol. 7, No. 4, December 2018 : 552 – 560
556
c. To construct a band-pass filter, the values of δ0 must be smaller than δ1 which set to (1) and
(2) respectively
( ) [ ( ) ( )] ( ) (1)
Where : ( ) : original image
( ) : blurred image
: Gaussian kernel function defined as :
( )
√
( )⁄
(2)
Table 2. Various descriptors in cross spectral matching
Publications Domain Modality Applications Feature Extraction Technique
[2] NIR vs. VL Iris recognition Real valued -Log Gabor Phase
[4] NIR vs. VL Face recognition LBP, DLBP
[10] NIR vs. VL Iris recognition DCT, Gabor wavelet
[11] NIR vs. VL Iris recognition Binarized Statistical Image Feature
[12] NIR vs. VL Face recognition uniform LBP and HOG
[16] NIR vs. VL Face recognition Log DoG LBP and Log DoG HOG
[19] NIR vs. VL Face recognition DoG, Center Surround Divisive Normalization
(CSDN), SIFT (Scale Invariant Feature Transform),
and Multiscale Local Binary Pattern (MLBP)
[21] NIR vs. VL, SWIR
vs. VL
Face recognition LBP and
Generalized LBP (GLBP)
[22] SWIR vs. VL Face recognition LBP and LTP
[23] NIR vs. VL Face recognition Directional binary code
[24] Thermal vs. VL Face recognition Pyramid Histogram of Oriented Gradient (PHOG),
PSFIT
[25] NIR vs. VL Face recognition Multi resolution LBP
[26] MWIR vs. VL
LWIR vs. VL
Face recognition Canonical Correlation Analysis (CCA)
[27] NIR vs. VL Face recognition Wavelet transform
[28] NIR vs. VL Face recognition Sparse dictionary
[29] SWIR vs. VL Face recognition Gabor filter, LBP, Simplified Weber Local
Descriptor (SWLD) and GLBP
[30] Thermal vs. VL Face recognition Deep Perceptual Mapping (DPM)
[31] SWIR vs. VL Face recognition Contrast Limited Adaptive Histogram Equalization
(CLAHE)
[32] NIR vs. VL Face recognition Gaussian filter and SIFT
4.2. Local Binary Pattern
At 1996, Local Binary Pattern (LBP) was first introduced by Ojala et al. [33]. The LBP operator is
a texture descriptor gray-scale invariant that analyzes the texture of an image based its texture spectrum
called Texture Unit (TU).
Texture spectrum is a distribution of texture units happening in a region. Originally, LBP uses 3x3
neighborhood and generates 8-bit code based on the number of 8 pixels around the center pixel. Texture unit
has 28=256 possibility histogram bin in describing the spatial pattern in a 3x3 neighborhood. They compute
by multiplying the weights of the corresponding pixel with the values of the pixels in the previous threshold
neighborhood. Then the pixel value of this neighborhood summed resulting the number of (169) texture unit
[34]. The LBP operator defined as:
( ) ∑ ( ) (3)
BEEI ISSN: 2302-9285 
Features for Cross Spectral Image Matching: A Survey (Maulisa Oktiana)
557
Where ( ) {
c, n represents a central pixel and 8 neighbors of the central pixel respectively.
The 3x3 neighbor's pixel is thresholding by center pixel. If the neighbor's pixel greater than center
pixel then the value is 1, and 0 otherwise. LBP operators further developed to accommodate variations of
texture scaling into circular neighborhoods (8.1), (16.2) and (8.2).
4.3. Binary Statistical Image Feature
Binary Statistical Image Feature (BSIF) is a texture descriptor which is inspired by Local Binary
pattern (LBP). BSIF uses a binary code to represent each pixel neighborhood in an image [35]. The binarized
feature is generated by convolving image with a set of linear filter. Thus, the response of a linear filter is
binarized with a threshold at zero. To construct the linear filter, independent component analysis (ICA) is
used. The statistical independence of the filter responses is maximizing by ICA from a training set of natural
image patches. The BSIF extraction process as described in [36]:
a. First, the response of the linear filter is constructed. Let X is image patch with size l x l pixels. is a
linear filter and represents the response of the filter.
∑ ( ) ( ) (4)
b. Binarized feature is obtained by :
{ (5)
4.4. Scale Invariant Feature Transform
The Scale Invariant Feature Transform (SIFT) is a robust descriptor that combinations of DoG
interest region detector with a corresponding feature descriptor. SIFT encodes image information in a
localized set of gradient orientation histograms. Thus, SIFT can accommodates illumination variations and
small positional shift [37].
4.5. Histogram of Oriented Gradient
Histogram of Oriented Gradient (HOG) also known as histogram normalization due to its ability to
normalize from local responses [38]. HOG is adopted human detection and used for object detection
applications. HOG resistant of shadowing process, illumination invariant, and reliable to photometric
variations. HOG does not have spatial image representation, therefore, HOG only computes the individual
pixel energy regardless spatial distribution of an image. The feature extraction process [39]:
a. The image is divided into a small area called cell (8x8 pixels).
b. Calculate the magnitude of the pixel orientation.
c. Interpolation of result no 2 into histogram orientation bin 20 degrees.
d. Cells are grouped into overlapping blocks.
e. Then overlapping blocks are normalized.
f. Finally the normalized histograms are concatenated resulted in vector features.
4.6. Additive Fusion Block Based
Additive fusion block based proposed by Varadarajan et al. that used block-based extraction process
[40]. Block based can maintain the optimum number of features that need to be extracted. Size and number of
blocks affect the resulting features in this techniques. The more blocks, the less feature is extracted because
of, the smaller block size causing the resultant block reduction. The ideal block sizes are 4,8, or 16 pixels.
Feature extraction process consists:
a. Image is divided into individual blocks of size 4.8, or 16 pixels
b. Each block is applied Chirp Z-Transform (CZT) and Goertzel algorithm for preprocessing and image
enhancement.
c. Individual blocks are summed to yield a single resultant block that contains the essential features of
each block.
d. This resultant block is then become a vector feature.
4.7. Discrete Cosine Transform
Discrete Cosine Transform (DCT) transforms spatial image information into frequency domain.
DCT consists cosine part of Fourier transform models [41]. DCT concentrates energy image into some DCT
 ISSN: 2302-9285
BEEI, Vol. 7, No. 4, December 2018 : 552 – 560
558
coefficients (energy compaction). Signal energy is concentrated on large DCT coefficient magnitudes (called
low frequencies) and located in the upper-left corner of the DCT array.
Low frequencies coefficients contain most of an essential image information. Therefore the original
image can be reconstructed only by using low frequencies coefficients. While less important information is
located in lower-right values of the DCT array (called high frequencies). This high-frequency coefficients
can be discharged through the quantization process without significantly affecting the image quality [42-43].
DCT feature extraction process [44]:
a. An original image is divided into 8x8 blocks.
b. Calculated DCT coefficients using (4) resulted in DCT coefficient arrays.
c. Quantized.the DCT coefficients.
d. The value that is in upper-left corner of the DCT is used as a feature vector (low frequency).
( ) ( ) ( ) ∑ ∑ [ ( )] [ ( )] ( )
(6)
( ) ( )
{
√
√
Where ; and ( ) represents intensity of the pixel in row x and
column y [45].
4.8. Radon Transform
Radon transform is one of the transformations that can enhance low-frequency components by
describing the integral line of an image. Radon transform is turned rotation into translation and often used in
face recognition. Radon feature extraction process [46] :
a. Calculated radon space using (5).
b. Discriminative information located in radon space which computes using projections for 0°–179°
orientations.
c. Calculated DCT of radon space to obtain feature frequencies.
d. Concatenated 25% of DCT coefficients resulted in feature vector.
( )[ ( )] ∫ ∫ ( ) ( ) (7)
Where ( ) represents Dirac function, perpendicular distance of a line from the origin represented r
ϵ[ ] and ϴ is an angle between X-axis and distance vector.
5. CONCLUSIONS
The literature relevant cross spectral matching is overgrowing, and many researchers have proposed
a cross spectral matching framework with better matching performance. This survey presents an overview
feature of thermal and visible images. Also, brief descriptions the current state of the art feature extraction
methods for cross spectral matching. The image features affect the performance of cross spectral matching.
Therefore, the selection of feature extraction methods that suitable for an application becomes essential issue.
Also, the researchers must consider visual properties of VL and thermal images.
ACKNOWLEDGEMENTS
This research was supported by the Ministry of Research, Technology, and Higher Education of the
Republic of Indonesia, under the scheme, namely PMDSU.
REFERENCES
[1] R. S. Ghias, et al., “Infrared Face Recognition: A Comprehensive Review of Methodologies and Databases,”
Pattern Recognition, vol. 47, pp. 2807–2824, 2014.
[2] S. Hu, et al., “Thermal-to-Visible Face Recognition using Partial Least Squares,” Journal of the Optical Society of
America, vol. 32, pp. 431-442, 2015.
BEEI ISSN: 2302-9285 
Features for Cross Spectral Image Matching: A Survey (Maulisa Oktiana)
559
[3] M. J. Burge and K. W. Bowyer. Handbook of Iris Recognition Advances in Computer Vision and Pattern
Recognition. Springer, 2013:15-24.
[4] R. Biswas et al., ,”A New Approach of Iris Detection and Recognition”, International Journal of Electrical and
Computer Engineering, vol. 7, pp. 2530–2536, 2017.
[5] F. Arnia and N. Pramita, “Enhancement of Iris Recognition System Based on Phase Only Correlation”,
TELKOMNIKA, vol. 9, pp. 387–394, 2011.
[6] F. Arnia, et al., “Ordinal Measure of Discrete Cosine Transform (DCT) Coefficients and Its Application to
Fingerprint Matching”, International Jounal of Electrical and Computer Engineering, vol. 3, pp. 823–830, 2013.
[7] T. S. Gunawan, et al., “Development of Efficient Iris Identification Algorithm using Wavelet Packets for
Smartphone Application”, Indonesian Jounal of Electrical Engineering and informatics, vol. 8, pp. 450–456, 2017.
[8] P. R. Nalla and A. Kumar, “Towards More Accurate Iris Recognition using Cross-Spectral Matching,” IEEE
Transaction on Image Processing, vol. 26, pp. 1-14, 2016.
[9] F. J. Xu, et al., “NIR-VIS Face Recognition Via Cross Spectral Joint Dictionary Learning and Reconstruction,” in
Proceedings of IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), 2015, pp.
141 – 150.
[10] M. Trokielewicz and E. Bartuzi, “Cross-spectral Iris Recognition for Mobile Applications using High-quality Color
Images,” Jounal of Telecommunications and Information Technology, vol. 3, pp. 91-97. 2016.
[11] M. A. M. Abdullah, et al., “A Novel Framework for Cross-Spectral Iris Matching,” IPSJ Transaction on Computer
Vision and Applications, vol. 8, pp. 1-12, 2016.
[12] B. F. Klare and A. K. Jain, “HeTerogeneous Face Recognition: Matching NIR to Visible Light Images,” in
Proceedings of 20th International Conference Pattern Recognition (ICPR), 2010, pp. 1513–1516.
[13] R. Schettini, et al., Feature Extraction for Content Based Image Retrieval. In: Liu L, Ozsu MT. Encyclopedia of
Database Systems. New York: Springer; 2009. 1115-1119.
[14] Y. C. Cabrera, et al., Iris Feature Extraction Methods. Centro de Aplicaciones de Tecnologıa de Avanzada
(CENATAV). 2142. 2013.
[15] E. S. Kumar, et al., “Feature Selection and Extraction for Content-Based Image Retrieval,” International Journal of
Mathematics Trends and Technology, vol. 3. Pp. 70–73, 2012.
[16] K. Anuradha and K. Sankaranarayanan, “Statistical Feature Extraction to Classify Oral Cancers,” Journal of Global
Research in Computer Science, vol. 4, pp. 8-12, 2013.
[17] P. Vithlani and C. K. Kumbharana, “Structural and Statistical Feature Extraction Methods for Character and Digit
Recognition,” International Journal of Computer Applications, vol. 120, pp. 43–47, 2015.
[18] J. Y. Zhu, et al., “Matching NIR Face to VIS Face Using Transduction,” IEEE Transaction on Information
Forensics and Security, vol. 9. Pp. 501–514, 2014.
[19] D. Kang, et al., “Nighttime Face Recognition at Large Standoff: Cross-Distance and Cross-Spectral Matching,”
Pattern Recognition, vol. 47, pp. 3750–3766, 2014.
[20] X. Tan and B. Triggs, “Enhanced Local Texture Feature Sets for Face Recognition Under Difficult Lighting
Conditions,” IEEE Transactions on Image Processing, vol. 16. pp. 1635 – 1650, 2010.
[21] Z. X. Cao and N. A. Schmid, “Recognition Performance of Cross-Spectral Periocular Biometrics and Partial Face
at Short and Long Standoff Distance,” Open Transaction on Information Processing, vol. 1, pp. 20-32, 2014.
[22] N. D. Kalka et al., “Cross-Spectral Face Recognition in Heterogeneous Environments: a Case Study on Matching
Visible to Short-Wave Infrared Imagery,” in 2011 International Joint Conference on Biometrics (IJCB).
Washington, DC. 2011, pp. 1-8.
[23] L. Shen, et al., “Face Recognition from Visible and Near-Infrared Images using Boosted Directional Binary Code”,
in International Conference on Intelligent Computing (ICIC). Zhengzhou, China. 2011, pp. 404-411.
[24] C. Chen and A. Ross, “Matching Thermal to Visible Face Images using Hidden Factor Analysis in A Cascaded
Subspace Learning Framework”, Pattern Recognition, vol. 72, pp. 25–32, 2016.
[25] J. Chen, et al., “Learning Mappings for Face Synthesis from Near Infrared to Visual Light Images”, in Proceedings
of IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Miami, USA. 2009, pp. 156–163.
[26] N. Osia and T. Bourlai., “On Matching Visible to Passive Infrared Face Images using Image Synthesis and
Denoising”, in Proceedings of 12th IEEE International Conference on Automatic Face & Gesture Recognition (FG
2017). Washington, DC, USA. 2017, pp. 904 - 911.
[27] A. Sea, et al., “Human Face Recognition using Random Forest Based Fusion of À-Trous Wavelet Transform
Coefficients from Thermal and Visible Images”, AEU - International Journal of Electronics and Communications,
vol. 70, pp. 1041–1049, 2016.
[28] P. Buyssens and M. Revenu, “Visible and Infrared Face Identification Via Sparse Representation”, ISRN Machine
Vision, vol. 1, pp. 1-11, 2013.
[29] F. Nicolo and N. A. Schmid, “Long Range Cross-Spectral Face Recognition: Matching SWIR Against Visible
Light Images”, IEEE Transaction on Information Forensics and Security, vol. 7, pp. 1717–1726, 2012.
[30] M. S. Sarfraz and R. Stiefelhagen, “Deep Perceptual Mapping for Thermal to Visible Face Recognition”, in
Proceedings of British Machine Vision Conference (BMVC). Swansea, UK .2015, pp. 1–11.
[31] T. Bourlai, et al., “Cross-Spectral Face Verification in the Short Wave Infrared (SWIR) Band”, in Proceedings of
International Conference on Pattern Recognition (ICPR). Istanbul, Turkey. 2010, pp. 1343–1347.
[32] H. Maeng, et al., “Nighttime Face Recognition at Long Distance: Cross Distance and Cross Spectral Matching”, in
Proceedings of Asian Conference on Computer Vision (ACCV). Taipei, Taiwan. 2013, pp. 708-721.
[33] T. Ojala, et al., “A comparative study of texture measures with classification based on feature distributions”,
Pattern Recognition. Vol. 29, pp. 51–59, 1996.
 ISSN: 2302-9285
BEEI, Vol. 7, No. 4, December 2018 : 552 – 560
560
[34] T. Ojala, et al., “Multiresolution Gray-Scale and Rotation Invariant Texture Classification with Local Binary
Patterns”, IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 24. pp. 971–987, 2002.
[35] A. Hyvarinen et al., Natural Image Statistics. Springer, 2009.
[36] J. Kannala and E. Rahtu, “BSIF: Binarized Statistical Image Features,” in 21st International Conference on Pattern
Recognition (ICPR) , Tsukuba, Japan. 2012, pp. 1363-1366.
[37] Z. Ghassabi, et al., “An Efficient Approach for Robust Multimodal Retinal Image Registration Based on UR-SIFT
Features and PIIFD Descriptors”, EURASIP Journal on Image and Video Processing, vol. 25, pp. 1-16, 2013.
[38] N. Dalal and B. Triggs, “Histograms of Oriented Gradients for Human Detection”, in Proceedings of IEEE
Computer Vision and Pattern Recognition (CVPR). San Diego. 2005, pp. 886-893.
[39] Z. R. Tan, et al., “Using Pyramid of Histogram of Oriented Gradients on Natural Scene Text Recognition”, in
Proceedings of International Conference on Image Processing (ICIP). Paris. 2014, pp. 2629 – 2633.
[40] K. K. Varadarajan, et al., “Face Recognition using Block Based Feature Extraction with CZT and Goertzel-
Algorithm as A Preprocessing Technique”, Procedia Computer Science, vol. 46, pp. 1458–1467, 2015.
[41] N. Ahmed, et al., “Discrete Cosine Transfom”, IEEE Transactions on Computers, vol. 23, pp. 90-93, 1974.
[42] S. A. Khayam. The Discrete Cosine Transform (DCT): Theory and Aplication. Departement of Electrical and
Computer Engineering Michigan State University. 2003.
[43] G. M. Deepa, et al., “Face Recognition Using Spectrum-Based Feature Extraction”, Applied Soft Computing
Joural, vol. 12, pp. 2913–2923, 2012.
[44] S. Dabbaghchian, et al., “Feature Extraction using Discrete Cosine Transform and Discrimination Power Analysis
with a Face Recognition Technology”, Pattern Recognition, vol. 43, pp. 1431–1440, 2010.
[45] R. M. Ramadan and R. F. A. Kader, “Face Recognition using Particle Swarm Optimization-Based Selected
Features”, International Journal of Signal Processing, Image Processing and Pattern Recognition, vol. 2, pp. 51–
66, 2009.
[46] D. V. Jadhav and R. S. Holambe, “Rotation, Illumination Invariant Polynomial Kernel Fisher Discriminant
Analysis using Radon and Discrete Cosine Transforms Based Features For Face Recognition”, Pattern
Recognition, vol. 31, pp. 1002–1009, 2010.
BIOGRAPHIES OF AUTHORS
Maulisa Oktiana received B.Eng degree in electrical engineering from Syiah Kuala University,
Banda Aceh, Indonesia, in 2013. She is currently a doctoral student at Graduate Program of
Engineering, Syiah Kuala University. Her research interests include image processing and
computer vision.
Fitri Arnia received B. Eng degree from Universitas Sumatera Utara (USU), Medan in 1997. She
finished her master and doctoral degree from Universsity of New South Wales (UNSW),
Sydney, Australia and Tokyo Metropolitan University, Japan in 2004 and 2008 respectively. She
has been with the Department of Electrical Engineering, Faculty of Engineering, Syiah Kuala
University since 1999. Dr. Arnia was a visiting scholar in Tokyo Metropolitan University
(TMU), Tokyo, Japan in 2013 and Suleyman Demirel University (SDU), Isparta, Turkey in
2017. She is a member of IEEE and APSIPA. Her research interests are signal, image and
multimedia information processing.
Yuwaldi Away received his B. Eng degree in Electrical-Computer Engineering from Sepuluh
Nopember Institute of Technology Surabaya, Indonesia in 1964, the M.Sc from Bandung
Institute of Technology (ITB) Bandung, Indonesia. He obtained his Ph.D. degrees in computer
technology from the National University of Malaysia. Since 1990, he joined Syiah Kuala
University as a teaching staff, and from 2007 until now he has been appointed as a Professor in
Electrical Engineering. His current research interest include microprocessor based system,
embedded system, FGPA, optimation and visualization.
Khairul Munadi received the B.E. degree from Sepuluh Nopember Institute of Technology,
Surabaya, Indonesia, in 1996, and the M.Eng. and Ph.D. degrees from Tokyo
MetropolitanUniversity (TMU), Japan, in 2004 and 2007 respectively, all in electrical
engineering. From1996 to 1999, he was with Alcatel Indonesia as a system engineer. Since 1999,
he joined SyiahKuala University, Banda Aceh, Indonesia, as a lecturer at the Electrical
Engineering Department.He was a visiting researcher at the Information and Communication
Systems Engineering,Faculty of System Design, TMU, Japan, from March 2007 to March 2008.
His research interests include multimedia signal processing and communications.

More Related Content

PDF
B42020710
PDF
F43053237
PDF
Property based fusion for multifocus images
PDF
Fpga implementation of image segmentation by using edge detection based on so...
PDF
Fpga implementation of image segmentation by using edge detection based on so...
PDF
A novel predicate for active region merging in automatic image segmentation
PDF
Development of stereo matching algorithm based on sum of absolute RGB color d...
PDF
Texture Classification
B42020710
F43053237
Property based fusion for multifocus images
Fpga implementation of image segmentation by using edge detection based on so...
Fpga implementation of image segmentation by using edge detection based on so...
A novel predicate for active region merging in automatic image segmentation
Development of stereo matching algorithm based on sum of absolute RGB color d...
Texture Classification

What's hot (19)

PDF
Spectral approach to image projection with cubic b spline interpolation
PDF
COLOUR BASED IMAGE SEGMENTATION USING HYBRID KMEANS WITH WATERSHED SEGMENTATION
PDF
Feature integration for image information retrieval using image mining techni...
PDF
FACE RECOGNITION USING DIFFERENT LOCAL FEATURES WITH DIFFERENT DISTANCE TECHN...
PDF
Cq32579584
PDF
06 17443 an neuro fuzzy...
PDF
OBJECT DETECTION, EXTRACTION AND CLASSIFICATION USING IMAGE PROCESSING TECHNIQUE
PDF
I04302068075
PDF
Image Segmentation Using Pairwise Correlation Clustering
PDF
Image segmentation based on color
PDF
Fuzzy Region Merging Using Fuzzy Similarity Measurement on Image Segmentation
PDF
QUALITY ASSESSMENT OF PIXEL-LEVEL IMAGE FUSION USING FUZZY LOGIC
PDF
An Improved Way of Segmentation and Classification of Remote Sensing Images U...
PDF
B018110915
PDF
PERFORMANCE ANALYSIS OF CLUSTERING BASED IMAGE SEGMENTATION AND OPTIMIZATION ...
PDF
FINGERPRINT MATCHING USING HYBRID SHAPE AND ORIENTATION DESCRIPTOR -AN IMPROV...
PDF
Image similarity using fourier transform
PDF
A novel embedded hybrid thinning algorithm for
PDF
Comparison of various Image Registration Techniques with the Proposed Hybrid ...
Spectral approach to image projection with cubic b spline interpolation
COLOUR BASED IMAGE SEGMENTATION USING HYBRID KMEANS WITH WATERSHED SEGMENTATION
Feature integration for image information retrieval using image mining techni...
FACE RECOGNITION USING DIFFERENT LOCAL FEATURES WITH DIFFERENT DISTANCE TECHN...
Cq32579584
06 17443 an neuro fuzzy...
OBJECT DETECTION, EXTRACTION AND CLASSIFICATION USING IMAGE PROCESSING TECHNIQUE
I04302068075
Image Segmentation Using Pairwise Correlation Clustering
Image segmentation based on color
Fuzzy Region Merging Using Fuzzy Similarity Measurement on Image Segmentation
QUALITY ASSESSMENT OF PIXEL-LEVEL IMAGE FUSION USING FUZZY LOGIC
An Improved Way of Segmentation and Classification of Remote Sensing Images U...
B018110915
PERFORMANCE ANALYSIS OF CLUSTERING BASED IMAGE SEGMENTATION AND OPTIMIZATION ...
FINGERPRINT MATCHING USING HYBRID SHAPE AND ORIENTATION DESCRIPTOR -AN IMPROV...
Image similarity using fourier transform
A novel embedded hybrid thinning algorithm for
Comparison of various Image Registration Techniques with the Proposed Hybrid ...
Ad

Similar to Features for Cross Spectral Image Matching: A Survey (20)

PDF
IRJET - Symmetric Image Registration based on Intensity and Spatial Informati...
PDF
A Survey on Image Retrieval By Different Features and Techniques
PDF
Video saliency detection using modified high efficiency video coding and back...
PDF
Rotation Invariant Face Recognition using RLBP, LPQ and CONTOURLET Transform
PDF
Ar4201293298
PDF
IRJET- Shape based Image Classification using Geometric ­–Properties
PDF
F44083035
PDF
Simplified Multimodal Biometric Identification
PDF
Biomedical Image Retrieval using LBWP
PPT
IEEE ICAPR 2009
PDF
Application of Image Retrieval Techniques to Understand Evolving Weather
PDF
COMPRESSION BASED FACE RECOGNITION USING DWT AND SVM
PDF
IRJET- Multi Image Morphing: A Review
PDF
Face skin color based recognition using local spectral and gray scale features
PDF
A Powerful Automated Image Indexing and Retrieval Tool for Social Media Sample
PDF
Image Fusion a survey Image Fusion a survey
PDF
Orientation Spectral Resolution Coding for Pattern Recognition
PDF
Medical Image Fusion Using Discrete Wavelet Transform
PDF
V.KARTHIKEYAN PUBLISHED ARTICLE 1
PDF
Implementation of Fuzzy Logic for the High-Resolution Remote Sensing Images w...
IRJET - Symmetric Image Registration based on Intensity and Spatial Informati...
A Survey on Image Retrieval By Different Features and Techniques
Video saliency detection using modified high efficiency video coding and back...
Rotation Invariant Face Recognition using RLBP, LPQ and CONTOURLET Transform
Ar4201293298
IRJET- Shape based Image Classification using Geometric ­–Properties
F44083035
Simplified Multimodal Biometric Identification
Biomedical Image Retrieval using LBWP
IEEE ICAPR 2009
Application of Image Retrieval Techniques to Understand Evolving Weather
COMPRESSION BASED FACE RECOGNITION USING DWT AND SVM
IRJET- Multi Image Morphing: A Review
Face skin color based recognition using local spectral and gray scale features
A Powerful Automated Image Indexing and Retrieval Tool for Social Media Sample
Image Fusion a survey Image Fusion a survey
Orientation Spectral Resolution Coding for Pattern Recognition
Medical Image Fusion Using Discrete Wavelet Transform
V.KARTHIKEYAN PUBLISHED ARTICLE 1
Implementation of Fuzzy Logic for the High-Resolution Remote Sensing Images w...
Ad

More from journalBEEI (20)

PDF
Square transposition: an approach to the transposition process in block cipher
PDF
Hyper-parameter optimization of convolutional neural network based on particl...
PDF
Supervised machine learning based liver disease prediction approach with LASS...
PDF
A secure and energy saving protocol for wireless sensor networks
PDF
Plant leaf identification system using convolutional neural network
PDF
Customized moodle-based learning management system for socially disadvantaged...
PDF
Understanding the role of individual learner in adaptive and personalized e-l...
PDF
Prototype mobile contactless transaction system in traditional markets to sup...
PDF
Wireless HART stack using multiprocessor technique with laxity algorithm
PDF
Implementation of double-layer loaded on octagon microstrip yagi antenna
PDF
The calculation of the field of an antenna located near the human head
PDF
Exact secure outage probability performance of uplinkdownlink multiple access...
PDF
Design of a dual-band antenna for energy harvesting application
PDF
Transforming data-centric eXtensible markup language into relational database...
PDF
Key performance requirement of future next wireless networks (6G)
PDF
Noise resistance territorial intensity-based optical flow using inverse confi...
PDF
Modeling climate phenomenon with software grids analysis and display system i...
PDF
An approach of re-organizing input dataset to enhance the quality of emotion ...
PDF
Parking detection system using background subtraction and HSV color segmentation
PDF
Quality of service performances of video and voice transmission in universal ...
Square transposition: an approach to the transposition process in block cipher
Hyper-parameter optimization of convolutional neural network based on particl...
Supervised machine learning based liver disease prediction approach with LASS...
A secure and energy saving protocol for wireless sensor networks
Plant leaf identification system using convolutional neural network
Customized moodle-based learning management system for socially disadvantaged...
Understanding the role of individual learner in adaptive and personalized e-l...
Prototype mobile contactless transaction system in traditional markets to sup...
Wireless HART stack using multiprocessor technique with laxity algorithm
Implementation of double-layer loaded on octagon microstrip yagi antenna
The calculation of the field of an antenna located near the human head
Exact secure outage probability performance of uplinkdownlink multiple access...
Design of a dual-band antenna for energy harvesting application
Transforming data-centric eXtensible markup language into relational database...
Key performance requirement of future next wireless networks (6G)
Noise resistance territorial intensity-based optical flow using inverse confi...
Modeling climate phenomenon with software grids analysis and display system i...
An approach of re-organizing input dataset to enhance the quality of emotion ...
Parking detection system using background subtraction and HSV color segmentation
Quality of service performances of video and voice transmission in universal ...

Recently uploaded (20)

PDF
Embodied AI: Ushering in the Next Era of Intelligent Systems
PPTX
FINAL REVIEW FOR COPD DIANOSIS FOR PULMONARY DISEASE.pptx
PPTX
M Tech Sem 1 Civil Engineering Environmental Sciences.pptx
PPTX
KTU 2019 -S7-MCN 401 MODULE 2-VINAY.pptx
PPTX
Welding lecture in detail for understanding
PDF
TFEC-4-2020-Design-Guide-for-Timber-Roof-Trusses.pdf
PPTX
web development for engineering and engineering
PDF
Well-logging-methods_new................
PDF
Mohammad Mahdi Farshadian CV - Prospective PhD Student 2026
PPTX
Sustainable Sites - Green Building Construction
PPT
Mechanical Engineering MATERIALS Selection
PPTX
CH1 Production IntroductoryConcepts.pptx
PPT
CRASH COURSE IN ALTERNATIVE PLUMBING CLASS
PDF
Model Code of Practice - Construction Work - 21102022 .pdf
PDF
SM_6th-Sem__Cse_Internet-of-Things.pdf IOT
PDF
BMEC211 - INTRODUCTION TO MECHATRONICS-1.pdf
PDF
keyrequirementskkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk
PDF
Automation-in-Manufacturing-Chapter-Introduction.pdf
PPTX
UNIT-1 - COAL BASED THERMAL POWER PLANTS
PPTX
UNIT 4 Total Quality Management .pptx
Embodied AI: Ushering in the Next Era of Intelligent Systems
FINAL REVIEW FOR COPD DIANOSIS FOR PULMONARY DISEASE.pptx
M Tech Sem 1 Civil Engineering Environmental Sciences.pptx
KTU 2019 -S7-MCN 401 MODULE 2-VINAY.pptx
Welding lecture in detail for understanding
TFEC-4-2020-Design-Guide-for-Timber-Roof-Trusses.pdf
web development for engineering and engineering
Well-logging-methods_new................
Mohammad Mahdi Farshadian CV - Prospective PhD Student 2026
Sustainable Sites - Green Building Construction
Mechanical Engineering MATERIALS Selection
CH1 Production IntroductoryConcepts.pptx
CRASH COURSE IN ALTERNATIVE PLUMBING CLASS
Model Code of Practice - Construction Work - 21102022 .pdf
SM_6th-Sem__Cse_Internet-of-Things.pdf IOT
BMEC211 - INTRODUCTION TO MECHATRONICS-1.pdf
keyrequirementskkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk
Automation-in-Manufacturing-Chapter-Introduction.pdf
UNIT-1 - COAL BASED THERMAL POWER PLANTS
UNIT 4 Total Quality Management .pptx

Features for Cross Spectral Image Matching: A Survey

  • 1. Bulletin of Electrical Engineering and Informatics Vol. 7, No. 4, December 2018, pp. 552~560 ISSN: 2302-9285, DOI: 10.11591/eei.v7i4.843  552 Journal homepage: http://journal.portalgaruda.org/index.php/EEI/index Features for Cross Spectral Image Matching: A Survey Maulisa Oktiana1 , Fitri Arnia2 , Yuwaldi Away3 , Khairul Munadi4 1,2,3,4 Post Graduate Program in Engineering, Universitas Syiah Kuala, Indonesia 2,3,4 Department of Electrical and Computer Engineering Universitas Syiah Kuala, Indonesia Article Info ABSTRACT Article history: Received Nov 11, 2017 Revised May 20, 2018 Accepted Oct 08, 2018 In recent years, cross spectral matching has been gaining attention in various biometric systems for identification and verification purposes. Cross spectral matching allows images taken under different electromagnetic spectrums to match each other. In cross spectral matching, one of the keys for successful matching is determined by the features used for representing an image. Therefore, the feature extraction step becomes an essential task. Researchers have improved matching accuracy by developing robust features. This paper presents most commonly selected features used in cross spectral matching. This survey covers basic concepts of cross spectral matching, visual and thermal features extraction, and state of the art descriptors. In the end, this paper provides a description of better feature selection methods in cross spectral matching. Keywords: Cross spectral matching Feature Robust features Thermal images Visible light images Copyright © 2018 Institute of Advanced Engineering and Science. All rights reserved. Corresponding Author: Khairul Munadi, Department of Electrical and Computer Engineering, Universitas Syiah Kuala, Jalan Teuku Nyak Arief, Darussalam, Banda Aceh 23111, Indonesia Email: khairul.munadi@unsyiah.ac.id 1. INTRODUCTION Cross-spectral matching also known as cross-spectrum matching is a matching process between two images taken on different electromagnetic spectrum [1]. Typically, we cross-matched the thermal spectrum images with visible light (VL) images. The thermal spectrum consists of four sub-bands, i.e., Near Infrared (SWIR), Short Wave Infrared (SWIR), Medium Wave Infrared (MWIR), and Longwave IR (LWIR) [2]. The study in the cross-spectral domain increases rapidly in line with applications of biometric systems [3]. Cross- spectral matching is widely used for security, national ID programs, also for personal identification and authentication using iris and face recognition [4-7]. By using cross-spectral image matching scheme, the identification and authentication process becomes more accurate and efficient because it utilises the additional information contained in both different spectrum and wavelength [8]. The performance of cross-spectral matching is dependent on features that can represent information from VL and thermal images. VL and thermal images represent information from the same subject even though the visual appearance and structure of the two images are different. Therefore the most challenging task in cross-spectral image matching is how to choose a representative feature for both VL and thermal images [9]. Various features are employed in cross-spectral image matching with high recognition performance. Trokielewicz [10] developed cross spectral mobile based verification using the Discrete Cosine Transform (DCT) and Gabor Wavelet features. DCT is used on Monro Iris Recognition Library (MIRLIN) software, whereas Gabor Wavelet is used on Open Source for IRIS (OSIRIS) software. DCT and Gabor wavelet features are suitable for cross spectral iris recognition with low Equal Error Rate (EER). Abdullah et al. [11] explored the Binarized Statistical Image Features (BSIF) to extract the statistical features from NIR and VL iris images. BSIF features able to represent the statistical properties of NIR and VL iris images appropriately with high iris recognition performance. Another interesting work was conducted by klare and
  • 2. BEEI ISSN: 2302-9285  Features for Cross Spectral Image Matching: A Survey (Maulisa Oktiana) 553 Jain [12] exploiting Histogram of Oriented Gradient (HOG) integrated with Local Binary Pattern (LBP) to describe the structure of the face in NIR and VL images. Experimental result showed that HOG and LBP have better performance in representing NIR and VIS images. A survey on state of the art feature in cross-spectral image matching is carried out in this paper. We describe several features based on the representation of VL and thermal image features. The main contribution of this paper is to provide a brief description about features in cross-spectral image matching so that it can help in selecting the most suitable feature used in the face and iris recognition. This paper is organized as follows. Section 2 describes basic background concepts in cross-spectral matching and the feature extraction, such as the definition of cross-spectral image matching, and block diagram matching process in the cross-spectral domain. Section 3 reviews the properties and characteristics of the VL and the thermal image features. Also, how to extracts features from the VL image and the NIR images for cross spectral matching are described in section 3. Section 4 reviews the robust feature used in cross-spectral matching including the feature extraction process. Conclusions are summarized in section 5. 2. CROSS SPECTRAL MATCHING Cross spectral matching represents the ability to recognize the objects presented in two different modalities. Cross spectral matching is illustrated in Figure 1. First, pre-processing was applied to prepare an image for further processing. The pre-processing step comprises image cropping, photometric and geometric normalization, and restoration. Next step is feature extraction which is carried out by taking unique features of thermal and VL image by using certain descriptors. The results of this process are used as inputs to the matching step using matching or classification algorithms. Figure 1. Cross spectral matching 3. VISIBLE AND THERMAL IMAGES FEATURES Image features are unique attributes of both numerical and alphanumerical that represent information about the content of a digital image. Numerical representation means any physical information of an image, such as heat, color, pressure, range, non visible wavelength, etc. replaced by numerical values that make it easier to be analyzed in various applications. In VL and thermal images, a feature can be either an image visual characteristics or interpretative response to the spatial, symbolic, semantic, or emotional image characteristics [13]. Feature extraction is an initial stage for all applications in image analysis. Feature extraction is known as the process of getting the unique characteristics of an image that distinguish one image with another. Feature extraction is used to indicate the relevant information of an image in completing the computational tasks associated with a particular application [14]. Feature extraction process for VL and thermal images are described in Figure 2. In cross spectral matching, performance and matching accuracy depends on proper feature extraction. VL and thermal images are retrieved based on the value of the vector feature, therefore, feature selection and feature extraction process are important to be considered. Feature selection is defined as a process of selecting the best features that suitable for a particular application [15]. The feature selection based on the visual properties of the image is shown in Table 1. Low level is defined as the image features can be extracted directly without considering the spatial relationship. While high-level concerns the spatial information. The feature selection process must consider [16]: Pre- processing Feature Extraction Probe Pre- processing Feature Extraction Matching Result Gallery
  • 3.  ISSN: 2302-9285 BEEI, Vol. 7, No. 4, December 2018 : 552 – 560 554 a. Similarities between two matched images, if the images similar the feature distance between this image is small. We can confirm that if VL and thermal images are similar the distance of feature vector of those two image is small. b. Computational task is not complex. c. Small feature dimensions do not affect the matching efficiency. Figure 2. Representation of image features Table 1. Visual properties of visible and thermal images Low level Middle level High level Color Shape Texture Regions Spatial relationship Localization Image categorization Objects identification d. The size of the dataset does not affect matching performance. e. Robust against variation illuminations and geometric transformation. Figure 3 presents several methods of feature extraction classification for cross spectral matching. Feature extraction methods divided into: a. Structural methods : this method identifies structural features of an image. Feature calculated based on topological and geometric properties [17]. b. Statistical methods : identifies statistical features of an image based on statistical distributions of pixels [17]. Visible Images Visual properties of VL images Numerical representation Feature extraction Feature vector Thermal Images Properties of thermal images Numerical representation Feature Extraction Feature vector Matching Indexing/Classify Similarity rate
  • 4. BEEI ISSN: 2302-9285  Features for Cross Spectral Image Matching: A Survey (Maulisa Oktiana) 555 c. Spectrum or transform based : feature calculated over spectral composition, low and high frequencies of an image. They can further be devided into : spatial, frequencies, and combined transform [14]. d. Block based : based on images block. This method combines the important feature of image into one block, consequently size and number of blocks affect the resulting features [14]. e. Combined methods : feature calculated by concatenated several individual methods to provide greater ability [14]. Figure 3. Classification of features extraction methods for cross spectral image 4. STATE OF THE ART FEATURE DESCRIPTORS In this section, we present some popular descriptors used in cross spectral matching. These descriptors frequently use because robust in describing VL and thermal image. The descriptors commonly used in the cross spectral matching are presented in Table 2. 4.1. Difference of Gaussian Filter The Difference of Gaussian (DoG) is a bandpass filter that used Gaussians filter to produce a normalized image [2]. DoG can suppress variations of noise and aliasing on the cross-spectral image caused by the frequency difference between VL and thermal images. Also, the DOG has a low computational complexity and popularly used in publications [18], [19]. DOG can accommodate difficult lighting condition by setting inner dan outer Gaussian values. For strong lighting variations datasets, the recognition gives the best result at outer Gaussian ≈ 2 pixels and up to about 4 for datasets less extreme lighting variations while the inner Gaussian is quite narrow at 1 pixel [20]. Band-pass filter is obtained by subtracting two Gaussians with different σ that eliminating all frequencies between frequencies cut-off of the two Gaussian. The image feature is located between this frequency band was extracted. The DOG feature extractions consists of three stages [20]: a. The input image is convolving with two Gaussian kernels having differing standard deviations as shown in (1) to produce a blurred image. b. Next two blurred images version is subtracted each other to obtain a normalized image. Fourier Transform Statistical Methods Local Binary Pattern (LBP) Structural / Geometric Histogram of Oriented Gradient (HOG) Combined Methods Log Gabor Wavelet Scale Invariant Feature Transform (SIFT) Gabor Local Binary Pattern (GLBP) Spatial Frequencies Spectrum Based Cumm ulative Sum based Fractal and Chaos Theory Color Texture Feature Gabor Transform Wavelet Transform Discrete Cosine Transform Cross Spectral Features Extraction Methods Binarized Statistical Image Features (BSIF) Block Based Additive Fusion Combined Transform Shift Invariant (SIFE) Fourier Mellin Radon Transform Local Ternary Pattern (LTP) Pyramid Histogram of Oriented Gradient (PHOG)
  • 5.  ISSN: 2302-9285 BEEI, Vol. 7, No. 4, December 2018 : 552 – 560 556 c. To construct a band-pass filter, the values of δ0 must be smaller than δ1 which set to (1) and (2) respectively ( ) [ ( ) ( )] ( ) (1) Where : ( ) : original image ( ) : blurred image : Gaussian kernel function defined as : ( ) √ ( )⁄ (2) Table 2. Various descriptors in cross spectral matching Publications Domain Modality Applications Feature Extraction Technique [2] NIR vs. VL Iris recognition Real valued -Log Gabor Phase [4] NIR vs. VL Face recognition LBP, DLBP [10] NIR vs. VL Iris recognition DCT, Gabor wavelet [11] NIR vs. VL Iris recognition Binarized Statistical Image Feature [12] NIR vs. VL Face recognition uniform LBP and HOG [16] NIR vs. VL Face recognition Log DoG LBP and Log DoG HOG [19] NIR vs. VL Face recognition DoG, Center Surround Divisive Normalization (CSDN), SIFT (Scale Invariant Feature Transform), and Multiscale Local Binary Pattern (MLBP) [21] NIR vs. VL, SWIR vs. VL Face recognition LBP and Generalized LBP (GLBP) [22] SWIR vs. VL Face recognition LBP and LTP [23] NIR vs. VL Face recognition Directional binary code [24] Thermal vs. VL Face recognition Pyramid Histogram of Oriented Gradient (PHOG), PSFIT [25] NIR vs. VL Face recognition Multi resolution LBP [26] MWIR vs. VL LWIR vs. VL Face recognition Canonical Correlation Analysis (CCA) [27] NIR vs. VL Face recognition Wavelet transform [28] NIR vs. VL Face recognition Sparse dictionary [29] SWIR vs. VL Face recognition Gabor filter, LBP, Simplified Weber Local Descriptor (SWLD) and GLBP [30] Thermal vs. VL Face recognition Deep Perceptual Mapping (DPM) [31] SWIR vs. VL Face recognition Contrast Limited Adaptive Histogram Equalization (CLAHE) [32] NIR vs. VL Face recognition Gaussian filter and SIFT 4.2. Local Binary Pattern At 1996, Local Binary Pattern (LBP) was first introduced by Ojala et al. [33]. The LBP operator is a texture descriptor gray-scale invariant that analyzes the texture of an image based its texture spectrum called Texture Unit (TU). Texture spectrum is a distribution of texture units happening in a region. Originally, LBP uses 3x3 neighborhood and generates 8-bit code based on the number of 8 pixels around the center pixel. Texture unit has 28=256 possibility histogram bin in describing the spatial pattern in a 3x3 neighborhood. They compute by multiplying the weights of the corresponding pixel with the values of the pixels in the previous threshold neighborhood. Then the pixel value of this neighborhood summed resulting the number of (169) texture unit [34]. The LBP operator defined as: ( ) ∑ ( ) (3)
  • 6. BEEI ISSN: 2302-9285  Features for Cross Spectral Image Matching: A Survey (Maulisa Oktiana) 557 Where ( ) { c, n represents a central pixel and 8 neighbors of the central pixel respectively. The 3x3 neighbor's pixel is thresholding by center pixel. If the neighbor's pixel greater than center pixel then the value is 1, and 0 otherwise. LBP operators further developed to accommodate variations of texture scaling into circular neighborhoods (8.1), (16.2) and (8.2). 4.3. Binary Statistical Image Feature Binary Statistical Image Feature (BSIF) is a texture descriptor which is inspired by Local Binary pattern (LBP). BSIF uses a binary code to represent each pixel neighborhood in an image [35]. The binarized feature is generated by convolving image with a set of linear filter. Thus, the response of a linear filter is binarized with a threshold at zero. To construct the linear filter, independent component analysis (ICA) is used. The statistical independence of the filter responses is maximizing by ICA from a training set of natural image patches. The BSIF extraction process as described in [36]: a. First, the response of the linear filter is constructed. Let X is image patch with size l x l pixels. is a linear filter and represents the response of the filter. ∑ ( ) ( ) (4) b. Binarized feature is obtained by : { (5) 4.4. Scale Invariant Feature Transform The Scale Invariant Feature Transform (SIFT) is a robust descriptor that combinations of DoG interest region detector with a corresponding feature descriptor. SIFT encodes image information in a localized set of gradient orientation histograms. Thus, SIFT can accommodates illumination variations and small positional shift [37]. 4.5. Histogram of Oriented Gradient Histogram of Oriented Gradient (HOG) also known as histogram normalization due to its ability to normalize from local responses [38]. HOG is adopted human detection and used for object detection applications. HOG resistant of shadowing process, illumination invariant, and reliable to photometric variations. HOG does not have spatial image representation, therefore, HOG only computes the individual pixel energy regardless spatial distribution of an image. The feature extraction process [39]: a. The image is divided into a small area called cell (8x8 pixels). b. Calculate the magnitude of the pixel orientation. c. Interpolation of result no 2 into histogram orientation bin 20 degrees. d. Cells are grouped into overlapping blocks. e. Then overlapping blocks are normalized. f. Finally the normalized histograms are concatenated resulted in vector features. 4.6. Additive Fusion Block Based Additive fusion block based proposed by Varadarajan et al. that used block-based extraction process [40]. Block based can maintain the optimum number of features that need to be extracted. Size and number of blocks affect the resulting features in this techniques. The more blocks, the less feature is extracted because of, the smaller block size causing the resultant block reduction. The ideal block sizes are 4,8, or 16 pixels. Feature extraction process consists: a. Image is divided into individual blocks of size 4.8, or 16 pixels b. Each block is applied Chirp Z-Transform (CZT) and Goertzel algorithm for preprocessing and image enhancement. c. Individual blocks are summed to yield a single resultant block that contains the essential features of each block. d. This resultant block is then become a vector feature. 4.7. Discrete Cosine Transform Discrete Cosine Transform (DCT) transforms spatial image information into frequency domain. DCT consists cosine part of Fourier transform models [41]. DCT concentrates energy image into some DCT
  • 7.  ISSN: 2302-9285 BEEI, Vol. 7, No. 4, December 2018 : 552 – 560 558 coefficients (energy compaction). Signal energy is concentrated on large DCT coefficient magnitudes (called low frequencies) and located in the upper-left corner of the DCT array. Low frequencies coefficients contain most of an essential image information. Therefore the original image can be reconstructed only by using low frequencies coefficients. While less important information is located in lower-right values of the DCT array (called high frequencies). This high-frequency coefficients can be discharged through the quantization process without significantly affecting the image quality [42-43]. DCT feature extraction process [44]: a. An original image is divided into 8x8 blocks. b. Calculated DCT coefficients using (4) resulted in DCT coefficient arrays. c. Quantized.the DCT coefficients. d. The value that is in upper-left corner of the DCT is used as a feature vector (low frequency). ( ) ( ) ( ) ∑ ∑ [ ( )] [ ( )] ( ) (6) ( ) ( ) { √ √ Where ; and ( ) represents intensity of the pixel in row x and column y [45]. 4.8. Radon Transform Radon transform is one of the transformations that can enhance low-frequency components by describing the integral line of an image. Radon transform is turned rotation into translation and often used in face recognition. Radon feature extraction process [46] : a. Calculated radon space using (5). b. Discriminative information located in radon space which computes using projections for 0°–179° orientations. c. Calculated DCT of radon space to obtain feature frequencies. d. Concatenated 25% of DCT coefficients resulted in feature vector. ( )[ ( )] ∫ ∫ ( ) ( ) (7) Where ( ) represents Dirac function, perpendicular distance of a line from the origin represented r ϵ[ ] and ϴ is an angle between X-axis and distance vector. 5. CONCLUSIONS The literature relevant cross spectral matching is overgrowing, and many researchers have proposed a cross spectral matching framework with better matching performance. This survey presents an overview feature of thermal and visible images. Also, brief descriptions the current state of the art feature extraction methods for cross spectral matching. The image features affect the performance of cross spectral matching. Therefore, the selection of feature extraction methods that suitable for an application becomes essential issue. Also, the researchers must consider visual properties of VL and thermal images. ACKNOWLEDGEMENTS This research was supported by the Ministry of Research, Technology, and Higher Education of the Republic of Indonesia, under the scheme, namely PMDSU. REFERENCES [1] R. S. Ghias, et al., “Infrared Face Recognition: A Comprehensive Review of Methodologies and Databases,” Pattern Recognition, vol. 47, pp. 2807–2824, 2014. [2] S. Hu, et al., “Thermal-to-Visible Face Recognition using Partial Least Squares,” Journal of the Optical Society of America, vol. 32, pp. 431-442, 2015.
  • 8. BEEI ISSN: 2302-9285  Features for Cross Spectral Image Matching: A Survey (Maulisa Oktiana) 559 [3] M. J. Burge and K. W. Bowyer. Handbook of Iris Recognition Advances in Computer Vision and Pattern Recognition. Springer, 2013:15-24. [4] R. Biswas et al., ,”A New Approach of Iris Detection and Recognition”, International Journal of Electrical and Computer Engineering, vol. 7, pp. 2530–2536, 2017. [5] F. Arnia and N. Pramita, “Enhancement of Iris Recognition System Based on Phase Only Correlation”, TELKOMNIKA, vol. 9, pp. 387–394, 2011. [6] F. Arnia, et al., “Ordinal Measure of Discrete Cosine Transform (DCT) Coefficients and Its Application to Fingerprint Matching”, International Jounal of Electrical and Computer Engineering, vol. 3, pp. 823–830, 2013. [7] T. S. Gunawan, et al., “Development of Efficient Iris Identification Algorithm using Wavelet Packets for Smartphone Application”, Indonesian Jounal of Electrical Engineering and informatics, vol. 8, pp. 450–456, 2017. [8] P. R. Nalla and A. Kumar, “Towards More Accurate Iris Recognition using Cross-Spectral Matching,” IEEE Transaction on Image Processing, vol. 26, pp. 1-14, 2016. [9] F. J. Xu, et al., “NIR-VIS Face Recognition Via Cross Spectral Joint Dictionary Learning and Reconstruction,” in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), 2015, pp. 141 – 150. [10] M. Trokielewicz and E. Bartuzi, “Cross-spectral Iris Recognition for Mobile Applications using High-quality Color Images,” Jounal of Telecommunications and Information Technology, vol. 3, pp. 91-97. 2016. [11] M. A. M. Abdullah, et al., “A Novel Framework for Cross-Spectral Iris Matching,” IPSJ Transaction on Computer Vision and Applications, vol. 8, pp. 1-12, 2016. [12] B. F. Klare and A. K. Jain, “HeTerogeneous Face Recognition: Matching NIR to Visible Light Images,” in Proceedings of 20th International Conference Pattern Recognition (ICPR), 2010, pp. 1513–1516. [13] R. Schettini, et al., Feature Extraction for Content Based Image Retrieval. In: Liu L, Ozsu MT. Encyclopedia of Database Systems. New York: Springer; 2009. 1115-1119. [14] Y. C. Cabrera, et al., Iris Feature Extraction Methods. Centro de Aplicaciones de Tecnologıa de Avanzada (CENATAV). 2142. 2013. [15] E. S. Kumar, et al., “Feature Selection and Extraction for Content-Based Image Retrieval,” International Journal of Mathematics Trends and Technology, vol. 3. Pp. 70–73, 2012. [16] K. Anuradha and K. Sankaranarayanan, “Statistical Feature Extraction to Classify Oral Cancers,” Journal of Global Research in Computer Science, vol. 4, pp. 8-12, 2013. [17] P. Vithlani and C. K. Kumbharana, “Structural and Statistical Feature Extraction Methods for Character and Digit Recognition,” International Journal of Computer Applications, vol. 120, pp. 43–47, 2015. [18] J. Y. Zhu, et al., “Matching NIR Face to VIS Face Using Transduction,” IEEE Transaction on Information Forensics and Security, vol. 9. Pp. 501–514, 2014. [19] D. Kang, et al., “Nighttime Face Recognition at Large Standoff: Cross-Distance and Cross-Spectral Matching,” Pattern Recognition, vol. 47, pp. 3750–3766, 2014. [20] X. Tan and B. Triggs, “Enhanced Local Texture Feature Sets for Face Recognition Under Difficult Lighting Conditions,” IEEE Transactions on Image Processing, vol. 16. pp. 1635 – 1650, 2010. [21] Z. X. Cao and N. A. Schmid, “Recognition Performance of Cross-Spectral Periocular Biometrics and Partial Face at Short and Long Standoff Distance,” Open Transaction on Information Processing, vol. 1, pp. 20-32, 2014. [22] N. D. Kalka et al., “Cross-Spectral Face Recognition in Heterogeneous Environments: a Case Study on Matching Visible to Short-Wave Infrared Imagery,” in 2011 International Joint Conference on Biometrics (IJCB). Washington, DC. 2011, pp. 1-8. [23] L. Shen, et al., “Face Recognition from Visible and Near-Infrared Images using Boosted Directional Binary Code”, in International Conference on Intelligent Computing (ICIC). Zhengzhou, China. 2011, pp. 404-411. [24] C. Chen and A. Ross, “Matching Thermal to Visible Face Images using Hidden Factor Analysis in A Cascaded Subspace Learning Framework”, Pattern Recognition, vol. 72, pp. 25–32, 2016. [25] J. Chen, et al., “Learning Mappings for Face Synthesis from Near Infrared to Visual Light Images”, in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Miami, USA. 2009, pp. 156–163. [26] N. Osia and T. Bourlai., “On Matching Visible to Passive Infrared Face Images using Image Synthesis and Denoising”, in Proceedings of 12th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2017). Washington, DC, USA. 2017, pp. 904 - 911. [27] A. Sea, et al., “Human Face Recognition using Random Forest Based Fusion of À-Trous Wavelet Transform Coefficients from Thermal and Visible Images”, AEU - International Journal of Electronics and Communications, vol. 70, pp. 1041–1049, 2016. [28] P. Buyssens and M. Revenu, “Visible and Infrared Face Identification Via Sparse Representation”, ISRN Machine Vision, vol. 1, pp. 1-11, 2013. [29] F. Nicolo and N. A. Schmid, “Long Range Cross-Spectral Face Recognition: Matching SWIR Against Visible Light Images”, IEEE Transaction on Information Forensics and Security, vol. 7, pp. 1717–1726, 2012. [30] M. S. Sarfraz and R. Stiefelhagen, “Deep Perceptual Mapping for Thermal to Visible Face Recognition”, in Proceedings of British Machine Vision Conference (BMVC). Swansea, UK .2015, pp. 1–11. [31] T. Bourlai, et al., “Cross-Spectral Face Verification in the Short Wave Infrared (SWIR) Band”, in Proceedings of International Conference on Pattern Recognition (ICPR). Istanbul, Turkey. 2010, pp. 1343–1347. [32] H. Maeng, et al., “Nighttime Face Recognition at Long Distance: Cross Distance and Cross Spectral Matching”, in Proceedings of Asian Conference on Computer Vision (ACCV). Taipei, Taiwan. 2013, pp. 708-721. [33] T. Ojala, et al., “A comparative study of texture measures with classification based on feature distributions”, Pattern Recognition. Vol. 29, pp. 51–59, 1996.
  • 9.  ISSN: 2302-9285 BEEI, Vol. 7, No. 4, December 2018 : 552 – 560 560 [34] T. Ojala, et al., “Multiresolution Gray-Scale and Rotation Invariant Texture Classification with Local Binary Patterns”, IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 24. pp. 971–987, 2002. [35] A. Hyvarinen et al., Natural Image Statistics. Springer, 2009. [36] J. Kannala and E. Rahtu, “BSIF: Binarized Statistical Image Features,” in 21st International Conference on Pattern Recognition (ICPR) , Tsukuba, Japan. 2012, pp. 1363-1366. [37] Z. Ghassabi, et al., “An Efficient Approach for Robust Multimodal Retinal Image Registration Based on UR-SIFT Features and PIIFD Descriptors”, EURASIP Journal on Image and Video Processing, vol. 25, pp. 1-16, 2013. [38] N. Dalal and B. Triggs, “Histograms of Oriented Gradients for Human Detection”, in Proceedings of IEEE Computer Vision and Pattern Recognition (CVPR). San Diego. 2005, pp. 886-893. [39] Z. R. Tan, et al., “Using Pyramid of Histogram of Oriented Gradients on Natural Scene Text Recognition”, in Proceedings of International Conference on Image Processing (ICIP). Paris. 2014, pp. 2629 – 2633. [40] K. K. Varadarajan, et al., “Face Recognition using Block Based Feature Extraction with CZT and Goertzel- Algorithm as A Preprocessing Technique”, Procedia Computer Science, vol. 46, pp. 1458–1467, 2015. [41] N. Ahmed, et al., “Discrete Cosine Transfom”, IEEE Transactions on Computers, vol. 23, pp. 90-93, 1974. [42] S. A. Khayam. The Discrete Cosine Transform (DCT): Theory and Aplication. Departement of Electrical and Computer Engineering Michigan State University. 2003. [43] G. M. Deepa, et al., “Face Recognition Using Spectrum-Based Feature Extraction”, Applied Soft Computing Joural, vol. 12, pp. 2913–2923, 2012. [44] S. Dabbaghchian, et al., “Feature Extraction using Discrete Cosine Transform and Discrimination Power Analysis with a Face Recognition Technology”, Pattern Recognition, vol. 43, pp. 1431–1440, 2010. [45] R. M. Ramadan and R. F. A. Kader, “Face Recognition using Particle Swarm Optimization-Based Selected Features”, International Journal of Signal Processing, Image Processing and Pattern Recognition, vol. 2, pp. 51– 66, 2009. [46] D. V. Jadhav and R. S. Holambe, “Rotation, Illumination Invariant Polynomial Kernel Fisher Discriminant Analysis using Radon and Discrete Cosine Transforms Based Features For Face Recognition”, Pattern Recognition, vol. 31, pp. 1002–1009, 2010. BIOGRAPHIES OF AUTHORS Maulisa Oktiana received B.Eng degree in electrical engineering from Syiah Kuala University, Banda Aceh, Indonesia, in 2013. She is currently a doctoral student at Graduate Program of Engineering, Syiah Kuala University. Her research interests include image processing and computer vision. Fitri Arnia received B. Eng degree from Universitas Sumatera Utara (USU), Medan in 1997. She finished her master and doctoral degree from Universsity of New South Wales (UNSW), Sydney, Australia and Tokyo Metropolitan University, Japan in 2004 and 2008 respectively. She has been with the Department of Electrical Engineering, Faculty of Engineering, Syiah Kuala University since 1999. Dr. Arnia was a visiting scholar in Tokyo Metropolitan University (TMU), Tokyo, Japan in 2013 and Suleyman Demirel University (SDU), Isparta, Turkey in 2017. She is a member of IEEE and APSIPA. Her research interests are signal, image and multimedia information processing. Yuwaldi Away received his B. Eng degree in Electrical-Computer Engineering from Sepuluh Nopember Institute of Technology Surabaya, Indonesia in 1964, the M.Sc from Bandung Institute of Technology (ITB) Bandung, Indonesia. He obtained his Ph.D. degrees in computer technology from the National University of Malaysia. Since 1990, he joined Syiah Kuala University as a teaching staff, and from 2007 until now he has been appointed as a Professor in Electrical Engineering. His current research interest include microprocessor based system, embedded system, FGPA, optimation and visualization. Khairul Munadi received the B.E. degree from Sepuluh Nopember Institute of Technology, Surabaya, Indonesia, in 1996, and the M.Eng. and Ph.D. degrees from Tokyo MetropolitanUniversity (TMU), Japan, in 2004 and 2007 respectively, all in electrical engineering. From1996 to 1999, he was with Alcatel Indonesia as a system engineer. Since 1999, he joined SyiahKuala University, Banda Aceh, Indonesia, as a lecturer at the Electrical Engineering Department.He was a visiting researcher at the Information and Communication Systems Engineering,Faculty of System Design, TMU, Japan, from March 2007 to March 2008. His research interests include multimedia signal processing and communications.