SlideShare a Scribd company logo
TELKOMNIKA, Vol.17, No.6, December 2019, pp.3110~3119
ISSN: 1693-6930, accredited First Grade by Kemenristekdikti, Decree No: 21/E/KPT/2018
DOI: 10.12928/TELKOMNIKA.v17i6.12857 ◼ 3110
Received April 6, 2019; Revised July 2, 2019; Accepted July 18, 2019
Regenerating face images from multi-spectral
palm images using multiple fusion methods
Raid Rafi Omar Al-Nima, Moatasem Yaseen Al-Ridha*, Farqad Hamid Abdulraheem
Technical Engineering College, Mosul, Northern Technical University, Mosul, Iraq
*Corresponding author, e-mail: moalridha@gmail.com
Abstract
This paper established a relationship between multi-spectral palm images and a face image based
on multiple fusion methods. The first fusion method to be considered is a feature extraction between different
multi-spectral palm images, where multi-spectral CASIA database was used. The second fusion method to
be considered is a score fusion between two parts of an output face image. Our method suggests that both
right and left hands are used, and that each hand aims to produce a significant part of
a face image by using a Multi-Layer Perceptron (MLP) network. This will lead to the second fusion part to
reconstruct the full-face image, in order to examine its appearance. This topology provided interesting results
of Equal Error Rate (EER) equal to 1.99%.
Keywords: face, fusion, MLP neural network, palmprint
Copyright © 2019 Universitas Ahmad Dahlan. All rights reserved.
1. Introduction
Biometric applications are currently extensively used in the case of recognition,
identification or authentication systems. That is because biological biometrics have unique
characteristics such as a face and palmprint [1]. Faces provide challenges, starting with extracting
the main and important parts, then using these parts to reconstruct images of the face
automatically. It is apparent that the image of the face has many details, for instance eyebrows,
eyes, nose, mouth, ears and the boundary of the face. Thus, auto-establishing the image of
the face to be clear enough to recognize is really complicated [2].
In the literature, several studies considere generating face features from another biometric
characteristic. This perhaps starts from the work of Sağiroğlu and Özkaya, where an intelligent
system of generating eyes features from Only fingerprints was presented [3]. Özkaya and
Sağiroğlu also presented a study of producing face borders from fingerprints by employing
the Artificial Neural Network (ANN) [4]. Then, Sağiroğlu and Özkaya published extended study for
producing main face features from fingerprints [5]. After that, Chitravanshi et al. proposed such an
interesting work of generating main face characteristics from palm features [6]. Al-Nima et al.
approached a novel work for predicting full face features from signatures, where this was the first
study of producing physiological biometrics from behavioural biometrics [7]. Yang et al.
reconstructed face images by utilizing a Manifold Constrained Convolutional Sparse Coding [8].
Lu et al. constructed high resolution face images from low resolution face images by using
conditional cycle Generative Adversarial Network (GAN) [9]. Jiang et al. considered a study of
generating 3D faces from the geometry details of a single face image [10]. Li et al. produced
photorealistic faces for recognizing facial sketches [11]. Mai et al. illustrates a Neighborly
de-convolutional neural Network (NbNet) method to reproduce face images from deep
templates [12]. Al-Nima et al. exploited hand-dorsal images to generate full face details by using
Back Propagation Neural (BPN) and Cascade-Forward Neural (CFN) Networks [13]. Up to now, it
has been noted that there is no reference in relation to reproduce face images from multi-spectral
hand images. Because vein patterns require substantial efforts to acquire, a system security can
be increased further. Therefore, this issue has been addressed in our work. The aims and
contributions of this paper are as follows:
- Proposing robust face regeneration system that can reconstruct the full details of face images.
- Exploiting muti-spectral images of right and left hands as required inputs. This would increase
the security and anti-spoofing of the suggested system.
TELKOMNIKA ISSN: 1693-6930 ◼
Regenerating face images from multi-spectral palm images... (Raid Rafi Omar Al-Nima)
3111
The employed objectives can be stated as follows: firstly, image preprocessing steps
were handled to extract and enhance the hand images. These preprocessings were eliminated
or cropped, morphological operations, adding top-hat characteristics and an unsharp filter.
Secondly, a feature fusion between multi-spectral images to combine the features, where Haar
wavelet fusion based on the mean rule was used. Thirdly, a wavelet transform was applied for
the enhanced and fused image. Fourthly, the MLP neural networks were trained by considering
a right hand to predict the inner face image and the left hand to predict the outer face image.
Fifthly, a score fusion was utilized to collect the face image according to the maximum or adding
rule. Sixthly, the same processings should be followed to test the MLPs using different data.
Finally, the last decision was taken and hence, Figures 1 and 2 demonstrate the block diagrams
for our proposed method. This new suggested topology will increase the security and effictiveness
of the biometric system.
This paper is organized as follows: the first section is the introduction, prior work and
the proposed method. The second section is the hand images extraction followed by
enhancement. The third section is in relation to the two types of fusions, features and the score.
The fourth section is for the MLPs artificial neural networks. The fifth section is for the results and
discussions, with the final section being the conclusion.
Figure 1. The block diagram of predicting face from hand images based on
MLP neural networks and multiple fusions
Figure 2. The block diagram of the preprocessing steps
2. Image Extraction and Enhancement
Data extraction and analysis are considered as the most critical and essential problems
in Image Processing (IM) and Artificial Intelligence (AI). In this paper, preprocessing steps are
adopted to extract, enhance and normalize the data in order to be prepared for the (MLPs)
networks. CASIA multi-spectral palm images database are employed to build a relationship with
the ORL face images database. The multi-spectral images of the palm consist of 7200 jpg images
for 100 different people. All these images are 8-bits gray-scale with six electromagnetic spectrums
starting with 460 nm, 630 nm, 700 nm, 850 nm, 940 nm and white respectively. This valuable
◼ ISSN: 1693-6930
TELKOMNIKA Vol. 17, No. 6, December 2019: 3110-3119
3112
database included images for both right and left hands. A CCD camera is positioned at the bottom
of a hand with some spectrum lights. There were no pegs or restrict positions for the palm in the
device, although it had a uniform background colour. Two sessions were organized to capture the
images. Each session took snap shots of three multi-spectrum images. The interval period
between each session was more than one month [14].
A number of morphological operations were adopted after the image elimination
to reduce any noise and maintain the hand image. To begin with, the cropping image was
an 8-bit grayscale denoted as 𝐺(𝑥, 𝑦): 𝑍2
→ [0,255] which needed to be converted to a binary
image defined as 𝐵(𝑥, 𝑦): 𝑍2
→ {0,1} [15]. Consequently, to convert the image from 8-bit grayscale
to a binary image a threshold  was executed in (1):
𝐵(𝑥, 𝑦) = {
1 𝑖𝑓 𝐺(𝑥, 𝑦) > 𝜃
0 𝑖𝑓 𝐺(𝑥, 𝑦) ≤ 𝜃
(1)
for the binary image 𝐵(𝑥, 𝑦) and a structuring element ℎ(𝑎, 𝑏), the erosion ⊝ and dilation ⨁ are
denoted as [16]:
(𝐵 ⊝ ℎ)(𝑥, 𝑦) = 𝑚𝑖𝑛{𝐵(𝑥 + 𝑎 , 𝑦 + 𝑏) − ℎ(𝑎, 𝑏)} (2)
(𝐵⨁ℎ)(𝑥, 𝑦) = 𝑚𝑎𝑥 {𝐵(𝑥 − 𝑎 , 𝑦 − 𝑏) + ℎ(𝑎, 𝑏)} (3)
small white objects should be removed from the binary image, while holding the very large area.
An open morphological operation may solve this issue. See (4):
(𝐵 ℎ) = ( 𝐵 ⊝ ℎ )⨁ ℎ 𝑖𝑓 𝐴𝑟𝑒𝑎 ≤ 𝜌 (4)
where:  is a specific area.
Nevertheless, there will still be some small objects connected to the largest hand area,
which will not be deleted by (4). To remove these objects, a complement image 𝐵̂ defined
as 𝐵̂(𝑥, 𝑦): 𝑍2
→ {1,0} was produced from the last operation. A major morphological operation was
performed consecutively, to clear the unexpected noise by setting them to 1's if
the neighbourhood majorities are ones [17, 18]. Hence, the palm image with fingers is easily
created by combining the original image with the complement image as described in (5):
𝐼𝑖𝑛(𝑥, 𝑦) = {
𝐺(𝑥, 𝑦) 𝑖𝑓 𝐵̂(𝑥, 𝑦) = 0
𝑠 + 𝐵̂(𝑥, 𝑦) 𝑖𝑓 𝐵̂(𝑥, 𝑦) = 1
(5)
where: 𝐼𝑖𝑛(𝑥, 𝑦) is the new created image, s is a small scalar value and 𝐵̂(𝑥, 𝑦) is the resulting
binary image after (4). An example of a hand image before and after the morphological
processings is given in Figure 3.
(a) (b) (c) (d) (e) (f)
Figure 3. A hand image before and after the morphological operations: (a) input image,
(b) eliminating and converting to the binary image, (c) removing the small areas,
(d) converting to the complement image, (e) removing the unexpected noise and
(d) combining the original image with a clear background
As each multi-spectral image has specific features, enhancements and feature
fusions are important to collect as many have characteristics. Obtaining top-hat values from each
TELKOMNIKA ISSN: 1693-6930 ◼
Regenerating face images from multi-spectral palm images... (Raid Rafi Omar Al-Nima)
3113
image, followed by adding them to the resulting original image 𝐼𝑖𝑛(𝑥, 𝑦) appeared to be
a high-quality enhancement. Initially, a structuring element is created as a disk shape of one’s
values [18]:
𝑆𝑒 = 𝑅2
× 𝜋 (6)
where: R is radius of pixels. Consequently, the top-hat details are isolated from the image as
shown in this equation [19]:
𝐼𝑚 𝑇 = 𝐼𝑖𝑛 – (𝐼𝑖𝑛 𝑆𝑒) (7)
where: 𝐼𝑚 𝑇 is the image after the top-hat filter and 𝑆𝑒 is the structuring element. Hence, these
top-hat features are added to the hand image according to (8) [18]:
𝐼𝑚 𝑂𝑇 = 𝐼𝑖𝑛 + 𝐼𝑚 𝑇 (8)
then, an unsharp filter is applied to enhance the 𝐼𝑚 𝑂𝑇 details of the edges. So, an 𝐼𝑚 𝑂𝑇𝑈 is
produced as follows [20]:
𝐼𝑚 𝑂𝑇𝑈(𝑥, 𝑦) = 𝐼𝑚 𝑂𝑇(𝑥, 𝑦) + 𝛽𝑈(𝑥, 𝑦) (9)
where: 𝛽 is a positive scale factor and 𝑈(𝑥, 𝑦) is the correction signal, calculated as an output of
a highpass filter [20]. An example of the image enhancement processing is given in Figure 4.
(a) (b) (c) (d)
Figure 4. An example of image enhancement processings: (a) a hand image needs to be
enhanced, (b) top-hat image details, (c) adding the top-hat details to the original hand image
and (d) sharpened image enhancement
3. Fusion
Fusion between different biometric feature acquirements could be considered as
a significant method to increase the ability and security of the systems [21]. Two fusion methods
are used: feature fusion based on the wavelet with mean rules to extract the hand features from
the multi-spectral images and score fusion to combine or collect the final face image.
3.1. Feature Fusion
A fusion between images is considered as an interesting technique, so as to increase the
level of efficiency of any biometric system. Collecting multiple information from different images
is a problem which can be solved by the fusion technique. Merging two images into one individual
image is a type of fusion ability that provides more data in a single image [22].
Multi-spectral hand images have many characteristics from palm, fingers and hand
geometry to vein, lines and small patterns. Fusion between each two multi-spectral image types
will maintain and cover the combination information. In this paper, image spectrums of 460 nm
were combined with the image spectrums of 630 nm. Similarly, 700 nm images were merged with
850 nm, whilst 940 nm were fused with the white light image. This feature fusion method is
implemented by wavelet fusion with the Haar signal, 4-levels and mean rule for the both
approximations and details parts. See Figure 5 (a).
According to this fusion, four coefficients will be generated for each image after
the wavelet transform (one approximation and three details). ImOTU1 and ImOTU2 are two
◼ ISSN: 1693-6930
TELKOMNIKA Vol. 17, No. 6, December 2019: 3110-3119
3114
multi-spectral palm images, (LL1) is the approximation and (LH1, HL1, HH1) are the details of
the first spectrum image. Likewise, (LL2) is the approximation and (LH2, HL2, HH2) are the details
of the second spectrum image. Hereafter, (10) is used to combine the information:
𝐼𝑚 𝑂𝑇𝑈3 = 𝐼𝐷𝑊𝑇 (𝐴𝑉(𝐿𝐿1, 𝐿𝐿2); 𝐴𝑉(𝐿𝐻1, 𝐻𝐿1, 𝐻𝐻1, 𝐿𝐻2, 𝐻𝐿2, 𝐻𝐻2)) (10)
where: IDWT is the inverse of the 2D wavelet transform and AV is the average.
(a)
(b)
Figure 5. Fusion methods: (a) feature fusion method between two spectral images
(940nm and white) based on the wavelet method and (b) score fusion method
between two face image parts after the MLP neural networks
3.2. Score Fusion
Score matching level fusion is a method that is used extensively with the multiple
biometric models [23]. It is performed after the authentication processing, where each output is
calculated individually and subsequently, a combination scoring level is applied [24]. The ORL
database of face images are used in this paper. This database is produced in AT&T laboratories
at Cambridge University through the collaboration of three groups (Speech, Vision and Robotics).
It consists of 400 images from 40 people and each person has a different expression [25].
The critical problem in our work is generating face images using score level fusion. Afterwards, a
decision is taken according to the most clear and distinct image. To explain in more details, two
essential parts of a face image are intended to be predicted by using MLP neural networks. This
is the middle part of a face image, which mainly consists of the eyes, nose and mouth. Thus,
the right-hand images are used to predict this middle part. The outer part of a face, which
commonly has the ears, hair and lower jaw employs the left-hand images to predict this boundary
part. Subsequently, a score fusion is performed by using the maximum or adding rule to construct
the face image. Figure 5 (b) shows the idea of the score fusion level.
Assume FA1 is the inner (middle) face image part and FA2 is the outer face image part.
Next, FA is the fused image from the middle and the boundary parts. See (11):
TELKOMNIKA ISSN: 1693-6930 ◼
Regenerating face images from multi-spectral palm images... (Raid Rafi Omar Al-Nima)
3115
𝐹𝐴 = 𝑀𝑎𝑥(𝐹𝐴1; 𝐹𝐴2) (11)
According to this technique the security of the system will increase because two hand images are
required to prove the face image.
4. Neural Network
Artificial Neural Network (ANN) is one of the most popular types of the Artificial
Intelligence (AI). In recent years, it has become widespread in different fields. Biometric
verification, identification and classification are examples of these fields. There are two main types
of ANN, supervised and unsupervised. Each one of these types attempts to simulate
a significant task in a human brain. Moreover, there are two essential stages in any ANN:
the learning stage and the testing stage. In the first stage, the network learns the inputs and
generates specific weights to manage the problem. In the second stage, the ANN deals with other
inputs which have not been seen before [26]. In our work, supervised MLP neural networks are
investigated and adapted to achieve their tasks.
First of all, the input data of each image need to be prepared for the MLP network. Thus,
each input image is segmented into specific matrices with different sizes of 5×5, 7×7, …, 13×13
pixels. This will ensure providing different overlaps between the matrices. A Coefficient of
variance is calculated to each segment as illuminated in (12)-(14) [27]:
𝑀𝑒𝑎𝑛 𝑠𝑒𝑔 =
1
𝑛
∑ 𝑠𝑒𝑔𝑖
𝑛
𝑖=1
(12)
𝑆𝐷𝑠𝑒𝑔 = √
1
𝑛 − 1
∑(𝑠𝑒𝑔𝑖 − 𝑀𝑒𝑎𝑛 𝑠𝑒𝑔)2
𝑛
𝑖=1
(13)
𝐶𝑉𝑠𝑒𝑔 =
𝑆𝐷𝑠𝑒𝑔
𝑀𝑒𝑎𝑛 𝑠𝑒𝑔
(14)
where: n is the number of pixels in each segment, seg is the matrix of 5×5, 7×7, 9×9, 11×11 or
13×13 pixels, Mean is the average, SD is the standard deviation and CV is the coefficient of
variance. The advantages of using the CV are: no dimension units can be considered, all values
are positive, the differences will be given as small ratio values (this will avoid the ANNs overload
in the next stage), the variances between the same vector type can be calculated (this will be
valuable for the same target in the training stage), and the variances between the different vector
types can be determined (this will be useful for the stage of fusion between two different types).
The second step, is arranging the CV values into a one-dimensional vector for each image.
The final input preparation is mapping the input data in [0,1] range as shown in (15) [28]:
𝐶𝑉𝑛𝑒𝑤 =
(max(𝐶𝑉𝑛𝑒𝑤) − min(𝐶𝑉𝑛𝑒𝑤)) × (𝐶𝑉 − min(𝐶𝑉))
(max(𝐶𝑉) − min(𝐶𝑉))
+ min(𝐶𝑉𝑛𝑒𝑤) (15)
5. Results and Discsussions
The performance of the proposed method is examined and compared with other
work. The databases are collected as well as organized into groups. An input group of
4020 multi-spectral image is used in the ANNs training stage and another input group of
804 multi-spectral images utilized in ANNs testing stage. In addition, each one of the two groups
have been separated into two other groups; the left hand and right-hand groups. In the training
stage, each hand group contained 2010 images. In the testing stage, each hand group consisted
of 402 images. Both training and testing stages of the MLP network attempted to predict a clear
and easily recognized part of a facial image, where each individual part has
its own MLP. All trainings have been accomplished by the algorithm of the Scaled Conjugate
Gradient (SCG), which is described in [29]. Examples of training curves are given in Figure 6.
◼ ISSN: 1693-6930
TELKOMNIKA Vol. 17, No. 6, December 2019: 3110-3119
3116
The minimum Mean Square Error (MSE) in all training is equal to 0.00001. Furthermore, training
for both the right and left hands succeeded in achieving the minimum error with the appropriate
number of epochs.
(a) (b)
Figure 6. An example of a training curve for: (a) the right hand multi-spectral images to predict
the inner part of a face image and (b) the left hand multi-spectral images to predict the outer
part of a face image
For the regression test, both trainings attained 45 degrees or Regression (R) equal
to 1 between the MLP outputs and targets. See Figure 7. To analyse Figure 7, the non-linear
relationships were established between two biometrics, which are the right and left hand with
the inner and outer part of a face, respectively. This relationship is the base for predicting a full
face image from inputs, which have never been seen before. From this point, predicting parts of
some face images are shown in Figures 8 (a and b). Whilst, the combination between each two
parts are displayed in Figure 8 (c).
(a) (b)
Figure 7. Regression test for: (a) a right-hand training and (b) a left-hand training
As could be observed, very clear face parts can be combined if new known vectors are
introduced to the trained MLPs. Conversely, the MLPs cannot recognize a face image clearly
when new unknown vectors are tested. Figures 9 (a, b and c) have examples of unclear face
images. As mentioned before, using the two main face parts will increase the security of
0 50 100 150
0
50
100
150
Target
Output~=1*Target+-1.5e-05
Training: R=1
Data
Fit
Y = T
0 50 100 150 200
0
50
100
150
200
Target
Output~=1*Target+-6.5e-06
Training: R=1
Data
Fit
Y = T
TELKOMNIKA ISSN: 1693-6930 ◼
Regenerating face images from multi-spectral palm images... (Raid Rafi Omar Al-Nima)
3117
the system. Moreover, producing a clear face image will assist any inexperienced individual to
easily authorize or identify the certain person.
In the final decision process, the image which is the most clear and nearest to
the specific vector will be considered as a 'true' and the image which is the most distorted and
furthest from the specific information will be considered as a 'false'. Thus, two types of
classification were determined. Hereafter, the three main parameters are calculated: False
Acceptance Rate (FAR), False Rejection Rate (FRR) and EER. The first two parameters are
calculated according to the following equations:
𝐹𝐴𝑅 =
𝑁𝑢𝑚𝑏𝑒𝑟 𝑜𝑓 𝑓𝑎𝑙𝑠𝑒 𝑎𝑐𝑐𝑒𝑝𝑡𝑒𝑑 𝑖𝑚𝑝𝑜𝑠𝑡𝑒𝑟𝑠
𝑇𝑜𝑡𝑎𝑙 𝑛𝑢𝑚𝑏𝑒𝑟 𝑡ℎ𝑒 𝑜𝑢𝑡𝑝𝑢𝑡 𝑠𝑎𝑚𝑝𝑙𝑒𝑠
(16)
𝐹𝑅𝑅 =
𝑁𝑢𝑚𝑏𝑒𝑟 𝑜𝑓 𝑓𝑎𝑙𝑠𝑒 𝑟𝑒𝑗𝑒𝑐𝑡𝑒𝑑 𝑔𝑒𝑛𝑢𝑖𝑛𝑒𝑠
𝑇𝑜𝑡𝑎𝑙 𝑛𝑢𝑚𝑏𝑒𝑟 𝑡ℎ𝑒 𝑜𝑢𝑡𝑝𝑢𝑡 𝑠𝑎𝑚𝑝𝑙𝑒𝑠
(17)
the EER parameter is calculated from the equality between FAR and FRR through different
threshold values, see Figure 10.
(a) (b) (c) (a) (b) (c)
Figure 8. Clear faces parts images:
(a) inner faces parts, (b) outer faces parts and
(c) a combination between the two parts
Figure 9. Distorted faces parts images:
(a) inner faces parts, (b) outer faces parts and
(c) a combination between the two parts
Figure 10. FAR versus FRR to acquire EER
To confirm the efficiency of our method, comparisons with other work including
the state-of-the-art have been established in Table 1. The reason of selecting the two studies
◼ ISSN: 1693-6930
TELKOMNIKA Vol. 17, No. 6, December 2019: 3110-3119
3118
in [7, 13] is that both of these works concentrated on regenerating full details of face images.
In addition, all of the reported works in Table 1 have used the ORL face images database. From
this table it can be seen that simple statistics were used with ANN techniques in [7, 13], where
the system strength level of the proposed systems is high. In this study, two fusion methods and
two multi-spectral hand images (right and left) have been employed to regenerate full face details.
Therefore, the strength level is very-high as spoofing the suggested system is so difficult.
The EER values have been recorded to 10% for [7] and 6.43% and 2.86% for [13].
In this proposed approach, the EER value has been recorded equal to (1.99%). So, our work
appears to have more accurate resutls than other studies.
Table 1. Comparisons of Various Face Regeneration Methods
Model Face Regenerating Methods System Strength Level EER
Al-Nima et al. [7] Simple statistics with the MLP High 10%
Al-Nima et al. [13]
Simple statistics with the BPN High 6.43%
Simple statistics with the DFN High 2.86%
Proposed approach Two fusion methods with the MLP Very-High 1.99%
6. Conclusion
A strategy to predict face image with high-level security was produced in this paper, where
two non-linear relationships have been established. Firstly, a relationship between
the right-hand features and the middle part of the facial image, which in general contain
the eyes, nose and mouth. Secondly, a relationship between the left-hand features and
the boundary part of the face image. In this paper, two levels of fusion are examined; the feature
level and the score level. The idea behind using the feature level is to combine and enhance
the multi-spectral hand characteristics, whilst, the score level is used on the face images to collect
and reconstruct clear details of a face.
The suggested approach structure confirmed its efficiency and robustness.
The performance of overall technique was benchmarked to EER =1.99% during the testing stage,
where full face details were reconstructed. In addition, the proposed system increases
the anti-spoofing, strength and security levels. This is because two multi-spectral images of the
two hands (left and right) are required to regenerate all face details.
Acknowledgment
“Portions of the research in this paper use the CASIA-MS-Palmprint V1 collected by
the Chinese Academy of Sciences' Institute of Automation (CASIA)”. In addition,
an acknowledgment is given to AT&T laboratories of Cambridge University.
References
[1] Liu YF, Lin CY, Guo JM. Impact of the Lips for Biometrics. IEEE Transactions on Image Processing.
2012; 21: 3092-3101.
[2] Özkaya N, Sağiroğlu Ş. Generating One Biometric Feature from Another: Faces from Fingerprints.
Sensors. 2010; 10(5): 4206-4237.
[3] Sağiroğlu Ş, Özkaya N. An Intelligent and Automatic Eye Generation System from Only Fingerprints.
Proceedings of Information Security and Cryptology Conference with International Participation.
Ankara. 2008: 231-236.
[4] Özkaya N, Sağiroğlu Ş. Intelligent face border generation system from fingerprints. 2008 IEEE
International Conference on Fuzzy Systems (IEEE World Congress on Computational Intelligence.
2008: 2169-2176.
[5] Sağiroğlu Ş, Özkaya N. An intelligent face features generation system from fingerprints. Turk J Elec
Eng & Comp Sci. 2009; 17(2): 183-203.
[6] Chitravanshi A, Singh A, Solanki S, Kumar A, Sharma R. Generating Face Features by Palm Scan:
Recognizing a Person Uniquely. International Journal of Applied Information Systems (IJAIS). 2012;
2(5): 36-41.
[7] Al-Nima RR, Dlay SS, Woo WL. A New Approach to Predicting Physical Biometrics from Behavioural
Biometrics. World Academy of Science, Engineering and Technology, International Journal of
Computer, Electrical, Automation, Control and Information Engineering. 2014; 8(11): 1996-2001.
TELKOMNIKA ISSN: 1693-6930 ◼
Regenerating face images from multi-spectral palm images... (Raid Rafi Omar Al-Nima)
3119
[8] Yang L, Li C, Han J, Chen C, Ye Q, Zhang B, Cao X, Liu W. Image Reconstruction via Manifold
Constrained Convolutional Sparse Coding for Image Sets. IEEE Journal of Selected Topics in Signal
Processing. 2017; 11(7): 1072-81.
[9] Lu Y, Tai YW, Tang CK. Conditional cyclegan for attribute guided face image generation. arXiv preprint
arXiv:1705.09966. 2017.
[10] Jiang L, Zhang J, Deng B, Li H, Liu L. Liu. 3D Face Reconstruction With Geometry Details From a
Single Image. IEEE Transactions on Image Processing. 2018; 27(10): 4756-70.
[11] Li X, Yang X, Su H, Zhou Q, Zheng S. Recognizing Facial Sketches by Generating Photorealistic Faces
Guided by Descriptive Attributes. IEEE Access. 2018; 6: 77568-80.
[12] Mai G, Cao K, Yuen PC, Jain AK. On the Reconstruction of Face Images from Deep Face Templates.
IEEE Transactions on Pattern Analysis and Machine Intelligence. 2019; 41(5): 1188-202.
[13] Al-Nima RRO, Abdulraheem FH, Al-Ridha MY. Using Hand-Dorsal Images to Reproduce Face Images
by Applying Back propagation and Cascade-Forward Neural Networks. 2nd International Conference
on Electrical, Communication, Computer, Power and Control Engineering ICECCPCE19. Mosul. 2019.
[14] Paul C, Courtney R, Sanson-Fisher R, Carey M, Hill D, Simmons J, Rose S. A randomized controlled
trial of the effectiveness of a pre-recruitment primer letter to increase participation in a study of
colorectal screening and surveillance. BMC Medical Research Methodology. 2014; 14(44).
[15] He Z. Hierarchical Colorant-Based Direct Binary Search Halftoning. IEEE Transactions on Image
Processing. 2010; 19(7): 1824-1836.
[16] Bai MR, Krishna VV, SreeDevi J. A new Morphological Approach for Noise Removal cum Edge
Detection. IJCSI International Journal of Computer Science Issues. 2010; 7(6):187.
[17] Jamil N, Sembok TM, Bakar ZA. Noise removal and enhancement of binary images using
morphological operations. International Symposium on Information Technology ITSim 2008. 2008; 4:
1-6.
[18] The MathWorks Inc. Image Processing Toolbox For Use with MATLAB, Ver.3 ed., 1993-2001.
[19] Kekre HB, Mukherjee P, Wadhwa S. Image Retrieval with Shape Features Extracted using
Morphological Operators with BTC. International Journal of Computer Applications. 2010; 12(3):
0975–8887.
[20] Polesel A, Ramponi G, Mathews VJ. Image enhancement via adaptive unsharp masking. IEEE
transactions on image processing. 2000; 9(3): 505-510.
[21] Al-Nima RRO. Signal processing and machine learning techniques for human verification based on
finger textures. PhD thesis. School of Engineering, Newcastle University. 2017.
[22] Sale D, Sonare P, Joshi MA. PCA Based Image Fusion for Multispectral Palm Enhancement.
International Journal of Advanced Research in Electrical, Electronics and Instrumentation Engineering.
2014; 3z2): 7501-7508.
[23] Bhokare R, Sale D, Joshi MA, Gaikwad MS. Multispectral Palm Image Fusion: A Critical Review.
International Journal of Advanced Research in Computer Engineering & Technology (IJARCET). 2013;
2(6): 2159-2164.
[24] Merkle J, Kevenaar T, Korte U. Multi-modal and multi-instance fusion for biometric cryptosystems. 2012
BIOSIG-Proceedings of the International Conference of the Biometrics Special Interest Group
(BIOSIG). 2012: 1-6.
[25] Reinhardt K, Weiss S, Rosenbauer J, Gärtner J, Von Kries R. Multiple sclerosis in children and
adolescents: incidence and clinical picture - new insights from the nationwide German surveillance
(2009-2011). European Journal of Neurology. 2014; 21(4): 654-9.
[26] Fausett L. Fundamental of Neural Networks, Architectures, Algorithms and Applications. Prentice Hall
Int. Snc. 1994.
[27] Li J, Yin G, Zhang G. Evaluation of tobacco mixing uniformity based on chemical composition. 2012
31st Chinese Control Conference (CCC). 2012: 7552-7555.
[28] Luz AG, Osis MJ, Ribeiro M, Cecatti JG, Amaral E. Perspectives of professionals participating in
the Brazilian Network for the Surveillance of Severe Maternal Morbidity regarding the implementation
of routine surveillance: a qualitative study. Reproductive Health. 2014; 11(1): 29.
[29] Møller MF. A scaled conjugate gradient algorithm for fast supervised learning. Neural Networks. 1993;
6(4): 525-533.

More Related Content

PDF
FPGA ARCHITECTURE FOR FACIAL-FEATURES AND COMPONENTS EXTRACTION
PDF
A survey on human face recognition invariant to illumination
PDF
A novel approach for efficient skull stripping using
PDF
A novel approach for efficient skull stripping using morphological reconstruc...
PDF
Fourier mellin transform based face recognition
PDF
Face Detection for identification of people in Images of Internet
PDF
2. 7698 8113-1-pb
PDF
Extraction of texture features by using gabor filter in wheat crop disease de...
FPGA ARCHITECTURE FOR FACIAL-FEATURES AND COMPONENTS EXTRACTION
A survey on human face recognition invariant to illumination
A novel approach for efficient skull stripping using
A novel approach for efficient skull stripping using morphological reconstruc...
Fourier mellin transform based face recognition
Face Detection for identification of people in Images of Internet
2. 7698 8113-1-pb
Extraction of texture features by using gabor filter in wheat crop disease de...

What's hot (17)

PDF
Automatic rectification of perspective distortion from a single image using p...
PDF
EV-SIFT - An Extended Scale Invariant Face Recognition for Plastic Surgery Fa...
PDF
H1802054851
PDF
Face Recognition Using Neural Network Based Fourier Gabor Filters & Random Pr...
PDF
Image Stitching Algorithm: An Optimization between Correlation-Based and Feat...
PDF
A Detailed Analysis on Feature Extraction Techniques of Panoramic Image Stitc...
PDF
Facial expression identification using four bit co- occurrence matrixfeatures...
PDF
Multi modal face recognition using block based curvelet features
PDF
Brain tumor segmentation using asymmetry based histogram thresholding and k m...
PDF
Pupil Detection Based on Color Difference and Circular Hough Transform
PDF
Ijetcas14 315
PDF
FACE RECOGNITION USING DIFFERENT LOCAL FEATURES WITH DIFFERENT DISTANCE TECHN...
PDF
H0334749
PDF
Content Based Image Retrieval Approach Based on Top-Hat Transform And Modifie...
PDF
40120140506007
PDF
IRJET- Digit Identification in Natural Images
PDF
proj525
Automatic rectification of perspective distortion from a single image using p...
EV-SIFT - An Extended Scale Invariant Face Recognition for Plastic Surgery Fa...
H1802054851
Face Recognition Using Neural Network Based Fourier Gabor Filters & Random Pr...
Image Stitching Algorithm: An Optimization between Correlation-Based and Feat...
A Detailed Analysis on Feature Extraction Techniques of Panoramic Image Stitc...
Facial expression identification using four bit co- occurrence matrixfeatures...
Multi modal face recognition using block based curvelet features
Brain tumor segmentation using asymmetry based histogram thresholding and k m...
Pupil Detection Based on Color Difference and Circular Hough Transform
Ijetcas14 315
FACE RECOGNITION USING DIFFERENT LOCAL FEATURES WITH DIFFERENT DISTANCE TECHN...
H0334749
Content Based Image Retrieval Approach Based on Top-Hat Transform And Modifie...
40120140506007
IRJET- Digit Identification in Natural Images
proj525
Ad

Similar to Regenerating face images from multi-spectral palm images using multiple fusion methods (20)

PDF
Volume 2-issue-6-2159-2164
PDF
Volume 2-issue-6-2159-2164
PDF
Palmprint Identification Using FRIT
PDF
Palm print recognition based on harmony search algorithm
PDF
Palmprint and Handgeometry Recognition using FAST features and Region properties
PDF
The International Journal of Engineering and Science (The IJES)
PPT
biometrics.ppt
PDF
Graph fusion of finger multimodal biometrics
PDF
HVDLP : HORIZONTAL VERTICAL DIAGONAL LOCAL PATTERN BASED FACE RECOGNITION
PDF
50120130405034
PDF
Palmprint verification using lagrangian decomposition and invariant interest
PDF
Palmprint verification using lagrangian decomposition and invariant interest
PDF
Hand geometry recognition: an approach for closed and separated fingers
PDF
A Spectral Domain Dominant Feature Extraction Algorithm for Palm-print Recogn...
PDF
Study of person identification using palmprint
PDF
Human identification using finger images
PDF
A hybrid learning scheme towards authenticating hand-geometry using multi-mo...
PDF
The International Journal of Engineering and Science (IJES)
PDF
A Textural Approach to Palmprint Identification
PDF
Image enhancement in palmprint recognition: a novel approach for improved bio...
Volume 2-issue-6-2159-2164
Volume 2-issue-6-2159-2164
Palmprint Identification Using FRIT
Palm print recognition based on harmony search algorithm
Palmprint and Handgeometry Recognition using FAST features and Region properties
The International Journal of Engineering and Science (The IJES)
biometrics.ppt
Graph fusion of finger multimodal biometrics
HVDLP : HORIZONTAL VERTICAL DIAGONAL LOCAL PATTERN BASED FACE RECOGNITION
50120130405034
Palmprint verification using lagrangian decomposition and invariant interest
Palmprint verification using lagrangian decomposition and invariant interest
Hand geometry recognition: an approach for closed and separated fingers
A Spectral Domain Dominant Feature Extraction Algorithm for Palm-print Recogn...
Study of person identification using palmprint
Human identification using finger images
A hybrid learning scheme towards authenticating hand-geometry using multi-mo...
The International Journal of Engineering and Science (IJES)
A Textural Approach to Palmprint Identification
Image enhancement in palmprint recognition: a novel approach for improved bio...
Ad

More from TELKOMNIKA JOURNAL (20)

PDF
Earthquake magnitude prediction based on radon cloud data near Grindulu fault...
PDF
Implementation of ICMP flood detection and mitigation system based on softwar...
PDF
Indonesian continuous speech recognition optimization with convolution bidir...
PDF
Recognition and understanding of construction safety signs by final year engi...
PDF
The use of dolomite to overcome grounding resistance in acidic swamp land
PDF
Clustering of swamp land types against soil resistivity and grounding resistance
PDF
Hybrid methodology for parameter algebraic identification in spatial/time dom...
PDF
Integration of image processing with 6-degrees-of-freedom robotic arm for adv...
PDF
Deep learning approaches for accurate wood species recognition
PDF
Neuromarketing case study: recognition of sweet and sour taste in beverage pr...
PDF
Reversible data hiding with selective bits difference expansion and modulus f...
PDF
Website-based: smart goat farm monitoring cages
PDF
Novel internet of things-spectroscopy methods for targeted water pollutants i...
PDF
XGBoost optimization using hybrid Bayesian optimization and nested cross vali...
PDF
Convolutional neural network-based real-time drowsy driver detection for acci...
PDF
Addressing overfitting in comparative study for deep learningbased classifica...
PDF
Integrating artificial intelligence into accounting systems: a qualitative st...
PDF
Leveraging technology to improve tuberculosis patient adherence: a comprehens...
PDF
Adulterated beef detection with redundant gas sensor using optimized convolut...
PDF
A 6G THz MIMO antenna with high gain and wide bandwidth for high-speed wirele...
Earthquake magnitude prediction based on radon cloud data near Grindulu fault...
Implementation of ICMP flood detection and mitigation system based on softwar...
Indonesian continuous speech recognition optimization with convolution bidir...
Recognition and understanding of construction safety signs by final year engi...
The use of dolomite to overcome grounding resistance in acidic swamp land
Clustering of swamp land types against soil resistivity and grounding resistance
Hybrid methodology for parameter algebraic identification in spatial/time dom...
Integration of image processing with 6-degrees-of-freedom robotic arm for adv...
Deep learning approaches for accurate wood species recognition
Neuromarketing case study: recognition of sweet and sour taste in beverage pr...
Reversible data hiding with selective bits difference expansion and modulus f...
Website-based: smart goat farm monitoring cages
Novel internet of things-spectroscopy methods for targeted water pollutants i...
XGBoost optimization using hybrid Bayesian optimization and nested cross vali...
Convolutional neural network-based real-time drowsy driver detection for acci...
Addressing overfitting in comparative study for deep learningbased classifica...
Integrating artificial intelligence into accounting systems: a qualitative st...
Leveraging technology to improve tuberculosis patient adherence: a comprehens...
Adulterated beef detection with redundant gas sensor using optimized convolut...
A 6G THz MIMO antenna with high gain and wide bandwidth for high-speed wirele...

Recently uploaded (20)

PPTX
CYBER-CRIMES AND SECURITY A guide to understanding
PPTX
Geodesy 1.pptx...............................................
PDF
PRIZ Academy - 9 Windows Thinking Where to Invest Today to Win Tomorrow.pdf
PDF
composite construction of structures.pdf
PDF
SM_6th-Sem__Cse_Internet-of-Things.pdf IOT
PDF
Well-logging-methods_new................
PDF
Evaluating the Democratization of the Turkish Armed Forces from a Normative P...
PPTX
Construction Project Organization Group 2.pptx
PPTX
IOT PPTs Week 10 Lecture Material.pptx of NPTEL Smart Cities contd
DOCX
573137875-Attendance-Management-System-original
PDF
Mitigating Risks through Effective Management for Enhancing Organizational Pe...
PDF
BMEC211 - INTRODUCTION TO MECHATRONICS-1.pdf
PDF
Embodied AI: Ushering in the Next Era of Intelligent Systems
PDF
PPT on Performance Review to get promotions
PPTX
Internet of Things (IOT) - A guide to understanding
PPTX
KTU 2019 -S7-MCN 401 MODULE 2-VINAY.pptx
PPTX
CARTOGRAPHY AND GEOINFORMATION VISUALIZATION chapter1 NPTE (2).pptx
PPTX
bas. eng. economics group 4 presentation 1.pptx
PDF
Automation-in-Manufacturing-Chapter-Introduction.pdf
PDF
R24 SURVEYING LAB MANUAL for civil enggi
CYBER-CRIMES AND SECURITY A guide to understanding
Geodesy 1.pptx...............................................
PRIZ Academy - 9 Windows Thinking Where to Invest Today to Win Tomorrow.pdf
composite construction of structures.pdf
SM_6th-Sem__Cse_Internet-of-Things.pdf IOT
Well-logging-methods_new................
Evaluating the Democratization of the Turkish Armed Forces from a Normative P...
Construction Project Organization Group 2.pptx
IOT PPTs Week 10 Lecture Material.pptx of NPTEL Smart Cities contd
573137875-Attendance-Management-System-original
Mitigating Risks through Effective Management for Enhancing Organizational Pe...
BMEC211 - INTRODUCTION TO MECHATRONICS-1.pdf
Embodied AI: Ushering in the Next Era of Intelligent Systems
PPT on Performance Review to get promotions
Internet of Things (IOT) - A guide to understanding
KTU 2019 -S7-MCN 401 MODULE 2-VINAY.pptx
CARTOGRAPHY AND GEOINFORMATION VISUALIZATION chapter1 NPTE (2).pptx
bas. eng. economics group 4 presentation 1.pptx
Automation-in-Manufacturing-Chapter-Introduction.pdf
R24 SURVEYING LAB MANUAL for civil enggi

Regenerating face images from multi-spectral palm images using multiple fusion methods

  • 1. TELKOMNIKA, Vol.17, No.6, December 2019, pp.3110~3119 ISSN: 1693-6930, accredited First Grade by Kemenristekdikti, Decree No: 21/E/KPT/2018 DOI: 10.12928/TELKOMNIKA.v17i6.12857 ◼ 3110 Received April 6, 2019; Revised July 2, 2019; Accepted July 18, 2019 Regenerating face images from multi-spectral palm images using multiple fusion methods Raid Rafi Omar Al-Nima, Moatasem Yaseen Al-Ridha*, Farqad Hamid Abdulraheem Technical Engineering College, Mosul, Northern Technical University, Mosul, Iraq *Corresponding author, e-mail: moalridha@gmail.com Abstract This paper established a relationship between multi-spectral palm images and a face image based on multiple fusion methods. The first fusion method to be considered is a feature extraction between different multi-spectral palm images, where multi-spectral CASIA database was used. The second fusion method to be considered is a score fusion between two parts of an output face image. Our method suggests that both right and left hands are used, and that each hand aims to produce a significant part of a face image by using a Multi-Layer Perceptron (MLP) network. This will lead to the second fusion part to reconstruct the full-face image, in order to examine its appearance. This topology provided interesting results of Equal Error Rate (EER) equal to 1.99%. Keywords: face, fusion, MLP neural network, palmprint Copyright © 2019 Universitas Ahmad Dahlan. All rights reserved. 1. Introduction Biometric applications are currently extensively used in the case of recognition, identification or authentication systems. That is because biological biometrics have unique characteristics such as a face and palmprint [1]. Faces provide challenges, starting with extracting the main and important parts, then using these parts to reconstruct images of the face automatically. It is apparent that the image of the face has many details, for instance eyebrows, eyes, nose, mouth, ears and the boundary of the face. Thus, auto-establishing the image of the face to be clear enough to recognize is really complicated [2]. In the literature, several studies considere generating face features from another biometric characteristic. This perhaps starts from the work of Sağiroğlu and Özkaya, where an intelligent system of generating eyes features from Only fingerprints was presented [3]. Özkaya and Sağiroğlu also presented a study of producing face borders from fingerprints by employing the Artificial Neural Network (ANN) [4]. Then, Sağiroğlu and Özkaya published extended study for producing main face features from fingerprints [5]. After that, Chitravanshi et al. proposed such an interesting work of generating main face characteristics from palm features [6]. Al-Nima et al. approached a novel work for predicting full face features from signatures, where this was the first study of producing physiological biometrics from behavioural biometrics [7]. Yang et al. reconstructed face images by utilizing a Manifold Constrained Convolutional Sparse Coding [8]. Lu et al. constructed high resolution face images from low resolution face images by using conditional cycle Generative Adversarial Network (GAN) [9]. Jiang et al. considered a study of generating 3D faces from the geometry details of a single face image [10]. Li et al. produced photorealistic faces for recognizing facial sketches [11]. Mai et al. illustrates a Neighborly de-convolutional neural Network (NbNet) method to reproduce face images from deep templates [12]. Al-Nima et al. exploited hand-dorsal images to generate full face details by using Back Propagation Neural (BPN) and Cascade-Forward Neural (CFN) Networks [13]. Up to now, it has been noted that there is no reference in relation to reproduce face images from multi-spectral hand images. Because vein patterns require substantial efforts to acquire, a system security can be increased further. Therefore, this issue has been addressed in our work. The aims and contributions of this paper are as follows: - Proposing robust face regeneration system that can reconstruct the full details of face images. - Exploiting muti-spectral images of right and left hands as required inputs. This would increase the security and anti-spoofing of the suggested system.
  • 2. TELKOMNIKA ISSN: 1693-6930 ◼ Regenerating face images from multi-spectral palm images... (Raid Rafi Omar Al-Nima) 3111 The employed objectives can be stated as follows: firstly, image preprocessing steps were handled to extract and enhance the hand images. These preprocessings were eliminated or cropped, morphological operations, adding top-hat characteristics and an unsharp filter. Secondly, a feature fusion between multi-spectral images to combine the features, where Haar wavelet fusion based on the mean rule was used. Thirdly, a wavelet transform was applied for the enhanced and fused image. Fourthly, the MLP neural networks were trained by considering a right hand to predict the inner face image and the left hand to predict the outer face image. Fifthly, a score fusion was utilized to collect the face image according to the maximum or adding rule. Sixthly, the same processings should be followed to test the MLPs using different data. Finally, the last decision was taken and hence, Figures 1 and 2 demonstrate the block diagrams for our proposed method. This new suggested topology will increase the security and effictiveness of the biometric system. This paper is organized as follows: the first section is the introduction, prior work and the proposed method. The second section is the hand images extraction followed by enhancement. The third section is in relation to the two types of fusions, features and the score. The fourth section is for the MLPs artificial neural networks. The fifth section is for the results and discussions, with the final section being the conclusion. Figure 1. The block diagram of predicting face from hand images based on MLP neural networks and multiple fusions Figure 2. The block diagram of the preprocessing steps 2. Image Extraction and Enhancement Data extraction and analysis are considered as the most critical and essential problems in Image Processing (IM) and Artificial Intelligence (AI). In this paper, preprocessing steps are adopted to extract, enhance and normalize the data in order to be prepared for the (MLPs) networks. CASIA multi-spectral palm images database are employed to build a relationship with the ORL face images database. The multi-spectral images of the palm consist of 7200 jpg images for 100 different people. All these images are 8-bits gray-scale with six electromagnetic spectrums starting with 460 nm, 630 nm, 700 nm, 850 nm, 940 nm and white respectively. This valuable
  • 3. ◼ ISSN: 1693-6930 TELKOMNIKA Vol. 17, No. 6, December 2019: 3110-3119 3112 database included images for both right and left hands. A CCD camera is positioned at the bottom of a hand with some spectrum lights. There were no pegs or restrict positions for the palm in the device, although it had a uniform background colour. Two sessions were organized to capture the images. Each session took snap shots of three multi-spectrum images. The interval period between each session was more than one month [14]. A number of morphological operations were adopted after the image elimination to reduce any noise and maintain the hand image. To begin with, the cropping image was an 8-bit grayscale denoted as 𝐺(𝑥, 𝑦): 𝑍2 → [0,255] which needed to be converted to a binary image defined as 𝐵(𝑥, 𝑦): 𝑍2 → {0,1} [15]. Consequently, to convert the image from 8-bit grayscale to a binary image a threshold  was executed in (1): 𝐵(𝑥, 𝑦) = { 1 𝑖𝑓 𝐺(𝑥, 𝑦) > 𝜃 0 𝑖𝑓 𝐺(𝑥, 𝑦) ≤ 𝜃 (1) for the binary image 𝐵(𝑥, 𝑦) and a structuring element ℎ(𝑎, 𝑏), the erosion ⊝ and dilation ⨁ are denoted as [16]: (𝐵 ⊝ ℎ)(𝑥, 𝑦) = 𝑚𝑖𝑛{𝐵(𝑥 + 𝑎 , 𝑦 + 𝑏) − ℎ(𝑎, 𝑏)} (2) (𝐵⨁ℎ)(𝑥, 𝑦) = 𝑚𝑎𝑥 {𝐵(𝑥 − 𝑎 , 𝑦 − 𝑏) + ℎ(𝑎, 𝑏)} (3) small white objects should be removed from the binary image, while holding the very large area. An open morphological operation may solve this issue. See (4): (𝐵 ℎ) = ( 𝐵 ⊝ ℎ )⨁ ℎ 𝑖𝑓 𝐴𝑟𝑒𝑎 ≤ 𝜌 (4) where:  is a specific area. Nevertheless, there will still be some small objects connected to the largest hand area, which will not be deleted by (4). To remove these objects, a complement image 𝐵̂ defined as 𝐵̂(𝑥, 𝑦): 𝑍2 → {1,0} was produced from the last operation. A major morphological operation was performed consecutively, to clear the unexpected noise by setting them to 1's if the neighbourhood majorities are ones [17, 18]. Hence, the palm image with fingers is easily created by combining the original image with the complement image as described in (5): 𝐼𝑖𝑛(𝑥, 𝑦) = { 𝐺(𝑥, 𝑦) 𝑖𝑓 𝐵̂(𝑥, 𝑦) = 0 𝑠 + 𝐵̂(𝑥, 𝑦) 𝑖𝑓 𝐵̂(𝑥, 𝑦) = 1 (5) where: 𝐼𝑖𝑛(𝑥, 𝑦) is the new created image, s is a small scalar value and 𝐵̂(𝑥, 𝑦) is the resulting binary image after (4). An example of a hand image before and after the morphological processings is given in Figure 3. (a) (b) (c) (d) (e) (f) Figure 3. A hand image before and after the morphological operations: (a) input image, (b) eliminating and converting to the binary image, (c) removing the small areas, (d) converting to the complement image, (e) removing the unexpected noise and (d) combining the original image with a clear background As each multi-spectral image has specific features, enhancements and feature fusions are important to collect as many have characteristics. Obtaining top-hat values from each
  • 4. TELKOMNIKA ISSN: 1693-6930 ◼ Regenerating face images from multi-spectral palm images... (Raid Rafi Omar Al-Nima) 3113 image, followed by adding them to the resulting original image 𝐼𝑖𝑛(𝑥, 𝑦) appeared to be a high-quality enhancement. Initially, a structuring element is created as a disk shape of one’s values [18]: 𝑆𝑒 = 𝑅2 × 𝜋 (6) where: R is radius of pixels. Consequently, the top-hat details are isolated from the image as shown in this equation [19]: 𝐼𝑚 𝑇 = 𝐼𝑖𝑛 – (𝐼𝑖𝑛 𝑆𝑒) (7) where: 𝐼𝑚 𝑇 is the image after the top-hat filter and 𝑆𝑒 is the structuring element. Hence, these top-hat features are added to the hand image according to (8) [18]: 𝐼𝑚 𝑂𝑇 = 𝐼𝑖𝑛 + 𝐼𝑚 𝑇 (8) then, an unsharp filter is applied to enhance the 𝐼𝑚 𝑂𝑇 details of the edges. So, an 𝐼𝑚 𝑂𝑇𝑈 is produced as follows [20]: 𝐼𝑚 𝑂𝑇𝑈(𝑥, 𝑦) = 𝐼𝑚 𝑂𝑇(𝑥, 𝑦) + 𝛽𝑈(𝑥, 𝑦) (9) where: 𝛽 is a positive scale factor and 𝑈(𝑥, 𝑦) is the correction signal, calculated as an output of a highpass filter [20]. An example of the image enhancement processing is given in Figure 4. (a) (b) (c) (d) Figure 4. An example of image enhancement processings: (a) a hand image needs to be enhanced, (b) top-hat image details, (c) adding the top-hat details to the original hand image and (d) sharpened image enhancement 3. Fusion Fusion between different biometric feature acquirements could be considered as a significant method to increase the ability and security of the systems [21]. Two fusion methods are used: feature fusion based on the wavelet with mean rules to extract the hand features from the multi-spectral images and score fusion to combine or collect the final face image. 3.1. Feature Fusion A fusion between images is considered as an interesting technique, so as to increase the level of efficiency of any biometric system. Collecting multiple information from different images is a problem which can be solved by the fusion technique. Merging two images into one individual image is a type of fusion ability that provides more data in a single image [22]. Multi-spectral hand images have many characteristics from palm, fingers and hand geometry to vein, lines and small patterns. Fusion between each two multi-spectral image types will maintain and cover the combination information. In this paper, image spectrums of 460 nm were combined with the image spectrums of 630 nm. Similarly, 700 nm images were merged with 850 nm, whilst 940 nm were fused with the white light image. This feature fusion method is implemented by wavelet fusion with the Haar signal, 4-levels and mean rule for the both approximations and details parts. See Figure 5 (a). According to this fusion, four coefficients will be generated for each image after the wavelet transform (one approximation and three details). ImOTU1 and ImOTU2 are two
  • 5. ◼ ISSN: 1693-6930 TELKOMNIKA Vol. 17, No. 6, December 2019: 3110-3119 3114 multi-spectral palm images, (LL1) is the approximation and (LH1, HL1, HH1) are the details of the first spectrum image. Likewise, (LL2) is the approximation and (LH2, HL2, HH2) are the details of the second spectrum image. Hereafter, (10) is used to combine the information: 𝐼𝑚 𝑂𝑇𝑈3 = 𝐼𝐷𝑊𝑇 (𝐴𝑉(𝐿𝐿1, 𝐿𝐿2); 𝐴𝑉(𝐿𝐻1, 𝐻𝐿1, 𝐻𝐻1, 𝐿𝐻2, 𝐻𝐿2, 𝐻𝐻2)) (10) where: IDWT is the inverse of the 2D wavelet transform and AV is the average. (a) (b) Figure 5. Fusion methods: (a) feature fusion method between two spectral images (940nm and white) based on the wavelet method and (b) score fusion method between two face image parts after the MLP neural networks 3.2. Score Fusion Score matching level fusion is a method that is used extensively with the multiple biometric models [23]. It is performed after the authentication processing, where each output is calculated individually and subsequently, a combination scoring level is applied [24]. The ORL database of face images are used in this paper. This database is produced in AT&T laboratories at Cambridge University through the collaboration of three groups (Speech, Vision and Robotics). It consists of 400 images from 40 people and each person has a different expression [25]. The critical problem in our work is generating face images using score level fusion. Afterwards, a decision is taken according to the most clear and distinct image. To explain in more details, two essential parts of a face image are intended to be predicted by using MLP neural networks. This is the middle part of a face image, which mainly consists of the eyes, nose and mouth. Thus, the right-hand images are used to predict this middle part. The outer part of a face, which commonly has the ears, hair and lower jaw employs the left-hand images to predict this boundary part. Subsequently, a score fusion is performed by using the maximum or adding rule to construct the face image. Figure 5 (b) shows the idea of the score fusion level. Assume FA1 is the inner (middle) face image part and FA2 is the outer face image part. Next, FA is the fused image from the middle and the boundary parts. See (11):
  • 6. TELKOMNIKA ISSN: 1693-6930 ◼ Regenerating face images from multi-spectral palm images... (Raid Rafi Omar Al-Nima) 3115 𝐹𝐴 = 𝑀𝑎𝑥(𝐹𝐴1; 𝐹𝐴2) (11) According to this technique the security of the system will increase because two hand images are required to prove the face image. 4. Neural Network Artificial Neural Network (ANN) is one of the most popular types of the Artificial Intelligence (AI). In recent years, it has become widespread in different fields. Biometric verification, identification and classification are examples of these fields. There are two main types of ANN, supervised and unsupervised. Each one of these types attempts to simulate a significant task in a human brain. Moreover, there are two essential stages in any ANN: the learning stage and the testing stage. In the first stage, the network learns the inputs and generates specific weights to manage the problem. In the second stage, the ANN deals with other inputs which have not been seen before [26]. In our work, supervised MLP neural networks are investigated and adapted to achieve their tasks. First of all, the input data of each image need to be prepared for the MLP network. Thus, each input image is segmented into specific matrices with different sizes of 5×5, 7×7, …, 13×13 pixels. This will ensure providing different overlaps between the matrices. A Coefficient of variance is calculated to each segment as illuminated in (12)-(14) [27]: 𝑀𝑒𝑎𝑛 𝑠𝑒𝑔 = 1 𝑛 ∑ 𝑠𝑒𝑔𝑖 𝑛 𝑖=1 (12) 𝑆𝐷𝑠𝑒𝑔 = √ 1 𝑛 − 1 ∑(𝑠𝑒𝑔𝑖 − 𝑀𝑒𝑎𝑛 𝑠𝑒𝑔)2 𝑛 𝑖=1 (13) 𝐶𝑉𝑠𝑒𝑔 = 𝑆𝐷𝑠𝑒𝑔 𝑀𝑒𝑎𝑛 𝑠𝑒𝑔 (14) where: n is the number of pixels in each segment, seg is the matrix of 5×5, 7×7, 9×9, 11×11 or 13×13 pixels, Mean is the average, SD is the standard deviation and CV is the coefficient of variance. The advantages of using the CV are: no dimension units can be considered, all values are positive, the differences will be given as small ratio values (this will avoid the ANNs overload in the next stage), the variances between the same vector type can be calculated (this will be valuable for the same target in the training stage), and the variances between the different vector types can be determined (this will be useful for the stage of fusion between two different types). The second step, is arranging the CV values into a one-dimensional vector for each image. The final input preparation is mapping the input data in [0,1] range as shown in (15) [28]: 𝐶𝑉𝑛𝑒𝑤 = (max(𝐶𝑉𝑛𝑒𝑤) − min(𝐶𝑉𝑛𝑒𝑤)) × (𝐶𝑉 − min(𝐶𝑉)) (max(𝐶𝑉) − min(𝐶𝑉)) + min(𝐶𝑉𝑛𝑒𝑤) (15) 5. Results and Discsussions The performance of the proposed method is examined and compared with other work. The databases are collected as well as organized into groups. An input group of 4020 multi-spectral image is used in the ANNs training stage and another input group of 804 multi-spectral images utilized in ANNs testing stage. In addition, each one of the two groups have been separated into two other groups; the left hand and right-hand groups. In the training stage, each hand group contained 2010 images. In the testing stage, each hand group consisted of 402 images. Both training and testing stages of the MLP network attempted to predict a clear and easily recognized part of a facial image, where each individual part has its own MLP. All trainings have been accomplished by the algorithm of the Scaled Conjugate Gradient (SCG), which is described in [29]. Examples of training curves are given in Figure 6.
  • 7. ◼ ISSN: 1693-6930 TELKOMNIKA Vol. 17, No. 6, December 2019: 3110-3119 3116 The minimum Mean Square Error (MSE) in all training is equal to 0.00001. Furthermore, training for both the right and left hands succeeded in achieving the minimum error with the appropriate number of epochs. (a) (b) Figure 6. An example of a training curve for: (a) the right hand multi-spectral images to predict the inner part of a face image and (b) the left hand multi-spectral images to predict the outer part of a face image For the regression test, both trainings attained 45 degrees or Regression (R) equal to 1 between the MLP outputs and targets. See Figure 7. To analyse Figure 7, the non-linear relationships were established between two biometrics, which are the right and left hand with the inner and outer part of a face, respectively. This relationship is the base for predicting a full face image from inputs, which have never been seen before. From this point, predicting parts of some face images are shown in Figures 8 (a and b). Whilst, the combination between each two parts are displayed in Figure 8 (c). (a) (b) Figure 7. Regression test for: (a) a right-hand training and (b) a left-hand training As could be observed, very clear face parts can be combined if new known vectors are introduced to the trained MLPs. Conversely, the MLPs cannot recognize a face image clearly when new unknown vectors are tested. Figures 9 (a, b and c) have examples of unclear face images. As mentioned before, using the two main face parts will increase the security of 0 50 100 150 0 50 100 150 Target Output~=1*Target+-1.5e-05 Training: R=1 Data Fit Y = T 0 50 100 150 200 0 50 100 150 200 Target Output~=1*Target+-6.5e-06 Training: R=1 Data Fit Y = T
  • 8. TELKOMNIKA ISSN: 1693-6930 ◼ Regenerating face images from multi-spectral palm images... (Raid Rafi Omar Al-Nima) 3117 the system. Moreover, producing a clear face image will assist any inexperienced individual to easily authorize or identify the certain person. In the final decision process, the image which is the most clear and nearest to the specific vector will be considered as a 'true' and the image which is the most distorted and furthest from the specific information will be considered as a 'false'. Thus, two types of classification were determined. Hereafter, the three main parameters are calculated: False Acceptance Rate (FAR), False Rejection Rate (FRR) and EER. The first two parameters are calculated according to the following equations: 𝐹𝐴𝑅 = 𝑁𝑢𝑚𝑏𝑒𝑟 𝑜𝑓 𝑓𝑎𝑙𝑠𝑒 𝑎𝑐𝑐𝑒𝑝𝑡𝑒𝑑 𝑖𝑚𝑝𝑜𝑠𝑡𝑒𝑟𝑠 𝑇𝑜𝑡𝑎𝑙 𝑛𝑢𝑚𝑏𝑒𝑟 𝑡ℎ𝑒 𝑜𝑢𝑡𝑝𝑢𝑡 𝑠𝑎𝑚𝑝𝑙𝑒𝑠 (16) 𝐹𝑅𝑅 = 𝑁𝑢𝑚𝑏𝑒𝑟 𝑜𝑓 𝑓𝑎𝑙𝑠𝑒 𝑟𝑒𝑗𝑒𝑐𝑡𝑒𝑑 𝑔𝑒𝑛𝑢𝑖𝑛𝑒𝑠 𝑇𝑜𝑡𝑎𝑙 𝑛𝑢𝑚𝑏𝑒𝑟 𝑡ℎ𝑒 𝑜𝑢𝑡𝑝𝑢𝑡 𝑠𝑎𝑚𝑝𝑙𝑒𝑠 (17) the EER parameter is calculated from the equality between FAR and FRR through different threshold values, see Figure 10. (a) (b) (c) (a) (b) (c) Figure 8. Clear faces parts images: (a) inner faces parts, (b) outer faces parts and (c) a combination between the two parts Figure 9. Distorted faces parts images: (a) inner faces parts, (b) outer faces parts and (c) a combination between the two parts Figure 10. FAR versus FRR to acquire EER To confirm the efficiency of our method, comparisons with other work including the state-of-the-art have been established in Table 1. The reason of selecting the two studies
  • 9. ◼ ISSN: 1693-6930 TELKOMNIKA Vol. 17, No. 6, December 2019: 3110-3119 3118 in [7, 13] is that both of these works concentrated on regenerating full details of face images. In addition, all of the reported works in Table 1 have used the ORL face images database. From this table it can be seen that simple statistics were used with ANN techniques in [7, 13], where the system strength level of the proposed systems is high. In this study, two fusion methods and two multi-spectral hand images (right and left) have been employed to regenerate full face details. Therefore, the strength level is very-high as spoofing the suggested system is so difficult. The EER values have been recorded to 10% for [7] and 6.43% and 2.86% for [13]. In this proposed approach, the EER value has been recorded equal to (1.99%). So, our work appears to have more accurate resutls than other studies. Table 1. Comparisons of Various Face Regeneration Methods Model Face Regenerating Methods System Strength Level EER Al-Nima et al. [7] Simple statistics with the MLP High 10% Al-Nima et al. [13] Simple statistics with the BPN High 6.43% Simple statistics with the DFN High 2.86% Proposed approach Two fusion methods with the MLP Very-High 1.99% 6. Conclusion A strategy to predict face image with high-level security was produced in this paper, where two non-linear relationships have been established. Firstly, a relationship between the right-hand features and the middle part of the facial image, which in general contain the eyes, nose and mouth. Secondly, a relationship between the left-hand features and the boundary part of the face image. In this paper, two levels of fusion are examined; the feature level and the score level. The idea behind using the feature level is to combine and enhance the multi-spectral hand characteristics, whilst, the score level is used on the face images to collect and reconstruct clear details of a face. The suggested approach structure confirmed its efficiency and robustness. The performance of overall technique was benchmarked to EER =1.99% during the testing stage, where full face details were reconstructed. In addition, the proposed system increases the anti-spoofing, strength and security levels. This is because two multi-spectral images of the two hands (left and right) are required to regenerate all face details. Acknowledgment “Portions of the research in this paper use the CASIA-MS-Palmprint V1 collected by the Chinese Academy of Sciences' Institute of Automation (CASIA)”. In addition, an acknowledgment is given to AT&T laboratories of Cambridge University. References [1] Liu YF, Lin CY, Guo JM. Impact of the Lips for Biometrics. IEEE Transactions on Image Processing. 2012; 21: 3092-3101. [2] Özkaya N, Sağiroğlu Ş. Generating One Biometric Feature from Another: Faces from Fingerprints. Sensors. 2010; 10(5): 4206-4237. [3] Sağiroğlu Ş, Özkaya N. An Intelligent and Automatic Eye Generation System from Only Fingerprints. Proceedings of Information Security and Cryptology Conference with International Participation. Ankara. 2008: 231-236. [4] Özkaya N, Sağiroğlu Ş. Intelligent face border generation system from fingerprints. 2008 IEEE International Conference on Fuzzy Systems (IEEE World Congress on Computational Intelligence. 2008: 2169-2176. [5] Sağiroğlu Ş, Özkaya N. An intelligent face features generation system from fingerprints. Turk J Elec Eng & Comp Sci. 2009; 17(2): 183-203. [6] Chitravanshi A, Singh A, Solanki S, Kumar A, Sharma R. Generating Face Features by Palm Scan: Recognizing a Person Uniquely. International Journal of Applied Information Systems (IJAIS). 2012; 2(5): 36-41. [7] Al-Nima RR, Dlay SS, Woo WL. A New Approach to Predicting Physical Biometrics from Behavioural Biometrics. World Academy of Science, Engineering and Technology, International Journal of Computer, Electrical, Automation, Control and Information Engineering. 2014; 8(11): 1996-2001.
  • 10. TELKOMNIKA ISSN: 1693-6930 ◼ Regenerating face images from multi-spectral palm images... (Raid Rafi Omar Al-Nima) 3119 [8] Yang L, Li C, Han J, Chen C, Ye Q, Zhang B, Cao X, Liu W. Image Reconstruction via Manifold Constrained Convolutional Sparse Coding for Image Sets. IEEE Journal of Selected Topics in Signal Processing. 2017; 11(7): 1072-81. [9] Lu Y, Tai YW, Tang CK. Conditional cyclegan for attribute guided face image generation. arXiv preprint arXiv:1705.09966. 2017. [10] Jiang L, Zhang J, Deng B, Li H, Liu L. Liu. 3D Face Reconstruction With Geometry Details From a Single Image. IEEE Transactions on Image Processing. 2018; 27(10): 4756-70. [11] Li X, Yang X, Su H, Zhou Q, Zheng S. Recognizing Facial Sketches by Generating Photorealistic Faces Guided by Descriptive Attributes. IEEE Access. 2018; 6: 77568-80. [12] Mai G, Cao K, Yuen PC, Jain AK. On the Reconstruction of Face Images from Deep Face Templates. IEEE Transactions on Pattern Analysis and Machine Intelligence. 2019; 41(5): 1188-202. [13] Al-Nima RRO, Abdulraheem FH, Al-Ridha MY. Using Hand-Dorsal Images to Reproduce Face Images by Applying Back propagation and Cascade-Forward Neural Networks. 2nd International Conference on Electrical, Communication, Computer, Power and Control Engineering ICECCPCE19. Mosul. 2019. [14] Paul C, Courtney R, Sanson-Fisher R, Carey M, Hill D, Simmons J, Rose S. A randomized controlled trial of the effectiveness of a pre-recruitment primer letter to increase participation in a study of colorectal screening and surveillance. BMC Medical Research Methodology. 2014; 14(44). [15] He Z. Hierarchical Colorant-Based Direct Binary Search Halftoning. IEEE Transactions on Image Processing. 2010; 19(7): 1824-1836. [16] Bai MR, Krishna VV, SreeDevi J. A new Morphological Approach for Noise Removal cum Edge Detection. IJCSI International Journal of Computer Science Issues. 2010; 7(6):187. [17] Jamil N, Sembok TM, Bakar ZA. Noise removal and enhancement of binary images using morphological operations. International Symposium on Information Technology ITSim 2008. 2008; 4: 1-6. [18] The MathWorks Inc. Image Processing Toolbox For Use with MATLAB, Ver.3 ed., 1993-2001. [19] Kekre HB, Mukherjee P, Wadhwa S. Image Retrieval with Shape Features Extracted using Morphological Operators with BTC. International Journal of Computer Applications. 2010; 12(3): 0975–8887. [20] Polesel A, Ramponi G, Mathews VJ. Image enhancement via adaptive unsharp masking. IEEE transactions on image processing. 2000; 9(3): 505-510. [21] Al-Nima RRO. Signal processing and machine learning techniques for human verification based on finger textures. PhD thesis. School of Engineering, Newcastle University. 2017. [22] Sale D, Sonare P, Joshi MA. PCA Based Image Fusion for Multispectral Palm Enhancement. International Journal of Advanced Research in Electrical, Electronics and Instrumentation Engineering. 2014; 3z2): 7501-7508. [23] Bhokare R, Sale D, Joshi MA, Gaikwad MS. Multispectral Palm Image Fusion: A Critical Review. International Journal of Advanced Research in Computer Engineering & Technology (IJARCET). 2013; 2(6): 2159-2164. [24] Merkle J, Kevenaar T, Korte U. Multi-modal and multi-instance fusion for biometric cryptosystems. 2012 BIOSIG-Proceedings of the International Conference of the Biometrics Special Interest Group (BIOSIG). 2012: 1-6. [25] Reinhardt K, Weiss S, Rosenbauer J, Gärtner J, Von Kries R. Multiple sclerosis in children and adolescents: incidence and clinical picture - new insights from the nationwide German surveillance (2009-2011). European Journal of Neurology. 2014; 21(4): 654-9. [26] Fausett L. Fundamental of Neural Networks, Architectures, Algorithms and Applications. Prentice Hall Int. Snc. 1994. [27] Li J, Yin G, Zhang G. Evaluation of tobacco mixing uniformity based on chemical composition. 2012 31st Chinese Control Conference (CCC). 2012: 7552-7555. [28] Luz AG, Osis MJ, Ribeiro M, Cecatti JG, Amaral E. Perspectives of professionals participating in the Brazilian Network for the Surveillance of Severe Maternal Morbidity regarding the implementation of routine surveillance: a qualitative study. Reproductive Health. 2014; 11(1): 29. [29] Møller MF. A scaled conjugate gradient algorithm for fast supervised learning. Neural Networks. 1993; 6(4): 525-533.