![216 Computer Science & Information Technology ( CS & IT )
extraction. For simulation and evaluation of the algorithm we have used our own database which
contains a total number of 336 images. All the images in the database are able to handle the
complicacies of the algorithm. This algorithm can operate on blurredness present in any portion
of the two input images.
2. STUDY ON IMAGE FUSION
2.1 WHY IMAGE FUSION IS REQUIRED?
The fused image contains greater information content for the scene than any one of the individual
image sources alone. The reliability and overall detail of the image is increased, because of the
addition of analogous and complementary information. Image fusion requires that images be
registered first before they are fused. Data fusion techniques combine data from different sources
together. The main objective of employing fusion is to produce a fused result that provides the
most detailed and reliable information possible. Fusing multiple information sources together also
produces a more efficient representation of the data. [6].
2.2 TYPES OF IMAGE FUSION
There are three main categories of fusion:
a. Pixel / Data level fusion
b. Feature level fusion
c. Decision level fusion
2.2.1 PIXEL LEVEL IMAGE FUSION
Pixel level fusion is the combination of the raw data from multiple source images into a single
image. In pixel level fusion the fused pixel is derived from a set of pixels in the various inputs
.The main advantage of pixel level fusion is that the original measured quantities are directly
involved in the fusion process [8]
2.2.2 FEATURE LEVEL IMAGE FUSION
Feature level fusion deals with the fusion of features such as edges or texture while decision level
fusion corresponds to combining decisions from several experts. In other word, Feature level
fusion requires the extraction of different features from the source data before features are merged
together.
2.2.3 DECISION LEVEL IMAGE FUSION
Decision-level fusion involves fusion of sensor information that is preliminary determinated by
the sensors. Examples of decision level Fusion methods include weighted decision methods,
classical inference, Bayesian inference, and Dempster–Shafer method. In decision level fusion the
results from multiple algorithms are combined together to yield a final fused decision.
2.3 ADVANTAGES OF IMAGE FUSION
Improve reliability (by redundant information) [8].
Improve capability (by complementary information) [8].](https://image.slidesharecdn.com/csit2321-180203051708/85/APPLICATION-OF-IMAGE-FUSION-FOR-ENHANCING-THE-QUALITY-OF-AN-IMAGE-2-320.jpg)
![Computer Science & Information Technology ( CS & IT ) 217
.3. REVIEW ON SOME IMPORTANT TERMS AND CONCEPTS
3.1 EIGEN FACES
Eigenfaces are a set of eigenvectors used in the computer vision problem of human face
recognition. The approach of using eigenfaces for recognition was developed by Sirovich and
Kirby (1987) and used by Matthew Turk and Alex Pentland in face classification. It is considered
the first successful example of facial recognition technology.
3.2 EIGEN VECTORS AND EIGEN VALUES
An eigenvector of a matrix is a vector such that, if multiplied with the matrix, the result is always
an integer multiple of that vector. This integer value is the corresponding eigenvalue of the
eigenvector. This relationship can be described by the equation M × u = λ × u, where u is an
eigenvector of the matrix M and λ is the corresponding eigenvalue.
3.3 VARIANCE
Variance is the measure of the variability or spread of data in a data set. In fact it is almost
identical to the standard deviation. Variance is simply the standard deviation squared. The
formula is this: [7]
3.4 COVARIANCE
Covariance is a measure of the extent to which corresponding elements from two sets of ordered
data move in the same direction. The formula for covariance is very similar to the formula for
variance. We use the following formula to compute covariance: [7]
3.5 EMPIRICAL MEAN
The mean subtracted is the average across each dimension. So, all the x values have x’ (the mean
of the x values of all the data points) subtracted, and all the y values have y’ subtracted from
them. For example, if we have a matrix of 3x2, then the empirical mean will be of dimension 1x2.
[7]
3.6 PRINCIPAL COMPONENT ANALYSIS (PCA)
The PCA involves a mathematical procedure that transforms a number of correlated variables into
a number of uncorrelated variables called principal components. It computes a compact and
optimal description of the data set. The first principal component accounts for as much of the
variance in the data as possible and each succeeding component accounts for as much of the
remaining variance as possible. First principal component is taken to be along the direction with](https://image.slidesharecdn.com/csit2321-180203051708/85/APPLICATION-OF-IMAGE-FUSION-FOR-ENHANCING-THE-QUALITY-OF-AN-IMAGE-3-320.jpg)
![218 Computer Science & Information Technology ( CS & IT )
the maximum variance. The second principal component is constrained to lie in the subspace
perpendicular of the first. Within this subspace, this component points the direction of maximum
variance. The third principal component is taken in the maximum variance direction in the
subspace perpendicular to the first two and so on. The PCA is also called as Karhunen-Loève
transform or the Hotelling transform. The PCA does not have a fixed set of basis vectors like
FFT, DCT and wavelet etc. and its basis vectors depend on the data set.
3.7 PEAK SIGNAL TO NOISE RATIO (PSNR)
PSNR computes the peak signal-to-noise ratio, in decibels, between two images. This ratio is used
as a quality measurement between the original and a reconstructed image. The higher the PSNR,
the better is the quality of the reconstructed image. To compute the PSNR, first we have to
compute the mean squared error (MSE) using the following equation:
MSE=ΣM, N [If (m, n) - Ii (m, n)]2
/ M*N
4. IMPLEMENTATION DETAILS OF THE PROPOSED ALGORITHM
4.1 PROPOSED ALGORITHM
I. Obtain the two blurred input images (images to be fused) I1(x, y) and I2(x, y).
II. The input images I1(x, y) and I2(x, y) are arranged in two column vectors and their empirical
means are subtracted.
III. The eigenvector and eigenvalues for this resulting vector are computed and the eigenvectors
corresponding to the larger eigenvalue is obtained.
IV. The normalized components P1 and P2 are computed from the obtained eigen vector using the
following equation:
P1 = V (1) / ΣV
and
P2 = V (2) / ΣV
V. The de-blurred fused image is calculated using the following equation:
If (x, y) = P1I1 (x, y) + P2I2 (x, y)
VI. Lastly, the image quality of the de-blurred fused image is calculated using PSNR by the
following equations:
MSE=ΣM, N [If (m, n) - Ii (m, n)]2
/ M*N
PSNR= 10log10 (R2
/ MSE)](https://image.slidesharecdn.com/csit2321-180203051708/85/APPLICATION-OF-IMAGE-FUSION-FOR-ENHANCING-THE-QUALITY-OF-AN-IMAGE-4-320.jpg)
![Computer Science & Information Technology ( CS & IT ) 219
4.2 SYSTEM DIAGRAM
4.3 PERFORMANCE ANALYSIS
We also see that the proposed algorithm is very time efficient as the estimated time was within 3
seconds for all the set of fused images. The PSNR values of all the fused images with respect to
their corresponding input images are also comparatively higher
Table 1: Peak Signal to Noise Ratio of the Fused Image w.r.t the two input images (50 fusion
samples)
5. CONCLUSION
Based on the various set of blurred input images and the de-blurred fused results, we come to
conclude that the proposed algorithm for image fusion using Principal Component Analysis
produces a comparatively better quality fused de-blurred image. Principal Component Analysis is
a mathematical procedure that transforms a number of correlated variables into a number of
uncorrelated variables called principal components. It computes a compact and optimal
description of the data set. [16] We also see that the resultant images are more illuminated and
require less amount of space for storage. Since, the proposed algorithm is based on PCA we see
that the algorithm has lower complexity and is very time efficient because the estimated time is](https://image.slidesharecdn.com/csit2321-180203051708/85/APPLICATION-OF-IMAGE-FUSION-FOR-ENHANCING-THE-QUALITY-OF-AN-IMAGE-5-320.jpg)
![220 Computer Science & Information Technology ( CS & IT )
within 3 seconds. The PSNR values of all the fused images with respect to their corresponding
input images are higher
6. FUTURE WORK
In future, we plan to fuse more than two images and produce a de-blurred fused image. We also
plan to concentrate on increasing the PSNR value of the fused image as compared to the current
PSNR value, as we know that more the value of the PSNR the better is the quality of the image.
REFERENCES
[1] http://en.wikipedia.org/wiki/Channel_(digital_image)
[2] http://www.cccure.org/Documents/HISM/041-044.html
[3] http://en.wikipedia.org/wiki/Facial_recognition_system
[4] http://en.wikipedia.org/wiki/Biometrics
[5] http://en.wikipedia.org/wiki/Image_fusion
[6] http://stattrek.com/matrix-algebra/variance.aspx
[7] http://www.ece.lehigh.edu/SPCRL/IF/image_fusion.htm
[8] http://www.face-rec.org/databases/
[9] http://www.equinoxsensors.com/products/HID.html
[10] Jeong-Seon Park, You Hwa Oh, Sang Chul Ahn, and Seong-Whan Lee, Sr. Member, IEEE, “Glasses
Removal from Facial Image Using Recursive Error Compensation”, 0162-8828/05/$20.00 _ 2005,
IEEE Published by the IEEE Computer Society, IEEE Transactions on Pattern analysis and Machine
intelligence”, VOL. 27, NO. 5, MAY 2005
[11] Diego A. Socolinsky† Andrea Selinger‡, †Equinox Corporation ‡Equinox Corporation, “Thermal
Face Recognition in an Operational Scenario”, 207 East Redwood St 9 West 57th St, Baltimore, MD
21202 New York, NY 10019, this research was supported in part by the DARPA Human
Identification at a Distance (HID) program.
[12] Xuerong Chen, Zhongliang Jing, Zhenhua Li, “Image fusion for face recognition”, proceeding of
2005 7th International Conference on Information Fusion, IEEE 2005, PP. (1226-1230).
[13] Jun-Ying Gan, Jun-Feng Liu, “Fusion and recognition of face and iris feature based on wavelet
feature and KFDA”, Proceedings of the 2009 International Conference on Wavelet Analysis and
Pattern Recognition, Baoding, 12-15 July 2009 IEEE, PP.(47-50).
[14] A tutorial on Principal component Analysis by Lindsay I Smith.
[15] A tutorial on Eigenface-based facial recognition by Dimitri Pissarenko.](https://image.slidesharecdn.com/csit2321-180203051708/85/APPLICATION-OF-IMAGE-FUSION-FOR-ENHANCING-THE-QUALITY-OF-AN-IMAGE-6-320.jpg)
APPLICATION OF IMAGE FUSION FOR ENHANCING THE QUALITY OF AN IMAGE
- 1. Natarajan Meghanathan, et al. (Eds): SIPM, FCST, ITCA, WSE, ACSIT, CS & IT 06, pp. 215–221, 2012. © CS & IT-CSCP 2012 DOI : 10.5121/csit.2012.2321 APPLICATION OF IMAGE FUSION FOR ENHANCING THE QUALITY OF AN IMAGE Suman Deb1 , Saptarshi Chakraborty2 , Taniya Bhattacharjee3 1 Assistant professor, Computer Science & Engg Dept, NIT Agartala. sumandebcs@gmail.com 2 Computer Science & Engg Dept, NIT Agartala. chakraborty0007@gmail.com 3 Assistant Professor, CSE Dept, MVJ College of Engg, Bangalore. taniya.bhattacharjee@gmail.com ABSTRACT Advances in technology have brought about extensive research in the field of image fusion. Image fusion is one of the most researched challenges of Face Recognition. Face Recognition (FR) is the process by which the brain and mind understand, interpret and identify or verify human faces.. Image fusion is the combination of two or more source images which vary in resolution, instrument modality, or image capture technique into a single composite representation. Thus, the source images are complementary in many ways, with no one input image being an adequate data representation of the scene. Therefore, the goal of an image fusion algorithm is to integrate the redundant and complementary information obtained from the source images in order to form a new image which provides a better description of the scene for human or machine perception. In this paper we have proposed a novel approach of pixel level image fusion using PCA that will remove the image blurredness in two images and reconstruct a new de-blurred fused image. The proposed approach is based on the calculation of Eigen faces with Principal Component Analysis (PCA). Principal Component Analysis (PCA) has been most widely used method for dimensionality reduction and feature extraction. KEYWORDS De-blurred fused image , Principal Component Analysis , Eigen faces ,empirical mean, peak signal to noise ratio (PSNR) 1. INTRODUCTION Image fusion produces a single image by combining information from a set of source images together, using data/ pixel, feature or decision level techniques. The goal of an image fusion algorithm is to integrate the redundant and complementary information obtained from the source images in order to form a new image which provides a better description of the scene for human or machine perception. Image fusion is essential for computer vision and robotics systems in which fusion results can be used to aid further processing steps for a given task. Image fusion techniques are practical and fruitful for many applications, including medical imaging, security, military, remote sensing, digital camera and consumer use. The proposed approach is based on the calculation of Eigen faces with Principal Component Analysis (PCA). Principal Component Analysis (PCA) has been most widely used method for dimensionality reduction and feature
- 2. 216 Computer Science & Information Technology ( CS & IT ) extraction. For simulation and evaluation of the algorithm we have used our own database which contains a total number of 336 images. All the images in the database are able to handle the complicacies of the algorithm. This algorithm can operate on blurredness present in any portion of the two input images. 2. STUDY ON IMAGE FUSION 2.1 WHY IMAGE FUSION IS REQUIRED? The fused image contains greater information content for the scene than any one of the individual image sources alone. The reliability and overall detail of the image is increased, because of the addition of analogous and complementary information. Image fusion requires that images be registered first before they are fused. Data fusion techniques combine data from different sources together. The main objective of employing fusion is to produce a fused result that provides the most detailed and reliable information possible. Fusing multiple information sources together also produces a more efficient representation of the data. [6]. 2.2 TYPES OF IMAGE FUSION There are three main categories of fusion: a. Pixel / Data level fusion b. Feature level fusion c. Decision level fusion 2.2.1 PIXEL LEVEL IMAGE FUSION Pixel level fusion is the combination of the raw data from multiple source images into a single image. In pixel level fusion the fused pixel is derived from a set of pixels in the various inputs .The main advantage of pixel level fusion is that the original measured quantities are directly involved in the fusion process [8] 2.2.2 FEATURE LEVEL IMAGE FUSION Feature level fusion deals with the fusion of features such as edges or texture while decision level fusion corresponds to combining decisions from several experts. In other word, Feature level fusion requires the extraction of different features from the source data before features are merged together. 2.2.3 DECISION LEVEL IMAGE FUSION Decision-level fusion involves fusion of sensor information that is preliminary determinated by the sensors. Examples of decision level Fusion methods include weighted decision methods, classical inference, Bayesian inference, and Dempster–Shafer method. In decision level fusion the results from multiple algorithms are combined together to yield a final fused decision. 2.3 ADVANTAGES OF IMAGE FUSION Improve reliability (by redundant information) [8]. Improve capability (by complementary information) [8].
- 3. Computer Science & Information Technology ( CS & IT ) 217 .3. REVIEW ON SOME IMPORTANT TERMS AND CONCEPTS 3.1 EIGEN FACES Eigenfaces are a set of eigenvectors used in the computer vision problem of human face recognition. The approach of using eigenfaces for recognition was developed by Sirovich and Kirby (1987) and used by Matthew Turk and Alex Pentland in face classification. It is considered the first successful example of facial recognition technology. 3.2 EIGEN VECTORS AND EIGEN VALUES An eigenvector of a matrix is a vector such that, if multiplied with the matrix, the result is always an integer multiple of that vector. This integer value is the corresponding eigenvalue of the eigenvector. This relationship can be described by the equation M × u = λ × u, where u is an eigenvector of the matrix M and λ is the corresponding eigenvalue. 3.3 VARIANCE Variance is the measure of the variability or spread of data in a data set. In fact it is almost identical to the standard deviation. Variance is simply the standard deviation squared. The formula is this: [7] 3.4 COVARIANCE Covariance is a measure of the extent to which corresponding elements from two sets of ordered data move in the same direction. The formula for covariance is very similar to the formula for variance. We use the following formula to compute covariance: [7] 3.5 EMPIRICAL MEAN The mean subtracted is the average across each dimension. So, all the x values have x’ (the mean of the x values of all the data points) subtracted, and all the y values have y’ subtracted from them. For example, if we have a matrix of 3x2, then the empirical mean will be of dimension 1x2. [7] 3.6 PRINCIPAL COMPONENT ANALYSIS (PCA) The PCA involves a mathematical procedure that transforms a number of correlated variables into a number of uncorrelated variables called principal components. It computes a compact and optimal description of the data set. The first principal component accounts for as much of the variance in the data as possible and each succeeding component accounts for as much of the remaining variance as possible. First principal component is taken to be along the direction with
- 4. 218 Computer Science & Information Technology ( CS & IT ) the maximum variance. The second principal component is constrained to lie in the subspace perpendicular of the first. Within this subspace, this component points the direction of maximum variance. The third principal component is taken in the maximum variance direction in the subspace perpendicular to the first two and so on. The PCA is also called as Karhunen-Loève transform or the Hotelling transform. The PCA does not have a fixed set of basis vectors like FFT, DCT and wavelet etc. and its basis vectors depend on the data set. 3.7 PEAK SIGNAL TO NOISE RATIO (PSNR) PSNR computes the peak signal-to-noise ratio, in decibels, between two images. This ratio is used as a quality measurement between the original and a reconstructed image. The higher the PSNR, the better is the quality of the reconstructed image. To compute the PSNR, first we have to compute the mean squared error (MSE) using the following equation: MSE=ΣM, N [If (m, n) - Ii (m, n)]2 / M*N 4. IMPLEMENTATION DETAILS OF THE PROPOSED ALGORITHM 4.1 PROPOSED ALGORITHM I. Obtain the two blurred input images (images to be fused) I1(x, y) and I2(x, y). II. The input images I1(x, y) and I2(x, y) are arranged in two column vectors and their empirical means are subtracted. III. The eigenvector and eigenvalues for this resulting vector are computed and the eigenvectors corresponding to the larger eigenvalue is obtained. IV. The normalized components P1 and P2 are computed from the obtained eigen vector using the following equation: P1 = V (1) / ΣV and P2 = V (2) / ΣV V. The de-blurred fused image is calculated using the following equation: If (x, y) = P1I1 (x, y) + P2I2 (x, y) VI. Lastly, the image quality of the de-blurred fused image is calculated using PSNR by the following equations: MSE=ΣM, N [If (m, n) - Ii (m, n)]2 / M*N PSNR= 10log10 (R2 / MSE)
- 5. Computer Science & Information Technology ( CS & IT ) 219 4.2 SYSTEM DIAGRAM 4.3 PERFORMANCE ANALYSIS We also see that the proposed algorithm is very time efficient as the estimated time was within 3 seconds for all the set of fused images. The PSNR values of all the fused images with respect to their corresponding input images are also comparatively higher Table 1: Peak Signal to Noise Ratio of the Fused Image w.r.t the two input images (50 fusion samples) 5. CONCLUSION Based on the various set of blurred input images and the de-blurred fused results, we come to conclude that the proposed algorithm for image fusion using Principal Component Analysis produces a comparatively better quality fused de-blurred image. Principal Component Analysis is a mathematical procedure that transforms a number of correlated variables into a number of uncorrelated variables called principal components. It computes a compact and optimal description of the data set. [16] We also see that the resultant images are more illuminated and require less amount of space for storage. Since, the proposed algorithm is based on PCA we see that the algorithm has lower complexity and is very time efficient because the estimated time is
- 6. 220 Computer Science & Information Technology ( CS & IT ) within 3 seconds. The PSNR values of all the fused images with respect to their corresponding input images are higher 6. FUTURE WORK In future, we plan to fuse more than two images and produce a de-blurred fused image. We also plan to concentrate on increasing the PSNR value of the fused image as compared to the current PSNR value, as we know that more the value of the PSNR the better is the quality of the image. REFERENCES [1] http://en.wikipedia.org/wiki/Channel_(digital_image) [2] http://www.cccure.org/Documents/HISM/041-044.html [3] http://en.wikipedia.org/wiki/Facial_recognition_system [4] http://en.wikipedia.org/wiki/Biometrics [5] http://en.wikipedia.org/wiki/Image_fusion [6] http://stattrek.com/matrix-algebra/variance.aspx [7] http://www.ece.lehigh.edu/SPCRL/IF/image_fusion.htm [8] http://www.face-rec.org/databases/ [9] http://www.equinoxsensors.com/products/HID.html [10] Jeong-Seon Park, You Hwa Oh, Sang Chul Ahn, and Seong-Whan Lee, Sr. Member, IEEE, “Glasses Removal from Facial Image Using Recursive Error Compensation”, 0162-8828/05/$20.00 _ 2005, IEEE Published by the IEEE Computer Society, IEEE Transactions on Pattern analysis and Machine intelligence”, VOL. 27, NO. 5, MAY 2005 [11] Diego A. Socolinsky† Andrea Selinger‡, †Equinox Corporation ‡Equinox Corporation, “Thermal Face Recognition in an Operational Scenario”, 207 East Redwood St 9 West 57th St, Baltimore, MD 21202 New York, NY 10019, this research was supported in part by the DARPA Human Identification at a Distance (HID) program. [12] Xuerong Chen, Zhongliang Jing, Zhenhua Li, “Image fusion for face recognition”, proceeding of 2005 7th International Conference on Information Fusion, IEEE 2005, PP. (1226-1230). [13] Jun-Ying Gan, Jun-Feng Liu, “Fusion and recognition of face and iris feature based on wavelet feature and KFDA”, Proceedings of the 2009 International Conference on Wavelet Analysis and Pattern Recognition, Baoding, 12-15 July 2009 IEEE, PP.(47-50). [14] A tutorial on Principal component Analysis by Lindsay I Smith. [15] A tutorial on Eigenface-based facial recognition by Dimitri Pissarenko.
- 7. Computer Science & Information Technology ( CS & IT ) 221 Authors Suman Deb presently working as Assistant Professor at CSE Department of National Institute of Technology Agartala, his research interest primarily in Human computer interaction, Soft Computing, Pattern Recognition and robotics. He has more than six years of teaching experience and have guided several projects. Saptarshi Chakraborty presently working at Technical Assistant at CSE Department Of National Institute Of Technology Agartala, his Research interest primarily in Image processing, Computer Networks,Wireless & Sensor Networks. He has completed his M Tech from Tripura University.