Image fusion using nsct denoising and target extraction for visual surveillance

IJRET: International Journal of Research in Engineering And Technology eISSN: 2319-1163 | pISSN: 2321-7308
__________________________________________________________________________________________
Volume: 03 Special Issue: 07 | May-2014, Available @ http://www.ijret.org 508
IMAGE FUSION USING NSCT: DENOISING AND TARGET
EXTRACTION FOR VISUAL SURVEILLANCE
Anjana Jayachandran1
, R.Gayathri2
1
Student, Applied electronics, Anna University, Paavai College of Engineering, Namakkal, TN, India
2
Professor, in Anna University, Paavai College of Engineering, Namakkal, TN, India
Abstract
Image fusion is a method of combining information from multiple images of same scene to get a composite image that is more suitable
for human visual perception or further image processing task. In this paper we propose a fusion framework based on Non-Subsampled
Contourlet Transform of infrared and visible images. The fused result contains the details of target present in the infrared image. For
the identification and extraction of the target from the IR image the threshold and watershed algorithm is used so that only the
relevant information from the IR image is introduced into the fused result so that the result becomes more accurate. A NLM filter is
then used to denoise the fused image.
Keywords— Denoising; Image fusion; Image Segmentation; Non Subsampled Contourlet Transform
------------------------------------------------------------------------***------------------------------------------------------------------
1. INTRODUCTION
With the development in the sensor field many surveillance
systems have been developed in the recent year. Infrared sen-
sors are employed in the field of military surveillance, medical
imaging and machine vision. Infrared sensors have the ability
to capture information under poor lightning conditions than
the visible cameras. The sensor differs in modalities so image
data acquired using different sensors exhibit different modali-
ties like thermal and viaual characteristics. A surveillance sys-
tem can perform better if data acquired from different sensors
are combined together. This is known as image fusion. Image
fusion is defined as the process of combining information
from two or more sensors to get a single image, which is more
precise and is suitable for humal viaual perception for further
image processing tasks.
In this study, we focus on fusion process of visible and infra-
red images that has high resolution and contains more textural
information. The infrared images are obtained in the low light
conditions, which is captured by the heat emitting from the
object. Thus combination of visible images and infrared in-
formation in the IR image is obtained as a single image. Many
more fusion methods are developed [1], [2], [3], [4], [5] dur-
ing the past two decades. The fusion algorithm can be of three
types pixel, feature and decision levels[13]. In pixel image
fusion, the visual information in the source images are com-
bined based on the original pixel information [14]. The pixel
level fusion algorithm can be categorized into sapatial domain
fusion and trandform domain fusion. The spatial domain tech-
nique uses local spatial frequency and local standard deriva-
tion for source image fusion. For transform domain methods,
the source images are projected onto localized bases, which
are designed to represent the sharpness and edges of the image
[3]. So the transformed coefficients are used in detecting fea-
tures of the source images to construct the fused image. So far,
many multi resolution image fusion techniquehave been pro-
posed and used with the development of different bases; py-
ramidal decomposition such as laplacian pyramid, gradient
pyramid, contrast pyramid [4], [5] fail to introduce spatial
oriented selectivity in decomposition process and results in
blocking effects. Another family of of multi resolution fusion
technique is wavelet-based method [6]. The problem with WT
[8] is that, it can provide better spatial and spectral localization
of image information but fails to provide spatial characteris-
tics. The advantage of DWT compared to pyramid fused im-
age is that it provide better SNR , but it lacks shift-invariant
properties and does not show singular curves. The curvelet
transform (CVT) and contourlet transform (CT) [10] over-
comes the problem of Wavelet Transform. The CT overcomes
the lack of geometrical structure in WT, but the CT is not
shift-invariant.
In this paper, we propose an improved contourlet transform
the Non Subsampled Contourlet Transform (NSCT). The
NSCT is fully shift invariant, multi scale and multi direction
expansion that has better directionality frequency localization
and a fast implementation. The filter design problem of NSCT
is much less constrained than that of CTs because of its redun-
dancy. This enables us to design filters with better frequency
selectivity thereby achieving a better subband decomposition.
The structure of this paper is as follows: section 2 introduces
the target extraction algorithm section 3 discusses the fusion
framework and section 4 discusses the results. Finally, the
conclusion is given in section 5.

__________________________________________________________________________________________
2. TARGET EXTRACTION ALGORITHM
There are number of image segmentation technique eg [13],
[14], [15] and [18]. Most of the methods first employ multi
scale transform to the source images and extract regions from
the transform coefficients. The final fusion performance de-
pends on the quality of segmentation process. In case of IR-
visible image fusion, proper segmentation map for all input
images is different nature of imaging sensors.
Only the infrared image has the detail of the targets so in this
methods instead of segmenting both source images only the
knowledge of the properties of IR images is used to extract
objects of interest for segmenting both infrared or visible im-
ages. In this work, a marker controlled watershed transforma-
tion is used to extract targets from the IR image. The marker
image is computed using the gradient modulus maxima of the
undecimated wavelet transform (UWT). The block diagram of
the target extraction algorithm is shown in Fig.1. It consists of
three parts: marker extraction, image simplification, and wa-
tershed transformation.
2.1 Marker Extraction
The direct application of watershed transform leads to over
segmentation of the input images. To improve the result, wa-
tershed transform is used along with the marker image, which
limits the segmentation process to some “marked ” areas.
Since the IR images are bounded by transient regions such as
edges, we first apply an undecimated wavelet transform based
edge detector to the input image [14]. Then three images are
obtained at each scale, the modulus of gradient vector, the
angle of steepest ascent of the gradient vector and a binary
image containing local modulus maxima of the gradient vec-
tor. Then the binary modulus maxima images are multiplied
Fig 1: Block diagram of the target extraction approach
with the corresponding gradient modulus images and first
threshold is applied. The binary image then contains only
those modulus maxima above the chosen threshold. After
combining the threshold image of 1st
and 2nd
decomposition
level, first course segmentation is obtained. From this segmen-
tation the starting point for the edge tracking operation is
computed. The proposed tracking algorithm takes a seed point
from the seed point list and follows target border in the direc-
tion perpendicular to the gradient angle. So in order to track a
target it is important that a single seed point or starting point is
located on target edge. This will minimize introduction of
false targets in the segmentation process and allows selection
of highest first threshold. At each new point, the tracking algo-
rithm multiplies the 8 connected neighborhood of the tracked
pixel with a directional mask and discards those pixels, which
does not agree the masks angle. The directional masks with
their corresponding directions are given in Fig.3. Form all
candidate pixels, tracking algorithm chooses the one with
highest gradient modulus and marks it as a tracked. All the 4
connected neighbors and remaining candidate pixels arising
from the previously tracked pixels are marked as discarded to
avoid the use of those pixels as candidate pixels again. The
tracking stops when, the new point is 8 connected to previous-
ly tracked point or the averaged gradient modulus of the new
point is below a particular threshold value. The fig 2 shows the
tracking operation. After tracking operation all the true targets
from the input image form the bounded region.
The post processing steps removes all edge segments, which
do not form a closed region, thereby cleans tracked image
from the wrongly tracked portions. Before performing marker
controlled WT, the original has to be simplified [15] by com-
puting morphological gradient of the source images and quan-
tizing it to 100 gray levels. After image simplification, it is
combined with the marker image and watershed transforma-
tion is applied. The result is shown in the fig 2.
Fig 2: Results of the target extraction: (Top-Left) Original
image, (Top-Right) Seed points. (Bottom left) Result of the
tracking operation. (Bottom-Right) Final result after applica-
tion of the watershed transformation

__________________________________________________________________________________________
3. FUSION FRAMEWORK
In this work an improved contourlet transform is used in the
fusion framework of infrared and visible images, the Non
Subsampled Contourlet Transform. NSCT is a multiscale and
multidirection fusion framework of the discrete images [16].
Fig 3(a) displays a high-level view of the NSCT. The structure
consists in a bank of filters that splin the 2-D frequency plane
in a bank of filters that splits the 2-D frequency plane in the
subbands illustrated in Fig 3(b). The transform is divided into
two parts that are both shift invariant; a non subsampled py-
ramid structure that ensures multiscale property and a direc-
tional filter bank structure that gives directionality. The multi
scale property of the NSCT is obtained from a shift-invariant
filtering structure that achieves subband decomposition simi-
lar to that of the Laplacian pyramid. This is achieved by
using two-channel non-subsampled 2-D filter banks. At each
NSP decomposition level one low frequency image and one
high frequency image is obtained. The subsequent NSP de-
composition are carried out to decompose the low frequency
component iteratively to capture the singularities in the image.
As a result NSP results in K+1 sub images, it consists of one
low frequency and k high frequency images having the same
size as the source image where k denotes the no.of decomposi-
tion levels. Fig 3(a) shows NSP decomposition with k=3 le-
vels. The NSDFB is a two-channel non subsampled filter bank
and are constructed by combining the directional decomposi-
tions with l stages in high frequency images from NSP at each
scale and produces 2l
directional sub images with same size as
source images. A
(a) (b)
Fig 3 The nonsubsampled contourlet transform. (a) Nonsub-
sampled filter bank structure that implements the NSCT. (b)
The idealized frequency partitioning obtained with the pro-
posed structure.
four channel NSDFB is constructed with two channel fan filter
banks. The result is a tree structured filter bank that splits the
frequency plane in the directional wedges as shown in Fig 4
(a).
Fig 4: Directional filter Bank (a) Ideal partitioning of the 2-D
frequency plane into 23 = 8 wedges. (b) Equivalent multi-
channel filter bank struture.
The NSDFB offers multidirection property and give more pre-
cise directional details information. The proposed system is
based on these two methods the fusion of infrared image and
visible image using the NSCT transformation and extracting
the target from the IR images using the marker ex traction and
watershed algorithm. The block diagram of the proposed sys-
tem is shown in figure 5.
The proposed system is based on these two methods the fusion
of infrared image and visible image using the NSCT transfor-
mation .
The step in the proposed system is as follows:
It should be noted that all the images should be registered be-
fore doing the fusion process so that the pixels match.
Step 1: The source images for the image fusion is an Infrared
image and the visible image
Step 2: The source images are prepossessed using Guassian
filter to remove the noise present in the images.
Step 3: The marker extraction and watershed algorithm are
applied to the Infrared image.
Step 4: To decompose the coefficients NSCT is applied to the
two source images.
Step 5: The lower coefficients of the source images are fused
using low frequency fusion rules and higher coefficients are
fused using high frequency fusion rules to get a fused low
frequency image and fused high frequency image.

__________________________________________________________________________________________
Fig 5: Block diagram of the proposed image fusion framework
Step 6: Inverse NSCT is applied to get the fused image with
target high lighting.
Step 7: A non local means filtering method is added to denoise
and to obtain a fused image.
The Infrared image and visible image is taken as the source
images. The source images are denoised using a guassian filter
to remove the noises present in the image. The decomposition
of the images are done by performing NSCT to get the lower
level coefficients and higher level cofficints. The fusion of
lower coefficients is done by using lower frequency rule to get
a fused low frequency image and the higher coeffients of both
images are fused by higher coefficients rules to get fused higf
frequency image. Inverse NSCT ia applied to the fused coeffi-
cients so that the fused image is obtained. The marker ex-
tracted target is then combined with the NSCT fused image so
that a target highlighted fused image is obtained. There are
certain noise introduced in the images during processing, to
remove these noise filtering process is done. In our work a
Non Local Mean filter is used for the post processing of the
image.
4. RESULTS
Some general requirements for fusion algorithm are: it should
be able to extract complimentary features from input images,
it must not introduce artifacts or inconsistencies according to
Human Visual System and it should be robust and reliable.
The fusion result of the proposed method is shown in Fig 6 (c)
and (d).
(a) (b)
(c) (d)
Fig 6 Source and image fusion results , (a) Infrared image; (b)
Visible image; (c) NSCT-based fused image; (d) Filtered im-
age
The performance of the proposed image fusion scheme was
compared to the fusion scheme obtained by using Non Sub-
sampled Contourlet Transform (NSCT). It can be seen that, by
including target information into the fusion process the result
of NSCT can be improved. This is more evident looking at the
fusion result shown in fig 7. It can be seen that the proposed
work produces fused images that show improved contrast
around the target regions. The proposed fusion work is also
used to artificially enhance the extracted target within the
fused image.

__________________________________________________________________________________________
Fig 7 Image fusion results , NSCT-based fused image (left);
Proposed image fusion with target extraction (right)
5. CONCLUSIONS
A novel image fusion framework is proposed for multisensor
images, which are based on Non Subsampled Contourlet
Transform along with a target extraction algorithm using wa-
tershed transform to get good results. The infrared target in the
natural scene can be clearly distinguished in the resulting
fused image. The infrared targets are highlighted using marker
extraction.This technique is very useful for visual surveillance.
ACKNOWLEDGMENTS
The authors wish to thank all those who helped in accomplish-
ing this task.
REFERENCES
[1] A.A. Goshtasby, S. Nikolov,Image fusion: advances in
the state of the art, Information Fusion 8 (2) (2007)
114–118.
[2] V.S. Petrovic, C.S. Xydeas, Gradient-based multireso-
lution image fusion, IEEE Transactions on Image
Processing 13 (2) (2004) 228–237
[3] N. Mitianoudis, T. Stathaki, Pixel-based and region-
based image fusion schemes using ICA bases, Informa-
tion Fusion 8 (2) (2007) 131–142.
[4] A. Toet, Image fusion by a ratio of low-pass pyramid,
Pattern Recognition Letters 9 (4) (1989)
[5] Z. Liu, K. Tsukada, K. Hanasaki, Y.K. Ho, Y.P. Dai,
Image fusion by using steerable pyramid, Pattern Rec-
ognition Letters 22 (9) (2001) 929–939.
[6] H. Li,B. Manjunath,S. Mitra, Multisensor image fusion
using the wavelet transform,Graphical Models and Im-
age Processing 57(3)(1995) 235–245.
[7] Z. Zhang, R.S. Blum, A categorization of multiscale-
decomposition-based image fusion schemes with a per-
formance study for a digital camera application, Pro-
ceedings of the IEEE 87 (8) (1999) 1315–1326.
[8] G.Pajares, J.Cruz, A wavelet-based image fusion tutori-
al,Pattern Recognition 37(9)(2004) 1855–1872.
[9] G. Piella,”A general framework for multiresolution
image fusion: from pixels to regions”, Information Fu-
sion 4 (4) (2003) 259–280.
[10] F. Nencini, A. Garzelli, S. Baronti, L. Alparone, “Re-
mote sensing image fusion using the curvelet trans-
form”, Information Fusion 8(2)(2007)143–156.
[11] S. Daneshvar, H. Ghassemian, MRI and PET image
fusion by combining IHS and retina-inspired models,
Information Fusion 11 (2) (2010) 114–123.
[12] M. Zribi, “Non-parametric and region-based image
fusion with Bootstrap sampling”, Information Fusion
11 (2) (2010) 85–94
[13] J.J. Lewis, R.J. O’Callaghan, S.G. Nikolov, D.R. Bull,
and N. Canagarajah, “Pixel- and region-based image
fusion with complex wavelets,” Information Fusion,
vol. 8, no. 2, pp. 119–130, April 2007.
[14] Z. Zhang and R.S. Blum, “Region-based image fusion
scheme for concealed weapon detection,” in Proceed-
ings of the 31st Annual Conference on Information
Sciences and Systems, March 1997, pp. 168–173.
[15] T. Wan, N. Canagarajah, and A. Achim, “Segmentation-
driven image fusion based on alpha-stable modeling of
wavelet coefficients,” IEEE Transactions on Multime-
dia, vol. 11, no. 4, pp. 624–633, June 2009.
[16] A. Ellmauthaler, E.A.B. da Silva, and C.L.Pagliari,
“Multiscale Image Fusion Using the Undecimated
Wavelet Transform With Spectral Factorization and
Non-Orthogonal Filter Banks,”
[17] C.S. Xydeas and V. Petrovic, “Objective image fusion
performance measure,” Electronics Letters, vol. 36, no.
4, pp. 308–309, February 2000.
[18] G. Piella, Adaptive Wavelets and their Applications to
Image Fusion And Compression, Ph.D. thesis, Univer-
sity of Amsterdam, 2003.

Image fusion using nsct denoising and target extraction for visual surveillance

More Related Content

What's hot (20)

Viewers also liked (20)

Similar to Image fusion using nsct denoising and target extraction for visual surveillance (20)

More from eSAT Publishing House (20)

Recently uploaded (20)

Image fusion using nsct denoising and target extraction for visual surveillance