Neighbour Local Variability for Multi-Focus Images Fusion

Signal & Image Processing: An International Journal (SIPIJ) Vol.11, No.6, December 2020
DOI: 10.5121/sipij.2020.11603 37
NEIGHBOUR LOCAL VARIABILITY FOR MULTI-
FOCUS IMAGES FUSION
Ias Sri Wahyuni1
and Rachid Sabre2
1
Universitas Gunadarma, Jl. Margonda Raya No. 100 Depok 16424, Indonesia
2
Laboratory Biogéosciences CNRS, University of Burgundy/Agrosup Dijon, France
ABSTRACT
The goal of multi-focus image fusion is to integrate images with different focus objects in order to obtain a
single image with all focus objects. In this paper, we give a new method based on neighbour local
variability (NLV) to fuse multi-focus images. At each pixel, the method uses the local variability calculated
from the quadratic difference between the value of the pixel and the value of all pixels in its
neighbourhood. It expresses the behaviour of the pixel with respect to its neighbours. The variability
preserves the edge function because it detects the sharp intensity of the image. The proposed fusion of each
pixel consists of weighting each pixel by the exponential of its local variability. The quality of this fusion
depends on the size of the neighbourhood region considered. The size depends on the variance and the size
of the blur filter. We start by modelling the value of the neighbourhood region size as a function of the
variance and the size of the blur filter. We compare our method to other methods given in the literature.
We show that our method gives a better result.
KEYWORDS
Neighbour Local Variability; Multi-focus image fusion; Root Mean Square Error (RMSE)
1. INTRODUCTION
The limitation of the depth of field of optical lenses often causes difficulty in capturing an image
containing all objects in focus on the scene. Only objects in depth of field are sharp, while other
objects are out of focus. Merging multi-focus images is the solution to this problem. There are
different approaches that have been carried out in the literature. These approaches can be
separated into two kinds: the spatial domain method and the multi-scale fusion method. The
spatial domain fusion method is performed directly on the source images. The techniques of the
spatial domain directly consider the pixels of the image. We quote some methods of fusion of the
spatial domain approaches: the mean, the principal component analysis (PCA) [1], the rule of
maximal selection, the methods bilateral gradient based [2] and guided image filter (GIF) method
[3] and maximum selection rule. We relate to the spatial domain approaches the fact that they
produce a spatial distortion in the merged image. Spatial distortion can be handled very well by
multiscale approaches to image fusion. The merging process in multi-scale merging methods is
done on the source images after their decomposition into several scales. we cite some examples
of these methods: The discrete wavelet transform (DWT) [4] - [9], fusion of images of the
Laplacian pyramid [10] - [17], discrete cosine transform with variance calculation (DCT + var)
[18], method based on salience detection (SD) [19] are examples of image fusion techniques
under transformation domain. The objective of this work is to propose a multi-focus image fusion
at the pixel level based on the neighboring local variability (NLV). It consists in calculating for
each pixel the quadratic distance which separates it from the other pixels belonging to its
neighborhood. It expresses the behavior of the pixel with respect to all the pixels belonging to its

38
neighborhood. The variability preserves the edge function because it detects the sharp intensity of
the image. The fusion of each pixel is given by a linear model of the values of the pixel in each
image hanging by the exponential of its local variability.
The precision of this fusion depends on the width of the region of pixels considered in the
neighbourhood. To determine this optimal width, we minimize the RMSE error. We propose a
model giving this width as a function of the variance and the size of the blur filter.
A comparative study of our method with other existing methods in the literature (DWT and LP-
DWT), it was carried out and showed that our method gave the best result using the mean
squared error (RMSE).
This paper is organized as follows: The second section reveals the stages of the process of fusing
the proposed method and a model giving the size of the neighbourhood. In section 3, we studied
the experimental results and compared our method to some recent methods. Section 4 gives the
conclusion of this work. In section 5, we give mathematical details to show the property of local
variability..
2. NEIGHBORING LOCAL VARIABILITY METHOD
Let be the fusion of two images, I1 and I2 that respectively have blurred parts B1 and B2. These
images have the same size: N1 x N2 where B1 and B2 are disjoint. The idea of the NLV merge
method is to add the pixel values of the two images weighted by the local variability of each
image. This local variability at (x, y) is calculated from the exponential of average of the square
difference between the value of the pixel (x, y) and the value of its neighbors. The NLV at is
defined as follows:
𝑣𝑎,𝑘(𝑥, 𝑦) = √
1
𝑅
∑ ∑ |𝐼𝑘(𝑥, 𝑦) − 𝐼𝑘
′ (𝑥 + 𝑚, 𝑦 + 𝑛)|
2
𝑎
𝑛=−𝑎
𝑎
𝑚=−𝑎
(1)
where k is the index of kth
source image (k = 1, 2), a is the size of neighborhood
  1 2
'
( , ), 1 1
,
( , ),
k
k
k
I x m y n if x m N and y n N
I x m y n
I x y otherwise
       

   
 ,
𝑅 = (2𝑎 + 1)2
− 𝑐𝑎𝑟𝑑(𝑆),
   
   
 
2 '
, , {(0,0)} , ( , ) .
k k
S m n a a I x m y n I x y
      
In the annex 1,we show that this local variability is small enoughwherethe location is on
theblurred area (B1 or B2).
In this work, we give a novel fusion method where each pixel of each image is weighting by
exponential of neighbour local variability. We calculate the neighbour local variabilityfrom the
quadratic difference between the value of the pixel and the all pixel values of its neighbours. The
idea came from the fact that the variation of the value in blurred region is smaller than the
variation of the value in focused region. We used the neighbour, with the size "a", of a pixel
defined as follows:
( , )
x i y j
  where , 1, , 1,
i a a a a
     and , 1, , 1,
j a a a a
     .

39
For example, the neighbor with the small size ("a" = 1) contains: ( 1, 1)
x y
  , ( 1, )
x y
 ,
( 1, 1)
x y
  , ( , 1)
x y  , ( , 1)
x y  , ( 1, 1)
x y
  , ( 1, )
x y
 ,( 1, 1)
x y
  .
(x-1, y-1) (x-1,y) (x-1,y+1)
(x,y-1) (x,y) (x,y+1)
(x+1,y-1) (x+1,y) (x+1,y+1)
Fig. 2. Pixel at (x,y) within its neighborhood, a = 1.
The fusion with size "a"follows the steps:
Consider M original source images, I1, ..., IM, with different focus and same size(𝑁1x𝑁2).
Step 1: For each pixel of each image, we calculated the neighbor local variability (NLV) of every
source image, va,k(x,y) defined in (1).
Step 2: The proposed fusion for the pixel (x,y) is F(x,y) calculated by the following model:
𝐹(𝑥, 𝑦) =
∑ exp(𝑣𝑎,𝑘(𝑥,𝑦))𝐼𝑘(𝑥,𝑦)
𝑀
𝑘=1
∑ exp(𝑣𝑎,𝑘(𝑥,𝑦))
𝑀
𝑖=1
(2)
We remark that method depends on the size "a". To determine the value of “a” , we find the value
minimizing the Root Mean Square Error (RMSE), where RMSE is defined in section 3. For that,
we showed that the value of "a" depends on the blurred area.
The choice of the size of the neighborhood "a" used in NLV method depends on variance (v) and
the size(s) of the blurring filter. Our problem is to have a model that gives the value of the "a"
according to the "v" and "s"; we take a sample of 1000 images that we blurred using Gaussian
filter with different values of v and s (v=1,2,3,...,35 and s=1,2,3,...,20).
After that, for each image we blurred with parameters "v" and "s", we applied our fusion method
with different values of "a"("a=1,2,...,17")and determined the value of "a" that gives the
minimum RMSE, denoted by
 
,
I
a v s
. Then, we took the mean of the
 
,
I
a v s
for 1000 images,
denoted
 
,
a v s
, because the coefficient of variation is smaller than 0,1.
Firstly, we have studied the variation of "a" in according to variance "v" for each fixed size of
blurring filter "s". We noted that this variation is logarithmic. For example, "s=8" on Fig. 4.By
using nonlinear regression, we obtained the model:
a=2.1096ln 2.8689
v .

40
Fig. 4. Graph between "a" and variance of blurring filter where "s"=8.
In general, the model is:
1 2
( )ln ( )
a c s v c s
  (3)
where the 1
c
and 2
c are functions that depend on "s". The graphs that describe 1
c
and 2
c ,
respectively, are revealed in Fig. 5. and Fig. 6.
Fig. 5. graph of 1( )
c s
Fig. 6. graph of 2 ( )
c s
.
By giving a model of 1
c
and a model of 2
c
and introducing these models in (3), we get the
general following model:
 
 
2
log 2.655551
3.0348761 75.062269
( , ) ln( ) 0.434 exp 0.5
1 29.0909139exp 0.5324955 1.225175 1.225175
s
a v s v
s s
 
 

   
    
   
     
  (4)
As "a" is integer, we have two choices of a. It is either the floor of
 
,
a v s
, denoted by
( , )
a v s
 
 
or the ceiling of a(v,s) , denoted by
( , )
a v s
 
  where
 
min |
x n n x
  
 
  and
 
max |
x m m x
  
 
  . Since the RMSE values of both "a" are very slightlydifferent, then
we can choose any "a" of them. We use "a" =
( , )
a v s
 
  in the remaining part of this paper.
To validate this model we applied it to 100 images (we generated 100 pairs of multi-focus images
with different variance and blur filter size values) and the result is as good as expected. To use
this NLV method, you must first estimate the variance and size of the blur filter from the blur
image. For this, there are some articles giving the methods to estimate the variance of the blur

41
filter and the blur detection as in [23] - [27]. subsequently, we developed another method in
which we combined the method of the Laplacian pyramid and the NLV method where the
Laplacian pyramid was used with NLV as selection rule, denoted LP-NLV.
3. EXPERIMENTAL STUDY
We have applied the NLV method on a dataset of images [26] using Matlab2013a. These images
were blurred using a Gaussian filter with many values of variance and size. To simplify the
reading of the article, we have presented only two examples in 256x256 format (N1 = N2 = 256).
The first “bird” image Fig.1 and the second “bottle” image Fig.2. All images consist of two
images with a different focus and a reference image.
We then compare the NLV method to other methods existing in the literature such as: PCA
method [1], Discrete Wavelet Transform (DWT) method [6], Laplacian Pyramid LP_PCA [15],
LP_DWT [17] and Bilateral gradient (BG) [2].
For that, we used the following the evaluation criteria frequently used:
Root Mean Square Error (RMSE)
RMSE gives the information how the pixel values of fused image F deviate from the reference
imageR. RMSE is defined as follows:
 
2
1 1
1
( , ) ( , )
c c
i j
RMSE R x y F x y
rc  
 

(5)
whereR is a reference image, F is a fused image,rx c is the size of the input image, and x, y
represents to the pixel locations. A smaller value of RMSE shows a good fusion result. If the
value of RMSE is equal to zero then it means the fused image is same the reference image.
For two images that are presented in this paper and blurred with variance = 10 and size of
blurring filter = 5, the model (5) gives the neighbour size "a" = 5 and "a" = 6. Here, we use "a" =
6 because it results the smaller RMSE compared to "a" = 5 however the RMSE values of "a" = 5
and "a" = 6 are very slightly different.

42
Blurred image 1 Blurred image 2
Figure1. Fusion by proposed method NLV
We have found that the NLV method better fusion compared to other methods, see Fig.1.
Table 1. Performance evaluation of image 'bird'
PCA DWT LP-
DWT
LP-
PCA
DCT+va
r
Bilatera
l
gradien
t
GIF SD NLV LP-
NLV
RMS
E
6.920
5
3.567
8
1.519
0
1.468
1
2.6860 8.8378 2.279
2
10.454
7
0.546
6
0.843
1
The table 1 gives the values of RMSE calculated for ten methods. As we can see on the Table 1
for image 'bird': the smallest RMSE is given by NLV method, the second smallest is LP-NLV,
the third is LP-PCA,. NLV method is the best method among the above methods and LP-NLV is
better than LP-PCA and LP-DWT.

43
Blurred image1 Blurred image2
Figure2. Fusion by proposed method (NLV)
We have found that the NLV method performs better compared to other methods, see Fig.2.
To confirm our visually result, we calculated the evaluation metrics: RMSE for ten methods see
Table 2. We can classify these methods from the smaller value of RMSE. The smallest value is
NLV, the second smallest is LP-NLV and the third smallest is LP-PCA.
Table 2. Performance evaluation of images of ‘the bottle'
PCA DWT LP-
DWT
LP-
PCA
DCT+va
r
Bilater
al
gradien
t
GIF SD NLV LP-
NLV
RMSE 15.00
5
5.384 2.528 2.485 2.642 20.380 3.681 16.9
19
0.902 1.584
According to the evaluation measure RMSE, the Table 3 gives the mean and standard deviation
of RMSE for the considered methods applied on 150 images.

44
Table 3. Statistic parameters of the sample (150 images)
Methods PC
A
DWT LP_DW
T
LP_PCA DCT_va
r
BG NLV LP_NLV
Mean 8,71
3
4,194 2,049 1,995 2,839 11,044 0,591 1,344
Standard
deviation
3,86
6
1,381 0,756 0,743 1,308 4,859 0,204 0,697
Time of
execution
by image
7s 5s 7s 7s 6s 6s 5s 7s
Table 3 shows that the proposed method (NLV) has a smaller average of the RMSE. The RMSE
histograms for 150 images by different methods (Figures 3, 4, 5, 6, 7, 8 and 9) show for almost
all methods that the RMSE values are almost symmetrically centered around the mean value.
Figure 3. Histogram of PCA method Figure 4. Histogram of LP_PCA method
Figure 5. The Histogram of DWT method Figure 6. Histogram of LP_DWT method
Figure 7. Histogram of Bilateral gradient method Figure 8. Histogram of NLV method

45
Figure 9. Histogram of LP_NLV method
In order to compare analytically the proposed method to other methods,a study of the Analysis of
variance (ANOVA) with dependent samples (dependence by image) is done. The software R
gives the following Anova table:
Table 4. Anova table with one factor: method
Df Sum Sq Mean Sq F value Pr(>F)
Method 9 25467 2829.6 742 <2e-16 ***
Residuals 1341 5114 3.8
As Pr(>F) is smaller than 1% , in table 4., the methods are signiﬁcantly different. Then, the
Newman Keuls test is performed for comparing the methods two-by-two and giving groups
having signiﬁcantly the same mean of RMSE. The software R shows the results of the test as
follows:
Table 5. Test of Newman Keuls
$groups
RMSE groups
SD 12.6072900 a
BG 11.0447767 b
PCA 8.7139600 c
DWT 4.1941660 d
DCT_var 2.8395233 e
GIF 2.5146353 e
LP_DWT 2.0496413 f
LP_PCA 1.9954953 f
LP_NLV 1.3446913 g
NLV 0.5921593 h
From table 5, we obtained the methods significantly different except the methods DCT_var and
GIF form the group “e” and the methods LP_DWT and LP_PCA form the group “f”.
The proposed method NLV in group “h” has a smaller mean (0.5921593) and significantly
different of the all methods. We conclude that the proposed method gives the better result than
other methods.
4. CONCLUSION
In this paper we propose an image fusion method based on local neighbour variability (NLV).
This method has been described in detail. Applying this method to 150 images gives a result of
significant improvement in both visual and quantitative image fusion compared to other fusion

46
methods. We used Laplacian pyramid with NLV as selection criterion, LP-NLV. Based on the
result of the experiment, we notice that LP-NLV is better than LP-DWT and DWT.
The proposed method has the advantage of taking into account the variability between each pixel
and its neighbours. Thus the coefficient of the pixels located in the focused part will be more
imposing. This method can be extended to multimodal images used in particular in medicine
(scanner, ultrasound, radiography, etc.) to give the presence of certain cancer cells seen in one
image and not visible in another image.
1) Drone is considered as a technological means allowing to capture images at different
altitudes. It opened up limitless possibilities for improving photography. The captured
images can be on the same scene by zooming in on different objects and at different
altitudes. It produces several images on the same scene but with different objects in focus.
2) In the food industry to control the quality of some products, cameras have been used to
take pictures. Each camera targets one of many objects placed on a conveyor belt to detect
an anomaly. To have a photo containing all the objects clearly, we can use the proposed
merge method which gives more details.
The perspectives of this work:
• We intend to extend our method to color images because color conveys additional
information.
• Encouraged by these results, we plan to extend the NLV method to the fusion of more than
two images, taking into account the local variability of each image (intra-variability) and
the variability between the images (inter-variability). Inter-variability can detect "abnormal
pixels" among images.
REFERENCES
[1] Naidu, V.P.S. and. Raol, J.R. (2008) “Pixel-level Image Fusion using Wavelets and Principal
Component Analysis”, Defence Science Journal, Vol. 58, No. 3, pp. 338-352.
[2] Tian, J., Chen, L., Ma, L., Yu, W., (2011) “Multi-focus image fusion using a bilateral gradient-based
sharpness criterion”, Optic Communications, 284, pp 80-87.
[3] Zhan, K., Teng, J., Li, Q., Shi, J. (2015) “A novel explicit multi-focus image fusion method”, Journal
of Information Hiding and Multimedia Signal Processing, vol. 6, no. 3, pp.600-612.
[4] Mallat, S.G. (1989) “A Theory for multiresolution signal decomposition: The wavelet
representation”, IEEE Trans. Pattern Anal. Mach. Intel., 11(7), 674-93.
[5] Pajares, G., Cruz, J.M. (2004) “A Wavelet-Based Image Fusion Tutorial”, Pattern Recognition 37.
Science Direct.
[6] Guihong, Q., Dali, Z., Pingfan, Y. (2001) “Medical image fusion by wavelet transform modulus
maxima”. Opt. Express 9, pp. 184-190.
[7] Indhumadhi, N., Padmavathi, G., (2011) “Enhanced Image Fusion Algorithm Using Laplacian
Pyramid and Spatial Frequency Based Wavelet Algorithm”, International Journal of Soft Computing
and Engineering (IJSCE). ISSN: 2231-2307, Vol. 1, Issue 5.
[8] Sabre, R. Wahyuni, I.S, (2020) “Wavelet Decomposition and Alpha Stable”, Signal and Image
Processing (SIPIJ), Vol. 11, No. 1. pp. 11-24.
[9] Jinjiang Li , Genji Yuan and Hui Fan (2019) “Multifocus Image Fusion Using Wavelet-Domain-
Based Deep CNN”, Computational Intelligence and Neuroscience, Vol. 2019 Article ID 4179397 |
https://doi.org/10.1155/2019/4179397
[10] Burt, P.J., Adelson, E.H. (1983) “The Laplacian Pyramid as a Compact Image Code”, IEEE
Transactions on communication, Vol.Com-31, No 40.

47
[11] Burt, P.J. (1984) “The Pyramid as a Structure for Efficient Computation. Multiresolution Image
Processing and Analysis”, A. Rosenfeld, Ed., Springer-Verlag. New York.
[12] Burt, P.J., Kolezynski, R.J. (1993) “Enhanced Image Capture Through Fusion”, in: International
Conference on Computer Vision, pp. 173-182.
[13] Wang, W., Chang, F. (2011) “A Multi-focus Image Fusion Method Based on Laplacian Pyramid”,
Journal of Computers, Vol.6, No 12.
[14] Zhao, P., Liu, G., Hu, C., Hu, and Huang, H. (2013) “Medical image fusion algorithm on the Laplace-
PCA”. Proc. 2013 Chinese Intelligent Automation Conference, pp. 787-794.
[15] Verma, S. Kaur K.,, Kumar M., R..(2016) “Hybrid image fusion algorithm using Laplacian Pyramid
and PCA method”, proceeding of the Second International Conference on Information and
Communication Technology for Competitive Strategies.
[16] Wahyuni, I.S, Sabre, R. (2019) “Multifocus Image Fusion Using Laplacian Pyramid Technique Based
on Alpha Stable Filter”, CRASE Vol. 5, No. 2. pp. 58-62.
[17] Wahyuni, I.S, Sabre, R. (2016) “Wavelet Decomposition in Laplacian Pyramid for Image Fusion”,
International Journal of Signal Processing Systems Vol. 4, No. 1. pp. 37-44.
[18] Haghighat, M.B.A, Aghagolzadeh, A., Seyedarabi, H. (2010) “Real-time fusion of multifocus images
for visual sensor networks”. Machine vision and image processing (MVIP), 2010 6th Iranian. 2010.
[19] Bavirisetti, D.P., and Dhuli, R.(2016) ‘‘Multi-focus image fusion using multi-scale image
decomposition and saliency detection’’, Ain Shams Eng. J., to be published. [Online]. Available:
http://dx.doi.org/ 10.1016/j.asej.2016.06.011.
[20] Petland, A. (1984) “A new sense for depth of field”, IEEE Transactions on Pattern Analysis and
Machine Intelligent, Vol. 9, No. 4, pp. 523-531.
[21] Nayar, S.K.(1992) ”Shape from Focus System”, Proc. of IEEE Conf. Computer Vision and Pattern
Recognition, pp. 302-308.
[22] Gonzales, R.C., Woods, R.E. (2002) “Ditigal Image Processing” 2nd edition. Prentice Hall.
[23] Liu,R., Li, Z., Jia, J.(2008) “Image Partial Blur Detection and Classification”, Computer Vision and
Pattern Recognition, CVPR 2008. IEEE Conference DOI: 10.1109/CVPR.2008.4587465
[24] Aslantas, V. (2007) “A depth estimation algorithm with a single image.” Optic Express, Vol. 15,
Issue 8. OSA Publishing.
[25] Elder, J.H., Zucker, S.W. (1998) “ Local Scale Control for Edge Detection and BlurEstimation.”
IEEE Transactions on Pattern Analysis and Machine Intelligence,Vol.20, No.7.
[26] www.rawsamples.ch. Accessed: 15 November 2017.
[27] Kumar, A., Paramesran, R., Lim, C. L., and Dass, S.C. (2016) “Tchebichef moment based restoration
of Gaussian blurred images”. Applied Optics, Vol. 55, Issue 32, pp. 9006-9016.
5. ANNEX 1
Consider, without loss the generality that we have a focus pixel (x, y) in image I1 and blurred
in image I2 as in Fig. 1.
Fig. 1. Two multi-focus images, the yellow part is blurred area and the white part is clear (focus) area.
The neighbor local variability of images I1 and I2, respectively is defined in (1) by:

48
𝑣𝑎,1(𝑥, 𝑦) = exp (√
1
𝑅
𝑟1(𝑥, 𝑦)) and 𝑣𝑎,2(𝑥, 𝑦) = exp (√
1
𝑅
𝑟2(𝑥, 𝑦)) where 𝑟1(𝑥, 𝑦)and
𝑟2(𝑥, 𝑦)
can be written as follows:
𝑟1(𝑥, 𝑦) = ∑ ∑ |𝐼1(𝑥, 𝑦) − 𝐼1(𝑥 + (𝑚 − 𝑎), 𝑦 + (𝑛 − 𝑎))|
2
2𝑎
𝑛=0
2𝑎
𝑚=0
(6)
𝑟2(𝑥, 𝑦) = ∑ ∑ |𝐼2(𝑥, 𝑦) − 𝐼2(𝑥 + (𝑚 − 𝑎), 𝑦 + (𝑛 − 𝑎))|
2
2𝑎
𝑛=0
2𝑎
𝑚=0
(7)
Let IR is the reference image of multi-focus images I1 and I2. Moreover, it is shown in [20] and
[21] that the blurred image can be seen as the product convolution between the reference image
and Gaussian filter:
1 1
1
1
* ( , ), ( , )
( , )
( , ), ( , )
R
R
w I x y x y B
I x y
I x y x y B


 


2 2
2
2
* ( , ), ( , )
( , )
( , ), ( , )
R
R
w I x y x y B
I x y
I x y x y B


 

 , (8)
where w1 and w2 are Gaussian filter defined by:
𝑤1(𝑘, 𝑙) = 𝑤1(𝑘, 𝑙) =
exp (−
𝑘2
+ 𝑙2
2𝜎1
2 )
∑ ∑ exp (−
𝑘2 + 𝑙2
2𝜎1
2 )
𝑠1
𝑙=−𝑠1
𝑠1
𝑘=−𝑠1
, (k ,l) ∈ [−𝑠1, 𝑠1]2
,
𝑤2(𝑘, 𝑙) =
exp (−
𝑘2
+ 𝑙2
2𝜎2
2 )
∑ ∑ exp (−
𝑘2 + 𝑙2
2𝜎2
2 )
𝑠2
𝑙=−𝑠2
𝑠2
𝑘=−𝑠2
, (𝑘, 𝑙) ∈ [−𝑠2, 𝑠2]2
The product convolution is defined as follows:
1 1
1 1
1 1
* ( , ) ( , ) ( , ),
s s
R R
k s l s
w I x y w k l I x k y l
 
  
 
2 2
2 2
2 2
* ( , ) ( , ) ( , ),
s s
R R
k s l s
w I x y w k l I x k y l
 
  
 
Put
2 2 2
1
1 ( , )
0 0
( , ) ( , )
a a
m n
m n
r x y D x y
 
 
and
2 2 2
2
2 ( , )
0 0
( , ) ( , )
a a
m n
m n
r x y D x y
 
 
(9)
where
   
 
1
( , ) 1 1
( , ) ( , ) ,
m n
D x y I x y I x m a y n a
     
(10)

49
   
 
2
( , ) 2 2
( , ) ( , ) ,
m n
D x y I x y I x m a y n a
     
(11)
Proposition:
The local variability on blurred part is smaller than the local variability on focused part. Let
2
( , )
x y B
 (the blurred part of I2) and 1
( , )
x y B
 (focus par of I1 ), then  
2 1
( , ) ( , )
r x y r x y

.
Proof:
For that, we use Plancherel theorem:
∑ ∑ |𝐷(𝑚,𝑛)
1
(𝑥, 𝑦)|
2
2𝑎
𝑛=0
2𝑎
𝑚=0
=
1
(2𝑎+1)2 ∑ ∑ |𝐷
̂(𝑛,𝑚)
1
(𝑥, 𝑦)|
2
2𝑎
𝑛=0
2𝑎
𝑚=0
(12)
where 𝐷
̂(𝑛,𝑚)
1
(𝑥, 𝑦) is Fourier transform of𝐷(𝑚,𝑛)
1
(𝑥, 𝑦).
𝐷
̂(𝑛,𝑚)
1
(𝑥, 𝑦) = 𝐹𝑇[𝐷(𝑚,𝑛)
1
(𝑥, 𝑦)] = 𝐹𝑇[𝐼1(𝑥, 𝑦) − 𝐼1(𝑥 + (𝑚 − 𝑎),𝑦 + (𝑛 − 𝑎))]
(13)
As 2
( , )
x y B

therefore 1
( , )
x y B

, from (12), equation (13) can be written as follows:
𝐷
̂(𝑛,𝑚)
1
(𝑥, 𝑦) = 𝐹𝑇[𝐼𝑅(𝑥, 𝑦) − 𝐼𝑅(𝑥 + (𝑚 − 𝑎),𝑦 + (𝑛 − 𝑎))] (14)
and
2 2
2 2
2 2
( , ) ( , )* ( , )
s s
R
k s l s
I x y w k l I x k y l
 
  
 
(15)
By using the definition of convolution, equation (15) can be written as:
𝐼2(𝑥, 𝑦) = ∑ ∑ 𝑤2(𝑘, 𝑙)1[−𝑠2,𝑠2]2𝐼𝑅(𝑥 − 𝑘, 𝑦 − 𝑙)
∞
𝑙=−∞
∞
𝑘=−∞
(16)
and
𝐼2(𝑥, 𝑦) = (𝑤21[−𝑠2,𝑠2]2) ∗ 𝐼𝑅(𝑥, 𝑦) (17)
where
1[−𝑠2,𝑠2]2(𝑘, 𝑙) = {
1, 𝑖𝑓 (k, l) ∈ [−𝑠2, 𝑠2]2
0, 𝑜𝑡ℎ𝑒𝑟𝑤𝑖𝑠𝑒
.
The Fourier transform of
2
( , ) ( , )
m n
D x y
is
𝐷
̂(𝑛,𝑚)
2
(𝑥, 𝑦) = 𝐹𝑇[𝑤21[𝑠2,𝑠2]2 ∗ 𝐼𝑅(𝑥, 𝑦) − 𝑤21[𝑠2,𝑠2]2 ∗ 𝐼𝑅(𝑥 + (𝑚 − 𝑎), 𝑦 + (𝑛 − 𝑎))]

50
= 𝐹𝑇 [𝑤21[𝑠2,𝑠2]2 ∗ (𝐼𝑅(𝑥, 𝑦) − 𝐼𝑅(𝑥 + (𝑚 − 𝑎),𝑦 + (𝑛 − 𝑎)))]
= 𝐹𝑇[𝑤21[𝑠2,𝑠2]2]𝐹𝑇[𝐼𝑅(𝑥, 𝑦) − 𝐼𝑅(𝑥 + (𝑚 − 𝑎), 𝑦 + (𝑛 − 𝑎))] (18)
Substitute (10) into (14), we get
𝐷
̂(𝑛,𝑚)
2
(𝑥, 𝑦) = 𝐹𝑇[𝑤21[𝑠2,𝑠2]2]𝐷
̂(𝑝,𝑞)
1
(𝑥, 𝑦)
= (∑ ∑ 𝑤2(𝑘, 𝑙)1[𝑠2,𝑠2]2
∞
𝑙=−∞
(𝑘, 𝑙)𝑒−𝑖2(𝑘𝑝+𝑙𝑞)
∞
𝑘=−∞
)𝐷
̂(𝑛,𝑚)
1
(𝑥, 𝑦) (19)
Hence, from equation (19), we can obtain
|𝐷
̂(𝑛,𝑚)
2
(𝑥, 𝑦)| =
|
|
|
∑∑
𝑒
−(
𝑘2+𝑙2
2𝜎2
2)
∑ ∑ 𝑒
−(
𝑘′2+𝑙′2
2𝜎2
2 )
𝑠2
𝑙′=−𝑠2
𝑠2
𝑘′=−𝑠2
𝑒−𝑖2(𝑘𝑛+𝑙𝑚)
𝑠2
𝑙=−𝑠2
𝑠2
𝑘=−𝑠2
𝐷
̂(𝑛,𝑚)
1
(𝑥, 𝑦)
|
|
|
=
≤ ∑ ∑
|
|
|
𝑒
−(
𝑘2+𝑙2
2𝜎2
2)
∑ ∑ 𝑒
−(
𝑘′2+𝑙′2
2𝜎2
2 )
𝑠2
𝑙′=−𝑠2
𝑠2
𝑘′=−𝑠2
|
|
|
𝑠2
𝑙=−𝑠2
𝑠2
𝑘=−𝑠2
|𝐷
̂(𝑛,𝑚)
1
(𝑥, 𝑦)| ≤ |𝐷
̂(𝑛,𝑚)
1
(𝑥, 𝑦)|
(20)
On the other hand, from equation (13) and Plancherel-Parseval's theorem, we have
𝑟2(𝑥, 𝑦) = ∑ ∑|𝐷(𝑚,𝑛)
2
(𝑥, 𝑦)|
2
2𝑎
𝑛=0
2𝑎
𝑚=0
=
1
(2𝑎 + 1)2
∑ ∑|𝐷
̂(𝑛,𝑚)
2
(𝑥, 𝑦)|
2
2𝑎
𝑛=0
2𝑎
𝑚=0
From (16), we get
𝑟2(𝑥, 𝑦) ≤
1
(2𝑎 + 1)2
∑ ∑|𝐷
̂(𝑝,𝑞)
1
(𝑥, 𝑦)|
2
2𝑎
𝑛=0
2𝑎
𝑚=0
≤ ∑ ∑|𝐷
̂(𝑚,𝑛)
1
(𝑥, 𝑦)|
2
2𝑎
𝑛=0
2𝑎
𝑚=0
𝑟2(𝑥, 𝑦) ≤ 𝑟1(𝑥, 𝑦)

51
This proves that the local variability in blurred part is smaller than local variability in focused
part.
AUTHORS
Rachid Sabre received the PhD degree in statistics from the University of Rouen, Rouen, France, in 1993
and Habilitation (HdR) from the University of Burgundy, Dijon, France, in 2003.He joined Agrosup Dijon,
Dijon, France, in 1995, where he is an Associate Professor. From 1998 through 2010, he served as a
member of “Institut de Mathématiques de Bourgogne”, France. He was a member of the Scientific Council
AgroSup Dijon from 2009 to 2013. In 2012, he has been a member of “ Laboratoire Electronique,
Informatique, et Image” (Le2i), France. Since 2019 has been a member of Laboratory Biogeosciences
UMR CNRS, University of Burgundy. He is author/co-author of numerous papers in scientific and
technical journals and conference proceedings. His research interests lie in areas of statistical process and
spectral analysis for signal and image processing.
Ias Sri Wahyuni was born in Jakarta, Indonesia, in 1986. She earned the B.Sc. and M.Sc. degrees in
mathematics from the University of Indonesia, Depok, Indonesia, in 2008 and 2011, respectively.In 2009,
she joined the Department of Informatics (COMPUTING) System, Gunadarma University, Depok,
Indonesia, as a Lecturer. She is currently a PhD student at University of Burgundy, Dijon, France. Her
current research interests include statistics and image processing.

Neighbour Local Variability for Multi-Focus Images Fusion

More Related Content

Similar to Neighbour Local Variability for Multi-Focus Images Fusion (20)

Recently uploaded (20)

Neighbour Local Variability for Multi-Focus Images Fusion