Adaptive Multiscale Stereo Images Matching Based on Wavelet Transform Modulus Maxima

Abdelhak Ezzine, Youssef Yazidi Alaoui & Achraf Cohen
International Journal of Image Processing (IJIP), Volume (6) : Issue (5) : 2012 373
Adaptive Multiscale Stereo Images Matching Based on Wavelet
Transform Modulus Maxima
Abdelhak EZZINE aezzine@uae.ac.ma
ENSAT/ SIC/LABTIC
Abdelmalek ESSAADI University
Tangier, 99000, Morocco
Youssef YazidiAlaoui yazidi17@gmail.com
ENSAT/ SIC/LABTIC
Achraf Cohen cohenachraf@gmail.com
ENSAT/ SIC/LABTIC
Abstract
In this paper we propose a multiscale stereo correspondence matching method based on
wavelets transform modulus maxima. Exploitation of maxima modulus chains has given us the
opportunity to refine the search for corresponding.Based on the wavelet transform we construct
maps of modules and phases for different scales, then extracted the maxima and then we build
chains of maxima. Points constituents maxima modulus chains will be considered as points of
interest in matching processes. The availability of all its multiscale information, allows searching
under geometric constraints, for each point of interest in the left image corresponding one of the
best points of constituent chains of the right image. The experiment results demonstrate that the
number of corresponding has a very clear decrease when the scale increases. In several tests we
obtained the uniqueness of the corresponding by browsing through the fine to coarse scales and
calculations remain very reasonable.
Keywords:Maxima modulus, stereo matching, wavelet transform, maxima modulus chains.
1 INTRODUCTION
Matching the extracted visual indexes of stereoscopic images is a very significant step within the
process of 3D version. It is made up of bringing together the primitive extracts of right and left
images so as to realize a 3D reconstruction of the scene. This is based on estimating the offset
between the positions of matched primitives and building a disparity map showing the scene by
highlighting the depth of scene objects. The disparity map generated from the correspondence
matching process, along with a stereo camera are used to estimate the depth map and produce
the 3D view of the scene.
This problematic is largely treated by a variety of researchers yet still there is no general method
[6,8,10,26], many approaches exist in the field and exploit generally the primitives such as
reference points, contours or areas but generally via the same characteristics which are either
photometric, geometric or either morphologic.
This technique remains difficult due to the multiplicity of essential parameters, mainly the
pertinence of considerable attributes, in the process of matching and the difficulty to find the good

corresponding primitive is subject to a number of potential problems in vision process like
occlusion, ambiguity, illuminative variations and radial distortion and all algorithms proposed are
adapted to control at least some of these problems. We find many class of stereo vision
algorithms, the first class is named global algorithms (GA) who deal with the correspondence
estimation process as global cost function optimization problem and the second one is the local
algorithms (LA) and are categorized into either area based or feature based algorithms [23]. The
area-based algorithm used the correlation of the two image functions over locally defined regions
and the feature-based algorithms establish correspondence between the selected primitives of
the image.
Our method is one of the new class of algorithms [5,6], it is based on the concept of wavelets
because the matching is the result of a multi-scale operation of the modulus and the phase of
wavelet coefficients. This class is considered in the middle way between the local and global
algorithm, these combine the best features of the LA and GA techniques and uses the
multiresolution concept to involves the matching of two images at different scales.
This method is based on the wavelets modulus maxima chain (CMI) and involves matching of the
different points forming the chain (CMI) to find the best corresponding in the right image for each
extremity of one chain in left image. The search of the best one is done by estimating a similarity
test between the points of each left chain and points of the right chains satisfying the geometric
constraint like epipolar and orientation. The disparity map generated has a multiscale character
as it is built based on the simultaneous treatment of space and scale. The multiscale character
will permit the refinement of the disparity map and possibility to refine the selection of the
potential candidates using a multiscale criterion based on normalized correlation and geometric
refinement. As proved in Mallat[7,22,12], the extremity of the chain coincides with a singularity of
the image characterized by its coefficient of Lipchitz, thus with a good choice for constructing
maxima chains, the proposed matching algorithm not only provides the precise results but it also
converges rapidly.
Firstly, we are going to recapitulate the main principles of wavelet transform modulus maxima
(MMW). Then, we move to present the method and its particularities. Finally, we synthesis the
major results of the method we intend to compare with other attributes.
2. WAVELETS TRANSFORM MODULUS MAXIMA
Based on the idea that in the image processing, visual data is put in a hierarchic way via the
scale, wavelets transform decompose image into elementary blocs that are well localized both on
scale and space. As a result of this decomposition, the image can be represented by the
approximation to the coarse resolution and detail to all intermediate scales.
Among the main interests of representation in the form of wavelets we state the coefficient
wavelet length exam of an image can inform us about it analysis. Indeed, in [2, 9], it is proved
that the regularity accentuates the decay of wavelet coefficients; we are talking about the local
regularity related to the analyzing wavelet support [17, 14].
Thereby, if the image is continuously regular everywhere except at a few isolated points the
estimated wavelet coefficients will be affected only if the support of the analyzing wavelet
contains these points or edges. In this sense, multiscale representations are better suited to focus
the information in an image whose regularity is not homogeneous because the wavelet
coefficients above a certain threshold focus only near the singularities (edges, single points, etc.).
One of the multiscale decompositions that exploits well the wavelet properties which permit
making a study, in a different levels, about the discontinuity points present in an image is wavelet
transform modulus maximas proposed by Mallat and Zhong[1,7].
The principle of wavelet modulus maxima can give the image of characteristics to identify it, they
transmit obvious features of the image depending on well defined models and well determined

directions. For more details about wavelets modulus maxima we refer to [2, 13,14, 21] and
present bellow the essential terms.
Suppose two wavelets ߰௫
and ߰௬
suh that ߰௫
ൌ െ
డீ
డ௫
and ߰௬
ൌ െ
డீ
డ௬
and ‫׬‬ ‫ܩ‬ሺ‫ݐ‬ሻ݀‫ݐ‬
ାஶ
ିஶ
് 0. For this
work we choose ‫ܩ‬ as a Gaussian wavelet to ensure both a valorization of wavelet maxima and
continuity of the wavelet maxima chains [4, 7]. Thus wavelet transform of an image I can be
written as a multiscale differential operator
ቆ
ܹ௫
‫ܫ‬ሺ‫,ݑ‬ ‫,ݒ‬ ݆ሻ
ܹ௬‫ܫ‬ሺ‫,ݑ‬ ‫,ݒ‬ ݆ሻ
ቇ ൌ ቆ
‫ܫ‬ ‫כ‬ ߰௝
௫
ሺെ‫,ݑ‬ െ‫ݒ‬ሻ
‫ܫ‬ ‫כ‬ ߰௝
௫ሺെ‫,ݑ‬ െ‫ݒ‬ሻ
ቇ ൌ 2௝
‫׏‬ሬሬԦ ሺ‫ܫ‬ ‫כ‬ ‫ܩ‬௝ሺെ‫,ݑ‬ െ‫ݒ‬ሻሻ ሺ1ሻ
The polarized representation of this gradient vector offers the wavelet transform modulus
‫ܫܯ‬ሺ‫,ݑ‬ ‫,ݒ‬ ݆ሻ ൌ ඥ|ܹ௫‫ܫ‬ሺ‫,ݑ‬ ‫,ݒ‬ ݆ሻ|ଶ ൅ |ܹ௬‫ܫ‬ሺ‫,ݑ‬ ‫,ݒ‬ ݆ሻ|ଶ ሺ2ሻ
and it is direction is defined by the angle ߠ‫ܫ‬ሺ‫,ݑ‬ ‫,ݒ‬ ݆ሻ ൌ tanିଵ
ቀ
ௐ೤ூሺ௨,௩,௝ሻ
ௐೣூሺ௨,௩,௝ሻ
ቁ . (3)
Then the local maximum of the wavelet transform modulus ‫ܫܯܯ‬ሺ‫,ݑ‬ ‫,ݒ‬ ݆ሻ can be found by solving
߲‫ܫܯ‬ሺ‫,ݑ‬ ‫,ݒ‬ ݆ሻ ൌ 0.
Otherwise on some scale j, the points ሺ‫ݑ‬଴, ‫ݒ‬଴ሻ such that ‫ܫܯܯ‬ሺ‫,ݑ‬ ‫,ݒ‬ ݆ሻ is a local maximum,
according the ‫ܮ‬ஶ norm, on a neighborhood of ሺ‫ݑ‬଴, ‫ݒ‬଴ሻ in the direction ߠ‫ܫܯ‬ሺ‫,ݑ‬ ‫,ݒ‬ ݆ሻ. We note that
for image color we consider the maxima according DiZenzo approach[11].
3. MULTILEVEL MATCHING METHOD
In works of Mallat and Zhong[7] is prove that a wavelet coefficient can be influenced by a
singularity resides within support of the analyzing wavelet; we talk about the cone of influence to
express the set of points where the wavelet coefficients may be influenced by a singularity at a
point ሺu଴, v଴ሻ. This singularity will generate local maxima of wavelet coefficient upon the cone and
till a particular scale. In order to characterize this singularity, we will observe the behavior of local
maxima in terms of scale; this will be much easier to follow if the maxima, between two
successive scales, are connected together to form a curve in the plane scale space so-called line
or chain maxima.
In practice, we start with an image of size ሺN, Nሻ, N ൌ 2J
. We obtain the wavelet transform
according dyadic approach [3,11,18] in which scale parameter is expressed by the power of
2, 2୨
, j ൌ J … L, L is the corsest scale.
This method generates on each scale, a wavelet transform with the same number of pixels as the
image I; in spite of that the support of the wavelet is increasingly large when scales grow. After
that we calculate the wavelets transform modulus and directions by equations (2) and (3). The
results are two matrix, the maxima modulus matrix MMIሺ. , . , jሻ and the angle matrix θMIሺ. , . , jሻ for
each scale j ൌ J െ 1 … L. The image I is then represented by ሺMMIሺ. , . , jሻሻ୨ୀJିଵ…L, positions and
values of the wavelet coefficients at each scale j when the module is a local maximum in the
direction .
We construct the line or chain of maxima by a chaining ൫‫ݔ‬௝, ‫ݕ‬௝൯, the locationof each element
ofMMIሺ. , . , jሻ, with ൫‫ݔ‬௝ିଵ, ‫ݕ‬௝ିଵ൯,the location of its successor in MMIሺ. , . , j െ 1ሻ[21,23]. The search
for these successors is limited to the neighborhood defined by the cone of influence and every
chain can be indexed byሺ‫ݔ‬௝, ‫ݕ‬௝ሻ௝ୀ௃…௠and noted‫ܫܯܥ‬൫‫ݔ‬௃
ூ
, ‫ݕ‬௃
ூ
, ݉൯[14,11]. The integer m indicate the
length of the chain, and is defined such that the last point of the chain is ones of maxima of the

wavelet transform at scale J-m. We note that each chain is composed of maxima and is
characterized by its extremity ‫ܫܯܯ‬ሺxJ, yJ, J at the level J that coincides with a singular point of the
image.
3.1 The matching approach
Given two rectified color images, from a vision system calibrated. we associate to each one its
maxima modulus chains representationሺ‫ݔ‬௝
௜
, ‫ݕ‬௝
௜
ሻ௝ୀ௃…௠, ݅ ൌ ݃for the left image and݅ ൌ ݀for the right
one. With different representations defined above, we are faced with three possible methods to
mapping. The first is to match directly the maxima modulus transforms‫ܫܯܯ‬௜ሺ. , . , jሻof the two
images at each scale݆ ൌ ‫ܬ‬ െ 1 … ‫ܮ‬ . The second performs the matching of the extremity of each
chain‫ܩܯܥ‬൫‫ݔ‬௃
௚
, ‫ݕ‬௃
௚
, ݉൯with an extremity of an chain‫ܦܯܥ‬൫‫ݔ‬௃
ௗ
, ‫ݕ‬௃
ௗ
, ݉Ԣ൯by an non-linear algorithm. The
third method proposes to consider the maxima chains as primitives.In this paper we develop the
second method and we will give an idea about the treatment of the third, we note that the first one
looks approximately like the work of Bhatti and we invite the reader to discover in this work [5].
To chain ‫ܩܯܥ‬൫‫ݔ‬௃
௚
, ‫ݕ‬௃
௚
, ݉൯of the left image , we will search for all Chain of the right image such that
their extremities respect the epipolar constraint with the pixel ሺ‫ݔ‬௃
௚
, ‫ݕ‬௃
௚
ሻ; all maxima of the left that
have no match in the right image verifying the epipolar constraint are dropped. We then perform a
test of similarity between ‫ܫܯܯ‬ሺ‫ݔ‬௃
௚
, ‫ݕ‬௃
௚
, Jሻand each one of the maxima modulus on extremities of
the rights chains selected before. We note ቄ൫‫ݔ‬௃,௞
ௗ
, ‫ݕ‬௃,௞
ௗ
൯
௞ୀଵ..௖
ቅthe set of extremity of the right chains
validating this test. If ܿ ൌ 1, we conclude that ሺ‫ݔ‬௃
௚
, ‫ݕ‬௃
௚
ሻis matched with ൫‫ݔ‬௃,ଵ
ௗ
, ‫ݕ‬௃,ଵ
ௗ
൯.If ܿ ൐ 1, we move
to the next scale and we redo the estimating test between the successors, i.e. ሺ‫ݔ‬௃ିଵ
௚
, ‫ݕ‬௃ିଵ
௚
ሻand
ቄ൫‫ݔ‬௃ିଵ,௞
ௗ
, ‫ݕ‬௃ିଵ,௞
ௗ
൯
௞ୀଵ..௖
ቅ. This step is repeated, recursively on the scale j, until the end of the chain
and it will result necessarily matching of ሺ‫ݔ‬௃
௚
, ‫ݕ‬௃
௚
ሻand ones of ቄ൫‫ݔ‬௃,௞
ௗ
, ‫ݕ‬௃,௞
ௗ
൯
௞ୀଵ..௖
ቅpoints.
The similarity is evaluated, under the orientation constraint, by using ones off different similarity
measure available in the literature. Major similarity measures used in matching methods are
measures of distortion (SAD) and (SSD) and the normalized cross correlation (NCC)[15]. It is
known that SAD and SSD are computationally fast then NCC who is more accurate and it allowed
us to refine the similarity test.
At fine scales, there are many edge points created by the image noise in the wavelet transform
maxima modulus representation. By reason of the presence of these wrong maxima and because
it is well known that the SSD and SAD are justified when the additive noise distribution is
Gaussian or exponential, we are led to combine the NCC and SAD to the similarity test for the
fine scales. We note that at larges scales the smoothing process removes most of the wrong
maxima and the NCC is sufficient.
To find the correspondence between pixels ofM‫ܩܯ‬ቀ‫ݔ‬୨
௚
, ‫ݕ‬୨
௚
, jቁ and M‫ܦܯ‬൫‫ݔ‬୨
ௗ
, ‫ݕ‬୨
ௗ
, j൯, one can
maximize a correlation or minimize the distortion measure. For each point൫‫ݔ‬௝
௚
, ‫ݕ‬௝
௚
൯, we search for
a pointሺ‫ݔ‬௝
ௗ
, ‫ݕ‬௝
ௗ
ሻ in ‫ܦܯܥ‬൫‫ݔ‬௃
ௗ
, ‫ݕ‬௃
ௗ
, ݉Ԣ൯, whose a neighborhood of a size s has a maximum correlation
or maximum distortion with the neighborhood of the pointሺ‫ݔ‬௝
௚
, ‫ݕ‬௝
௚
ሻ in ‫ܦܯܥ‬൫‫ݔ‬௃
௚
, ‫ݕ‬௃
௚
, ݉൯.Due to the
epipolar constraint we have‫ݕ‬௝
ௗ
ൌ ‫ݕ‬௝
௚
and the search is reduced to points൫‫ݔ‬௝
ௗ
, ‫ݕ‬௝
௚
൯on the epipolar
line.
4. EXPERIMENT RESULTS
The algorithm presented in this paper is tested on the images from web site
[10,10b]www.middlebury.edu/stereo/. These images are created for the stereo matching
algorithms we present the results obtained for four of them with the same illumination. In order to
give visual results, the performance of the proposed nonlinear algorithm is presented for different

Abdelhak Ezzine, Youssef Yazidi A
International Journal of Image Processing (IJIP), Volume (6) : Issue (5) : 2012
scales j with the dyadic approach wavelet
on different scales.
The experimental results were generated using the Gaussian wavelets.
thefigure 1, we present the left images
the right view of the same images. In the second and fourth column we present the
modulus on scale J=4.
Into the third columnare presented the number
validating the similarity test with
present these results of the simulation
great maxima of each one of the tree images.
and for each maxima
number of corresponding maxima of the right image on each scale j.
Left Image Maxima
modulus for
scale j:4
Figure 1: in third column we observe the decreasing behavior of number of corresponding
maxima in right image at scale j:4 for everyone of maxima
maxima and z axis: number of corresponding)
We observe that the number of
are more than 6 corresponding in the first scale, our method allows the reduction of this number
to 1 for several maximas. We observe also that
require further exploration scales.
This observation is more important and
category of singularities and the speed of convergence of this method
this method with other work and
the disparity map based on the
direction of our research.
Alaoui & Achraf Cohen
Image Processing (IJIP), Volume (6) : Issue (5) : 2012
scales j with the dyadic approach wavelet transform so we can follow the behavior of a maximum
The experimental results were generated using the Gaussian wavelets. In the first
left images cones, teddy and tsukuba. In the fifth column we present
the right view of the same images. In the second and fourth column we present the
e presented the number (Z axis) of maxima of the right imag
validating the similarity test with each one of the maxima (X axis) of the left image. We
of the simulationonly for 50 points (on the X axis) represents the
each one of the tree images. The third axis (Y axis) represents
or each maxima of the left image we represent the behavior of the
maxima of the right image on each scale j.
modulus for
Behavior of
corresponding maxima
Maxima
modulus for
scale j:4
Right image
: in third column we observe the decreasing behavior of number of corresponding
maxima in right image at scale j:4 for everyone of maxima in left image (x axis :scale; y axis:
maxima and z axis: number of corresponding)
We observe that the number of corresponding decrease clearly when the scale increases.
to 1 for several maximas. We observe also that maxima corresponding to the strong singularity
require further exploration scales.
This observation is more important and we are working to prove the relationship between the
category of singularities and the speed of convergence of this method. To compare the results of
this method with other work and especially by platform Middleburry[10,10b], we need to
the disparity map based on the points of interest of our method and this is one of the future
377
transform so we can follow the behavior of a maximum
In the first column of
ddy and tsukuba. In the fifth column we present
the right view of the same images. In the second and fourth column we present the maxima
of the right image
of the left image. We
represents the
represents the scale
we represent the behavior of the
Right image
: in third column we observe the decreasing behavior of number of corresponding
in left image (x axis :scale; y axis:
when the scale increases. There
g to the strong singularity
we are working to prove the relationship between the
To compare the results of
, we need to validate
oints of interest of our method and this is one of the future

5. CONCLUSION
This paper introduced a nonlinear algorithm matching based on wavelet transform maxima
modulus. The maxima modulus chains are used to find the best corresponding maxima in the
right image of each maximum in left image. This process exploits intelligently the multiscale
character and offers an accurate and fast algorithm. We note also that the first results of the
construction of disparity maps are very encouraging.
Following this work is, firstly, affine the disparity map and compare it to techniques benchmarked
and published in the Middleburry database; and secondly integer the Lipschitz regularity of pixels
[2,16,17] in the similarity process as an additive information, because it was proven that there is
a strong relationship between maxima and this regularity.
This will allow us to exploit the nature of the maxima as edge points in the image and propose a
non-linear feature-based algorithm. The next future direction on this work is to explore different
wavelets basis and observe the behavior of the nonlinear algorithm of this paper according the
choice of wavelets basis.
6. REFERENCES
[1] S. Zhong, “Edge Representation from Wavelet Transform Maxima”. PhD thesis, New York
University, September 1990.
[2] Y. Meyer, “ Ondelettesetopérateurs “, vol. I. Paris: Hermann, 1990.
[3] M. Shensa, “ Wedding the à trous and Mallat algorithms, “ IEEE Trans. Signal Processing,
vol. 40, no. 10, pp. 2464-2482, 1992.
[4] J. Canny, “ computational approach to edge detection “, IEEE Trans. on PAMI, vol. 8, no. 6,
pp. 679-698, 1986.
[5] A. Bhatti,S. Nahavandi “Stereo Image matching Using Wavelet Scale-Space Representation”,
Proceeding of CGIV’06, pp 267-272, 2006
[6] P. B. Zadeh, C. V. Serdean “ A Hierarchical multiwavelet based stereo correspondence
matching techniques “, EURASIP’2011, pp 393-397
[7] S. Mallat and S. Zhong, “ Characterization of signals from multiscale edges “, IEEE Trans. on
PAMI, vol. 14, pp. 710-732, July 1992.
[8] G. Qu, D. Zhang and P. Yan “ Medical image fusion by wavelet transform modulus maxima”,
Optics Express Vol 9, No 4 pp 184-190, 2001.
[9] A. Cohen and A. Ezzine “Quadratures singulières et fonctions d'echelle “, CRAS, t.323, Serie
I, 829-834, 1996.
[10] D. Scharstein and R. Szeliski, www. Middlebury.edu/stereo/,
[10b] D. Scharstein and R. Szeliski, Stereo matching with nonlinear diffusion. Int. J. of Computer
vision, 28(2), 1998.
[11] S. Dizenzo, “ A note on the gradient of a multi-image “, CVGIP, vol. 33, pp. 116-125, 1986.
[12] S. Mallat and W. Hwang, “ Singularity detection and processing with wavelets “, IEEE Trans.
Information Theory, vol. 38, no. 2, pp.617-643, 1992.

[13] P. Carré, “ Maxima d’ondelettes couleur, colorwavelet maxima “, traitement du signal 2004
vol 21 special L’image numérique couleur 623-632.
[14] S. Mallat, “ Wavelets for a vision “, Processing of IEEE, vol. 84, NO 4, pp. 604-614, April
1996.
[15] L. Distefano, Stefano Mattocia, “ Fast template matching using bounded partial correlation “,
Machine vision and application, NO 13, pp. 213-221, 2003.
[16] S. Jaffard, “ Exposants de Holder en des points données et coefficients d’ondelettes”, CRAS
France,308ser I ,pp79-81,1989.
[17] I. Daubechies, Ten lectures on Wavelets, SIAM, Philadelphia, PA, 1992.
[18] S. Mallat, a wavelet tour of signal processing, Academic Press, second edition, 1998.
[19] J. Canny A computational approach to edge detection, IEEE Trans. Patt. Anal. and Mach.
Intell.,vol PAMI-8 no. 6, pp. 679-698, Novembre 1986.
[20] M. Holschneider, R. Kronland-Martinet, J. Morlet, P. Tchamitchian Wavelets, Time-
Frequency Methods, and Phase Space, Springer-Verlag, Berlin, 1989.
[21] O. Lecadet Méthodes d’ondelettes pour la segmentation d’images. Applications à l’imagerie
médicale et au tatouage d’images, these de doctorat, 2003.
[22] S. Meignen, S. Achard, Time Localization of Transients with Wavelet Maxima Lines, IEEE.
Trans. sign. Proc., accepted, 2004.
[23] Y. Ruichek, Perception de l’environnement par stereovision , Application à la sécurité dans
les systèmes de transports terrestres, Phdthesis, USTL, France 2006.

Adaptive Multiscale Stereo Images Matching Based on Wavelet Transform Modulus Maxima

More Related Content

What's hot (18)

Viewers also liked (17)

Similar to Adaptive Multiscale Stereo Images Matching Based on Wavelet Transform Modulus Maxima (20)

Recently uploaded (20)

Adaptive Multiscale Stereo Images Matching Based on Wavelet Transform Modulus Maxima