A Region-Based Randomized Voting Scheme for Stereo Matching

International Symposium on Visual Computing 2010

A Region-Based Randomized
Voting Scheme for Stereo Matching
Guillaume GALES, Alain CROUZIL
TOULOUSE INSTITUTE OF COMPUTER SCIENCE RESEARCH (IRIT)
Sylvie CHAMBON
FRENCH PUBLIC WORKS LABORATORY (LCPC)

Monday, December 6, 2010

Problematic
Y
Scene

• 3D reconstruction from stereo pairs Z
X
• Acquisition jl jr
• Pixel matching il
P
pl ir pr
• Calibration yl yr
• Reconstruction zl zr
Left image Right image
xl xr
Left camera Right camera

Left image Right image Disparity map

G. GALES, A. CROUZIL, S. CHAMBON 2 ISVC 2010


Outline

• Pixel matching methods
• Proposed algorithm based on a randomized voting scheme
• Results
• Conclusion



Pixel matching

• Local methods - based on correlation measures in the vicinity of
pixels
• Global methods - based on mathematical optimization techniques
(belief propagation, graph cut, simulated annealing, etc.) to
minimize errors made by the computed disparities
• Region-based methods - pixels within a homogeneous color region
of an image belong to the same surface



Region-based methods
Left Right
image image

Segmentation Local matching

Region Initial
map disparity map
Estimation of the
parameters of a
surface model

Surface model-
based disparity map

Global optimization

Final
disparity map


Proposed algorithm
Left Right
image image
Local matching
Segmentations with robust
correlation

Region Region Initial
map 1 map 2 ... disparity map

Surface Surface
model-based model-based
disparity map disparity map
Randomized
surface model
computation Surface Surface
model-based model-based
disparity map disparity map
...
Voting scheme
Final disparity
map


Initial disparities
• Do not need to be dense, however a good repartition of correct
initial disparities is needed within each region
• A robust correlation measure is used to improve results near
occluded areas
Match
Match
Left image Right image

G. GALES, A. CROUZIL, S. CHAMBON Outliers 7 ISVC 2010


Multi-segmentations

• Over-segmentation (more regions) - hard to ﬁnd enough correct
initial disparities in some segments
• Under-segmentation (less regions) - errors are more important
when, for instance, a plane model is used to ﬁt a segment from a
conic surface

Region boundaries

Initial disparities

False initial disparities

Over-segmentation Under-segmentation



Randomized voting scheme

• Estimation of a surface model-based disparity map:
• For each region map :
• For each region :
• Randomly select 3 points (in,jn,dn) in the disparity space
within the region, n={1,2,3}
• Compute for this region the plane parameters a,b,c such as
ain+bjn+dn=c
• Assign to each pixel (k,l) within the region its estimated
disparity
pk,l ← c − ak − bl



Region map 1 Region map 2 ...

...

...

... ... ... ...

Region boundaries
Surface model-
based disparities
False disparities




• Estimation of the disparity density function
• n surface model disparity maps are computed. For each pixel,
each computed disparity represents a vote
• Final disparity is the most voted one
• Sub-pixel accuracy can be obtained by computing a disparity
density function using a kernel density estimation method
• Let xi be one vote

n

1 3 1 − x − xi 2 if x − xi ≤ 1
pk,l ← argmax 4
x≤dmax n i=1 0 otherwise



egmentations (S1−2 ). Region boundaries as well as errors are shown in w
map presents different errors. However, these errors are approximations o
disparities. Let’s take a closer look at the cone in front of the mask. The
l does not fit the whole cone, nevertheless, at each random selection, a diff
of the cone obtains correct disparities. As a final result, we take the mode
rity density function computed from all the approximations.

ˆ
disparity density function
f
0.5
0.3
0.1
disparity= x
disparity
33 34 35 36 37 38 39 = dmax

argmax
4. Density function built fromxi the different estimated disparities (black ci
given pixel. The mode (36.3 here) is shown with a bold vertical line.

rateALES, A.number HAMBON
G. G
vs. CROUZIL, S. C of segmentations We evaluate the result of our 2010
12 ISVC
algor

Results

• Data set and evaluation - Middlebury protocol. The error rate
is given by the percentage of bad pixels:
1
BP = (|d(i, j) − dtheoretical (i, j)| threshold)
N i,j

• BP versus the number of region maps
• BP versus the number of random selections (votes)



Results (cont’d)
A Region-Based Randomized Voting Scheme for Stereo Matching

error rate (%) error rate (%)
3.75 3.75
3.5 3.5
3.25 3.25
3 3
2.75 2.75
2.5 2.5
1 2 3 4 5 15 25 50 100
# segmentations # random selections
# region maps # random selections
g. 5. Error rate for the images cones with t 1 over the non-occluded pixels versu
number of segmentations (left) and versus the number of random selections (right)
e mean values are CHAMBONby the black circles and the standard deviation ISVC 2010
G. GALES, A. CROUZIL, S. given 14 are given
the vertical 6, 2010
Monday, December
lines.

Results (cont’d)
5 15 25 50 100
nonocc 9.89 ± 0.94 7.33 ± 0.71 6.74 ± 0.34 5.91 ± 0.28 5.47 ± 0.18
tsukuba G. Gales, A. Crouzil and S.± 0.68 7.49 ± 0.35 6.65 ± 0.27 6.22 ± 0.18
10 all 10.66 ± 0.93 8.05 Chambon
disc 19.88 ± 1.24 17.58 ± 0.82 17.45 ± 0.75 16.45 ± 0.63 16.22 ± 0.46
nonocc tsukuba 0.31 0.20venus
0.49 ± ± 0.03 0.17 ± teddy 0.16 ± 0.01cones ± 0.01
0.02 0.15
venus all 0.89 ± 0.34 0.55 ± 0.06 0.50 ± 0.04 0.48 ± 0.02 0.48 ± 0.02
disc 3.21 ± 0.82 2.56 ± 0.37 2.22 ± 0.27 2.13 ± 0.18 2.09 ± 0.16
nonocc 7.17 ± 1.03 6.04 ± 0.25 5.95 ± 0.21 5.74 ± 0.15 5.65 ± 0.12
teddy all 11.56 ± 1.30 10.26 ± 0.38 10.13 ± 0.28 9.77 ± 0.28 9.80 ± 0.26
disc 17.38 ± 1.20 15.70 ± 0.52 15.62 ± 0.42 15.29 ± 0.39 15.19 ± 0.27
nonocc 3.05 ± 0.12 2.75 ± 0.09 2.68 ± 0.07 2.64 ± 0.04 2.61 ± 0.02
cones 0.5 all
t
8.88 ± 0.22 8.30 ± 0.18 8.12 ± 0.12 8.02 ± 0.08 7.94 ± 0.04
disc 8.58 ± 0.29 7.88 ± 0.23 7.71 ± 0.18 7.60 ± 0.10 7.52 ± 0.06
Table 2. Error rate and standard deviation for the 4 stereo pairs (t 1) ver-
sus the number of random selections (5–100). The evaluation is performed over the
non-occluded pixels (nonocc), all the pixels (all ) and the pixels within the depth-
discontinuity areas (disc) with 4 segmentations. The best result of each row is written
t1
in bold.

t Fig. 6.tsukuba
The first row shows thevenus disparity maps computed by our algorithm. The
final teddy cones
nonocc and third rows nonoccthe all
second all disc show error maps with t 0.5 and disc 1 nonocc =all
disc nonocc all t (black disc
errors,
t0.5 t1 16(34) 16.7(34) 27.1(73) 2.48(10) 2.95(8) 8.13(8) 8.73(3) 13.9(4) 22.1(3) 4.42(1) 10.2(1) 11.5(2)
grey = errors in occluded areas).
t1 t 9.83(27)10.6(26)23.8(64) 0.39(6) 0.78(9) 3.97(13) 6.41(7) 11.1(6) 17.1(9) 2.95(2) 8.40(4) 8.45(3)
2
t3 4.85(77) 5.54(70) 17.7(74) 0.13(6) 0.45(13) 1.86(8) 5.40(12)9.54(12)14.8(13)2.62(4)7.93 (7)7.54(6)
Table 3. Middlebury error rates and ranking into parentheses with different thresholds
t: t1 2. 0.5, t2 Z.F., Zheng, Z.G.: 1. The best ranking for each talgorithm using cooper-
= Wang, = 0.75 and t3 = A region based stereo matching is shown in bold.
G. GALES, A. CROUZIL,optimization. In: CVPR. (2008)
ative S. CHAMBON 15 ISVC 2010
3. Klaus, A., Sormann, M., Karner, K.: Segment-based stereo matching using belief

Results (cont’d)



Conclusion

• Our algorithm gives good results at sub-pixel accuracy even with
non polyhedral objects
• Easy to implement yet very effective
• Algorithm highly parallelizable
• We are working on improvements by using a confidence measure to
make a weighted random selection (to avoid the selection of initial
disparities with low confidence)



A Region-Based Randomized Voting Scheme for Stereo Matching

More Related Content

Similar to A Region-Based Randomized Voting Scheme for Stereo Matching (20)

Recently uploaded (20)

A Region-Based Randomized Voting Scheme for Stereo Matching