Joint 3D Shape Matching
December 2015
Abstract
In this paper we extend the global optimization-based
approach of jointly matching a set of images to jointly
matching a set of 3D meshes. The estimated correspon-
dences simultaneously maximize pairwise feature affini-
ties and cycle consistency across multiple models. We
show that the low-rank matrix recovery problem can be
efficiently applied to the 3D meshes as well. The fast
alternating minimization algorithm helps to handle real
world practical problems with thousands of features. Ex-
perimental results show that, unlike the state-of-the-art
algorithm which rely on semi-definite programming, our
algorithm provides an order of magnitude speed-up along
with competitive performance. Along with the joint shape
matching we propose an approach to apply a distortion
term in pairwise matching, which helps in successfully
matching the reflexive sub-parts of two models distinc-
tively. In the end, we demonstrate the applicability of
the algorithm to match a set of 3D meshes of the SCAPE
benchmark database.
1 Introduction
Finding feature correspondences between two meshes is a
new problem in computer vision with various applications
such as structure from motion, model registration, shape
analysis, to name a few. While previous efforts were
mostly focused on matching a pair of images, many tasks
require to find correspondences across multiple meshes in
fact. A typical example is nonrigid structure from motion
[4], where one can hardly reconstruct a nonrigid shape
from two frames. Furthermore, recent work has shown
that leveraging multi-way information can dramatically
improve matching results compared to pairwise match-
ing. The most important constraint for joint matching
is the cycle consistency, i.e., the composition of matches
along a loop of meshes should be identity, as illustrated
in Figure 1. Given pairwise matches, one can possibly
identify true or false matches by checking all cycles in
the mesh collection. But there are many difficulties for
this approach. For example, the input pairwise matches
are often very noisy Figure 1. An illustration of consis-
tent multi-mesh matching. With many false matches and
missing matches, and the features detected from different
meshes may only have a partial overlap even if the same
feature detector is applied. Therefore, it is likely that very
few consistent cycles can be found. Moreover, how to
sample cycles is not straightforward due to the huge num-
ber of possibilities. Recent work on joint matching has
shown that, if all feature correspondences within multi-
ple meshes are denoted by a large binary matrix, the cy-
cle consistency can be translated into the fact that such
a matrix should be positive semi-definite and low-rank .
Based on this observation, convex optimization-based al-
gorithms were proposed, which achieved the state-of-the-
art performances with theoretical guarantees . But these
algorithms rely on semi-definite programming (SDP) [9],
which is not computationally efficient to handle image
matching problems in practice.
In this paper, we make use of MatchALS [16], a novel
algorithm for multi image matching, and use it for 3D
model meshes. The inputs to our algorithm are orig-
inal similarities between feature descriptors which we
used Heat Kernel Signature as. The outputs are fea-
ture correspondences between all pairs of images. Un-
like many previous methods starting from quantized pair-
wise matches , we postpone the decision until we opti-
mize for both pairwise affinities and multi-mesh consis-
tency. Instead of using Semi Definite Programming (SDP)
relaxation, we formulate the problem as a low-rank ma-
1
Figure 1: Illustration of Consistent Multi-Image Match-
ing
trix recovery problem and employ the nuclear-norm re-
laxation for rank minimization. We show that the posi-
tive semi-definiteness of a desired solution can be spon-
taneously fulfilled. Moreover, we derive a fast alternat-
ing minimization algorithm to globally solve the problem
in the low-dimensional variable space . Besides validat-
ing our method on both simulated and real benchmark
data sets, we also demonstrate the applicability of the pro-
posed method combined with Re-weighted Random Walk
(RRWM) [7] graph matching to match meshes with dif-
ferent models. We introduce a random walk view on the
problem and propose a robust graph matching algorithm
against outliers and deformation. Matching between two
graphs is formulated as node selection on an association
graph whose nodes represent candidate correspondences
between the two graphs. The solution is obtained by sim-
ulating random walks with re-weighting jumps enforcing
the matching constraints on the association graph. This
algorithm achieves noise-robust graph matching by iter-
atively updating and exploiting the confidences of candi-
date correspondences. In a practical sense, this RRWM
Distortion term is of particular importance since the real-
world matching problem is made difficult by the presence
of noise and outliers. Extensive and comparative experi-
ments demonstrate that it outperforms the state-of-the-art
graph matching algorithms especially in the presence of
outliers and deformation.
2 Preliminaries and notation
Suppose we have n models and pi features from each
model i. The objective is to find feature correspondences
between all pairs of models. Before introducing the pro-
posed method, we first give a brief introduction to pair-
wise matching techniques and the definition of cycle con-
sistency.
2.1 Pairwise matching
To match an image pair (i,j), one can compute similarities
for all pairs of feature points from two images and store
them in a matrix Sij ∈ Rpi×pj
. We represent the feature
correspondences for mesh pair (i,j) by a partial permuta-
tion matrix Xij ∈ 0, 1pi×pj
, which satisfies the doubly
stochastic constraints:
0 ≤ Xij1 ≤ 1, 0 ≤ XT
ij1 ≤ 1 (1)
To find Xij , we can maximize the inner product be-
tween Xij and Sij subject to the constraints in (1) result-
ing in a linear assignment problem, which has been well
studied and can be efficiently solved by the Hungarian al-
gorithm. In image matching, spatial rigidity is usually
preferred, i.e., the relative location between two features
in an mesh should be similar to that between their cor-
respondences in the other mesh. This problem is well
known as graph matching and formulated as a quadratic
assignment problem (QAP). While QAP is NP-hard [2],
many efficient algorithms have been proposed to solve it
approximately, e.g. Those solvers basically relax the bi-
nary constraint on the permutation matrix, solve the opti-
mization, and output the confidence of a candidate match
being correct. We refer readers to the related literature
for details. Here we aim to emphasize that the outputs
of graph matching solvers are basically optimized affinity
scores of candidate matches, which consider both feature
similarity and spatial rigidity. We will use these scores
(saved in Sij) as our input in some cases.
2.2 Cycle Consistency
For any three models (i,j,z), cycle consistency can be de-
scribed as
Xij = XizXzj, (2)
and can be extended to the case with more than three
models.The cycle consistency can be described more con-
cisely by introducing a virtual universe that is defined as
the set of unique features that appear in the image collec-
tion [13]. Each point in the universe may be observed by
2
several images and the corresponding image points should
be matched. In this way, consistent matching should sat-
isfy Xij = AiAT
j , where Ai ∈ {0, 1}
pi×k
denotes the
map from Image i to the universe, k is the number of
points in the universe, and k ≥ pi for all i.
Suppose the correspondences for all m =
n
i=1 pi
features in the image collection is denoted by X ∈
{0, 1}
m×m
and all Ais are concatenated as rows in a ma-
trix A ∈ {0, 1}
m×k
. Then one can write X as
X =





X11 X12 . . . X1n
X21 X22 . . . X2n
...
...
...
...
Xn1 Xn2 . . . Xnn





, (3)
X = AAT
(4)
From (4), it is clear to see that desired X should be both
positive semi-definite and low rank, which effectively im-
pose cycle consistency without checking all cycles of pair-
wise matches.
X 0, rank(X) ≤ k (5)
3 Joint Matching via Rank Mini-
mization
Given the affinity scores {Sij|1 ≤ i, j ≤ n} for all mod-
els, we aim to find globally consistent matches X. Affin-
ity scores can be computed from either feature similarities
or graph matching solvers according to specific scenarios,
For our experiments we use Heat Kernel Signature (HKS)
feature descriptor to define the affinity score matrices.
3.1 Cost Function
To formulate the cost function, we treat the problem as
a low rank matrix recovery problem. The inner product
between Xij and Sij is maximized for all i, j as multiple
linear assignment problems. Alongside, to enforce cycle
consistency, the rank of X is minimized. We make fol-
lowing two relaxations on X to optimize the solution:
• X is treated as a real matrix X ∈ [0, 1]
m×m
in-
stead of a binary matrix. Experimental results show
that the solution values can be stably quantized by a
threshold of 0.5.
• Rank of X is replaced by nuclear norm ||X||∗ (sum
of singular values), which is a tight convex relaxation
proven to be very effective in various low-rank prob-
lems such as matrix completion [6].
The estimated X should be sparse since at most one
value in each row of Xij can be nonzero. To induce spar-
sity, we minimize the sum of values in X. Combining all
three terms, we obtain the following cost function:
f(X) = −
n
i=1
n
j=1
Sij, Xij + α 1, X + λ||X||∗,
= − S − α1, X + λ||X||∗, (6)
where ., . denotes the inner product, S ∈ Rm×m
is a
collection of Sijs, α is is the weight of sparsity, λ controls
the weight of the nuclear norm.
Besides the doubly stochastic constraints in (1), addi-
tional constraints shall be imposed on X after relaxation:
Xii = Ipi
, 1 ≤ i ≤ n, (7)
Xij = XT
ji, 1 ≤ i, j ≤ n, i = j, (8)
0 ≤ X ≤ 1, (9)
Finally, we obtain the following optimization problem:
min
X
W, X + λ||X||∗,
s.t.X ∈ C, (10)
where W = α1 − S and C denotes a set of matrices
satisfying the constraints given in (1), (7), (8), and (9).
4 Fast Alternating Minimization
The state-of-the-art methods to solve the convex nuclear
norm minimizatiion in (10) are proximal method [14] or
ADMM [3] based on iterative single value thresholding
[5]. However, singular value decomposition (SVD) needs
to be performed in each iteration, which is extremely ex-
pensive even for a mediumsized problem.
3
Similar low-rank optimization was solved in [16] for
2D images more efficiently via a change of variables
X = ABT
, where A, B ∈ Rm×k
are new variables with
a smaller dimension k < m.
We use the following lowrank factorization-based for-
mulation implemented for 2D images in [16] and extend
it to 3D mesh based models:
min
A,B
W, ABT
+ λ||ABT
||∗,
s.t.ABT
∈ C, (11)
Now rewriting the nuclear norm minimization problem
as SDP [15],
||X||∗ = min
A,B:ABT =X
1
2
||A||2
F + ||B||2
F (12)
Finally the following formulation is obtained:
min
A,B
W, ABT
+
λ
2
||A||2
F +
λ
2
||B||2
F ,
s.t.ABT
∈ C, (13)
The selection of matrix dimension k is critical to the
success of change of variables, while it directly affects
the computational complexity.
4.1 Algorithm
We extend the following algorithm described in [16] for
2D Images to 3D models. The algorithm applies ADMM
to solve (13). The augmented Lagrangian of (13) is as
follows:
£µ(X, A, B, Y ) = W, X +
λ
2
||A||2
F +
λ
2
||B||2
F
+ Y X − ABT
+
µ
2
||X − ABT
||2
F , (14)
where Y is the dual variable and µ is a parameter control-
ling the step size in optimization. The ADMM alternately
updates each primal variable by minimizing £µ and up-
dates the dual variable via gradient ascent while fixing all
other variables.
Result: Globally consistent matches X
randomly initialize A and B, Y = 0 ;
W = α1 − S ;
while not converged do
A ← X + 1
µ Y B BT
B + λ
µ I
τ
;
B ← X + 1
µ Y A AT
A + λ
µ I
τ
;
X ← Pc ABT
− 1
µ (W + Y ) ;
Y ← Y k
+ µ X − ABT
;
end
quantize X with a threshold equal to 0.5;
Algorithm 1: Matching via Alternating Least Squares
(MatchALS)
5 RRWM
Introduction to problem: Graph matching is an essen-
tial problem in computer vision and machine learning. In
this paper, we implemented a random walk view on the
problem and use a robust graph matching algorithm [7]
against outliers and deformation. Matching between two
graphs is formulated as node selection on an association
graph whose nodes represent candidate correspondences
between the two graphs. The solution is obtained by sim-
ulating random walks with re-weighting jumps enforcing
the matching constraints on the association graph. Our
algorithm achieves noise-robust graph matching by iter-
atively updating and exploiting the confidences of candi-
date correspondences. In a practical sense, our work is of
particular importance since the real-world matching prob-
lem is made difficult by the presence of noise and outliers.
Extensive and comparative experiments demonstrate that
it outperforms the state-of-the-art graph matching algo-
rithms especially in the presence of outliers and deforma-
tion.
Recent resurgence of combinatorial optimization ap-
proaches to feature matching has changed the situation
and firmly settled graph matching formulations based on
Integer Quadratic Programming (IQP), which is a gener-
alization of the classical graph matching problems. IQP
explicitly takes into consideration both unary and pair-
wise terms reflecting the compatibilities in local appear-
ance as well as the pair-wise geometric relationships be-
tween the matching features. Since IQP is known to be
4
NP-hard, approximate solutions are required. This pro-
vides a novel interpretation of graph matching in a ran-
dom walk view and relates it to the IQP formulation. In-
troducing an association graph constructed with nodes as
candidate correspondences and edges as pair-wise com-
patibilities between candidate correspondences, we show
that the search for correspondences between the given
two graphs can be cast as a node ranking [11] and selec-
tion problem in the association graph. For this ranking,
we introduce an affinity-preserving random walk and de-
rive a ranking based on its quasi-stationary distribution,
and prove its equivalence to the spectral relaxation for
the IQP formulation. Then, in this random walk view,
we adopt the personalization strategy of Web ranking al-
gorithms [8] and propose the re-weighted random walk
algorithm by re-weighting jumps for the graph matching
constraints. It achieves noise-robust graph matching by si-
multaneously updating and exploiting the confidences of
candidate correspondences. In a practical sense, our work
is of particular importance since the real-world matching
problem is made difficult by the presence of deformation
and outliers.
5.1 Problem Formulation
The objective of graph matching is to determine the cor-
rect correspondences between two attributed graphs Gp =
(Vp, Ep, Ap) and Gq = (Vq, Eq, Aq) where V represents
a set of nodes, E, edges, and A, attributes. Each node
vP
i ∈ V P
or edge eP
i j ∈ EP
has an associated attribute
vector aP
i ∈ AP
or aP
i j ∈ AP
. In feature correspondence
problems, a node attribute aP
i usually describes a local ap-
pearance of feature i in model P, and an edge attribute
aP
i j represents the geometric relationship between fea-
tures i and j in the image P. For each pair of edges eP
i j ∈
EP
and eQ
a b ∈ EQ
, there is an affinity or compatibility
Wia; jb = f aP
i , aP
j , aP
i j, aQ
a , aQ
b , aQ
a b that measures
the mutual consistency of attributes between the pairs
of candidate correspondences vP
i , vQ
a and vP
j , vQ
b .
Thus, using a matrix form W, a non-diagonal element
Wia; jb contains a pair-wise affinity between two corre-
spondences vP
i , vQ
a and vP
j , vQ
b , and a diagonal term
Wia; ia represents a uanry affinity of a correspondence
vP
i , vQ
a . Representing the correspondence with an as-
signment or permutation matrix X ∈ {0, 1}
nP
nQ
is com-
mon, such that Xia = 1 implies that node vP
i corresponds
to node vQ
a , e.g., feature i in the image P is matched to
feature a in the image Q, and Xia = 0 otherwise. In
this paper, we denote x ∈ {0, 1}
nP
nQ
as a column-wise
vectorized replica of X. The graph matching problem
can be formulated as an integer quadratic program (IQP),
that is, finding the indicator vector x∗
that maximizes the
quadratic score function as follows.
x∗
= argmax(xT
Wx) (15)
s.t.x ∈ {0, 1}
nP
nQ
, ∀i
nq
a=1
xiq ≤ 1, ∀a
np
i=1
xia ≤ 1
where the two-way constraints refer to the one-to-one
matching from GP
to GQ
. In general, no efficient al-
gorithm exists that can guarantee the optimality bounds
since the IQP is NP-hard, thus approximate solutions are
required.
Figure 2: Association Graphs for Graph Matching by
Random Walks
5.2 Algorithnm
In the previous affinity-preserving random walks, the
matching constraints are ignored and not reflected in the
random walk process. Inducing the matching constraints
only as a post-processing discretization step like leads
to a weak local optimum. How then can we reflect the
two-way matching constraints in the affinity-preserving
random walk? We adopts the personalization approach
widely used in Web ranking methods, which strength-
ens the effects of reliable nodes in random walks. This
is achieved by adopting a jump or teleport in the ran-
dom walk: the random walker moves by traversing an
edge with probability alpha(α) or by performing a jump
5
to some constrained nodes with probability 1 - α. α rep-
resents the bias between the two possible actions, i.e., fol-
lowing an edge or jumping. To address the lack of person-
alization or users focus, Web ranking algorithms adopted
this approach in topic-sensitive or query dependent vari-
ants. In our formulation, adopting the personalized jump,
the probability distribution is updated using the following
equation:
x(n+1)T
x
(n+1)
abs = α x(n)T
x
(n)
abs P + (1 − α) rT
(16)
where a re-weighting jump vector r is added to the
affinity-preserving random walk of. In this approach, we
use the jumps for generating a biased random walk to the
matching constraints.
6 Experimental Results
We evaluate the performance of the proposed method us-
ing SCAPE benchmark data [1]. We use the blended in-
trisic maps described in [10] to evaluate the performance
of MatchALS for 3D meshes with RRWM Distortion term
added and compare the matching errors for different mod-
els. The size of universe is fixed as 100 points and in each
mesh a random sample of the points are observed with a
probability denoted by ρo. The number of meshes is de-
noted by n, which is generic in our code but for this sim-
ulation purposes it was kept to 4. Then, the ground-truth
pairwise matches are established, and random corruptions
are simulated by removing some true matches and adding
some false matches to achieve an error rate of ρe. Fi-
nally, the corrupted permutation matrix is fed into our Al-
gorithm as the input affinity scores.
Figure 3 shows the results we obtained before we ap-
plied the RRWM distortion term. We observed 1-to-many
feature correspondences prior to RRWM application, and
this can bee seen by the same color of Figure 3 meshes at
the limbs which give false matches sometimes due to the
inherent geometrical symmetry of the object.
Next we demonstrate the results obtained from run-
ning the MatchALS for 3D Meshes in Figure 4 using the
RRWM Distortion term, and the results can be clearly
seen to weed out the false positive matches in Figure 4.
Given the weight matrix W, the reweight factor α,
and the inflation factor β ;
Prevent conflicting walks by setting Wia; jb = 0for
all conflicting match pairs ;
Set the maximum degree
dmax = maxia jb Wia;jb ;
Initialize the transition matrix P = W/dmax, the
starting probability x as uniform ;
while x not converged do
( Affinity-preserving random walking by edges ) ;
xT
= xT
P ;
( Reweighting with two-way constraints ) ;
yT
= exp (βx/maxx) ;
while y not converged do
normalize across rows by
yai = yai/
I
i=1 yai;
normalize across columns by
yai = yai/
A
q=1 yai;
end
y = y/ yai ;
( Affinity-preserving random walking with
reweighted jumps) ;
xT
= αxT
+ (1 − α) yT
;
x = x/ xai ;
end
Discretize x by the matching constraints;
Algorithm 2: Reweighted Random Walk Graph Match-
ing
Preciously colored the same, the limbs are now of dif-
ferent colors, which signifies that RRWM allows one-to-
many matches that appear due to inherent symmetry of
models, to be filtered out in a very precise manner.
Figure 5 next shows the 1-to-1 correspondences we
have achieved between one pair of meshes we matched
and the run time for the simulation was 4.53 seconds
Next we demonstrate the matching errors between the
4 meshes we obtained. For each pair of possible match-
ing the evaluation was performed by calculating the av-
erage Geodesic Error. For the SCAPE Data-set [12] pro-
vides the ground truth vertex-to-vertex correspondence.
To evaluate performance of our algorithm we map every
vertex that has a ground truth correspondence. A geodesic
distance between the predicted correspondence and the
6
(a) Mesh1 matched to Mesh3 (b) Mesh2 matched to Mesh3 (c) Mesh3 matched to Mesh4
Figure 4: Results of MatchALS combined with RRWM
Figure 3: Results of MatchALS performed on 4 Meshes
from SCAPE Benchmark Dataset
true correspondence is recorded. Then the error per sam-
ple is averaged to calculate the Average Geodesic Error
of a mapping between a pair of meshes. Table 1 shows
the evaluation results of the pairwise matching of the 4
meshes we used to demonstrate our results.
Mesh1 Mesh2 Mesh3 Mesh4
Mesh1 0 1.904% 2.637% 13.909%
Mesh2 1.904% 0 6.824% 18.363%
Mesh3 2.637% 6.824% 0 8.236%
Mesh4 13.909% 18.636% 8.236% 0
Table 1: Average Geodesic Error for matching a pair of
Meshes using MatchALS and RRWM
Figure 5: Illustration of 1 to 1 Mapping of feature corre-
spondences in 2 Meshes
7 Conclusion
In this paper, we extended the joint matching algorithm
MatchALS for 2D images to 3D mesh matching. We
use pairwise feature similarities, the heat kernel signature
feature descriptor, as inputs and obtain accurate matches
by an efficient algorithm (MatchALS) that globally op-
timizes for both feature affinities and cycle consistency
of matches. The experiments not only validate the effec-
tiveness of the proposed method but also demonstrate that
joint matching is a promising approach to matching a col-
lection of meshes obtained from crowd sourced 3D model
collections MatchALS is a powerful algorithm that can be
7
extended to 3-D mesh models and the performance gen-
erated by the algorithm is almost the same as the state-
of-the art algorithm but with exponential orders of incre-
ment in speed, as we ourselves observed in our simula-
tion. Coupling the MatchALS with Re-weighted Random
Walk Matching is something that has not been done be-
fore and our novel implementation of stitching the two
algorithms has given very fruitful and promise-bearing re-
sults applicable to real world 3-D model matching appli-
cations .The results reveal that the matching accuracy in
the challenging situations largely depends on the effective
exploitation of the matching constraints.
References
[1] D. Anguelov, P. Srinivasan, D. Koller, S. Thrun,
J. Rodgers, and J. Davis. Scape: shape completion
and animation of people. In ACM Transactions on
Graphics (TOG), volume 24, pages 408–416. ACM,
2005. 6
[2] A. C. Berg, T. L. Berg, and J. Malik. Shape matching
and object recognition using low distortion corre-
spondences. In Computer Vision and Pattern Recog-
nition, 2005. CVPR 2005. IEEE Computer Society
Conference on, volume 1, pages 26–33. IEEE, 2005.
2
[3] S. Boyd, N. Parikh, E. Chu, B. Peleato, and J. Eck-
stein. Distributed optimization and statistical learn-
ing via the alternating direction method of multipli-
ers. Foundations and Trends in Machine Learning,
3(1):1–122, 2011. 3
[4] C. Bregler, A. Hertzmann, and H. Biermann. Re-
covering non-rigid 3d shape from image streams.
In Computer Vision and Pattern Recognition, 2000.
Proceedings. IEEE Conference on, volume 2, pages
690–696. IEEE, 2000. 1
[5] J.-F. Cai, E. J. Cand`es, and Z. Shen. A singular value
thresholding algorithm for matrix completion. SIAM
Journal on Optimization, 20(4):1956–1982, 2010. 3
[6] E. J. Cand`es and B. Recht. Exact matrix completion
via convex optimization. Foundations of Computa-
tional mathematics, 9(6):717–772, 2009. 3
[7] M. Cho, J. Lee, and K. M. Lee. Reweighted ran-
dom walks for graph matching. In Computer Vision–
ECCV 2010, pages 492–505. Springer, 2010. 2, 4
[8] T. H. Haveliwala. Topic-sensitive pagerank. In
Proceedings of the 11th international conference on
World Wide Web, pages 517–526. ACM, 2002. 5
[9] Q.-X. Huang and L. Guibas. Consistent shape maps
via semidefinite programming. In Computer Graph-
ics Forum, volume 32, pages 177–186. Wiley Online
Library, 2013. 1
[10] V. G. Kim, Y. Lipman, and T. Funkhouser. Blended
Intrinsic Maps. Transactions on Graphics (Proc. of
SIGGRAPH), 30(4), 2011. 6
[11] J. M. Kleinberg. Authoritative sources in a hyper-
linked environment. Journal of the ACM (JACM),
46(5):604–632, 1999. 5
[12] Y. Lipman and T. Funkhouser. M¨obius voting for
surface correspondence. In ACM Transactions on
Graphics (TOG), volume 28, page 72. ACM, 2009.
6
[13] D. Pachauri, R. Kondor, and V. Singh. Solving the
multi-way matching problem by permutation syn-
chronization. In Advances in neural information
processing systems, pages 1860–1868, 2013. 3
[14] N. Parikh and S. Boyd. Proximal algorithms. Foun-
dations and Trends in optimization, 1(3):123–231,
2013. 3
[15] B. Recht, M. Fazel, and P. A. Parrilo. Guaran-
teed minimum-rank solutions of linear matrix equa-
tions via nuclear norm minimization. SIAM review,
52(3):471–501, 2010. 4
[16] X. Zhou, M. Zhu, and K. Daniilidis. Multi-image
matching via fast alternating minimization. arXiv
preprint arXiv:1505.04845, 2015. 1, 4
8

More Related Content

PDF
GREY LEVEL CO-OCCURRENCE MATRICES: GENERALISATION AND SOME NEW FEATURES
PDF
Illustration Clamor Echelon Evaluation via Prime Piece Psychotherapy
PDF
Image Restitution Using Non-Locally Centralized Sparse Representation Model
PDF
ALEXANDER FRACTIONAL INTEGRAL FILTERING OF WAVELET COEFFICIENTS FOR IMAGE DEN...
PPTX
Snakes in Images (Active contour tutorial)
PDF
Lecture 19: Implementation of Histogram Image Operation
PDF
Macrocanonical models for texture synthesis
PDF
Quantitative Propagation of Chaos for SGD in Wide Neural Networks
GREY LEVEL CO-OCCURRENCE MATRICES: GENERALISATION AND SOME NEW FEATURES
Illustration Clamor Echelon Evaluation via Prime Piece Psychotherapy
Image Restitution Using Non-Locally Centralized Sparse Representation Model
ALEXANDER FRACTIONAL INTEGRAL FILTERING OF WAVELET COEFFICIENTS FOR IMAGE DEN...
Snakes in Images (Active contour tutorial)
Lecture 19: Implementation of Histogram Image Operation
Macrocanonical models for texture synthesis
Quantitative Propagation of Chaos for SGD in Wide Neural Networks

What's hot (20)

PDF
A Correlative Information-Theoretic Measure for Image Similarity
PDF
Gtti 10032021
PDF
PDF
Conformal symmetry transformations and nonlinear Maxwell equations, G.A.Goldi...
PDF
IMPROVING THE RELIABILITY OF DETECTION OF LSB REPLACEMENT STEGANOGRAPHY
PPT
Free vibration analysis of composite plates with uncertain properties
PDF
Arity shape of polyadic algebraic structures
PDF
Adaptive Multiscale Stereo Images Matching Based on Wavelet Transform Modulus...
PDF
Dt31811815
PDF
Perimetric Complexity of Binary Digital Images
PDF
Steven Duplij, "Polyadic Hopf algebras and quantum groups"
PDF
PDF
PhysRevE.89.042911
PDF
Projection methods for stochastic structural dynamics
PDF
S. Duplij. Polyadic algebraic structures and their applications
PDF
Effect of Rigidity on Trilateration Technique for Localization in Wireless Se...
PDF
A lattice-based consensus clustering
PDF
An efficient approach to wavelet image Denoising
PDF
The Bayesian Optimization Algorithm with Substructural Local Search
PDF
Bd32360363
A Correlative Information-Theoretic Measure for Image Similarity
Gtti 10032021
Conformal symmetry transformations and nonlinear Maxwell equations, G.A.Goldi...
IMPROVING THE RELIABILITY OF DETECTION OF LSB REPLACEMENT STEGANOGRAPHY
Free vibration analysis of composite plates with uncertain properties
Arity shape of polyadic algebraic structures
Adaptive Multiscale Stereo Images Matching Based on Wavelet Transform Modulus...
Dt31811815
Perimetric Complexity of Binary Digital Images
Steven Duplij, "Polyadic Hopf algebras and quantum groups"
PhysRevE.89.042911
Projection methods for stochastic structural dynamics
S. Duplij. Polyadic algebraic structures and their applications
Effect of Rigidity on Trilateration Technique for Localization in Wireless Se...
A lattice-based consensus clustering
An efficient approach to wavelet image Denoising
The Bayesian Optimization Algorithm with Substructural Local Search
Bd32360363
Ad

Similar to Joint3DShapeMatching (20)

PDF
Joint3DShapeMatching - a fast approach to 3D model matching using MatchALS 3...
PDF
Classification of handwritten characters by their symmetry features
PDF
Tutorial on Markov Random Fields (MRFs) for Computer Vision Applications
PDF
Image Processing
PDF
Image Restoration UsingNonlocally Centralized Sparse Representation and histo...
DOC
Face recognition using laplacianfaces (synopsis)
PDF
Application of Graphic LASSO in Portfolio Optimization_Yixuan Chen & Mengxi J...
PDF
COMPARISON OF VOLUME AND DISTANCE CONSTRAINT ON HYPERSPECTRAL UNMIXING
PDF
Parallel Batch-Dynamic Graphs: Algorithms and Lower Bounds
PDF
Parallel Batch-Dynamic Graphs: Algorithms and Lower Bounds
PDF
Quantum algorithm for solving linear systems of equations
PDF
Image Super-Resolution Reconstruction Based On Multi-Dictionary Learning
PDF
A Condensation-Projection Method For The Generalized Eigenvalue Problem
PDF
A simple framework for contrastive learning of visual representations
PDF
PERFORMANCE EVALUATION OF DIFFERENT TECHNIQUES FOR TEXTURE CLASSIFICATION
DOCX
M Jamee Raza (BSE-23S-056)LA project.docx
PDF
Report on Efficient Estimation for High Similarities using Odd Sketches
PPTX
A CONVERGENCE ANALYSIS OF GRADIENT_version1
PDF
Bag of Pursuits and Neural Gas for Improved Sparse Codin
PDF
SASUM: A Sharing-based Approach to Fast Approximate Subgraph Matching for Lar...
Joint3DShapeMatching - a fast approach to 3D model matching using MatchALS 3...
Classification of handwritten characters by their symmetry features
Tutorial on Markov Random Fields (MRFs) for Computer Vision Applications
Image Processing
Image Restoration UsingNonlocally Centralized Sparse Representation and histo...
Face recognition using laplacianfaces (synopsis)
Application of Graphic LASSO in Portfolio Optimization_Yixuan Chen & Mengxi J...
COMPARISON OF VOLUME AND DISTANCE CONSTRAINT ON HYPERSPECTRAL UNMIXING
Parallel Batch-Dynamic Graphs: Algorithms and Lower Bounds
Parallel Batch-Dynamic Graphs: Algorithms and Lower Bounds
Quantum algorithm for solving linear systems of equations
Image Super-Resolution Reconstruction Based On Multi-Dictionary Learning
A Condensation-Projection Method For The Generalized Eigenvalue Problem
A simple framework for contrastive learning of visual representations
PERFORMANCE EVALUATION OF DIFFERENT TECHNIQUES FOR TEXTURE CLASSIFICATION
M Jamee Raza (BSE-23S-056)LA project.docx
Report on Efficient Estimation for High Similarities using Odd Sketches
A CONVERGENCE ANALYSIS OF GRADIENT_version1
Bag of Pursuits and Neural Gas for Improved Sparse Codin
SASUM: A Sharing-based Approach to Fast Approximate Subgraph Matching for Lar...
Ad

More from Mamoon Ismail Khalid (20)

PDF
Caring.ai - AI + Voice Agent co-pilot for all things dementia
PPTX
REMOTE SOLAR MONITORING SYSTEM - A solution to make battery life extend by 300%
PPTX
Network Traffic Adaptable Image Codec - A solution to make streaming faster
PDF
Hospital Management and Inventory Control Solution for Public Hospitals in De...
PDF
ATLAS - Product Requirement Document.pdf
PDF
T(X) Innoway - Prediction Algorithm design.pdf
PDF
Golf Swing Analysis and Posture Correction System
PDF
24 ideas to revive any developing country.pdf
PDF
#2 - Smart Bins - Returnable Plastic Ecosystem.pdf
PDF
PyTorch to detect Humans Eating Food.pdf
PDF
Future of agriculture agriculture - technology is a necessity in 2020 and beyond
PDF
PDF
Real estate in blockchain (2)
PDF
Cohort analysis saa s (1)
PDF
ISA backed technology skills platform
PDF
Start up valuation methods
PDF
Analysis mvp factory
DOCX
Detect Negative and Positive sentiment in user reviews using python word2vec ...
PDF
Detect spam comments youtube videos and app store reviews
PPTX
Start Up deal/interaction management workflow
Caring.ai - AI + Voice Agent co-pilot for all things dementia
REMOTE SOLAR MONITORING SYSTEM - A solution to make battery life extend by 300%
Network Traffic Adaptable Image Codec - A solution to make streaming faster
Hospital Management and Inventory Control Solution for Public Hospitals in De...
ATLAS - Product Requirement Document.pdf
T(X) Innoway - Prediction Algorithm design.pdf
Golf Swing Analysis and Posture Correction System
24 ideas to revive any developing country.pdf
#2 - Smart Bins - Returnable Plastic Ecosystem.pdf
PyTorch to detect Humans Eating Food.pdf
Future of agriculture agriculture - technology is a necessity in 2020 and beyond
Real estate in blockchain (2)
Cohort analysis saa s (1)
ISA backed technology skills platform
Start up valuation methods
Analysis mvp factory
Detect Negative and Positive sentiment in user reviews using python word2vec ...
Detect spam comments youtube videos and app store reviews
Start Up deal/interaction management workflow

Joint3DShapeMatching

  • 1. Joint 3D Shape Matching December 2015 Abstract In this paper we extend the global optimization-based approach of jointly matching a set of images to jointly matching a set of 3D meshes. The estimated correspon- dences simultaneously maximize pairwise feature affini- ties and cycle consistency across multiple models. We show that the low-rank matrix recovery problem can be efficiently applied to the 3D meshes as well. The fast alternating minimization algorithm helps to handle real world practical problems with thousands of features. Ex- perimental results show that, unlike the state-of-the-art algorithm which rely on semi-definite programming, our algorithm provides an order of magnitude speed-up along with competitive performance. Along with the joint shape matching we propose an approach to apply a distortion term in pairwise matching, which helps in successfully matching the reflexive sub-parts of two models distinc- tively. In the end, we demonstrate the applicability of the algorithm to match a set of 3D meshes of the SCAPE benchmark database. 1 Introduction Finding feature correspondences between two meshes is a new problem in computer vision with various applications such as structure from motion, model registration, shape analysis, to name a few. While previous efforts were mostly focused on matching a pair of images, many tasks require to find correspondences across multiple meshes in fact. A typical example is nonrigid structure from motion [4], where one can hardly reconstruct a nonrigid shape from two frames. Furthermore, recent work has shown that leveraging multi-way information can dramatically improve matching results compared to pairwise match- ing. The most important constraint for joint matching is the cycle consistency, i.e., the composition of matches along a loop of meshes should be identity, as illustrated in Figure 1. Given pairwise matches, one can possibly identify true or false matches by checking all cycles in the mesh collection. But there are many difficulties for this approach. For example, the input pairwise matches are often very noisy Figure 1. An illustration of consis- tent multi-mesh matching. With many false matches and missing matches, and the features detected from different meshes may only have a partial overlap even if the same feature detector is applied. Therefore, it is likely that very few consistent cycles can be found. Moreover, how to sample cycles is not straightforward due to the huge num- ber of possibilities. Recent work on joint matching has shown that, if all feature correspondences within multi- ple meshes are denoted by a large binary matrix, the cy- cle consistency can be translated into the fact that such a matrix should be positive semi-definite and low-rank . Based on this observation, convex optimization-based al- gorithms were proposed, which achieved the state-of-the- art performances with theoretical guarantees . But these algorithms rely on semi-definite programming (SDP) [9], which is not computationally efficient to handle image matching problems in practice. In this paper, we make use of MatchALS [16], a novel algorithm for multi image matching, and use it for 3D model meshes. The inputs to our algorithm are orig- inal similarities between feature descriptors which we used Heat Kernel Signature as. The outputs are fea- ture correspondences between all pairs of images. Un- like many previous methods starting from quantized pair- wise matches , we postpone the decision until we opti- mize for both pairwise affinities and multi-mesh consis- tency. Instead of using Semi Definite Programming (SDP) relaxation, we formulate the problem as a low-rank ma- 1
  • 2. Figure 1: Illustration of Consistent Multi-Image Match- ing trix recovery problem and employ the nuclear-norm re- laxation for rank minimization. We show that the posi- tive semi-definiteness of a desired solution can be spon- taneously fulfilled. Moreover, we derive a fast alternat- ing minimization algorithm to globally solve the problem in the low-dimensional variable space . Besides validat- ing our method on both simulated and real benchmark data sets, we also demonstrate the applicability of the pro- posed method combined with Re-weighted Random Walk (RRWM) [7] graph matching to match meshes with dif- ferent models. We introduce a random walk view on the problem and propose a robust graph matching algorithm against outliers and deformation. Matching between two graphs is formulated as node selection on an association graph whose nodes represent candidate correspondences between the two graphs. The solution is obtained by sim- ulating random walks with re-weighting jumps enforcing the matching constraints on the association graph. This algorithm achieves noise-robust graph matching by iter- atively updating and exploiting the confidences of candi- date correspondences. In a practical sense, this RRWM Distortion term is of particular importance since the real- world matching problem is made difficult by the presence of noise and outliers. Extensive and comparative experi- ments demonstrate that it outperforms the state-of-the-art graph matching algorithms especially in the presence of outliers and deformation. 2 Preliminaries and notation Suppose we have n models and pi features from each model i. The objective is to find feature correspondences between all pairs of models. Before introducing the pro- posed method, we first give a brief introduction to pair- wise matching techniques and the definition of cycle con- sistency. 2.1 Pairwise matching To match an image pair (i,j), one can compute similarities for all pairs of feature points from two images and store them in a matrix Sij ∈ Rpi×pj . We represent the feature correspondences for mesh pair (i,j) by a partial permuta- tion matrix Xij ∈ 0, 1pi×pj , which satisfies the doubly stochastic constraints: 0 ≤ Xij1 ≤ 1, 0 ≤ XT ij1 ≤ 1 (1) To find Xij , we can maximize the inner product be- tween Xij and Sij subject to the constraints in (1) result- ing in a linear assignment problem, which has been well studied and can be efficiently solved by the Hungarian al- gorithm. In image matching, spatial rigidity is usually preferred, i.e., the relative location between two features in an mesh should be similar to that between their cor- respondences in the other mesh. This problem is well known as graph matching and formulated as a quadratic assignment problem (QAP). While QAP is NP-hard [2], many efficient algorithms have been proposed to solve it approximately, e.g. Those solvers basically relax the bi- nary constraint on the permutation matrix, solve the opti- mization, and output the confidence of a candidate match being correct. We refer readers to the related literature for details. Here we aim to emphasize that the outputs of graph matching solvers are basically optimized affinity scores of candidate matches, which consider both feature similarity and spatial rigidity. We will use these scores (saved in Sij) as our input in some cases. 2.2 Cycle Consistency For any three models (i,j,z), cycle consistency can be de- scribed as Xij = XizXzj, (2) and can be extended to the case with more than three models.The cycle consistency can be described more con- cisely by introducing a virtual universe that is defined as the set of unique features that appear in the image collec- tion [13]. Each point in the universe may be observed by 2
  • 3. several images and the corresponding image points should be matched. In this way, consistent matching should sat- isfy Xij = AiAT j , where Ai ∈ {0, 1} pi×k denotes the map from Image i to the universe, k is the number of points in the universe, and k ≥ pi for all i. Suppose the correspondences for all m = n i=1 pi features in the image collection is denoted by X ∈ {0, 1} m×m and all Ais are concatenated as rows in a ma- trix A ∈ {0, 1} m×k . Then one can write X as X =      X11 X12 . . . X1n X21 X22 . . . X2n ... ... ... ... Xn1 Xn2 . . . Xnn      , (3) X = AAT (4) From (4), it is clear to see that desired X should be both positive semi-definite and low rank, which effectively im- pose cycle consistency without checking all cycles of pair- wise matches. X 0, rank(X) ≤ k (5) 3 Joint Matching via Rank Mini- mization Given the affinity scores {Sij|1 ≤ i, j ≤ n} for all mod- els, we aim to find globally consistent matches X. Affin- ity scores can be computed from either feature similarities or graph matching solvers according to specific scenarios, For our experiments we use Heat Kernel Signature (HKS) feature descriptor to define the affinity score matrices. 3.1 Cost Function To formulate the cost function, we treat the problem as a low rank matrix recovery problem. The inner product between Xij and Sij is maximized for all i, j as multiple linear assignment problems. Alongside, to enforce cycle consistency, the rank of X is minimized. We make fol- lowing two relaxations on X to optimize the solution: • X is treated as a real matrix X ∈ [0, 1] m×m in- stead of a binary matrix. Experimental results show that the solution values can be stably quantized by a threshold of 0.5. • Rank of X is replaced by nuclear norm ||X||∗ (sum of singular values), which is a tight convex relaxation proven to be very effective in various low-rank prob- lems such as matrix completion [6]. The estimated X should be sparse since at most one value in each row of Xij can be nonzero. To induce spar- sity, we minimize the sum of values in X. Combining all three terms, we obtain the following cost function: f(X) = − n i=1 n j=1 Sij, Xij + α 1, X + λ||X||∗, = − S − α1, X + λ||X||∗, (6) where ., . denotes the inner product, S ∈ Rm×m is a collection of Sijs, α is is the weight of sparsity, λ controls the weight of the nuclear norm. Besides the doubly stochastic constraints in (1), addi- tional constraints shall be imposed on X after relaxation: Xii = Ipi , 1 ≤ i ≤ n, (7) Xij = XT ji, 1 ≤ i, j ≤ n, i = j, (8) 0 ≤ X ≤ 1, (9) Finally, we obtain the following optimization problem: min X W, X + λ||X||∗, s.t.X ∈ C, (10) where W = α1 − S and C denotes a set of matrices satisfying the constraints given in (1), (7), (8), and (9). 4 Fast Alternating Minimization The state-of-the-art methods to solve the convex nuclear norm minimizatiion in (10) are proximal method [14] or ADMM [3] based on iterative single value thresholding [5]. However, singular value decomposition (SVD) needs to be performed in each iteration, which is extremely ex- pensive even for a mediumsized problem. 3
  • 4. Similar low-rank optimization was solved in [16] for 2D images more efficiently via a change of variables X = ABT , where A, B ∈ Rm×k are new variables with a smaller dimension k < m. We use the following lowrank factorization-based for- mulation implemented for 2D images in [16] and extend it to 3D mesh based models: min A,B W, ABT + λ||ABT ||∗, s.t.ABT ∈ C, (11) Now rewriting the nuclear norm minimization problem as SDP [15], ||X||∗ = min A,B:ABT =X 1 2 ||A||2 F + ||B||2 F (12) Finally the following formulation is obtained: min A,B W, ABT + λ 2 ||A||2 F + λ 2 ||B||2 F , s.t.ABT ∈ C, (13) The selection of matrix dimension k is critical to the success of change of variables, while it directly affects the computational complexity. 4.1 Algorithm We extend the following algorithm described in [16] for 2D Images to 3D models. The algorithm applies ADMM to solve (13). The augmented Lagrangian of (13) is as follows: £µ(X, A, B, Y ) = W, X + λ 2 ||A||2 F + λ 2 ||B||2 F + Y X − ABT + µ 2 ||X − ABT ||2 F , (14) where Y is the dual variable and µ is a parameter control- ling the step size in optimization. The ADMM alternately updates each primal variable by minimizing £µ and up- dates the dual variable via gradient ascent while fixing all other variables. Result: Globally consistent matches X randomly initialize A and B, Y = 0 ; W = α1 − S ; while not converged do A ← X + 1 µ Y B BT B + λ µ I τ ; B ← X + 1 µ Y A AT A + λ µ I τ ; X ← Pc ABT − 1 µ (W + Y ) ; Y ← Y k + µ X − ABT ; end quantize X with a threshold equal to 0.5; Algorithm 1: Matching via Alternating Least Squares (MatchALS) 5 RRWM Introduction to problem: Graph matching is an essen- tial problem in computer vision and machine learning. In this paper, we implemented a random walk view on the problem and use a robust graph matching algorithm [7] against outliers and deformation. Matching between two graphs is formulated as node selection on an association graph whose nodes represent candidate correspondences between the two graphs. The solution is obtained by sim- ulating random walks with re-weighting jumps enforcing the matching constraints on the association graph. Our algorithm achieves noise-robust graph matching by iter- atively updating and exploiting the confidences of candi- date correspondences. In a practical sense, our work is of particular importance since the real-world matching prob- lem is made difficult by the presence of noise and outliers. Extensive and comparative experiments demonstrate that it outperforms the state-of-the-art graph matching algo- rithms especially in the presence of outliers and deforma- tion. Recent resurgence of combinatorial optimization ap- proaches to feature matching has changed the situation and firmly settled graph matching formulations based on Integer Quadratic Programming (IQP), which is a gener- alization of the classical graph matching problems. IQP explicitly takes into consideration both unary and pair- wise terms reflecting the compatibilities in local appear- ance as well as the pair-wise geometric relationships be- tween the matching features. Since IQP is known to be 4
  • 5. NP-hard, approximate solutions are required. This pro- vides a novel interpretation of graph matching in a ran- dom walk view and relates it to the IQP formulation. In- troducing an association graph constructed with nodes as candidate correspondences and edges as pair-wise com- patibilities between candidate correspondences, we show that the search for correspondences between the given two graphs can be cast as a node ranking [11] and selec- tion problem in the association graph. For this ranking, we introduce an affinity-preserving random walk and de- rive a ranking based on its quasi-stationary distribution, and prove its equivalence to the spectral relaxation for the IQP formulation. Then, in this random walk view, we adopt the personalization strategy of Web ranking al- gorithms [8] and propose the re-weighted random walk algorithm by re-weighting jumps for the graph matching constraints. It achieves noise-robust graph matching by si- multaneously updating and exploiting the confidences of candidate correspondences. In a practical sense, our work is of particular importance since the real-world matching problem is made difficult by the presence of deformation and outliers. 5.1 Problem Formulation The objective of graph matching is to determine the cor- rect correspondences between two attributed graphs Gp = (Vp, Ep, Ap) and Gq = (Vq, Eq, Aq) where V represents a set of nodes, E, edges, and A, attributes. Each node vP i ∈ V P or edge eP i j ∈ EP has an associated attribute vector aP i ∈ AP or aP i j ∈ AP . In feature correspondence problems, a node attribute aP i usually describes a local ap- pearance of feature i in model P, and an edge attribute aP i j represents the geometric relationship between fea- tures i and j in the image P. For each pair of edges eP i j ∈ EP and eQ a b ∈ EQ , there is an affinity or compatibility Wia; jb = f aP i , aP j , aP i j, aQ a , aQ b , aQ a b that measures the mutual consistency of attributes between the pairs of candidate correspondences vP i , vQ a and vP j , vQ b . Thus, using a matrix form W, a non-diagonal element Wia; jb contains a pair-wise affinity between two corre- spondences vP i , vQ a and vP j , vQ b , and a diagonal term Wia; ia represents a uanry affinity of a correspondence vP i , vQ a . Representing the correspondence with an as- signment or permutation matrix X ∈ {0, 1} nP nQ is com- mon, such that Xia = 1 implies that node vP i corresponds to node vQ a , e.g., feature i in the image P is matched to feature a in the image Q, and Xia = 0 otherwise. In this paper, we denote x ∈ {0, 1} nP nQ as a column-wise vectorized replica of X. The graph matching problem can be formulated as an integer quadratic program (IQP), that is, finding the indicator vector x∗ that maximizes the quadratic score function as follows. x∗ = argmax(xT Wx) (15) s.t.x ∈ {0, 1} nP nQ , ∀i nq a=1 xiq ≤ 1, ∀a np i=1 xia ≤ 1 where the two-way constraints refer to the one-to-one matching from GP to GQ . In general, no efficient al- gorithm exists that can guarantee the optimality bounds since the IQP is NP-hard, thus approximate solutions are required. Figure 2: Association Graphs for Graph Matching by Random Walks 5.2 Algorithnm In the previous affinity-preserving random walks, the matching constraints are ignored and not reflected in the random walk process. Inducing the matching constraints only as a post-processing discretization step like leads to a weak local optimum. How then can we reflect the two-way matching constraints in the affinity-preserving random walk? We adopts the personalization approach widely used in Web ranking methods, which strength- ens the effects of reliable nodes in random walks. This is achieved by adopting a jump or teleport in the ran- dom walk: the random walker moves by traversing an edge with probability alpha(α) or by performing a jump 5
  • 6. to some constrained nodes with probability 1 - α. α rep- resents the bias between the two possible actions, i.e., fol- lowing an edge or jumping. To address the lack of person- alization or users focus, Web ranking algorithms adopted this approach in topic-sensitive or query dependent vari- ants. In our formulation, adopting the personalized jump, the probability distribution is updated using the following equation: x(n+1)T x (n+1) abs = α x(n)T x (n) abs P + (1 − α) rT (16) where a re-weighting jump vector r is added to the affinity-preserving random walk of. In this approach, we use the jumps for generating a biased random walk to the matching constraints. 6 Experimental Results We evaluate the performance of the proposed method us- ing SCAPE benchmark data [1]. We use the blended in- trisic maps described in [10] to evaluate the performance of MatchALS for 3D meshes with RRWM Distortion term added and compare the matching errors for different mod- els. The size of universe is fixed as 100 points and in each mesh a random sample of the points are observed with a probability denoted by ρo. The number of meshes is de- noted by n, which is generic in our code but for this sim- ulation purposes it was kept to 4. Then, the ground-truth pairwise matches are established, and random corruptions are simulated by removing some true matches and adding some false matches to achieve an error rate of ρe. Fi- nally, the corrupted permutation matrix is fed into our Al- gorithm as the input affinity scores. Figure 3 shows the results we obtained before we ap- plied the RRWM distortion term. We observed 1-to-many feature correspondences prior to RRWM application, and this can bee seen by the same color of Figure 3 meshes at the limbs which give false matches sometimes due to the inherent geometrical symmetry of the object. Next we demonstrate the results obtained from run- ning the MatchALS for 3D Meshes in Figure 4 using the RRWM Distortion term, and the results can be clearly seen to weed out the false positive matches in Figure 4. Given the weight matrix W, the reweight factor α, and the inflation factor β ; Prevent conflicting walks by setting Wia; jb = 0for all conflicting match pairs ; Set the maximum degree dmax = maxia jb Wia;jb ; Initialize the transition matrix P = W/dmax, the starting probability x as uniform ; while x not converged do ( Affinity-preserving random walking by edges ) ; xT = xT P ; ( Reweighting with two-way constraints ) ; yT = exp (βx/maxx) ; while y not converged do normalize across rows by yai = yai/ I i=1 yai; normalize across columns by yai = yai/ A q=1 yai; end y = y/ yai ; ( Affinity-preserving random walking with reweighted jumps) ; xT = αxT + (1 − α) yT ; x = x/ xai ; end Discretize x by the matching constraints; Algorithm 2: Reweighted Random Walk Graph Match- ing Preciously colored the same, the limbs are now of dif- ferent colors, which signifies that RRWM allows one-to- many matches that appear due to inherent symmetry of models, to be filtered out in a very precise manner. Figure 5 next shows the 1-to-1 correspondences we have achieved between one pair of meshes we matched and the run time for the simulation was 4.53 seconds Next we demonstrate the matching errors between the 4 meshes we obtained. For each pair of possible match- ing the evaluation was performed by calculating the av- erage Geodesic Error. For the SCAPE Data-set [12] pro- vides the ground truth vertex-to-vertex correspondence. To evaluate performance of our algorithm we map every vertex that has a ground truth correspondence. A geodesic distance between the predicted correspondence and the 6
  • 7. (a) Mesh1 matched to Mesh3 (b) Mesh2 matched to Mesh3 (c) Mesh3 matched to Mesh4 Figure 4: Results of MatchALS combined with RRWM Figure 3: Results of MatchALS performed on 4 Meshes from SCAPE Benchmark Dataset true correspondence is recorded. Then the error per sam- ple is averaged to calculate the Average Geodesic Error of a mapping between a pair of meshes. Table 1 shows the evaluation results of the pairwise matching of the 4 meshes we used to demonstrate our results. Mesh1 Mesh2 Mesh3 Mesh4 Mesh1 0 1.904% 2.637% 13.909% Mesh2 1.904% 0 6.824% 18.363% Mesh3 2.637% 6.824% 0 8.236% Mesh4 13.909% 18.636% 8.236% 0 Table 1: Average Geodesic Error for matching a pair of Meshes using MatchALS and RRWM Figure 5: Illustration of 1 to 1 Mapping of feature corre- spondences in 2 Meshes 7 Conclusion In this paper, we extended the joint matching algorithm MatchALS for 2D images to 3D mesh matching. We use pairwise feature similarities, the heat kernel signature feature descriptor, as inputs and obtain accurate matches by an efficient algorithm (MatchALS) that globally op- timizes for both feature affinities and cycle consistency of matches. The experiments not only validate the effec- tiveness of the proposed method but also demonstrate that joint matching is a promising approach to matching a col- lection of meshes obtained from crowd sourced 3D model collections MatchALS is a powerful algorithm that can be 7
  • 8. extended to 3-D mesh models and the performance gen- erated by the algorithm is almost the same as the state- of-the art algorithm but with exponential orders of incre- ment in speed, as we ourselves observed in our simula- tion. Coupling the MatchALS with Re-weighted Random Walk Matching is something that has not been done be- fore and our novel implementation of stitching the two algorithms has given very fruitful and promise-bearing re- sults applicable to real world 3-D model matching appli- cations .The results reveal that the matching accuracy in the challenging situations largely depends on the effective exploitation of the matching constraints. References [1] D. Anguelov, P. Srinivasan, D. Koller, S. Thrun, J. Rodgers, and J. Davis. Scape: shape completion and animation of people. In ACM Transactions on Graphics (TOG), volume 24, pages 408–416. ACM, 2005. 6 [2] A. C. Berg, T. L. Berg, and J. Malik. Shape matching and object recognition using low distortion corre- spondences. In Computer Vision and Pattern Recog- nition, 2005. CVPR 2005. IEEE Computer Society Conference on, volume 1, pages 26–33. IEEE, 2005. 2 [3] S. Boyd, N. Parikh, E. Chu, B. Peleato, and J. Eck- stein. Distributed optimization and statistical learn- ing via the alternating direction method of multipli- ers. Foundations and Trends in Machine Learning, 3(1):1–122, 2011. 3 [4] C. Bregler, A. Hertzmann, and H. Biermann. Re- covering non-rigid 3d shape from image streams. In Computer Vision and Pattern Recognition, 2000. Proceedings. IEEE Conference on, volume 2, pages 690–696. IEEE, 2000. 1 [5] J.-F. Cai, E. J. Cand`es, and Z. Shen. A singular value thresholding algorithm for matrix completion. SIAM Journal on Optimization, 20(4):1956–1982, 2010. 3 [6] E. J. Cand`es and B. Recht. Exact matrix completion via convex optimization. Foundations of Computa- tional mathematics, 9(6):717–772, 2009. 3 [7] M. Cho, J. Lee, and K. M. Lee. Reweighted ran- dom walks for graph matching. In Computer Vision– ECCV 2010, pages 492–505. Springer, 2010. 2, 4 [8] T. H. Haveliwala. Topic-sensitive pagerank. In Proceedings of the 11th international conference on World Wide Web, pages 517–526. ACM, 2002. 5 [9] Q.-X. Huang and L. Guibas. Consistent shape maps via semidefinite programming. In Computer Graph- ics Forum, volume 32, pages 177–186. Wiley Online Library, 2013. 1 [10] V. G. Kim, Y. Lipman, and T. Funkhouser. Blended Intrinsic Maps. Transactions on Graphics (Proc. of SIGGRAPH), 30(4), 2011. 6 [11] J. M. Kleinberg. Authoritative sources in a hyper- linked environment. Journal of the ACM (JACM), 46(5):604–632, 1999. 5 [12] Y. Lipman and T. Funkhouser. M¨obius voting for surface correspondence. In ACM Transactions on Graphics (TOG), volume 28, page 72. ACM, 2009. 6 [13] D. Pachauri, R. Kondor, and V. Singh. Solving the multi-way matching problem by permutation syn- chronization. In Advances in neural information processing systems, pages 1860–1868, 2013. 3 [14] N. Parikh and S. Boyd. Proximal algorithms. Foun- dations and Trends in optimization, 1(3):123–231, 2013. 3 [15] B. Recht, M. Fazel, and P. A. Parrilo. Guaran- teed minimum-rank solutions of linear matrix equa- tions via nuclear norm minimization. SIAM review, 52(3):471–501, 2010. 4 [16] X. Zhou, M. Zhu, and K. Daniilidis. Multi-image matching via fast alternating minimization. arXiv preprint arXiv:1505.04845, 2015. 1, 4 8