2012.09.25 - Local and non-metric similarities between images - why, how and what for ?

Local and Non-Metric Similarities
Between Images
-
Why, How and What for ?
Frédéric Morain-Nicolier
CReSTIC - URCA - Troyes

2012.09.25

1

Outlines

• Local similarities
• Non metric similarities
• Conclusion :
• Local non-metric similarities ?
• solutions
• open problems

2

Local Similarities

3

Local Similarities,
Why ?

4

3D Acquisition

?

[Troyes Library - Early Rennaissance Collection
- Bibliothèque Bleue]

5

3D Acquisition

[Thanks to Le2I - Le Creusot Team]
6

3D Acquisition : Economical Solution

7

3D Acquisition : An Example

ANALYSIS AND CONSERVATION OF ANCIENT WOODEN STAMPS

ANALYSIS AND CONSERVATION OF ANCIENT WOODEN STAMPS 115

7 Range image (‘Pisces’ stamp) computed from projec-
6 3D View of acquired points of ‘Pisces’ stamp: point tion of point cloud
cloud acquired with Minolta scanner
Else t5Mzk*s (M.m and the neigh-
a stamp, the printing zones are high elevation ones;
7 Range image (‘Pisces’ stamp) computed
bourhood is in a low zone)
6 3D View of acquired points of ‘Pisces’range imagespoint
the high grey-level pixels in the stamp: have to tion of point cloud
End if
be binarized as black (pixel50). The non-printing If pixel.t Then pixel50
cloud acquired with Minolta scanner
zones must therefore be binarized as white (pixel51). Else pixel51

Stamp 3D model Range Image
A modified Niblack’s algorithm is used to binarize End if Else t5Mzk*s (M.m an
range images.11,12 The main task is to adapt the End Do
a stamp, the printing zones are high elevation ones;
threshold over the image. The threshold is deter- bourhood is in a low zone
The local threshold computing can be summarized
the high grey-level pixels from the local meanimageslocal standard
mined in the range and the have to End if
by equation (1)
deviation, computed on a restricted neighbourhood t~max(M,m)zk Ã s (1)
be binarized as blackeach pixel.
for
(pixel50). The non-printing If pixel.t Then pixel50
By modifying the k parameter, it is possible to
zones must therefore be binarized as white (pixel51).
Define: Else pixel51
simulate the inking and printing process for various
m the local mean computed on a [w6w]
8
A modified Niblack’s algorithm is used to binarize
neighbourhood End if
conditions (ink quantity, paper quality or humidity,
11,12 ink fluidity, exerted pressure etc.). Figure 8 shows the

3D Acquisition ➙ Virtual Printing

RVATION OF ANCIENT WOODEN STAMPS 115

local threshold

7 Range image (‘Pisces’ stamp) computed from projec-
point tion of point cloud

Else t5Mzk*s (M.m and the neigh-
ones; bourhood is in a low zone)
ave to End if
inting If pixel.t Then pixel50
el51). Else pixel51
narize End if
pt the End Do
deter- The local threshold computing can be summarized
ndard by equation (1)
rhood t~max(M,m)zk Ã s (1)
By modifying the k parameter, it is possible to
simulate the inking and printing process for various
w6w]
conditions (ink quantity, paper quality or humidity,
ink ﬂuidity, exerted pressure etc.). Figure 8 shows the
mplete
same range image as that used in Fig. 7, binarized for
k50.1–0.8. Note the ‘inking’ variations produced by
d on a
the k value modiﬁcation.

3.3 Comparison between virtual and real stamping
In order to test the proposed method, the results of 9
virtual printing were compared with real ones. For

3D : Virtual vs. Real Fidelity ?

Virtual Resolution ! Real

10

A Local Comparison is needed !

11

A Local Comparison is Needed !

12


small diffs

scattered diffs

big localised diffs 13


14

Local Similarities,
How ?

15

Local Dissimilarity Map (LDM)

Φ( , )

Which measure ?

Which size ?
e 7.4 – Comparaison de deux groupes de lettres. L’image de la CDL indique clai
CDL indique cla
lisations des dissimilarités. La comparaison est également quantifiée.
quantifiée.
LDM

on de deux groupes de lettres. L’image de la CDL indique clairement
n de deux L’image de la CDL indique clairement
milarités. La comparaison est également16quantifiée.
milarités. également quantifiée.

Local Dissimilarity Map
Which measure
between small
images ?
• MSE - PSNR?

➡ pixel to pixel diffs
A
dA,B (p) = |A(p) B(p)|

|A(p) B(p)|
➡ low information
and
hard to interpret B

17

Which measure
between small
images ? DH(A, B) = max (h(A, B), h(B, A))
✓ ◆
• Binary image = set of h(A, B) = max min d(a, b)
pixels (foreground) a2A b2B
h(B, A)
➡ Hausdorff distance DH(A, B)
[Huttenlocher 1993]
DH(A, Tv A) = kvk
➡ numerous variations, A
including partial HD
ieme
B
hK (A, B) = Ka2A d(a, B)

18


Which size for the sliding
window ?

➡ adaptative

➡ must encompass
the «diffs» but no
more
A B
(a) (a) (b) (b) (c
➡ increase until the
Figure 7.2 – Illustration de la notion de dissimilarité locale. Les Les imag
Figure 7.2 – Illustration de la notion de dissimilarité locale. image A
measure equals itsdissimilaires. Les Les fenêtrescentrepetite et AetetABetcar carc), petites.perme
dissimilaires. fenêtres de taille
de taille petitemoyenne (en (en ne permetten
la dissimilarité locale au centre des des images
la dissimilarité locale au images
moyenne
B trop
c), ne
trop petites.
theoretical max ⇒ stopping criterion
Nous pouvons donner uneune idée générale. les pixels situés dans la
Nous pouvons donner idée générale. Si Si les pixels situés dans
partiennent à des des traits grossiers,fenêtre doitdoit avoir une taille su⇥
partiennent à traits grossiers, la la fenêtre avoir une taille su⇥san
écarts résultant de la comparaison de ces ces traits. les traits sont ﬁns,
écarts résultant de la comparaison de traits. Si Si les traits sont ﬁ
fenêtre soitsoit trop grande. Dans le cas contraire, des écarts qui sont plu
fenêtre trop grande. Dans le cas contraire, des écarts qui ne ne sont
19

rmax = max r|DHW (p,r) (A, B) = r . (7.27)
r>0

En pratique ce théorème indique que tant que la distance de Hausdor locale est égale au
rayon de la fenêtre, la mesure optimale n’est pas atteinte.

In the Hausdorff distancedissimilarités locales
7.1.4.4 Déﬁnition de la carte des
case :
La carte des dissimilarités locales est déﬁnie maintenant aisément. Elle regroupe l’ensemble
des mesures de dissimilarités locales réalisés pour di érentes positions. L’algorithme général
lorsque la distance locale est basée sur la distance de Hausdor est le suivant :

Algorithme 7.1 Algorithme itératif de calcul de la carte des dissimilarités locales (CDL)
entre deux images binaires A et B. W (p, n) désigne la fenêtre carrée centrée au pixel p et de
rayon n.
Pour chaque pixel p, faire
1. n ⇤ 1
2. tant que DHW (p,n) (A, B) = n et n ⇥ DH(A, B), faire
n⇤n+1
3. CDLA,B (p) = DHW (p,n 1) (A, B) =n 1

84

20


With the Hausdorff distance : fast computation

CDLA,B (p) = |A(p) B(p)| max (d(p, A), d(p, B))

Distance transform (or function) based
TDA (p) = d(p, A)
(distance to the nearest foreground pixel : very fast with a chamfer distance)

⇒ linear expression (binary images) :

CDLA,B = A.TDB + B.TDA

[Baudrier PhD - Pattern Recognition 2008]
21

Local Dissimilarity Map : Toy Examples

CDLa,b

Figure 7.3 – Comportement de la CDL avec des motifs simp
comparer. d est la CDL entre a et b, e est la CDL entre b et c e
CDL de
le niveau b,c gris est foncé, plus grande est la valeur locale de la
FigureFigure 7.3 – Comportement de la CDL avec des simples ; a,b,c sont lessont les images à
7.3 – Comportement de la CDL avec des motifs motifs simples ; a,b,c images à
comparer. d est la d est la CDL et b, e est b, e est la CDL entreet fet cCDL la CDL entre Plus c. Plus
comparer. CDL entre a entre a et la CDL entre b et c b la et f entre a et c. a et
le niveau de gris est foncé, plus grande est la valeur locale de la mesure.mesure.
le niveau de gris est foncé, plus grande est la valeur locale de la
Cet algorithme est coûteux en temps de calcul car itératif. En
Figure 7.3 – Comportement de la CDL avec des motifs simples ; a,b,c sont le
est en O(m4 ) pour deux images composées deet f⇥ m pixels. No
comparer. d est la CDL entre a et b, e est la CDL entre b et c m la CDL entre
A quantified and localized information
le niveau de gris est distance de grandeEn elaet, la complexité de mesure.
la foncé, plus Hausdor est utilisée dans de mesure locale de dis
locale la la calcul
Cet algorithme est coûteuxcoûteux en temps de calcul car itératif.valeur et, la complexité de calcul
en temps de calcul car itératif. est
7.3 – Comportement de algorithme est des motifs simples ; a,b,c sont les imagese à
Cet la CDL avec très rapide. En
. d est laest en entreen et b, 4 ) est la deux images composées de m ⇥ m pixels.c. Plus avons montré que lorsque
4 ) pour deux images composées de m ⇥ m pixels. Nous avons montré que lorsque
CDL O(m a O(m e pour CDL entre b et c et f la CDL entre a et Nous
est
de gris est distancedistance deest estvaleur locale Théorème mesure locale dedes dissimilaritéscalcul peut être
la foncé,la de grande Hausdor est dansde la mesure.
plus Hausdor la utilisée utilisée dans la locale La dissimilarité, le calcul peut être entre deux
la mesure 7.8. de carte dissimilarité, le locales
22


➡ structural informations
23

3.7. G´n´ralisation aux images en niveaux de gris
e e 85


➡How to save time during holidays ?

24

Local Similarities,
What for ?

25

Ancients Printings

É. Baudrier et al. al. / Pattern Recognition 41 (2008) 1461 – 1478
É. Baudrier et / Pattern Recognition 41 (2008) 1461 – 1478
et 1471
1471
É. É. Baudrier al. al.Pattern Recognition 41 41 (2008) 1461 – 1478
Baudrier et / / Pattern Recognition (2008) 1461 – 1478 14711471
É. Baudrier et al. al. / Pattern Recognition 41 (2008) 1461 – 1478
É. Baudrier et / Pattern Recognition 41 (2008) 1461 – 1478 1471
1471
É. É. Baudrier al. al.Pattern Recognition 41 41 (2008) 1461 – 1478
Baudrier et et / / Pattern Recognition (2008) 1461 – 1478 14711471

20 20 1616
20 20
20 20 1616
16 1416
18 18
20 20
18 1416
16 14
14 14
18 18
1816 14 12
14
16 18
18 16
16
14 12
12
16 16
14 14
16 16
14
1412
12 12
12 10
12
12 10
14 14
14 14 10
10 10
12 12
12 10 10
10 8 8
12 12
10 10
12 12
10810 8
88 8
10 10
10 10 8 668
8 8
8 6
66 6
8 8
6 6
8 8
64 6 6 44
4
6
6 6 26
6 6 44 4
4 4
4 4 224
4 4
2 2
4 4
2 2 22

Ancients Printings
É. Baudrier et al. / Pattern Recognition 41 (2008) 1461 – 1478 1471
É. Baudrier et al. / Pattern Recognition 41 (2008) 1461 – 1478

20 16
20 16
18
20 1614
18
20 14
16
18
16 14
12
18 14
12
14
16
14 12
16 12
1010
12
14
12
14 10
10
12
10
12 88
10
810
10
8 88
66
68
86
É. Baudrier et al. 46 /4Pattern Recognition 41 (2008) 1461 – 1478 66
44
É. Baudrier et al. / Pattern Recognition 41 (2008) 1461 – 1478 6 1471
24
42 44
22
É. Baudrier et al. 0 Pattern Recognition 41 (2008) 1461 – 1478
02/ 22
00
É. Baudrier et al. / Pattern Recognition 41 (2008) 1461 – 1478 2 1471
20 0 16
0
16 00
20
16
20 18
16 18
18 14
20 18 14 1616
18 1614
16 16
18
18 12
18 14 16
12 16
1416
16 1412
14 14
16 12 14
16
16
14 10
14 12 10 14
1214
12
14 12
10 12
14
14
12
12 10
10 10 8 8 12
10
12 10
12 12
10
10 88
88 12
10 6
10 6
8 10
8810
88 666 810
10
6 44
6 88
66
66 4
44
4 68
8
22
4 66
44
44 2 2 46
6
2 02
20 00
2 4
24
2 24
4 2
00 00 0
2 2
02
2
0 0
1818 0 0
0 16
16 0
18
18 Fig. 8. Medieval impressions and their LDMaps. Here are four medieval impressions. Imp. 1, Imp. 2 and Imp. 3 illustrate the same scene with a di
1616 and their LDMaps. Here are four medieval impressions. Imp. 1, Imp. 2 and Imp. 3 illustrate the same scene with a
Fig. 8. Medieval impressions16 16
16
16 kind 8. Medieval impressions Imp. their LDMaps. Here aare four medieval 14
Comparaison d’illustrations 12
14
14
Figure 20 – 14 Imp. their LDMaps. Here aare four 16 anciennes.Imp. 1, Imp. 22 and Imp. 3illustrate the same16 scene with a
Fig. of grass and helmets in and 3. Imp. 4 illustrates distinct scene. impressions. Imp. 1, images a, b3et c représentent la a di
7.9 14 and 3. Imp. 4 illustrates distinct medieval impressions. Les Imp. and Imp. illustrate the same with m scene
kind 8. Medieval impressions
Fig. of grass and helmets in 14in Imp. 3. Imp. 4 illustrates a distinct 16
scene. 20 16
kindFigure 18CDL Imp.(f) CDL d’illustrations anciennes. Les images a, b et cde scènes dissimila
of grass and 7.9 –12 Comparaison ; (g) CDLscene. (h) CDL . La comparaison représentent la m
14
14 kind of grass and helmets
20 12 12 20
18 14
scène. (e)18 10 helmets12 ; 3. Imp. 4 illustrates a distinct14
in
12 scene. ; 10
14 18 14
12
12 a,b
10 a,c a,d 10 16 c,d 12
10
10 scène. (e)methods are (f) CDLa,c ; (g)réparties; sur CDLc,d .the measure result is h).image comparaison
produit 14
16
CDLa,b ; importantes ones12a,d (h) whether La comparaison de scènes dissimila
8
10
8 8 16
comparisondes valeurscompared with the CDLobtained 8 toute l’image (en g et an La (case of the LD
16 10 12 14
14
12
10
comparison methods86are compared with the ones 10obtained 6 whether LSDMap) or aresultvalue (case of(case ofof the L
comparisonThe five classification methods réparties sur toute l’image (en result is h).image comparaison
14
ones obtained6
methods are compared with the are the follo-
8 10 whether the measure realet is an image 10 HD and LD
12 the the measure an (case the its
88
produit 10 valeurs importantes
manually. des 12 and
10 faible measure g
12
scènes methods66 produit with the ones 8obtained 4 and the LSDMap) or a real value ou très8the HD and i
12
manually. The10 five 64classification des valeurs importanteswhetherLSDMap) or a resultvalue(case of 8(case of is itsL
comparison similaires classificationmethods are the follo-4 La the
en the In the first case,(en is an (case of method the
the nombrereal e) image the HD and
the classification localisées
6 are compared methods are the follo- 8 10
6 wing ones: The five
manually. 4 and
ations). a
44 scènes similaires produit methods are the follo-2 2 ations). Inin Sectioncase, In theclassificationmethod is is
manually. The
wing ones:
wing ones: 6
8 five42classification des valeurs importantes enthe LSDMap) or a real valueou très6localisées
8
2 4 6 6 and 88 faible nombre (en e) (case of the HD and 6
described the first case, the classification 4 an empirica
ations).
66 first 6.2. the second case, method a
2 6 20 4
2 wing ones: 44 based on the LDMap,
• our method 02 4 4 0
0 describedforthe first case,In the second case, an empirica
ations). Inin each class C In and C
Section 6.2. the classification method is
described Section 6.2. sim the second is computed from
44
tribution case, an empiri
0 0 2
In and Cdissim 2 0
2 dissim
0 • ourthe so-calledbased on the LDMap,
• our method Local the LDMap,
method22 based on Simple Difference Map (LSDMap) us-
0 22 2
describedfor eachthe modes of the dissim isis computed fr
in each class Csim and C empirical distribution
Section 6.2. sim the second case, an empiri
tribution set. different C
Medieval impressions and their LDMaps. Here are four medieval on the LDMap, Imp. 2 and Imp. 30illustrate the same 0 learning a As class
tribution 0 computed fro
• and are four illustrateLocal SimpleImp. theasimple difference locallythe same0 scene set. eachthe modes of the empiricalcomputed f
ourImp. the medievalthe Simple with 1, Imp.Map (LSDMap) us-
• the the
• so-called
edieval impressions and their LDMaps.2Here ing so-called based impressions. Imp. 1, different (LSDMap) us-
impressions.
method00 Localmap, but Difference 2Map Imp. 316
distance same sceneDifference and
0illustrate
scene with
tribution with As the easy of the dissim 16 distribution
learning for a different Csim and C empirical distributi
class is
16
ure andimpressions. Imp. Imp.Imp.d’illustrations anciennes.(F, G)imagesdifferencelocally learning ladefined, anmodesand efficient classification me
f medieval impressions. Imp. 1, Imp. 2 and Imp.33 illustrate the same scene with a different b 16 c représentent
grass 7.9 – Comparaisonillustrates
4 20
medieval helmets Imp. 3. 3. 1, 4 illustrates a a distinctscene.
in Imp. scene. with
20Local Simple Difference Map a,
quite well
20
Les simple ∩ W − Get W |,us- learning set.mêmeanmodesand efficient classification m
• ing instead ofcthe HD: but with la = |F difference locally 20
ons anciennes. Les images a, bdistance map, H SD W the même (LSDMap)
ass and helmets in
t scene.
scene.
Imp. the ing the distance map, but with the simple
so-called
distinct
the et 18 représentent
18
18 14
14
∩ quite well defined, an easy method. empirical distributi
quite maximum likelihood and the
the
18
18
18 As the easy of efficient 14
is 18 well defined, 14
classification
16
e. (e) CDLa,b ; (f) CDLa,c ;insteadglobal the map,SD W (F, G)La=comparaisonW |, scènes16 well defined, an easymethod.
(g) CDL16 HD: H but with . G) |F difference W de
• instead of ; (h) CDL (F, simple W − G 12 |,
18
16
ingthe of theHD,HD: H SD Wc,dthe = |F ∩∩ W − G 12locally
the distance16
a,d 16
∩ ∩
16 16dissimilaires
quite maximum likelihood and efficient 14
16
is 14 maximum likelihood method.
is the
the
16
16
12
classification
12
La,d ; methods are c,d . La comparaison14HD, H SD W (F, G) =measure result1014 an image (case of the LDMap
(h) CDL compared withinsteadglobal HD: 14de scènes dissimilaires
16
arisondes valeurs importantes theones the sur whether the |F ∩ W g 14 W |, La comparaison likelihood method. 14 14
duitmethods are comparedmeasuretherépartiesimage touteof the LDMap − Get12h). imageis 12 maximum de
son
• of 14
• the global HD,
PHD,14 obtained
12
12 l’image 27 result10 an
(en ∩is 14
14
the
12 of the LDMap
10
12
10
12
rties sur toute l’image (enresult isobtained comparaison de
obtained whether the with the g et 10 La (case
ones h). an whether the measure 12is
8 (case
12
12
6.3.2. Results
10 8

Ancients Printings

• LDM classification

• similar

• dissimilar

➡ SVM

Bonne classification (en %) CDL DPP DH PHD MHD
pour Csim 98 90 60 83 77
pour Cdissim 97 92 75 81 83

Table 7.1 – Performances en classification d’images similaires issues de la base d’impressions
anciennes. CDL = carte des dissimilarités locales, DPP : di érence pixel à pixel, DH : distance
de Hausdor , PHD : distance de Hausdor partielle, MHD : distance de Hausdor modifiée
(voir 7.2.1.4).
[Baudrier PhD]
28

Tumor Evolution

t1 t2 t3
(a) (b) (c)
? ?
Fig. 2. Segmented MRI.
7.12 – Un exemple de la segmentation d’une tumeur. Seule une coupe d’u
représentée, à trois dates di érentes.
values in (f) do not reﬂect et al. 2007] in this case. As
[Nicolier the similarity
a short conclusion, the LDMap is a useful tool for non-
29

(S1) Coupes de la segmentation 1
A. Segmentation Method R ESULTS
III.
using SVM with RBF kernel.
(j)
Tumor Evolution
A. MRI images are acquired on a 1.5T GE (General
Segmentation Method
Electric Co.) machine using an axial1.5T IR (Inversion
on a 3D Slices 18 to 23 (a-f) of the Local Distance Volu
MRI images are acquired Fig. 5. GE (General
Recuperation) machine using an axialan axial FSE (Fast
Electric Co.) T1-weighted sequence, 3D IR (Inversion
Spin Echo) T2-weighted, ansequence, anPD-weighted 3) and 2 (fig 4). The distances are absolute
volumes 1 (fig se-
Recuperation) T1-weighted axial FSE axial FSE (Fast
quence and T2-weighted, an axial FSE examination, we distance histogram (logarithmic scale in g
Spin Echo) an axial FLAIR. eq. one PD-weighted se-
For (3). (j) is the (a) (b) (c)
have 24and an axial FLAIR.signals with a voxel size
quence slices of the four colormap of images (a-f).
For one examination, we
of 0.47 ⇥slices ⇥ 5.5 mm3. signals with a and all size
0.47 of the four All the slices voxel the (a) (b) (c)
have 24
examinations are⇥ 5.5 mm3. All SPM software. all the
of 0.47 ⇥ 0.47 registrated using the slices and
examinations are registrated using SPM software. using
We use the first examination for training SVM
RBF kernelthe first examination for training SVM using
We use [10]. The training set was obtained from one
(a) slice(b) using mouse to(c)
(c) RBF by choose ten pixels into the tumour
kernel [10]. The training set was obtained from one (d) (e) (f)
(a) and (b) outside. We perform theten pixels into the tumour
sliceten using mouse to choose first segmentation of this
by (c)
(c)
volume by usingWe perform the first segmentation of this Fig. 4. Segmentation of the de (e) volume (slices from(f) to 23, a-f)
and (b) outside. the SVM model obtained. So, we build using SVM with RBF kernel. second segmentation 18
(d) Coupes
(S2) la 2
(c) t1 (a) ten (c)hundred points into the tumour
automatically about one model obtained. So, we build Fig. 4. Segmentation of the second volume (slices from 18 to 23, a-f)
volume by using the SVM
his case. As and outside from all one hundred slices. into retraining a using SVM with RBF kernel.
automatically about the tumoral points We the tumour
second SVM fromuse it fortumoral slices. Wesegmentation (a) (b)
ol for non-
his case. As and outside and all the perform a second retraining a
for improve the first it for perform this last SVM model, As a true distance is used to compute the LDV, the
). It is non-
for thus
ol case. As (d) second SVM and use result. With a second segmentation
(e) (f)
his we perform the segmentationWith given scalar are true physical distance in mm. The given
). It is non-
thus (d) for improve the first result. of others examinations. At
Fig. 3. Segmentation of the first volume (slices from (f) to 23, a-f) this last SVM model,
(e) 18 As a true distance is used to compute the LDV, the
ol for using SVM with RBF kernel. eachperform the segmentation of others examinations. At given scalar are true physical there are mm. more low
we segmentation of examination, we use this 2 steps
distances histogram indicates distance inmuch The given
). It is thus (d) Coupesprocess for improve of examination, we use this 2 steps distances than (b) distances. This a coherent fact as non-
Fig. 3. Segmentation of thede (e) segmentation(f) to 23, Fig 2 contains an example
(S1) each la segmentation the 1
first volume (slices from 18 result. a-f) (a) distances histogram indicates there are much more low
high (c)
using SVM with RBF kernel. zero LDV values aredistances. in theaintersection of as non-
Fig. 3. Segmentation of thede la obtained segmentation. All the nine slices of two
the for (slices from result. a-f)
process segmentation18 distances high obtained This coherent fact the two
(S1) Coupesof first volumeimprove the 1 to 23, Fig 2 contains an example volumes. than intersection is locally filled with increasing
segmented volumes are given All the and slices of two zero LDVThe are obtained in the intersection of the two
using SVM with RBF kernel. the obtained segmentation. in figs 3 nine 4.
of values (d) (e)
E (General values, starting from zero locally filledmaximum local
segmented volumesDetails are given in figs 3 and 4. volumes. The intersection is up to the with increasing
E (Inversion B. Implementation distance starting from volumes. to the maximum 7. Us Fig.
(General
l FSE (Fast Fig. 6. a new mageJ is 15.56mm. This represent the higher straight distance
values, between the zero the The maximum tumor hasvo
A three-dimensional view of up LDV between re local
distance
E (Inversion
(General B. The computation DetailsLDV is done with
Implementation of the distance between the volumes. The maximum has progresse distance
leighted se-
FSE (Fast
R (Inversion
2.
plugin computation2GHz opteron, donecomparison mageJ between the two volumes. the higher straight directed dista
The [6]. With a of the LDV is the with a new of the is 15.56mm. This represent
(d) (e) (f) distance
t
weighted we 2
ination, se-
l voxel (Fast
FSE
(a) (b) ⇥ 512 ⇥ 9 volumes is done in 39 seconds.
plugin [6].
(c) The proposed Local Distance Volume can be used to
two 512 With a 2GHz opteron, the comparison of the between the two volumes. and 4). (b) is
(j)
ination, size
we (a)
(VDM) - coupes duprecisely the variations between two volumes. S
track more volume des dissimilarités can be used to
locales entre
weighted these-
and all size two (b) ⇥ (c)
C. Results 512 ⇥ 9 volumes is done in 39 seconds.
512 The proposed Local Distance Volume
voxel we
ination,
Fig. 5. Slices 18 to 23 (a-f) of the
The Hausdorff Distance in a window (eq. (3)) is defined as
track more precisely the variations between two2 volumes. ds
C.The LDV is computed between Figure 1 and – Exemple de volume des dissimilarités (j) is the distance histogram (l
volumes 7.13 2. The the maximum of two directed distance. In(3)) is Les deux
locales.
are.
and all size (a) (b) (c) volumes 1 (fig 3) and (fig 4). The
voxel the
SVM using
Results The Hausdorff Distance in a window (eq. the present case
eq. (3). defined as
are.
and all one results LDVpresented in betweenS2)the z-resolution is L’histogramme indique la useful information.(a-f). (A, B)
are is computed fig 5. As sont 1 and 2. The the maximum of two directed distance.of imagespresent case(e
comparées. the directed distances carry répartition des distances
clearly seen (by high negative distances). colormap hW
d from the
Local Dissimilary Volume
The volumes In the
SVM using
are. tumour slightly greather than the xfigy 5. As the (with a ratio of carry the information on voxels sont expriméesW (A, B)
results are presented in resolutions z-resolution is the directedniveaux de gris, present in vol. 1h ennot in
Les distances, traduites par des distances carry useful information. and mm.
(j)
the
d from one
SVMof this
using (d) 11.7), the obtained distance depends only(with a ratiothe vol. 2. SymmetricallyonW (B, A) carry in vol. 1 and not on
(e) (f) information h voxels Volume the information in
slightly greather than the x y resolutions lightly on of carry the (a-f) of the Local Distance present between
ationtumour
the Fig. 5. Slices 18 to 23
z-axis information. distance dependsobtained distance is voxels present indistances andabsolute accordinginformation B)
Only when the only lightly vol. 2. (fig 4). The vol. hWare not carry the to hW (A, on
2 (B, A) in vol. 1. So
o, from this
d we build
one (d) Coupesgreathersegmentation the z-information has beenon the(fig 3) and 2 Symmetricallytumor has regressed and h (B, A)
Fig. 4. Segmentation of the11.7), the obtained from(f) to 23, a-f)
(S2) de la 2 volumes 1
the of
ationtumour
the tumour using SVM with RBF kernel.
(e) than 11.7mm, 18
second volume (slices
Fig. 4. Segmentation of the de (e) volume (slices froma18 to 23, a-f)
IV. CONCLUSION
z-axis information. Only when the obtained distance (j) is the distance histogram the 2 and notinin vol. 1. the hW (A, B)
taken indicates where (logarithmic scale gray) and So W
eq. (3). is voxels present in vol.
ation build
o, weof this (d) Coupesinto account. Fig 6 is (f) z-information has been taken images (a-f). where the tumor has regressedillustrated(B, fig.
(S2) second segmentation 2
la than 11.7mm, the
greather colormap of
three-dimensional representation where the tumor has progressed. This is and hW in A)
indicates
retraining a
the tumour using SVM with RBF kernel.
o, we build of the LDV. Fig is 18 to 23, a-f)
Fig. 4. Segmentation of theinto account. (slices6from a three-dimensional representation
second volume (a) (b) (c) 97
7 and fig. 8. The has progressed. This central occlusion is
where the tumor augmentation of the is illustrated in fig.
egmentation
retraining a
VM tumour
the model, using SVM with RBF kernel. the LDV.
of
As a true distance is used to compute the LDV, the A distance measure between volumes has
30
7 and fig. 8. The augmentation of the central occlusion is

Binary Pattern Localization

x

y
P I(x,y)
I(x,y)

P

CDLP,I(x,y)

I

31


Local dissimilarities aggregation :

XX
MDGI,P = CDLI,P (k, l)
k l

with CDLI,P = I.TDP + P.TDI

) DI,P = TD2 P +I TD2
I P sum of two
oriented measures
Chamfer score [Borgefors,
1988]: how much
I looks like à P?

32

Binary Pattern Localization : Example

Fig. 2.Chamfer score MDG
In (c), image response by Borgefors chamfer matcher. – In (d), image
obtained with symmetric LDM-matcher. – A good match with the reference
pattern is reported by low values (with dark gray levels).
ce pattern, the ideal location in (a)

33

Binary Pattern Localization : Example

I

[Morain-Nicolier et al. 2009]
34

('%+*
"#$
Brain Internal Structures Segmentation
!
(',-*(','*
!
! !
! !
!

!
/01!! !
!!
"#$!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!"%$!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!"&$!
"#$!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!"%$!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!"&$!
"#$!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!"%$!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!"&$!
"#$! !"%$! !"&$ !
'()*+,!-./!"0$!12!3*405!#+0(5!066,0+05%,!"#$!%7+7508!"%$!09(08!"&$!:0)(;;08!
"#$!
"#$! !"%$!
!"%$! !"&$
!"&$ ! !
'()*+,!-./!"0$!12!3*405!#+0(5!066,0+05%,!"#$!%7+7508!"%$!09(08!"&$!:0)(;;08!
'()*+,!-./!"0$!12!3*405!#+0(5!066,0+05%,!"#$!%7+7508!"%$!09(08!"&$!:0)(;;08!
35


caudate
putamen putamen

thalamus

"#$!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!"%$!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!"&$!
36

!
"#. !
! !
!
!
!
!
! Deformation ?
!
!
!
Atlas %/'! "#$!
Test Volume
%+'

! !
%&'! !"($! "#. !! )*+ %/'! !%+'
37
!"#$! !!!


Optimal transformation :
regularization
T ⇤ = argminT 2 {Esim (B, A T ) + Ereg (T )}

similarity measure
Classic solution :
1 2
Esim (B, A T ) = kB A Tk
2
Displacement field :
(A T B)(p)
u(p) = 2 rB(p)
(A T B)2 (p) + krB(p)k

38

representation of the corresponding structure in the deformed deformed shape map shapeaccording to the following p
representation of the corresponding structure in the unified unified MðpÞ map MðpÞ according to the
8
atlas after the transformation T. Under the constraint of the shape the shape > Fs ðpÞ
atlas after the transformation T. Under the constraint of 8
< i > Fsif Fsi ðpÞ Z0 Fsi ðpÞ Z0
< i
ðpÞ if
similarity term, the optimal transform would leadwould lead to the final maxðF ðpÞÞ maxðF ðpÞ o0if and ðpÞ o0F ðpÞÞj
similarity term, the optimal transform to the final

MðpÞ ¼ MðpÞs¼
i
if Fssi ðpÞÞ
i
Fsi jmaxð and
si
segmented structure shape as closer as as closer asatlas.in the atlas. Therefore >
segmented structure shape that in the that Therefore :0 >
: 0 if F ðpÞ o0if and ðpÞ o0F ðpÞÞj
si Fsi jmaxð and
si
the above overall costoverall cost function can be modified as
the above function can be modified as
where e is the threshold, ethreshold, e ¼ Fsi ðpÞÞg. Ther
E ¼ Esim ðB; A3TÞ¼ Esim ðB; A3TÞ þ Ereg ðB; A3TÞ þ Eshape ðFS ðAÞ; Fshape ðFSþ Ereg ðTÞ
E þ Ereg ðTÞ ¼ Eintensity ðTÞ ¼ Eintensity ðB; A3TÞ þ ESSD
where e is the ¼ mini fmaxp ð mini fmaxp ðF
SSD SSD SSD S ðA3TÞÞ ðAÞ; FS ðA3TÞÞ þ Ereg ðTÞ
in Eqs. (7),in Eqs. (7), (8)should be should be rep
(8) and (10) and (10) replaced by M
ð8Þ ð8Þ
implementation procedure. procedure.
implementation
By extending the originalthe original Demons registration[25],
By extending Demons registration algorithm algorithm [25],

Shape constraint introduction :
an optimal an optimal solution can be obtained by the alternating strategy.
solution can be obtained by the alternating strategy.
The displacement vectors related to the intensity and the shape at
The displacement vectors related to the intensity and the shape at
3.2. Topology correction strategy
3.2. Topology correction strategy

the point p theinterest regions are regions are
of point p of interest
As is mentioned, mentioned, topology preservatio
As is topology preservation of a defor
ðA3TðpÞÀBðpÞÞðA3TðpÞÀBðpÞÞ is important in registration-segmentation method
is important in registration-segmentati
uintensity ðpÞ ¼ À
uintensity ðpÞ ¼ À 2 rBðpÞ rBðpÞ ð9Þ ð9Þ
8 ðA3TðpÞÀBðpÞÞ 22
ðA3TðpÞÀBðpÞÞ þ JrBðpÞJ þ JrBðpÞJ 2 brain case.brain case. bijectivity bijectivity and
Although Although and smoothing
adopted in optimizing the cost function like Eq. (4) l
adopted in optimizing the cost function o
>0
<
ushape ðpÞ ¼ À shape ðpÞ ¼ À
u
ðFS ðA3TðpÞÞÀFSSðA3TðpÞÞÀFS ðAðpÞÞÞ
ð F ðAðpÞÞÞ
2 2 2
p⇥C helpful for preventing topology change, it ischange, it
rFS ðAðpÞÞ 2 rFS ðAðpÞÞ helpful for preventing topology hard to b
ðFS ðA3TðpÞÞÀFSSðA3TðpÞÞÀFrFS ðAðpÞÞJ rFS ðAðpÞÞJ
ðF ðAðpÞÞÞ þJ S ðAðpÞÞÞ þJ theory. The topology preservation problem of t
theory. The topology preservation pro

S (p) = d(p, C) p⇥S
ð10Þ algorithm has been investigated in recent years [3
ð10Þ algorithm has been investigated in rece
cases without topology preservation using this metho
cases without topology preservation usin
>
:
The combined displacement vector is vector is
The combined displacement found in some published reliable experiments [3
found in some published reliable exp
d(p, C)
uðpÞ ¼ ð1ÀbuðpÞ ¼ ð1ÀbÞuintensityðpÞ þ bushape ðpÞ
Þuintensity ðpÞ þ bushape p ⇥ ¬S
ð11Þ experiments. By optimizing optimizing the cost fu
ð11Þ experiments. By the cost function over
diffeomorphism, a diffeomorphic Demons algorith
diffeomorphism, a diffeomorphic Demo
posed [36,37]. In this paper, a different simple topolog
posed [36,37]. In this paper, a different sim
method is proposedis proposed based onfield vector
method based on the vector the analys
Let T ¼ ðX; Y; ZÞ T ¼ ðX; Y; ZÞ deformation field, whe
Let denote the denote the deformatio
the new point p of point p ðx; y; zÞ afte
the new position of position ðx; y; zÞ after deformati

1 β1 β

0.5 0.5

− + − +
x
x0 x 0 x 0 x0 x 0 x 0
Fig. 1. An example of the shape representationrepresentation (a) (b) its shape (b) its shape
Fig. 1. An example of the shape (a) the putamen the putamen
S
distance representationrepresentation map.
distance map.
S Fig. 2. The function parameter b.
Fig. 2. The function of the balance of the balance

39


Shape cost function :

forme 1 2
Esim (A, A T ) = k A A Tk
2
Displacement field :
u(p) = (1 )uintensite (p) + uforme (p)

with
(A T B)(p)
uintensite (p) = 2 rB(p)
(A T B)2 (p) + krB(p)k
( A T A )(p)
uforme (p) =
2 (p) 2r A (p)
( A T A) + kr A (p)k

40

&

!"#$%$&'($!)'*!& !"#$%$&'($!)'*!&

/.&18&6;/&12<=<87>&<86/8:<6?&97:/.&818+2<=<.&7>=12<6;3&:54/2<341:/.&9?&<6:&9158.72?@&%6&A78&
./0123/.&45673/8&97:/.&18&6;/&12<=<87>&<86/8:<6?&97:/.&818+2<=<.&7>=12<6;3&:54/2<341:/.&9?&<6:&9158.72?@&%6&A78&

376A;&9/6B//8&6;/&>/06&76>7:&45673/8&78.&<6:&./0123/.&C/2:<18&<:&816&C/2?&=11.@&D8./2&6;/&
9/&://8&6;76&6;/&:;74/&376A;&9/6B//8&6;/&>/06&76>7:&45673/8&78.&<6:&./0123/.&C/2:<18&<:&816&C/2?&=11.@&D8./2&6;/&

;&6;/&<86/8:<6?&78.&6;/&:;74/F&6;/&2/0/2/8A/&:;74/&78.&6;/&672=/6&:;74/&B<>>&376A;&7:&35A;&7:&
E1<86&A18:627<8:&10&916;&6;/&<86/8:<6?&78.&6;/&:;74/F&6;/&2/0/2/8A/&:;74/&78.&6;/&672=/6&:;74/&B<>>&376A;&7:&35A;&7:&

<8=&6;/&42141:/.&3/6;1.&B<6;&6;/&<86/=276/.&:;74/&3/62<AF&7&9/66/2&:/=3/8676<18&<:&1967<8/.F&
41::<9>/@&!;/2/012/F&5:<8=&6;/&42141:/.&3/6;1.&B<6;&6;/&<86/=276/.&:;74/&3/62<AF&7&9/66/2&:/=3/8676<18&<:&1967<8/.F&

=52/&H@&IJ0K@&
B;<A;&<:&:;1B/.&<8&G<=52/&H@&IJ0K@&

&
Volume segmentation putamen shape&
Figure 8.4 – Exemple de segmentation obtenue avec la con
G<=52/H@& I& $/=3/8676<18& 2/:5>6:L& A13472<:18& 9/6B//8& M/318:& 818+2<=<.& 2/=<:
rence (atlas) où les structures d’intérêt sont surlignées ; (b) re
2/0/2/8A/&<37=/&:54/2<341:/.&9?&6;/&76>7:&10&:59A126<A7>&:625A652/:&J9K&6;/&:;74/&2/
gauche de l’atlas ; (c) image à segmenter ; (d) segmentation
:54/241:/.&9?&6;/&:625A652/N:&9158.72?&JAK&6;/&672=/6&<37=/&J.K&:/=3/8676<18&9?&4
d’intensité ; (e) représentation de forme du putamen déform
6;/&:;74/&2/42/:/8676<18&10&6;/&./0123/.&>/06&45673/8&:54/241:/.&9?&6;/&:625A652
obtenue en incluant la contrainte de forme.
6;/&42141:/.&3/6;1.@&

& & &
& &
*1347276<C/& :65.</:& 9/6B//8& 6;/& 12<=<87>& M/318:& 3/6;1.& 78.& 6;
8& 2/:5>6:L&Figure 8.4 AtlasM/318:& 818+2<=<.& 2/=<:6276<18& 78.& 818+2<=<.& 2/=<:6276<18& 78.& 6;/& 42141:/.& 3/6;1.@& réfé-
ExempleA13472<:18& 9/6B//8& obtenue segmentation M/318:& 6;/& 42141:/.& 3/6;1.@& réfé-
G<=52/H@& de$/=3/8676<18& 2/:5>6:L& A13472<:18& la contrainte de formela (a) image J7K& 6;/&
– Exemple de avec 9/6B//8& obtenue avec : contrainte de forme : (a) image J7K& 6;/&
I& segmentation &
putamen shape
ù les structures d’intérêtles structures d’intérêt sont surlignées ;forme du putamen de forme du putamen
rence (atlas) où sont surlignées ; (b) représentation de (b) représentation segmentation
41:/.&9?&6;/&76>7:&10&:59A126<A7>&:625A652/:&J9K&6;/&:;74/&2/42/:/8676<18&10&6;/&>/06&45673/8&<8&6;/&76>7:& &
2/0/2/8A/&<37=/&:54/2<341:/.&9?&6;/&76>7:&10&:59A126<A7>&:625A652/:&J9K&6;/&:;74/&2/42/:/8676<18&10&6;/&>/06&45673/8&<8&6;/&76>7:&
:/=3/86<8=&927<8&<86/287>&:625A652/:&0213&7>>&C1>53/:@&)/:5>6&18&7&6?4<A7
gauche de l’atlas ; (c) image à segmenter ; (d) segmentation obtenue avec la seule contrainte
las ; (c) image à segmenter ; (d) segmentation obtenue avec la ExempleA13472<:18&
Figure 8.4 – seule contrainte
:54/241:/.&9?&6;/&:625A652/N:&9158.72?&JAK&6;/&672=/6&<37=/&J.K&:/=3/8676<18&9?&452/&<86/8:<6?&97:/.&818+2<=<.&2/=<:6276<18&J/K& la contrainte de forme : (
G<=52/H@& I& $/=3/8676<18& 2/:5>6:L& de segmentation obtenue avec
52/N:&9158.72?&JAK&6;/&672=/6&<37=/&J.K&:/=3/8676<18&9?&452/&<86/8:<6?&97:/.&818+2<=<.&2/=<:6276<18&J/K& 9/6B//8& M/318:& 818+2<=<.& 2/=<:6276<18& 78.& 6;/& 42141:/
) représentation de (e) représentation de forme rencede (d) ; où les structures d’intérêt sont surlignées ; (b) représentation de form
d’intensité ; forme du putamen déformé du putamen déformé issu de (d) ; (f) segmentation
issu (atlas) (f) segmentation
6;2//&4>78/:&:54/241:/.&B<6;&6;/&:/=3/86/.&:59A126<A7>&:625A652/:&78.&6;/&0
2/0/2/8A/&<37=/&:54/2<341:/.&9?&6;/&76>7:&10&:59A126<A7>&:625A652/:&J9K&6;/&:;74/&2/42/:/8676<18&10&6;/&>/06&4
6;/&:;74/&2/42/:/8676<18&10&6;/&./0123/.&>/06&45673/8&:54/241:/.&9?&6;/&:625A652/N:&9158.72?&J0K&6;/&:/=3/8676<18&97:/.&18&
10&6;/&./0123/.&>/06&45673/8&:54/241:/.&9?&6;/&:625A652/N:&9158.72?&J0K&6;/&:/=3/8676<18&97:/.&18&
cluant laobtenue en incluant la contrainte de forme.
contrainte de forme. gauche de l’atlas ; (c) image à segmenter ; (d) segmentation obtenue avec la se
6;/&42141:/.&3/6;1.@& 114
:54/241:/.&9?&6;/&:625A652/N:&9158.72?&JAK&6;/&672=/6&<37=/&J.K&:/=3/8676<18&9?&452/&<86/8:<6?&97:/.&818+2
d’intensité ; (e) représentation de forme du putamen déformé issu de (d) ; (f)
41
& obtenue en incluant la contrainte de forme. +&,-&+&
6;/&:;74/&2/42/:/8676<18&10&6;/&./0123/.&>/06&45673/8&:54/241:/.&9?&6;/&:625A652/N:&9158.72?&J0K&6;/&:/=


Shape constraint contribution (15 segmentations) :

Structure Caudate g. Caudate d. Putamen g. Putamen d. Thalam g. Thalam d.
Méthode avec sans avec sans avec sans avec sans avec sans avec sans
max 0,812 0,729 0,813 0,691 0,831 0,752 0,829 0,759 0,857 0,806 0,850 0,788
min 0,539 0,537 0,567 0,571 0,694 0,633 0,743 0,666 0,735 0,680 0,747 0,665
moy. 0,739 0,699 0,717 0,658 0,767 0,725 0,783 0,739 0,809 0,740 0,801 0,730
écart-type 0,072 0,049 0,062 0,031 0,037 0,029 0,031 0,027 0,031 0,032 0,030 0,032

able 8.2 – Comparaison quantitative des segmentations obtenues avec et sans contrainte de forme (ligne « Méthode ») po
usieurs structures. Les valeurs minimale, maximale et moyenne de l’indice sont données. En gras sont indiquées les meille
sultats. 2 ⇥ VP
=
2 ⇥ VP + FN + FP

[Lin PhD - Pattern Recognition 2010]
42

Local Similarities,
Future Directions

43

Gray-Level Local Dissimilarity

Distance
transforms

CDLA,B (p) = |A B| max (TDA , TDB )

Distance transform generalizations :
1
dGW D (a, b) = (I(a) + I(b)) ⇥ ||a b|| CDL
2 q SSIM
2
dW DOCS (a, b) = (I(a) + I(b)) + ||a b||2

Foreground / background
distinction needed

[Morain-Nicolier et al. 2009]
44


Distance
transforms

CDLA,B (p) = |A B| max (TDA , TDB )
Hypograph [Molchanov, 2003] :
(other solutions are available)
It = {(p, t) ⇤ R2 R : I(p) ⇥ t}
b
X
1
TDI = TDIi i
b a i=a

No foreground / background

[Work in Progress - Morain-Nicolier / Bouchot]
45

2. Outils d’analyse de séquences d’images fonctionnelles

Gray-Level Loc. Dissim. : application
La radiothérapie et le suivi des patients pendant et après traitement sont l’unes des
applications bénéficiant des séquences d’images fonctionnelles. Typiquement, pour un
examen, nous pouvons avoir plusieurs centaines d’images. Les outils d’analyse sont alors
nécessaires pour extraire efficacement l’information pertinente des images sous forme

18FDG PET Images
synthétique et faciliter son stockage.

Volume d’images
t1 t2 t3

y … z

x
t
t4 t5 t6

Figure 6. Quantité énorme de données pour un seul examen

t1 t2 t3 t7 t8 t9

t10 t11
t4 t5 t6

Figure. Séquence d’images dynamiques TEP
46

Gray-Level Loc. Dissim. : application

18FDG PET Images

8000

7000

6000

5000
Tumeur
4000
Aorte
3000

2000

1000

0
1 2 3 4 5 6 7 8 9 10 11

47

Gray-Level Local Dissim. : application

Figure. Résultat de la segmentation

LDM between mean
Figure. Résultat de la carte de la dissimilarité
Segmentation
images

[Ketata PhD - in progress]
48

fenêtre ;
(9.11) : lorsque la fenêtre croît, le mesure locale croît. mesure locale ne repose pas sur les propriétés de
Gray-Leveldéfinir une dépendent de la partir d’une mesure quelconque sous ré
Local Dissimilarity Dans le
Cependant le principe de la
De nombreux critères di érents sont envisageables et mesure locale à mesure choisie.
possible de
cas de la distance de Hausdor , le critère est :propriétés [Baudrier, la fenêtre est maximale et
vérifie certaines si la mesure dans 2005].
que la valeur maximale K n’est pas atteinte, alors la fenêtre est agrandie. Ce critère convient
bien pour une distance max-min. Il peut Carte des dissimilarités locales généralisée moins
Other internal 9.1.2.1
être assoupli pour des mesures qui atteignent
facilement la valeur maximale. Un exemple 9.2.critère possible est le suivant. et B et W (A, B) une mesure de
Théorème de Soient deux images binaires A
measures dans une fenêtre W , qui vérifie les propriétés suivantes :
locale,
Corollaire 9.3. Soit un paramètre ⌥ [0, 1], si la mesure dans la fenêtre est supérieure à
M , où M est la valeur maximale possible dans la fenêtre et que(A, B) = 0 maximale K n’est W
W la valeur ⇧⌃ A W =B
pas atteinte, alors la fenêtre est agrandie. K > 0, W (A, B) K,
W1 ⇤ W2 ⌃ W1 (A, B) W2 (A, B),
Algorithme Pour chaque pixel p, faire
1. n ⌅ k (initialisation de lail est alorsla fenêtre) définir un critère pour fixer la mesure locale des dissimila
taille de possible de
2. tant que W (p,n) (A, B) ⇥ M , faire
Compatible with pixel to
L’interprétation des propriétés est la suivante :
n⌅n+1 (9.9)
pixel diffs (e.g. PSNR)
: si la valeur de la mesure locale est nulle, alors les extraits des deux
3. CDLA,B (p) = W (p,n 1) (A, B) identiques ;
(9.10) : les valeurs de la mesure locale sont majorées par une valeur indépe
fenêtre ;
(9.11) : lorsque la fenêtre croît, le mesure locale croît.
128
De nombreux critères di érents sont envisageables et dépendent de la mesure ch
cas de la distance de Hausdor , le critère est : si la mesure dans la fenêtre est
que la valeur maximale K n’est pas atteinte, alors la fenêtre est agrandie. Ce cri
49


other internal • Segmentations
representations
• Sparse
representation
✓ ◆
DH(A, B) = max max d(p, A), max d(p, B)
p2A p2B • SIFT

Image = set of pixels
• internal distance
Image = set of graphical needed = distance
elements between two
elements (x, y,
small image/

50

Non-Metric Similarities

a work in progress ...

51

Non-Metric Similarities,
Why ?

52


x

y
P I(x,y)
I(x,y)

P

CDLP,I(x,y)

I

53


Local dissimilarites aggregation :

XX
MDGI,P = CDLI,P (k, l)
k l

with CDLI,P = I.TDP + P.TDI

) DI,P = TD2 P +I TD2
I P sum of two
oriented measures
Chamfer score [Borgefors,
1988]: how much
I looks like à P?

54

Non-Metric Similarity

Some classical similarity = distance ?
similarities :
• Minkowski
Identity
• Jeffrey divergence
d(x, y) = 0 , x = y
• χ2
Symmetry
• Levenstein distance d(x, y) = d(y, x)
• Earth Mover Distance
Triangular inequality
• Hausdorff distance
d(x, z)  d(x, y) + d(y, z)
• ...

55

Non-Metric Similarity : Psychological Aspects

• Recognition task :
Identity
d(x, y) = 0 , x = y P(A|A) < P(A|B)
Symmetry ⇒ d(A,A) > d(A,B) !
d(x, y) = d(y, x)
• auto-similarity
d(x, z)  d(x, y) + d(y, z) = prototypality
≈ complexity,
number of
attributes, ...

56


Identity
d(x, y) = 0 , x = y

Symmetry
d(x, y) = d(y, x)

Triangular inequality Prototypality ≈ «Good
d(x, z)  d(x, y) + d(y, z) shape»
[Tversky 1977]
⇒ symmetric, regular,
stable, harmonious,
homogeneous, ...
[Gestalt]
57


Identity
d(x, y) = 0 , x = y

Symmetry
d(x, y) = d(y, x)

d(x, z)  d(x, y) + d(y, z) B. Pitt looks like my brother

(B. Pitt is more prototypical)

My brother looks like B. Pitt

58


Identity
d(x, y) = 0 , x = y

Symmetry
d(x, y) = d(y, x)

Triangular inequality [Veltkamp et Hagedoorn, 2000]
Figure 9.8 – Un exemple de violation de la transitivité de la
d(x, z)  d(x, y) + d(y, z)

9.2.4.3 • Transitity not verified
Non-vériﬁcation de l’inégalité triangulaire
• Suspicions on triangular
Trouver des exemples inequality [Ashbyl’inégalité triangulaire
de violation de et Perrin, 1988]
➡ inégalité à tester, si D est
[Ashby et Perrin, 1988]. Cette 4 inequalities [Tversky] : une mesur
«corner inequality»
verse d’une ressemblance) est
59

How ?

60

Non-Metric Similarity : How ?

Psychology inspired measures

S(A, B) = f (A B) f (A B) ⇥f (B A) [Tversky]
D(A, B) = F [d(A, B) + r(A) + c(B)] [Nosofsky]

A connection with Local Dissimilarities :
A B A.TD(B)

B.TD(A)

CRL(A, B) = AB max(TD¬A , TD¬B ) A.TDB ⇥B.TDA

61

What for ?

62

perform those with a higher weight on the target compound tives. For example, these actives may come from HTS screen-
in terms of the ability to retrieve active analogues. This can ing or serendipity, and we want to quickly identify and test
serve as evidence of the presence of asymmetry in chemical their analogues to confirm the series and hopefully find some

NM Similarity : A Practical Motivation
similarity measures. The observed asymmetry can be interpret- preliminary SARs. Then, highly symmetric similarity measures
ed by the directionality or inequality that is inherent in a should be the choice for this purpose, based on the results of
chemical similarity search: the probe compounds used in prac- this study. Another goal we often seek is to identify remotely
tice are always active, whereas the target compounds can be similar analogues. For example, when we follow the lead struc-
either active or inactive so that their unique structural descrip- tures reported by our competitors, we may wish to identify
tors may not be as important as those of the probe com- some new structures that are similar to these lead structures

Let’s do some chimistry :
pounds regarding their contribution to biological activity. so that they will have better chances of being active as well,
Figures 2 and 3 also clearly indicate that the degree of asym- yet not so similar that they fall under protection of the com-
metry is positively correlated to the degree of similarity itself, petitors’ patents. In this case, we now know highly asymmetric

«similar structure» ≈ «similar» properties
in general. In other words, the red region becomes ever more similarity measures should be adopted.
inclined to the right side with an increasing number of nearest Asymmetry can be easily introduced into many other popu-
neighbors. Therefore, to retrieve more remotely similar active lar similarity measures, such as that recently proposed by
compounds, a more asymmetric similarity measure should be Flinger et al.[15] We expect to see more studies on this concept
used, that is, more weight should be put on the probe com- and more applications of the related techniques in computer-
pound. On the other hand, to retrieve more highly similar assisted drug discovery in the future.
active compounds, approximately equal weights should be put

Table 2. Optimal a values.[a]

MACCS TGD PDR-FP
BEN 0.6 0.5 –
CAT 0.6 0.7 –
HH2 0.8 0.6 –
NNI 0.2 0.1 –
TNF 0.6 0.8 –

[a] a values producing minimal overlap between intraclass and class-NCI
Tversky similarity value distributions are shown as determined by graphi-
cal analysis of Figure 4. PDR-FP calculations are independent of a values
because of its constant bit density. Therefore the overlap is also constant
(see Figure 4 c).
Figure 2. Hit-rate maps for the NCI anti-AIDS database using a) atom pairs, b) Daylight fingerprints, and c) MACCS keys as structural descriptors. Average hit
rate is represented in the color scale shown.

[Wang et al., 2007] ChemMedChem 2007, 2, 180 – 182 [Chen et al., 2007]
2007 Wiley-VCH Verlag GmbH Co. KGaA, Weinheim www.chemmedchem.org 181

tives including moderately active compounds (see Table 3).
These numbers were very similar but not identical to those re-
S(A, B) = f (A B)
ported by Chen and Brown, which we attribute to the use of f (A B) ⇥f (B A)
slightly different (updated) versions of NCI. We then calculated
fingerprint bit densities for active and inactive NCI compounds
(Table 3). For MACCS and TGD, on average five more bits were
set on in active than in inactive compounds, which rationalizes
the preference for a values greater than 0.5 and explains the
slight asymmetry observed in the hit-rate maps of Chen and 63

Local and Non-Metric Similarities

some conclusions

64

Local Similarity

Where is the information ?

• A good solution with Local Dissimilarity Map

• Global - local
or
Local - global ?

• Open problem
• What is the local intrinsic scale of
information in an image ?
65


• Many problems are intrinsically asymmetric !
• Stereovision : point correspondence
(occlusions)
• Database request (CBIR)
• Pattern matching :

66


• Solutions
• Psychological high level models
(simple rule : do not take into account only common
information or missing information, but mix them -
with orientation)
• De-symmetrizing some measures
(e.g. scalar product, pixel to pixel diff)
• Using naturally asymmetric measures
(e.g. conditional probability, ...)

67


• Open problems
• How to classify with such measures ?
(SVM ? KNN ?)
• How to make an efficient search [T. Skopal
works] ?

68

Local Non-Metric Similarities

• High level LDM (pixel - graphical element) ?

• Semantic gap ?
(working on similarity measures and not on
descriptors)

69

Local Non-Metric Similarity

frederic.nicolier@univ-reims.fr

http://pixel-shaker.fr
(papers, discussions and reference
implementation)

70

2012.09.25 - Local and non-metric similarities between images - why, how and what for ?

More Related Content

What's hot (20)

Viewers also liked (20)

Similar to 2012.09.25 - Local and non-metric similarities between images - why, how and what for ? (20)

Recently uploaded (20)

2012.09.25 - Local and non-metric similarities between images - why, how and what for ?