Tuto cvpr part4

Sparse Coding and Dictionary Learning
for Image Analysis
Part IV: Recent Advances in Computer Vision and New Models

Francis Bach, Julien Mairal, Jean Ponce and Guillermo Sapiro

CVPR’10 tutorial, San Francisco, 14th June 2010

Francis Bach, Julien Mairal, Jean Ponce and Guillermo Sapiro Part IV: Recent Advances in Computer Vision 1/45

What this part is about
Learning dictionaries for discriminative tasks. . .
. . . and adapted to image classiﬁcation tasks.
Structured Sparse Models.


Learning dictionaries with a discriminative cost function
Idea:
Let us consider 2 sets S− , S+ of signals representing 2 diﬀerent classes.
Each set should admit a dictionary best adapted to its reconstruction.

Classiﬁcation procedure for a signal x ∈ Rn :
min(R (x, D− ), R (x, D+ ))
where
R (x, D) = minp ||x − Dα||2 s.t. ||α||0 ≤ L.
2
α∈R

“Reconstructive” training
minD− i∈S− R (xi , D− )
minD+ i∈S+ R (xi , D+ )

[Grosse et al., 2007], [Huang and Aviyente, 2006],
[Sprechmann et al., 2010b] for unsupervised clustering (CVPR ’10)

“Discriminative” training
[Mairal, Bach, Ponce, Sapiro, and Zisserman, 2008a]

min C λzi R (xi , D− ) − R (xi , D+ ) ,
D− ,D+
i

where zi ∈ {−1, +1} is the label of xi .

Logistic regression function


Mixed approach
min C λzi R (xi , D− ) − R (xi , D+ ) + µR (xi , Dzi ),
D− ,D+
i

where zi ∈ {−1, +1} is the label of xi .

Keys of the optimization framework
Alternation of sparse coding and dictionary updates.
Continuation path with decreasing values of µ.
OMP to address the NP-hard sparse coding problem. . .
. . . or LARS when using 1.
Use softmax instead of logistic regression for N > 2 classes.


Examples of dictionaries

Top: reconstructive, Bottom: discriminative, Left: Bicycle, Right:
Background.

Texture segmentation


Pixelwise classiﬁcation


Multiscale scheme


weakly-supervised pixel classiﬁcation


Application to edge detection and classiﬁcation
[Mairal, Leordeanu, Bach, Hebert, and Ponce, 2008b]

Good edges Bad edges


Berkeley segmentation benchmark

Raw edge detection on the right



Raw edge detection on the right


Contour-based classiﬁer: [Leordeanu, Hebert, and Sukthankar, 2007]

Is there a bike, a motorbike, a car or a person on this
image?

Performance gain due to the preﬁltering
Ours + [Leordeanu ’07] [Leordeanu ’07] [Winn ’05]
96.8% 89.4% 76.9%
Recognition rates for the same experiment as [Winn et al., 2005] on
VOC 2005.
Category Ours+[Leordeanu ’07] [Leordeanu ’07]
Aeroplane 71.9% 61.9%
Boat 67.1% 56.4%
Cat 82.6% 53.4%
Cow 68.7% 59.2%
Horse 76.0% 67%
Motorbike 80.6% 73.6%
Sheep 72.9% 58.4%
Tvmonitor 87.7% 83.8%
Average 75.9% 64.2 %

Recognition performance at equal error rate for 8 classes on a subset of
images from Pascal 07.

Digital Art Authentiﬁcation
Data Courtesy of Hugues, Graham, and Rockmore [2009]

Authentic Fake



Authentic Fake

Fake



Authentic Fake

Authentic


Image Half-Toning


Learning Codebooks for Image Classiﬁcation

Idea
Replacing Vector Quantization by Learned Dictionaries!
unsupervised: [Yang et al., 2009]
supervised: [Boureau et al., 2010, Yang et al., 2010] (CVPR ’10)


Let an image be represented by a set of low-level descriptors xi at N
locations identiﬁed with their indices i = 1, . . . , N.
hard-quantization:
p
p
xi ≈ Dαi , αi ∈ {0, 1} and αi [j] = 1
j=1

soft-quantization:
2
e −β xi −dj 2
αi [j] = p 2
−β xi −dk
k=1 e
2

sparse coding:
1 2
xi ≈ Dαi , αi = arg min xi − Dα 2 +λ α 1
α 2


Table from Boureau et al. [2010]

Yang et al. [2009] have won the PASCAL VOC’09 challenge using this
kind of techniques.

Summary so far
Learned dictionaries are well adapted to model images.
They can be used to learn dictionaries of SIFT features.
They are also adapted to discriminative tasks.


Sparse Structured Linear Model
We focus again on linear models

x ≈ Dα.
x ∈ Rm , vector of m observations.
D ∈ Rm×p , dictionary or data matrix.
α ∈ Rp , loading vector.
Assumptions:
α is sparse, i.e., it has a small support

|Γ| p, Γ = {j ∈ {1, . . . , p}; αj = 0}.
The support, or nonzero pattern, Γ is structured:
Γ reﬂects spatial/geometrical/temporal. . . information.
e.g., 2-D grid for features associated to the pixels of an image.


Sparsity-Inducing Norms (1/2)
data ﬁtting term

min f (α) + λ ψ(α)
α∈Rp
sparsity-inducing norm

Standard approach to enforce sparsity in learning procedures:
Regularizing by a sparsity-inducing norm ψ.
The eﬀect of ψ is to set some αj ’s to zero, depending on the
regularization parameter λ ≥ 0.

The most popular choice for ψ:
p
The 1 norm, α 1 = j=1 |αj |.
For the square loss, Lasso [Tibshirani, 1996].
However, the 1 norm encodes poor information, just cardinality!

Sparsity-Inducing Norms (2/2)
Another popular choice for ψ:
The 1- 2 norm,
1/2
αG 2 = α2
j , with G a partition of {1, . . . , p}.
G ∈G G ∈G j∈G

The 1 - 2 norm sets to zero groups of non-overlapping variables
(as opposed to single variables for the 1 norm).
For the square loss, group Lasso [Yuan and Lin, 2006].
However, the 1 - 2 norm encodes ﬁxed/static prior information,
requires to know in advance how to group the variables !

Questions:
What happen if the set of groups G is not a partition anymore?
What is the relationship between G and the sparsifying eﬀect of ψ?


Structured Sparsity
[Jenatton et al., 2009]

Case of general overlapping groups.

When penalizing by the 1- 2 norm,
1/2
αG 2 = α2
j
G ∈G G ∈G j∈G

The 1 norm induces sparsity at the group level:
Some αG ’s are set to zero.
Inside the groups, the 2 norm does not promote sparsity.
Intuitively, variables belonging to the same groups are encouraged
to be set to zero together.
Optimization via reweighted least-squares, proximal methods, etc. . .


Examples of set of groups G (1/3)

Selection of contiguous patterns on a sequence, p = 6.

G is the set of blue groups.

Any union of blue groups set to zero leads to the selection of a
contiguous pattern.



Selection of rectangles on a 2-D grids, p = 25.

G is the set of blue/green groups (with their not displayed
complements).

Any union of blue/green groups set to zero leads to the selection of
a rectangle.


Selection of diamond-shaped patterns on a 2-D grids, p = 25.

It is possible to extent such settings to 3-D space, or more complex
topologies.


Overview of other work on structured sparsity

Speciﬁc hierarchical structure [Zhao et al., 2009, Bach, 2008].
Union-closed (as opposed to intersection-closed) family of nonzero
patterns [Baraniuk et al., 2010, Jacob et al., 2009].
Nonconvex penalties based on information-theoretic criteria with
greedy optimization [Huang et al., 2009].
Structure expressed through a Bayesian prior, e.g., [He and Carin,
2009].


Hierarchical Dictionaries
[Jenatton, Mairal, Obozinski, and Bach, 2010]

A node can be active only if its ancestors are active.
The selected patterns are rooted subtrees.

Optimization via eﬃcient proximal methods (same cost as 1)


Hierarchical Dictionaries
[Jenatton, Mairal, Obozinski, and Bach, 2010]


Group Lasso + 1 = Collaborative Hierarchical Lasso
[Sprechmann, Ramirez, Sapiro, and Eldar, 2010a]

Optimization also via proximal methods


Topographic Dictionaries
“Topographic” dictionaries [Hyvarinen and Hoyer, 2001, Kavukcuoglu
et al., 2009] are a speciﬁc case of dictionaries learned with a structured
sparsity regularization for α.

Figure: Image obtained from [Kavukcuoglu et al., 2009]

References I
F. Bach. Exploring large feature spaces with hierarchical multiple kernel learning. In
Advances in Neural Information Processing Systems, 2008.
R. G. Baraniuk, V. Cevher, M. Duarte, and C. Hegde. Model-based compressive
sensing. IEEE Transactions on Information Theory, 2010. to appear.
Y-L. Boureau, F. Bach, Y. Lecun, and J. Ponce. Learning mid-level features for
recognition. In Proceedings of the IEEE Conference on Computer Vision and
Pattern Recognition (CVPR), 2010.
R. Grosse, R. Raina, H. Kwong, and A. Y. Ng. Shift-invariant sparse coding for audio
classification. In Proceedings of the Twenty-third Conference on Uncertainty in
Artificial Intelligence, 2007.
L. He and L. Carin. Exploiting structure in wavelet-based Bayesian compressive
sensing. IEEE Transactions on Signal Processing, 57:3488–3497, 2009.
J. Huang, Z. Zhang, and D. Metaxas. Learning with structured sparsity. In
Proceedings of the International Conference on Machine Learning (ICML), 2009.
K. Huang and S. Aviyente. Sparse representation for signal classification. In Advances
in Neural Information Processing Systems, Vancouver, Canada, December 2006.


References II
J. M. Hugues, D. J. Graham, and D. N. Rockmore. Quantification of artistic style
through sparse coding analysis in the drawings of Pieter Bruegel the Elder.
Proceedings of the National Academy of Science, TODO USA, 107(4):1279–1283,
2009.
A. Hyvarinen and P. Hoyer. A two-layer sparse coding model learns simple and
complex cell receptive fields and topography from natural images. Vision Research,
41(18):2413–2423, 2001.
L. Jacob, G. Obozinski, and J.-P. Vert. Group Lasso with overlap and graph Lasso. In
Proceedings of the International Conference on Machine Learning (ICML), 2009.
R. Jenatton, J-Y. Audibert, and F. Bach. Structured variable selection with
sparsity-inducing norms. Technical report, 2009. preprint arXiv:0904.3523v1.
R. Jenatton, J. Mairal, G. Obozinski, and F. Bach. Proximal methods for sparse
hierarchical dictionary learning. In Proceedings of the International Conference on
Machine Learning (ICML), 2010.
K. Kavukcuoglu, M. Ranzato, R. Fergus, and Y. LeCun. Learning invariant features
through topographic filter maps. In Proceedings of CVPR, 2009.


References III
M. Leordeanu, M. Hebert, and R. Sukthankar. Beyond local appearance: Category
recognition from pairwise interactions of simple features. In Proceedings of the
IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2007.
J. Mairal, F. Bach, J. Ponce, G. Sapiro, and A. Zisserman. Discriminative learned
dictionaries for local image analysis. In Proceedings of the IEEE Conference on
Computer Vision and Pattern Recognition (CVPR), 2008a.
J. Mairal, M. Leordeanu, F. Bach, M. Hebert, and J. Ponce. Discriminative sparse
image models for class-speciﬁc edge detection and image interpretation. In
Proceedings of the European Conference on Computer Vision (ECCV), 2008b.
P. Sprechmann, I. Ramirez, G. Sapiro, and Y. C. Eldar. Collaborative hierarchical
sparse modeling. Technical report, 2010a. Preprint arXiv:1003.0400v1.
P. Sprechmann, I. Ramirez, G. Sapiro, and Y. C. Eldar. Collaborative hierarchical
sparse modeling. Technical report, 2010b. Preprint arXiv:1003.0400v1.
R. Tibshirani. Regression shrinkage and selection via the Lasso. Journal of the Royal
Statistical Society. Series B, 58(1):267–288, 1996.
J. Winn, A. Criminisi, and T. Minka. Object categorization by learned universal visual
dictionary. In Proceedings of the IEEE International Conference on Computer
Vision (ICCV), 2005.


References IV
J. Yang, K. Yu, Y. Gong, and T. Huang. Linear spatial pyramid matching using sparse
coding for image classiﬁcation. In Proceedings of the IEEE Conference on
Computer Vision and Pattern Recognition (CVPR), 2009.
J. Yang, K. Yu, , and T. Huang. Supervised translation-invariant sparse coding. In
Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition
(CVPR), 2010.
M. Yuan and Y. Lin. Model selection and estimation in regression with grouped
variables. Journal of the Royal Statistical Society Series B, 68:49–67, 2006.
P. Zhao, G. Rocha, and B. Yu. The composite absolute penalties family for grouped
and hierarchical variable selection. 37(6A):3468–3497, 2009.


Tuto cvpr part4

More Related Content

Featured (20)

Tuto cvpr part4