ECCV2010: feature learning for image classification, part 1

Part 1: Classical Image Classification Methods Kai Yu Dept. of Media Analytics NEC Laboratories America Andrew Ng Computer Science Dept. Stanford University

Outline of Part 2 05/13/11 Local Features, Sampling, Visual Words Discriminative Methods Bag-of-Words (BoW) representation Spatial pyramid matching (SPM) Generative Methods P art-based methods Topic models

Local features 05/13/11 Distinctive descriptors of local image patches Invariant to local translation, scale, … and sometimes rotation or general affine transformations The most famous choice is the SIFT feature

Sampling local features from images 05/13/11 Image credits: F-F. Li, E. Nowak, J. Sivic A set of points

Visual words 05/13/11 Similar points are grouped into one visual word Algorithms: k-means, agglomerative clustering, … Points from different images are then more easily compared. Slide credit: Kristen Grauman

Outline of Part 2 05/13/11 Local Features, Sampling, Visual Words, … Discriminative Methods Bag-of-Words (BoW) representation Spatial pyramid matching (SPM) Generative Methods P art-based methods Topic models

Bag-of-words (BoW) representation 05/13/11 Adapted from tutorial slides by Fei-Fei et al. Analogy to documents

BoW for object categorization 05/13/11 Works pretty well for whole-image classification Slide credit: Svetlana Lazebnik Csurka et al. (2004), Willamowski et al. (2005), Grauman & Darrell (2005), Sivic et al. (2003, 2005)

Unsupervised Dictionary Learning 05/13/11 image database Sample local features from images Run k-mean or other clustering algorithm to get dictionary Dictionary is also called “codebook” SIFT space R1 R2 R3

Compute BoW histogram for each image 05/13/11 Assign sift features into clusters BoW histogram representations R1 R2 R3 Compute the frequency of each cluster within an image R1 R2 R3

Indication of BoW histogram 05/13/11 Summarize entire image based on its distribution of visual word occurrences Turn bags of different sizes into a fixed length vector Analogous to bag of words representation commonly used for text categorization.

Image classification based on BoW histogram 05/13/11 dog bird Decision boundary BoW histogram vector space Learn a classification model to determine the decision boundary Nonlinear SVMs are commonly applied.

Issues 05/13/11 Sampling strategy Learning codebook: size? supervised?, … Classification: which method? scalability? Scalability: how to handle millions of data? How to use spatial information?

Spatial information 05/13/11 The BoW removes spatial layout. This increases the invariance to scale, translation, and deformation, B ut sacrifices discriminative power, especially when the spatial layout is important . Slide adapted from Bill Freeman

Spatial pyramid matching 05/13/11 Compute BoW for image regions at different locations in various scales Figure credit: Svetlana Lazebnik

A common pipeline for discriminative image classification using BoW 05/13/11 K-means Dense/Sparse SIFT dictionary Dictionary Learning VQ Coding Dense/Sparse SIFT Spatial Pyramid Pooling Nonlinear SVM Image Classification

Combining multiple descriptors 05/13/11 Multiple Feature Detectors Multiple Descriptors: SIFT, shape, color, … VQ Coding and Spatial Pooling Nonlinear SVM Diagram from SurreyUVA_SRKDA, winner team in PASCAL VOC 2008

Topic models for images 05/13/11 Latent Dirichlet Allocation (LDA) Fei-Fei et al. ICCV 2005 Slide credit Fei-Fei Li w N c z D  “ beach”

Part-based Model 05/13/11 Fischler & Elschlager 1973 Rob Fergus ICCV09 Tutorial

For a comprehensive coverage of object categorization models , please visit 05/13/11 Recognizing and Learning Object Categories Li Fei-Fei (Stanford), Rob Fergus (NYU), Antonio Torralba (MIT) http://people.csail.mit.edu/torralba/shortCourseRLOC/

ECCV2010: feature learning for image classification, part 1

More Related Content

More from zukun (20)

Recently uploaded (20)

ECCV2010: feature learning for image classification, part 1