Image semantic coding using OTB

Competence Centre on Information Extraction
and Image Understanding for Earth Observation

July 2009
using OTB

Télécom ParisTech
Marie Liénou - Marine Campedel
Image Semantic Coding

1

SUMMARY

COC
Notion of
semantic Coding
A promising
approach

Semantic Coding
OTB tool

Development of
Conclusion and an OTB tool
perspectives

2

COC = COmpetence Centre…

 Tripartite agreement between CNES – DLR and Télécom
ParisTech
 Signed in June 2005

 Goal : joint action on image understanding
 SAR/Optical, HR and VHR, temporal series
 Feature extraction, modeling, indexing, compression,
(interactive) classification, interpretation, knowledge
representation, reasoning, …
 Means
 ~ 4 new phds / year
 ~10 permanent researchers partially involved
 financial support for specific actions (studentships, engineers,
post-docs)

3

Image Semantic coding

Semantic Coding

Meaning Compression
Understanding Reduce data while
Interpretation ensuring informational
Image to text? content

[Barnard et al., 2003 ; Jeon et al., 2003]
[Li et Bretschneider, 2006]

Goal: find an image representation able to
capture the contained semantics
Idea: use text indexing approach + active learning

4

Image Semantic coding
Visual interaction

Manual annotation

Feature
extraction
Where is
Quantization semantics?
Automatic
annotation
« visual words »
Active learning

Indexing Mining
5

Image Semantic coding vs KIM

« Design and evaluation of HMC for Image Information Mining »
Daschiel and Datcu
IEEE transaction on multimedia, vol 7, no6, dec. 2005

6

A promising approach

 Feature extraction
 Segmentation, arbitrar regions
 “Classical” signature: color, texture, shape descriptors
 Experiments: intensity mean and variance in each spectral band

 Quantization
 K-Means: each estimated cluster corresponds to one “visual word”
 K estimated using MDL (Minimum Description Length) descriptor

 Bag-of-words signature for semantics identification
 Count visual words on image regions which will be annotated
 Normalize (tf-idf)

 Exploitation using machine learning (SVM, LDA)
7

A promising approach

 Marie Lienou PhD work (march 2009)
 Tested on several VHR (multispectral) images
 Compared to other classification approachs (GMM, SVM)

Visual word
production
Feature Classification
Quantization Count words
extraction SVM, LDA
Annotations

Feature Classification
Majority rule
extraction GMM, SVM
Low level
annotations

 Recognition accuracy demonstrated for “semantically complex”
classes Ex: “urban area”
 LDA = fast + does not need negative examples
8

OTB tool: cocSemanticCoding

 Feature extraction
 Vectorial image with as many components as feature dimension
 Exploitation of OTB extractors at each pixel
 Quantization
 Use of K-Means filter
 Bag-of-words signature
 Count visual words on image regions which will be annotated
 Normalization (tf-idf)

 Learning from manual annotation
 Fluid interface facilities
 Learn LDA from only target samples
 Learn SVM from target samples and counter examples
 Classify the whole image
 Iterate
9


10


11


12


13


14


15


Learning and classification tools
LDA on occurrence data
SVM on TFiDF data (features)
Both results can be obtained with same labeling for comparison
Difficulty for the user : compute features adapted to the underlying
semantics

16

Conclusion

 OTB useful features
 Vectorial image representation
 Great diversity of available filters (extractors, classifiers)
 New = LDA classifier + estimator
 Visualization tools

 cocSemanticCoding tool availability
 www.tsi.enst.fr/~campedel/
 will be updated

 Necessity to valorize research results
 Engineering process (C++ programming)
 Not easy but OTB is a nice initiative to help researchers
 In the future: centralize processing tools (in OTB) + easy their
exploitation (documentations, interfaces)
17

Perspectives

 Other COC tools should be integrated in
cocSemanticCoding
 MDL to estimate of visual words number
 new feature extractors (QMF-based texture descriptors)
 Feature selection
 Complete relevance feedback framework

 New approaches for image interpretation
 From semantics to knowledge?
 Knowledge engineering: modeling (ontologies) + reasoning
 Several works on characterizing relations between identified
concepts and/or image objects

18

Image semantic coding using OTB

More Related Content

What's hot (20)

Viewers also liked (20)

Similar to Image semantic coding using OTB (20)

More from melaneum (9)

Recently uploaded (20)

Image semantic coding using OTB