IEEE DSP Workshop 2011

QUANTIZATION FOR
CLASSIFICATION ACCURACY IN
HIGH-RATE QUANTIZERS
Behzad M. Dogahe
Manohar N. Murthi
Department of Electrical and Computer
Engineering
IEEE DSP Workshop, January 2011

Outline
• Motivation
• Background
• Problem Statement and Solution
• Simulations
• Concluding Remarks

Motivation
• Quantization of signals is required for many applications
• The original signal is quantized at the encoder and at the
decoder side a replica that should resemble the original signal
in some sense is recovered
• Present quantizers make an effort to reduce the distortion of
the signal in the sense of reproduction fidelity
• Consider scenarios in which signals are generated from
multiple classes. The encoder focuses on the task of
quantization without any regards to the class of the signal
• The quantized signal reaches the decoder where not only the
recovery of the signal should take place but also a decision is
to be made on the class of the signal based on the quantized
version of the signal only

Motivation
• Goal: Design of a quantizer that is optimized for the task of
classification at the decoder
• Application Scenarios:
 Want to have good sound fidelity (good voice/audio
quality) but also want to be able to perform speaker
recognition
 Sensor network where the sensors have low complexity,
simple quantizers, but the decoder/sensor sink node does
more sophisticated processing (so the raw signal value is
needed, but we also want to be able to classify the sensed
signal)

Background
Quantizerx )(ˆ xQx 
x
xˆ
x
)(xp
x
)(x
x
xˆ
x
)(xp
x
)(x
In high-rate theory point density function represents the density of codebook
points in any region for a quantizer. The design of a quantizer is equivalent to design
of the optimal point density function.
)(xp : Probability Density Function

Background
• Design of Quantizer involves minimizing:
where is Distortion Measure
• Examples of Distortion Measure:
 MSE
 Log Spectral Distortion
• High-Rate Theory:
2
ˆ)ˆ,( xxxxd 
Optimization Problem

Background
• Following the steps in [Gardner and Rao] point density function
will be derived as
(n is the dimension of x)
W.R. Gardner and B.D. Rao, “Theoretical analysis of the high-rate vector quantization of lpc
parameters,” Speech and Audio Processing, IEEE Transactions on, vol. 3, no. 5, pp. 367 –381,
sep 1995.

Problem Statement
• We are looking for a point density function that is representative of a
quantizer that performs well in the classification task
• We have to select a distortion measure that is well defined for
classification purposes
• We chose the symmetric Kullback-Leibler divergence measure
between probability of class given the signal before and after
quantization

Problem Statement & Solution
We assume a generative
model for classifier. Hence
and are known
a priori.
Trade-off Distortion Measure:

Simulations
• Signal is from two
classes with known
conditional PDFs
• Dashed lines represent
the decision boundaries
• Point density
function dedicates
codebook points to the
boundaries

Simulations
• only dedicates
codebook points where
the signal is concentrated
• By introducing tradeoff
between MSE and
classification, codebook
points move to the
classification boundaries

Simulations
0
0.05
0.1
0.15
0.2
0.25
0.3
0.35
0.4
0.45
0.5
KL Tradeoff (a = 0.2)Tradeoff (a = 0.8) MSE
10 Bits
8 Bits
6 Bits
Classification
Error (%)
• The higher the bit
rate of quantizer the
better classification
accuracy
• As we move from
MSE to KL, the
classification
accuracy improves

Simulations
-50
-45
-40
-35
-30
-25
-20
-15
-10
-5
0
KL Tradeoff (a = 0.2) Tradeoff (a = 0.8) MSE
10 Bits
8 Bits
6 Bits
Distortion
(dB)
• Pure KL performs
poorly as far as the
distortion of the signal
• However, introducing
the slightest tradeoff
with MSE improves
distortion significantly

Concluding Remarks
• A solution for quantization of signals for the purpose of obtaining
a more accurate classification at the decoder was proposed
• High-rate theory for quantizer design was employed
• An optimal point density function was derived
• The performance of this method on synthetically generated data
was examined and observed to be superior in the task of
classification of signals at the decoder
• The tradeoff between the reproduction fidelity and classification
accuracy was studied as well
• In our future work, we will study the practical vector quantizer
design based on the high-rate theory

IEEE DSP Workshop 2011

More Related Content

What's hot (19)

Similar to IEEE DSP Workshop 2011 (20)

Recently uploaded (20)

IEEE DSP Workshop 2011