DSP2_slides_04_adaptievefiltering.pdf

Digital Signal Processing 2
Les 4: Adaptieve filtering
Prof. dr. ir. Toon van Waterschoot
Faculteit Industriële Ingenieurswetenschappen
ESAT – Departement Elektrotechniek
KU Leuven, Belgium

Digital Signal Processing 2: Vakinhoud
• Les 1: Eindige woordlengte
• Les 2: Lineaire predictie
• Les 3: Optimale filtering
• Les 4: Adaptieve filtering
• Les 5: Detectieproblemen
• Les 6: Spectrale signaalanalyse
• Les 7: Schattingsproblemen 1
• Les 8: Schattingsproblemen 2
• Les 9: Sigma-Deltamodulatie
• Les 10: Transformatiecodering

• Linear adaptive filtering algorithms
RLS, steepest descent, LMS, …
• Case study: Adaptive notch filters for acoustic
feedback control
acoustic feedback problem, acoustic feedback control, adaptive
notch filters, ANF-LMS algorithm …

• Linear adaptive filtering algorithms
S. V. Vaseghi, Multimedia Signal Processing
－ Ch. 9, “Adaptive Filters: Kalman, RLS, LMS”
• Section 9.3, “Sample Adaptive Filters”
• Section 9.4, “RLS Adaptive Filters”
• Section 9.5, “The Steepest-Descent Method”
• Section 9.6, “LMS Filter”
• Case study: Adaptive notch filters for acoustic
feedback control
Course notes:
T. van Waterschoot, “Adaptive notch filters for acoustic feedback
control”, Course Notes Digital Signal Processing-2, KU Leuven,
Faculty of Engineering Technology, Dept. ESAT, Oct. 2014.

Linear adaptive filtering algorithms
• Adaptive filtering concept
• Recursive Least Squares (RLS) adaptive filters
• Steepest Descent method
• Least Mean Squares (LMS) adaptive filters
• Computational complexity

Adaptive filtering concept (1)
• FIR adaptive filter
－ signal flow graph
of the algorithm to initial parameter states.
Optimisation criteria: In this chapter two optimality criteria are used. One is based on the minim
ation of the mean of squared error (used in LMS, RLS and Kalman filter) and the other is bas
on constrained minimisation of the norm of the incremental change in the filter coefficients wh
esults in normalised LMS (NLMS). In Chapter 12, adaptive filters with non-linear object
unctions are considered for independent component analysis.
Adaptation
algorithm
“Desired” or “target”
signal x(m)
Input y(m)
z–1 . . .
y(m – 1) y(m – 2) y(m – P – 1)
x(m)
ˆ
w0 w1
Transversal filter
e(m)
z–1 z–1
w2
wP – 1
this part is different
from Wiener filter

Adaptive filtering concept (2)
• Adaptive filtering concept
－ adaptive filter = time-varying optimal filter
－ filter coefficients are updated whenever new input/desired
signal sample (or block of samples) is provided
－ general updating scheme:
optimal filter (time t) = optimal filter (time t-1) + adaptation gain * error
• Design choices:
－ FIR/IIR structure
－ filter order
－ cost function
－ adaptation algorithm

Recursive Least Squares (RLS) algorithm (1)
• Online Wiener/LS filter implementation
－ starting point: Wiener filter or least squares estimate
－ how can we implement this filter in online applications?
• at time m, only data {x(0),y(0),…,x(m),y(m)} are available
• optimal filter coefficients w might be time-varying
w = R̂ 1
yy r̂yx = (YT
Y) 1
YT
x
w =
2
6
6
6
4
w0
w1
.
.
.
wP 1
3
7
7
7
5
, Y =
2
6
6
6
4
yT
(0)
yT
(1)
.
.
.
yT
(N 1)
3
7
7
7
5
, y(m) =
2
6
6
6
4
y(m)
y(m 1)
.
.
.
y(m P + 1)
3
7
7
7
5
, x =
2
6
6
6
4
x(0)
x(1)
.
.
.
x(N 1)
3
7
7
7
5

• Recursive time update of correlation matrix/vector
－ consider the LS estimate at time m:
－ the correlation matrix/vector can be computed recursively as
－ if optimal filter w is time-varying, use “forgetting” mechanism:
w(m) = R̂ 1
yy (m)r̂yx(m)
R̂yy(m) =
m
X
n=0
y(n)yT
(n) = R̂yy(m 1) + y(m)yT
(m)
r̂yx(m) =
m
X
n=0
y(n)x(n) = r̂yx(m 1) + y(m)x(m)
r̂yx(m) =
m
X
n=0
m n
y(n)x(n) = r̂yx(m 1) + y(m)x(m)
R̂yy(m) =
m
X
n=0
m n
y(n)yT
(n) = R̂yy(m 1) + y(m)yT
(m)
0 ⌧ < 1

• Matrix inversion lemma (MIL)
－ since autocorrelation matrix needs to be inverted at each
time m, recursive computation of inverse matrix is desired
－ define inverse autocorrelation matrix
－ Matrix inversion lemma:
with gain vector
yy(m) = R̂ 1
yy (m)
k(m) =
1
yy(m 1)y(m)
1 + 1yT (m) yy(m 1)y(m)
R̂yy(m) = R̂yy(m 1) + y(m)yT
(m)
m
yy(m) = 1
yy(m 1) 1
k(m)yT
(m) yy(m 1)

• Recursive time update of filter coefficients
－ plug recursive update for into optimal filter solution:
－ replacing inverse autocorrelation matrix by
results in a recursive time update of filter coefficients
r̂yx(m)
w(m) = yy(m)[ ryx(m 1) + y(m)x(m)]
yy(m) = 1
yy(m 1) 1
k(m)yT
(m) yy(m 1)
w(m) = yy(m 1)ryx(m 1)
| {z }
w(m 1)
+k(m) [x(m) yT
(m)w(m 1)]
| {z }
e(m)

• RLS algorithm
－ input:
－ initialization:
－ recursion (for m = 1,2,…):
• adaptation gain:
• error signal:
• filter coefficient update:
• inverse input autocorrelation matrix update
y(m), x(m)
yy(0) = I, w(0) = 0
k(m) =
1
yy(m 1)y(m)
1 + 1yT (m) yy(m 1)y(m)
e(m) = x(m) wT
(m 1)y(m)
w(m) = w(m 1) + k(m)e(m)
yy(m) = 1
yy(m 1) 1
k(m)yT
(m) yy(m 1)

Steepest Descent method (1)
• Steepest Descent (SD) method
－ RLS algorithm
• recursive implementation of LS optimal filter
• calcuation of RLS adaptation gain is computationally expensive
－ Steepest Descent method
• iterative implementation of Wiener filter
• autocorrelation matrix inversion avoided to reduce complexity
－ idea: step-wise minimization of MSE cost function
314 ADAPTIVE FILTERS: KALMAN, RLS, LMS
w(i – 2)
w(i – 1)
w(i)
woptimal
E[e2(m)]
w
Figure 9.6 Illustration of gradient search of the mean square error surface for the minimum error point.

• Steepest Descent (SD) method
－ idea: step-wise minimization of MSE cost function
－ optimal step = “steepest descent” direction
= negative gradient direction
314 ADAPTIVE FILTERS: KALMAN, RLS, LMS
w(i – 2)
w(i – 1)
w(i)
woptimal
E[e2(m)]
w
Figure 9.6 Illustration of gradient search of the mean square error surface for the minimum error point.
w(m + 1) = w(m) + µ

@E{e2
(m)}
@w(m)
= w(m) + µ [ryx Ryyw(m)]

• SD convergence & step size
－ consider SD filter error vector w.r.t. optimal Wiener filter
－ SD method produces filter estimates resulting in error update
－ SD convergence properties thus depend on
• step size
• input autocorrelation matrix
w̃(m) = w(m) w0
w̃(m + 1) = [I µRyy]w̃(m)
Ryy
µ

• SD convergence & step size
－ consider eigenvalue decomposition of autocorrelation matrix
－ stable adaptation (i.e. ) is guaranteed if
－ convergence rate is inversely proportional to
－ note: eigenvalue spread = measure for magnitude of power
spectrum variations
Ryy = Q⇤QT
⇤ = diag{ max, . . . , min}
w̃(m + 1) < w̃(m)
0 < µ <
2
max
eigenvalue spread =
max
min

Least Mean Squares (LMS) algorithm (1)
• LMS filter
－ Steepest Descent method
• iterative implementation of Wiener filter
• correlation matrix/vector assumed to be known a priori
－ Least Mean Squares (LMS) algorithm
• recursive implementation of Steepest Descent method
• gradient of instantaneous squared error i.o. mean squared error
• surprisingly simple algorithm!
w(m + 1) = w(m) + µ

@e2
(m)
@w(m)
= w(m) + µ [y(m)e(m)]

Least Mean Squares (LMS) algorithm (2)
• LMS variations
－ Leaky LMS algorithm
• leakage factor α < 1 results in improved stability and tracking
－ Normalized LMS (NLMS) algorithm
• step size normalization results in power-independent adaptation
• small regularization parameter δ avoids division by zero
• input power =
w(m + 1) = ↵w(m) + µ [y(m)e(m)]
w(m + 1) = w(m) +
µ
yT (m)y(m) +
[y(m)e(m)]
yT
(m)y(m) = ky(m)k2
2 =
P 1
X
k=0
y2
(m k)

Computational complexity
RLS LMS Leaky LMS NLMS
number of
multiplications O(P2
) 4P + 2
3P + 1 4P + 1

Case study: Adaptive notch filters
• Introduction
－ sound reinforcement
－ acoustic feedback
• Acoustic feedback control
• Adaptive notch filtering
• ANF-LMS algorithm

Introduction (1): Sound reinforcement (1)
Goal: to deliver sufficiently high sound level
and best possible sound quality to audience
• sound sources
• microphones
• mixer & amp
• loudspeakers
• monitors
• room
• audience

• We will restrict ourselves to the single-channel case
(= single loudspeaker, single microphone)

• Assumptions:
– loudspeaker has linear & flat response
– microphone has linear & flat response
– forward path (amp) has linear & flat response
– acoustic feedback path has linear response
• But: acoustic feedback path has non-flat response

• Acoustic feedback path response: example room (36 m3)
impulse response frequency magnitude response
direct
coupling
early
reflections
diffuse
sound field
peaks/dips = anti-nodes/nodes of standing waves
peaks ~10 dB above average, and separated by ~10 Hz

• “Desired” system transfer function:
• Closed-loop system transfer function:
– spectral coloration
– acoustic echoes
– risk of instability
• “Loop response”:
– loop gain
– loop phase
Introduction (5): Acoustic feedback (1)
U(z)
V (z)
= G(z)
U(z)
V (z)
=
G(z)
1 G(z)F(z)
|G(ei!
)F(ei!
)|
G(ei!
)F(ei!
)

• Nyquist stability criterion:
– if there exists a radial frequency ω for which
then the closed-loop system is unstable
– if the unstable system is excited at the critical frequency ω,
then an oscillation at this frequency will occur = howling
• Maximum stable gain (MSG):
– maximum forward path gain before instability
– primarily determined by peaks in frequency magnitude
response of the room
– 2-3 dB gain margin is desirable to avoid ringing
⇢
|G(ei!
)F(ei!
)| 1
G(ei!
)F(ei!
) = n2⇡, n 2 Z
|F(ei!
)|

• Example of closed-loop system instability:
loop gain loudspeaker spectrogram
|G(ei!
)F(ei!
)| U(ei!
, t)

Case study: Adaptive notch filters
• Introduction
• Acoustic feedback control
• Adaptive notch filtering
• ANF-LMS algorithm

Acoustic feedback control (1)
• Goal of acoustic feedback control
= to solve the acoustic feedback problem
– either completely (to remove acoustic coupling)
– or partially (to remove howling from loudspeaker signal)
• Manual acoustic feedback control:
– proper microphone/loudspeaker selection & positioning
– a priori room equalization using 1/3 octave graphic EQ filters
– ad-hoc discrete room modes suppression using notch filters
• Automatic acoustic feedback control:
– no intervention of sound engineer required
– different approaches can be classified into four categories

Acoustic feedback control (2)
1. phase modulation (PM) methods
– smoothing of “loop gain” (= closed-loop magnitude response)
– phase/frequency/delay modulation, frequency shifting
– well suited for reverberation enhancement systems (low gain)
2. spatial filtering methods
– (adaptive) microphone beamforming for reducing direct coupling
3. gain reduction methods
– (frequency-dependent) gain reduction after howling detection
– most popular method for sound reinforcement applications
4. room modeling methods
– adaptive inverse filtering (AIF): adaptive equalization of acoustic
feedback path response
– adaptive feedback cancellation (AFC): adaptive prediction and
subtraction of feedback (≠howling) component in microphone signal

Adaptive notch filtering (1)
• Gain reduction methods
– automation of the actions a sound engineer would undertake
• Classification of gain reduction methods
– automatic gain control (full-band gain reduction)
– automatic equalization (1/3 octave bandstop filters)
– notch filtering (NF) (1/10-1/60 octave filters)
gain margin
G’(z)F(z)
f
Winst in stabiele versterking
door notch filter
Notch−filterkarakteristiek
gain margin
G(z)F(z)
H(z)
f
f
Notch filter characteristic
Increase of stable gain due to notch filter
G F
x(t)
v(t)
y(t)
e(t)
u(t)
Notch Filter

• Adaptive notch filter
– filter that automatically finds & removes narrowband signals
– based on constrained second-order pole-zero filter structure
– constraint 1: poles and zeros lie on same radial lines
−1 −0.5 0 0.5 1
−1
−0.8
−0.6
−0.4
−0.2
0
0.2
0.4
0.6
0.8
1
Real Part
Imaginary
Part
α ri
r
i
Bode Diagram
Frequency (rad/sec)
Phase
(deg)
Magnitude
(dB)
−15
−10
−5
0
5
10
3
10
4
−60
−30
0
30
60
Figure 3: Pole-zero map (left) and Bode plot (right) of an 2nd
order ANF with
eros z = r e±j!i
and poles p = ↵r e±j!i
−1 −0.5
−1
−0.8
−0.6
−0.4
−0.2
0
0.2
0.4
0.6
Imaginary
Part
Figure 3: Pole-zero
zeros zi = rie±j!i
an
3.1 Adaptive
3.1.1 Overview
The ANF filter s
conceived by Rao et
signals buried in bro
−1 −0.5 0 0.5 1
−1
−0.8
−0.6
−0.4
−0.2
0
0.2
0.4
0.6
0.8
1
Real Part
Imaginary
Part
α ri
ri
Phase
(deg)
Magnitude
(dB)
−15
−10
−5
0
10
−60
−30
0
30
60
Figure 3: Pole-zero map (left) and Bode pl
zeros zi = rie±j!i
and poles pi = ↵rie±j!i
3.1 Adaptive Notch Filter (AN
3.1.1 Overview of the literature
The ANF filter structure. The Adap
conceived by Rao et al. [15] as a means for

• ANF transfer function
– cascade of constrained second-order pole-zero filters:
– constraint 2: zeros are forced to lie on unit circle
– “pole radius” ρ = “debiasing parameter” α
ANF transfer function in the z-domain looks like
H(z 1
) =
2n
Y
i=1
(1 ziz 1
)
2n
Y
i=1
(1 ↵ziz 1
)
where 0  ↵ < 1 (3)
=
1 + a1z 1
+ a2z 2
+ ... + a2nz 2n
1 + ↵a1z 1 + ↵2a2z 2 + ... + ↵2na2nz 2n
(4)
=
A(z 1
)
A(↵z 1)
(5)
With this structure a filter of order 2n has 2n unknown parameters and may
suppress at most n narrow-band components. A Bode plot of a 2nd
order ANF
is shown on the right in Figure 3.
Nehorai [17] proposed an adaptive notch filter with half as much parameters
by imposing a second constraint: the zeros zi should lie on the unit circle. A
necessary condition to meet this constraint is that the numerator coefficients
1
2n
Y
i=1
(1 ↵ziz 1
)
=
1 + a1z 1
+ a2z 2
+ ... + a2nz 2n
1 + ↵a1z 1 + ↵2a2z 2 + ... + ↵2na2nz 2n
(4)
=
A(z 1
)
A(↵z 1)
(5)
With this structure a filter of order 2n has 2n unknown parameters and may
suppress at most n narrow-band components. A Bode plot of a 2nd
order ANF
is shown on the right in Figure 3.
Nehorai [17] proposed an adaptive notch filter with half as much parameters
by imposing a second constraint: the zeros zi should lie on the unit circle. A
necessary condition to meet this constraint is that the numerator coefficients
have a mirror symmetric form (i.e. when zi is a zero, 1
zi
will also be a zero). A
2nth
order ANF with n unknown parameters thus has a transfer function
H(z 1
) =
1 + a1z 1
+ ... + anz n
+ ... + a1z 2n+1
+ z 2n
1 + ⇢a1z 1 + ... + ⇢nanz n + ... + ⇢2n 1a1z 2n+1 + ⇢2nz 2n
(6)
=
A(z 1
)
A(⇢z 1)
(7)
where the debiasing parameter ↵ has been replaced by the “pole radius ⇢”.
Estimating the filter coefficients. Including an adaptive notch filter in the

• ANF coefficient estimation
– coefficient vector
– least squares (LS) cost function:
=
A(z )
A(⇢z 1)
(7)
the debiasing parameter ↵ has been replaced by the “pole radius ⇢”.
ating the filter coefficients. Including an adaptive notch filter in the
d path of the 1-microphone/1-loudspeaker setup results in the scheme
ed in Figure 4. An estimate of parameter vector ✓ = [a1 a2 ... an]T
is
ed by minimizing the cost function VN (✓):
ˆ
✓ = arg min
✓
VN (✓) (8)
= arg min
✓
N
X
t=1
e2
(✓, t) (9)
= arg min
✓
N
X
t=1

A(✓, z 1
)
A(✓, ⇢z 1)
y(t)
2
(10)
the ANF order is chosen approximately twice the number of expected
w-band components, minimizing the square of the filter output e(✓, t) with
er stucture as defined in (6) results in a filter with n notches at the desired
ncies. As for the bandwidth of the notches, the pole radius ⇢ plays a
role: the closer ⇢ is to 1, the narrower the notches will be. Choosing ⇢
ter ↵ has been replaced by the “pole radius ⇢”.
fficients. Including an adaptive notch filter in the
ophone/1-loudspeaker setup results in the scheme
estimate of parameter vector ✓ = [a1 a2 ... an]T
is
cost function VN (✓):
arg min
✓
VN (✓) (8)
arg min
✓
N
X
t=1
e2
(✓, t) (9)
arg min
✓
N
X
t=1

A(✓, z 1
)
A(✓, ⇢z 1)
y(t)
2
(10)
osen approximately twice the number of expected
inimizing the square of the filter output e(✓, t) with
n (6) results in a filter with n notches at the desired
dwidth of the notches, the pole radius ⇢ plays a

ANF-LMS algorithm (1)
• 2nd order constrained ANF implementation
– Direct-Form II implementation of second-order ANF
A 2nd
order ANF of the type described above was applied to hearing aids
y Kates [20], only to detect oscillations due to acoustic feedback. Later on,
Maxwell et al. [21] employed Kates’ algorithm to suppress feedback oscillations
or comparison with adaptive feedback cancellation techniques. Their imple-
mentation in Direct Form II follows directly from the ANF tranfer function
6):
x(t) = y(t) + ⇢(t)a(t 1)x(t 1) ⇢2
(t)x(t 2) (12)
e(t) = x(t) a(t 1)x(t 1) + x(t 2) (13)
here y(t) and e(t) represent the ANF input resp. output as before, x(t) is
ntroduced as an auxiliary variable and some signs have been changed. This
nd
order ANF has only one parameter a(t) that appears in both transfer func-
on numerator and denominator. Instead of solving the nonlinear minimization
roblem (10) the filter coefficient update is done in an approximate way as sug-
ested by Travassos-Romano et al. [19]. Only the FIR portion of the filter is
dapted to track the frequency of the narrow-band components and the coef-
cients are then copied to the IIR portion of the filter. The filter coefficient
7
È
È
z 1
z 1
È
È
y(t) x(t)
x(t 1)
x(t 2)
e(t)
⇢(t)a(t 1)
⇢2
(t)
a(t 1)

• ANF filter coefficient update
– adaptation strategy: only FIR portion of filter is adapted,
coefficients are then copied to IIR portion of filter
– LMS filter coefficient update:
y Kates [20], only to detect oscillations due to acoustic feedback. Later on,
Maxwell et al. [21] employed Kates’ algorithm to suppress feedback oscillations
or comparison with adaptive feedback cancellation techniques. Their imple-
mentation in Direct Form II follows directly from the ANF tranfer function
6):
x(t) = y(t) + ⇢(t)a(t 1)x(t 1) ⇢2
(t)x(t 2) (12)
e(t) = x(t) a(t 1)x(t 1) + x(t 2) (13)
here y(t) and e(t) represent the ANF input resp. output as before, x(t) is
ntroduced as an auxiliary variable and some signs have been changed. This
nd
order ANF has only one parameter a(t) that appears in both transfer func-
on numerator and denominator. Instead of solving the nonlinear minimization
roblem (10) the filter coefficient update is done in an approximate way as sug-
ested by Travassos-Romano et al. [19]. Only the FIR portion of the filter is
dapted to track the frequency of the narrow-band components and the coef-
cients are then copied to the IIR portion of the filter. The filter coefficient
7
ate can thus be calculated as follows:
aupd(t) = arg min
a
(e2
(t)) (14)
= arg
✓
d
da
e2
(t) = 0
◆
(15)
= arg
✓
2e(t)
d
da
e(t) = 0
◆
(16)
= arg
⇣
2e(t)( x(t 1)) = 0
⌘
(17)
re the last equality follows from (13). In this way we obtain an LMS filter

• 2nd order ANF-LMS algorithm
= arg 2e(t)( x(t 1)) = 0 (17)
where the last equality follows from (13). In this way we obtain an LMS filter
update which completes the filter implementation given by (12)-(13):
a(t) = a(t 1) + 2µe(t)x(t 1) (18)
The 2nd
order ANF-LMS algorithm is summarized in Algorithm 1.
Algorithm 1 : 2nd
order ANF-LMS algorithm
Input step size µ, initial pole radius ⇢(0), final pole radius ⇢(1), expo-
nential decay time constant , input data {y(t)}N
t=1, initial conditions
x(0), x( 1), a(0)
Output 2nd
order ANF parameter {a(t)}N
t=1
1: for t = 1, . . . , N do
2: ⇢(t) = ⇢(t 1) + (1 )⇢(1)
3: x(t) = y(t) + ⇢(t)a(t 1)x(t 1) ⇢2
(t)x(t 2)
4: e(t) = x(t) a(t 1)x(t 1) + x(t 2)
5: a(t) = a(t 1) + 2µe(t)x(t 1)
6: end for
Higher order ANF’s can be implemented in a similar way. As an example
we give the di↵erence equations describing an 8th
order ANF with LMS update:
x(t) = y(t) + ⇢a1(t 1)x(t 1) ⇢2
a2(t 1)x(t 2) + ⇢3
a3(t 1)x(t 3)

DSP2_slides_04_adaptievefiltering.pdf

More Related Content

Similar to DSP2_slides_04_adaptievefiltering.pdf (20)

Recently uploaded (20)

DSP2_slides_04_adaptievefiltering.pdf