Av 738- Adaptive Filtering - Wiener Filters[wk 3]

Av-738
Adaptive Filter Theory
Lecture 3- Optimum Filtering
[Weiner Filters]
Dr. Bilal A. Siddiqui
Air University (PAC Campus)
Spring 2018

Linear Optimum Filtering
• Consider a linear discrete-time filter
• The filter is linear (to make mathematical analysis easier to handle)
• Filter operates in discrete time (makes implementation easier)
• The requirement is to make the filter error e(n) as small as possible in some statistical sense
• Statistical “sense” means filter output should be optimum (minimizing some criteria). Candidate
criteria are expectation of:
• Absolute value of estimation error
• Square of absolute value (or mean-square value) of estimation error
• Third or higher power of absolute value of estimation error
• Option 1 has a discontinuous first derivative. Option 3 is also mathematically cumbersome
• Option 2 has a unique minimum, and has a second order dependence on filter weights, which
makes optimization possible.
𝐽 = 𝐸 𝑒 𝑛 𝑒∗ 𝑛 = 𝐸 𝑒 𝑛 2
• The linear optimal filtering problem is then formally stated as “design a discrete-time linear filter
whose output y(n) provides an estimate of the desired response d(n) such that mean-square value of
estimation error is minimized.”

Weiner filter – Derivation ,part 1
• The problem is to minimize J by choosing the optimum weights w0
• Since filter coefficients are also (in general complex), 𝑤 𝑘 = 𝑎 𝑘 + 𝑗𝑏 𝑘
• Let’s define the gradient operator (general derivative), 𝛻𝑘 =
𝜕
𝜕𝑎 𝑘
+ 𝑗
𝜕
𝜕𝑏 𝑘
• Multi-dimensional gradient of the (scalar and real) cost function is, therefore 𝛻𝑘 𝐽 =
𝜕𝐽
𝜕𝑎 𝑘
+ 𝑗
𝜕𝐽
𝜕𝑏 𝑘
• For J to attain a minimum, the necessary condition is 𝛻𝑘 𝐽 = 0
𝛻𝑘 𝐽 = 𝛻𝑘 𝐸 𝑒 𝑛 𝑒∗ 𝑛 = 𝐸 𝛻𝑘 𝑒 𝑛 𝑒∗ 𝑛
𝛻𝑘 𝐽 = 𝐸
𝜕𝑒 𝑛
𝜕𝑎 𝑘
𝑒∗
𝑛 +
𝜕𝑒∗
𝑛
𝜕𝑎 𝑘
𝑒 𝑛 + 𝑗
𝜕𝑒 𝑛
𝜕𝑏 𝑘
𝑒∗
𝑛 +
𝜕𝑒∗
𝑛
𝜕𝑏 𝑘
𝑒 𝑛

Weiner filter – derivation 2
𝛻𝑘 𝐽 = 𝐸
𝜕𝑒 𝑛
𝜕𝑎 𝑘
𝑒∗ 𝑛 +
𝜕𝑒∗ 𝑛
𝜕𝑎 𝑘
𝑒 𝑛 +
𝜕𝑒 𝑛
𝜕𝑏 𝑘
𝑗𝑒∗ 𝑛 +
𝜕𝑒∗ 𝑛
𝜕𝑏 𝑘
𝑗𝑒 𝑛
• Since, 𝑒 𝑛 = 𝑑 𝑛 − 𝑘=0
𝑀−1
𝑤 𝑘
∗
𝑢 𝑛 − 𝑘 = 𝑑 𝑛 + 𝑘=0
𝑀−1
−𝑎 𝑘 + 𝑗𝑏 𝑘 𝑢 𝑛 − 𝑘
and d(n) is not a function of wk
𝛻𝑘 𝐽 = −2𝐸 𝑢 𝑛 − 𝑘 𝑒∗
𝑛
• If e0 is the special value of estimation error when filter operates in optimum condition
𝐸 𝑢 𝑛 − 𝑘 𝑒0
∗
𝑛 = 0
• This is an optimum result. It states that, “The necessary and sufficient condition for the cost
function J to attain its minimum, the error value at the operating condition is orthogonal (i.e.
unaffected) by all the samples of input used to calculate that estimate.”
• Since 𝐸 𝑦 𝑛 𝑒∗
𝑛 = 𝐸 𝑘=0
𝑀−1
𝑤 𝑘
∗
𝑢 𝑛 − 𝑘 𝑒∗
𝑛 = 𝑘=0
𝑀−1
𝑤 𝑘
∗
𝐸 𝑢 𝑛 − 𝑘 𝑒∗
𝑛
• This means 𝐸 𝑦0 𝑛 𝑒0
∗
𝑛 = 0, i.e. optimum output is orthogonal to min. error

Weiner Filter - Orthogonality
• For a filter of order 2, this is pictorially depicted below

Weiner-Hopf Equation
• Let’s re-write the orthogonality condition in terms of optimal weights (k=0,1,2,…M-1)
𝐸 𝑢 𝑛 − 𝑘 𝑒0
∗
𝑛 = 𝐸 𝑢 𝑛 − 𝑘 𝑑∗
−
𝑙=0
𝑀−1
𝑤𝑙,0 𝑢∗
𝑛 − 𝑙 = 0
Expanding the equation for
𝑙=0
𝑀−1
𝑤𝑙,0 𝐸 𝑢 𝑛 − 𝑘 𝑢∗
𝑛 − 𝑙 = 𝐸 𝑢 𝑛 − 𝑘 𝑑∗
𝑛
Recognizing autocorrelation 𝑟 𝑙 − 𝑘 = 𝐸 𝑢 𝑛 − 𝑘 𝑢∗ 𝑛 − 𝑙 and cross-correlation 𝑝 −𝑘 = 𝐸 𝑢 𝑛 − 𝑘 𝑑∗ 𝑛
𝑙=0
𝑀−1
𝑤𝑙,0 𝑟 𝑙 − 𝑘 = 𝑝 −𝑘
This can be written in matrix form for 𝑢 𝑛 = [𝑢 𝑛 𝑢 𝑛 − 1 … 𝑢 𝑛 − 𝑀 − 1 ], 𝑹 𝒖 = 𝐸 𝑢 𝑛 𝑢 𝐻
𝑛 ,
𝒑 𝒅𝒖 = 𝑝 0 𝑝 1 … 𝑝 𝑀 − 1 and 𝒘 𝟎 = 𝑤0,0 𝑤1,0 … 𝑤 𝑀−1,0
𝒘 𝟎 = 𝑹 𝒖
−𝟏
𝒑 𝒅𝒖
This is known as the Weiner-Hopf equation

Mean Square Error Surface
• Let the desired filter be of order M (number of tap lines)
𝒚 𝒏 = 𝑘=0
𝑀−1
𝑤 𝑘
∗
𝑢 𝑛 − 𝑘 = 𝒘 𝑯 𝒖(𝒏), 𝒆 𝒏 = 𝒅 𝒏 − 𝒚 𝒏
• The mean square error is
𝐽 𝑤 = 𝐸 𝑒 𝑛 𝑒∗
𝑛 = 𝐸 𝒅 𝒏 − 𝒘 𝑯
𝒖 𝒏 𝒅 𝒏 − 𝒘 𝑯
𝒖 𝒏 𝐻
= 𝐸 𝒅 𝟐
𝒏 − 2𝒘 𝑯
𝐸 𝒅 𝒏 𝒖 𝒏 + 𝒘 𝑯
𝐸 𝒖 𝒏 𝒖 𝑯
𝒏 𝒘
⇒ 𝐽 𝑤 = 𝜎 𝑑
2
− 2𝒘 𝑯
𝒑 𝒅𝒖 + 𝒘 𝑯
𝑹 𝒖 𝒘
Also, for optimal solution, we known that 𝒘 𝟎 = 𝑹 𝒖
−𝟏 𝒑 𝒅𝒖
𝐽0 𝑤0 = 𝜎 𝑑
2
− 2𝒑 𝒅𝒖
𝑯
𝑹 𝒖
−𝟏
𝒑 𝒅𝒖

Example: System Identification
• Find the optimum filter coefficients wo and w1 of the Wiener filter,
which approximates (models) the unknown FIR system with
coefficients bo= 1.5 and b1= 0.5 and contaminated with additive white
uniformly distributed noise of 0.02 variance. Input is white Gaussian
noise of variance 1.
clc;clear;
n =0.5*(rand(1,200)-0.5);%n=noise vector with zero mean and variance 0.02
u=randn(1,200);% u=data vector entering the system
y=filter( [1.5 0.5],1,u); %filter output
d=y + n; % desired output
[ru,lagsru]=xcorr(u,1,'unbiased') ;
Ru=toeplitz(ru(1:2));
[pdu,lagsdu]=xcorr(u,d,1,'unbiased') ;
W_opt=inv(Ru) *pdu(1:2)' ; % optimum Weiner filter weights
sigma2d=xcorr(d,d,0);%autocorrelation of d at zero lag
jmin=mean((d-filter(w_opt,1,u)).^2);

Error Surface
w=[linspace(-3*w_opt(1),3*w_opt(1),50);linspace(-9*w_opt(2),9*w_opt(2),50)];
[w1,w2]=meshgrid(w(1,:),w(2,:));
for k=1:length(w1)
for l=1:length(w2)
J(k,l)=mean((d-filter([w1(k,l) w2(k,l)],1,u)).^2);
end
end
surf(w1,w2,J);
xlabel('W_1');ylabel('W_2');zlabel('Cost Function, J');

Schematic of the previous example

Noise Cancellation with Weiner Filter
• In many practical applications we need to cancel the noise added to a signal.
• E.g., when pilots in planes and helicopters try to communicate, or tank drivers
try to do the same, noise from engine etc. is added to original speech.

The two mic problem
• In this case, we will have two mics. One near pilot’s mouth, the other away from it,
probably both mounted on the helmet.
• We will measure v2 to estimate v1. Assume both are zero mean.
• In this case the Wiener filter is (since desired signal is v2)
𝑅 𝑣2 𝑤𝑜 = 𝑝 𝑣1,𝑣2
• v2 is being measured so Rv2 can be easily calculated. But, v1 is not measured. Clearly v1
and v2 are correlated as they emanate from same source but follow different paths.
𝑝 𝑣1
,𝑣2
𝑘 = 𝐸 𝑣1 𝑛 𝑣2 𝑛 − 𝑘 = 𝐸 𝑥 𝑛 − 𝑑(𝑛) 𝑣2 𝑛 − 𝑘
= 𝐸 𝑥 𝑛 𝑣2 𝑛 − 𝑘 − 𝐸 𝑑(𝑛)𝑣2 𝑛 − 𝑘
• Since d and v2 are not correlated, 𝐸 𝑑(𝑛)𝑣2 𝑛 − 𝑘 = 𝐸 𝑑 𝑛 𝐸(𝑣2(𝑛 − 𝑘)=0
• Therefore, 𝑝 𝑣1
,𝑣2
𝑘 = 𝐸 𝑥 𝑛 𝑣2 𝑛 − 𝑘 =𝑝 𝑥,𝑣2
𝑘 which can be estimated.

Example of Noise Cancellation
Let 𝑑 𝑛 = sin 0.1𝑛𝜋 + 0.2𝜋 , 𝑣1 𝑛 = 0.8𝑣1 𝑛 − 1 + 𝑣(𝑛) and
𝑣1 𝑛 = −0.95𝑣2 𝑛 − 1 + 𝑣(𝑛), where v(n) is white noise with zero
mean value and unit variance.
n=1:500;
d=sin(0.1*n*pi+0.2*pi);
v=randn(1,length(n));
v1=filter(0.8,1,v);
v2=filter(-0.95,1,v);
x=d+v1;
plot(n,d,n,x);
legend('original signal','noisy signal')

M=16;
r_v2=xcorr(v2,M,'unbiased');%autocorrelation
of v
Rv2=toeplitz(r_v2(1:M+1)) ;
p_xv2=xcorr(x,v2,M,'unbiased');
w=inv(Rv2)*p_xv2(1:M+1)' ;
y=filter(w,1,v2) ;
d_est=x-y;
figure;
plot(n,d,n,d_est);
legend('original signal','Weinner filtered')

Homework 1 (b)
• Find the Wiener coefficients wo , w1 and w2 that approximates the
unknown system coefficients which are bo= 0.7 and b1 =0.5. Let the
noise v(n) be white with zero mean and variance 0.15. Further, we
assume that the input data sequence x(n) is stationary white
process with zero mean and variance 1. In addition, v(n) and x(n) are
uncorrelated and v(n) is added to the output of the system under
study. Also find Jmin using the orthogonality principle

Av 738- Adaptive Filtering - Wiener Filters[wk 3]

More Related Content

What's hot (19)

Similar to Av 738- Adaptive Filtering - Wiener Filters[wk 3] (20)

More from Dr. Bilal Siddiqui, C.Eng., MIMechE, FRAeS (19)

Recently uploaded (20)

Av 738- Adaptive Filtering - Wiener Filters[wk 3]