![Fourier theorem. [Fourier, Dirichlet, Riemann] Any (sufficiently smooth)
periodic function can be expressed as the sum of a series of sinusoids.
Fourier analysis
𝑦 𝑡 =
2
𝜋
𝑘=1
𝑁
𝑆𝑖𝑛 𝑘𝑡
𝑘
𝑁 = 100
Sinusoids. Sum of sine and cosines = sum of complex
exponentials.
13 May 2016 By:Muhammad Umer Javed 2](https://image.slidesharecdn.com/fftdftalgorithm-160513125430/85/FFT-and-DFT-algorithm-2-320.jpg)
![Signal. [touch tone button 1]
𝑦 𝑡 =
1
2
sin 2𝜋 ⋅ 697 𝑡 +
1
2
sin(2𝜋 ⋅ 1209 𝑡)
Time domain.
Frequency domain.
Time domain vs. frequency domain
13 May 2016 By:Muhammad Umer Javed 3](https://image.slidesharecdn.com/fftdftalgorithm-160513125430/85/FFT-and-DFT-algorithm-3-320.jpg)
![Polynomial. [coefficient representation]
𝐴 𝑥 = 𝑎0 + 𝑎1 𝑥 + 𝑎2 𝑥2
+ ⋯ + 𝑎 𝑛−1 𝑥 𝑛−1
𝐵 𝑥 = 𝑏0 + 𝑏1 𝑥 + 𝑏2 𝑥2
+ ⋯ + 𝑏 𝑛−1 𝑥 𝑛−1
Add. O(n) arithmetic operations.
𝐴 𝑥 + 𝐵 𝑥 = 𝑎0 + 𝑏0 + 𝑎1 + 𝑏1 𝑥 + ⋯ + 𝑎 𝑛−1 + 𝑏 𝑛−1 𝑥 𝑛−1
Evaluate. O(n) using Horner's method.
𝐴(𝑥) = 𝑎0 + (𝑥 (𝑎1 + 𝑥 (𝑎2 + ⋯ + 𝑥 𝑎 𝑛−2 + 𝑥 𝑎 𝑛−1 ⋯ ))
Multiply (convolve). 𝑂(𝑛2
) using brute force.
𝐴 𝑥 × 𝐵 𝑥 =
𝑖=0
2𝑛−2
𝑐𝑖 𝑥 𝑖
, 𝑤ℎ𝑒𝑟𝑒 𝑐𝑖 =
𝑗=0
𝑖
𝑎𝑗 𝑏𝑖−𝑗
Polynomials: coefficient representation
13 May 2016 By:Muhammad Umer Javed 5](https://image.slidesharecdn.com/fftdftalgorithm-160513125430/85/FFT-and-DFT-algorithm-5-320.jpg)
![Polynomial. [point-value representation]
𝐴 𝑥 = 𝑥0, 𝑦0 , ⋯ , 𝑥 𝑛−1, 𝑦 𝑛−1
𝐵 𝑥 = 𝑥0, 𝑧0 , ⋯ , (𝑥 𝑛−1, 𝑧 𝑛−1)
Add. O(n) arithmetic operations.
𝐴 𝑥 + 𝐵 𝑥 = 𝑥0, 𝑦0 + 𝑧0 , ⋯ , (𝑥 𝑛−1, 𝑦 𝑛−1 + 𝑧 𝑛−1)
Multiply (convolve). O(n), but need 2n – 1 points.
𝐴 𝑥 × 𝐵 𝑥 = 𝑥0, 𝑦0 × 𝑧0 , ⋯ , (𝑥2𝑛−1, 𝑦2𝑛−1 × 𝑧2𝑛−1)
Evaluate. O(n2) using Lagrange's formula.
𝐴 𝑥 =
𝑘=0
𝑛−1
𝑦 𝑘
𝑗≠𝑘 𝑥 − 𝑥𝑗
𝑗≠𝑘 𝑥 𝑘 − 𝑥𝑗
Polynomials: point value representation
13 May 2016 By:Muhammad Umer Javed 7](https://image.slidesharecdn.com/fftdftalgorithm-160513125430/85/FFT-and-DFT-algorithm-7-320.jpg)
![𝑋 𝑘 =
𝑛=0
𝑁−1
𝑥[𝑛]𝑒−
𝑖2𝜋𝑘𝑛
𝑛 ; 𝑘 = 0, ⋯ , 𝑁 − 1
For each k
𝑁 𝑐𝑜𝑚𝑝𝑒𝑙𝑒𝑡𝑒 𝑚𝑢𝑙𝑡𝑠
𝑁 − 1 𝑐𝑜𝑚𝑝𝑙𝑒𝑥 𝑎𝑑𝑑𝑠
• 𝑂(𝑁2) for DFT
• 𝑂(𝑁𝑙𝑜𝑔𝑁) for FFT …….
How! lets see
As 𝜔 𝑁 = 𝑒−
𝑖2𝜋
𝑁 where 𝜔 𝑁
𝑘𝑁
= 𝑒−𝑖2𝜋𝑘 = 1
Fast Fourier Transform
13 May 2016 By:Muhammad Umer Javed 15](https://image.slidesharecdn.com/fftdftalgorithm-160513125430/85/FFT-and-DFT-algorithm-15-320.jpg)
![FFT
For indices
𝑒𝑣𝑒𝑛 𝑛 = 2𝑟
𝑜𝑑𝑑 𝑛 = 2𝑟 + 1
where
𝑟 = 0, 1, ⋯ ,
𝑁
2
− 1
𝑋 𝑘 =
𝑟=0
𝑁
2
−1
𝑥 2𝑟 𝑊𝑁
2𝑟𝑘
+
𝑟=0
𝑁
2
−1
𝑥 2𝑟 + 1 𝑊𝑁
(2𝑟+1)𝑘
𝑋 𝑘
=
𝑟=0
𝑁
2
−1
𝑥 2𝑟 𝑊𝑁
2 𝑘𝑟
+ 𝑊𝑁
𝑘
𝑟=0
𝑁
2
−1
𝑥 2𝑟 + 1 𝑊𝑁
2 𝑘𝑟
𝑋 𝑘 =
𝑟=0
𝑁
2
−1
𝑥 2𝑟 𝑊𝑁
2
𝑘𝑟
+ 𝑊𝑁
𝑘
𝑟=0
𝑁
2
−1
𝑥 2𝑟 + 1 𝑊𝑁
2
𝑘𝑟
So we get
𝑋 𝑘 = 𝑋 𝑒 𝑘 + 𝑊𝑁
𝑘
𝑋 𝑜[𝑘]
also
𝑋[𝑘 + 𝑁/2] = 𝑋 𝑒[𝑘] − 𝑊𝑁
𝑘
𝑋 𝑜[𝑘],
𝑵
𝟐
DFT of even
samples 𝑿 𝒆[𝒌]
𝑵
𝟐
DFT of odd
samples 𝑿 𝒐[𝒌]
But 𝑾 𝒏
𝟐
= 𝒆−
𝒊𝟐𝝅
𝑵
𝟐
= 𝒆
−
𝒊𝟐𝝅
𝑵
𝟐 = 𝐖𝐍
𝟐
13 May 2016 By:Muhammad Umer Javed 19](https://image.slidesharecdn.com/fftdftalgorithm-160513125430/85/FFT-and-DFT-algorithm-19-320.jpg)
FFT and DFT algorithm
- 1. FFT & DFT A Divide and Conquer Algorithm
- 2. Fourier theorem. [Fourier, Dirichlet, Riemann] Any (sufficiently smooth) periodic function can be expressed as the sum of a series of sinusoids. Fourier analysis 𝑦 𝑡 = 2 𝜋 𝑘=1 𝑁 𝑆𝑖𝑛 𝑘𝑡 𝑘 𝑁 = 100 Sinusoids. Sum of sine and cosines = sum of complex exponentials. 13 May 2016 By:Muhammad Umer Javed 2
- 3. Signal. [touch tone button 1] 𝑦 𝑡 = 1 2 sin 2𝜋 ⋅ 697 𝑡 + 1 2 sin(2𝜋 ⋅ 1209 𝑡) Time domain. Frequency domain. Time domain vs. frequency domain 13 May 2016 By:Muhammad Umer Javed 3
- 4. FFT. Fast way to convert between time-domain and frequency-domain. Alternate viewpoint. Fast way to multiply and evaluate polynomials. There are two ways to represent polynomials • Coefficient representation • Point value representation Fast Fourier transform We take this approach 13 May 2016 By:Muhammad Umer Javed 4
- 5. Polynomial. [coefficient representation] 𝐴 𝑥 = 𝑎0 + 𝑎1 𝑥 + 𝑎2 𝑥2 + ⋯ + 𝑎 𝑛−1 𝑥 𝑛−1 𝐵 𝑥 = 𝑏0 + 𝑏1 𝑥 + 𝑏2 𝑥2 + ⋯ + 𝑏 𝑛−1 𝑥 𝑛−1 Add. O(n) arithmetic operations. 𝐴 𝑥 + 𝐵 𝑥 = 𝑎0 + 𝑏0 + 𝑎1 + 𝑏1 𝑥 + ⋯ + 𝑎 𝑛−1 + 𝑏 𝑛−1 𝑥 𝑛−1 Evaluate. O(n) using Horner's method. 𝐴(𝑥) = 𝑎0 + (𝑥 (𝑎1 + 𝑥 (𝑎2 + ⋯ + 𝑥 𝑎 𝑛−2 + 𝑥 𝑎 𝑛−1 ⋯ )) Multiply (convolve). 𝑂(𝑛2 ) using brute force. 𝐴 𝑥 × 𝐵 𝑥 = 𝑖=0 2𝑛−2 𝑐𝑖 𝑥 𝑖 , 𝑤ℎ𝑒𝑟𝑒 𝑐𝑖 = 𝑗=0 𝑖 𝑎𝑗 𝑏𝑖−𝑗 Polynomials: coefficient representation 13 May 2016 By:Muhammad Umer Javed 5
- 6. Fundamental theorem of algebra. A degree n polynomial with complex coefficients has exactly n complex roots. Corollary. A degree n – 1 polynomial A(x) is uniquely specified by its evaluation at n distinct values of x Polynomials: point-value representation 13 May 2016 By:Muhammad Umer Javed 6
- 7. Polynomial. [point-value representation] 𝐴 𝑥 = 𝑥0, 𝑦0 , ⋯ , 𝑥 𝑛−1, 𝑦 𝑛−1 𝐵 𝑥 = 𝑥0, 𝑧0 , ⋯ , (𝑥 𝑛−1, 𝑧 𝑛−1) Add. O(n) arithmetic operations. 𝐴 𝑥 + 𝐵 𝑥 = 𝑥0, 𝑦0 + 𝑧0 , ⋯ , (𝑥 𝑛−1, 𝑦 𝑛−1 + 𝑧 𝑛−1) Multiply (convolve). O(n), but need 2n – 1 points. 𝐴 𝑥 × 𝐵 𝑥 = 𝑥0, 𝑦0 × 𝑧0 , ⋯ , (𝑥2𝑛−1, 𝑦2𝑛−1 × 𝑧2𝑛−1) Evaluate. O(n2) using Lagrange's formula. 𝐴 𝑥 = 𝑘=0 𝑛−1 𝑦 𝑘 𝑗≠𝑘 𝑥 − 𝑥𝑗 𝑗≠𝑘 𝑥 𝑘 − 𝑥𝑗 Polynomials: point value representation 13 May 2016 By:Muhammad Umer Javed 7
- 8. Point-value => coefficient . Given 𝑛 distinct points 𝑥0, ⋯ , 𝑥 𝑛−1 𝑎0 + 𝑎1 𝑥 + ⋯ + 𝑎 𝑛−1 𝑥 𝑛−1, Evaluate it at 𝑛 distinct points Running time. O(n2) for matrix-vector multiply (or n Horner's).. Converting between two representations: brute force 13 May 2016 By:Muhammad Umer Javed 8
- 9. Coefficient => point-value. Given 𝑛 distinct points 𝑥0, ⋯ , 𝑥 𝑛−1 and values 𝑦0, ⋯ , 𝑦 𝑛−1 , find unique polynomial 𝑎0 + 𝑎1 𝑥 + ⋯ + 𝑎 𝑛−1 𝑥 𝑛−1, that has given values at given points. Running time. 𝑂(𝑛3 ) for Gaussian elimination. Converting between two representations: brute force Vandermonde matrix is invertible iff xi distinct 13 May 2016 By:Muhammad Umer Javed 9
- 10. Tradeoff. Fast evaluation or fast multiplication. We want both! Converting between two representations Goal. Efficient conversion between two representations ⇒ all ops fast. 13 May 2016 By:Muhammad Umer Javed 10
- 11. Example Equation 13 May 2016 By:Muhammad Umer Javed 11
- 12. DISCRETE FOURIER TRANSFORM 13 May 2016 By:Muhammad Umer Javed 12
- 13. Coefficient => point-value. Given a polynomial 𝑎0 + 𝑎1 𝑥 + ⋯ + 𝑎 𝑛−1 𝑥 𝑛−1 evaluate it at n distinct points 𝑥0, ⋯ , 𝑥 𝑛−1 Key idea. Choose 𝑥 𝑘 = 𝜔 𝑘 = 𝑒 2𝜋𝑖𝑘 𝑛 , 0 ≤ 𝑘 < 𝑛 where 𝜔 is principal 𝑛𝑡ℎ root of unity. Discrete Fourier transform 𝜔3 2 𝜔2 1 𝜔1 0 1 + 𝜔1 + 𝜔2 = 0 Cube root of unity8th root of unity will be = 𝟏 + 𝝎 𝟏 + 𝝎 𝟐 + 𝝎 𝟑 + 𝝎 𝟒 + 𝝎 𝟓 + 𝝎 𝟔 + 𝝎 𝟕 13 May 2016 By:Muhammad Umer Javed 13
- 14. Goal. Evaluate a degree 𝑛 – 1 polynomial 𝐴(𝑥) = 𝑎0 + ⋯ + 𝑎 𝑛–1 𝑥 𝑛–1 at its 𝑛𝑡ℎ roots of unity: 𝜔0, 𝜔1, … , 𝜔𝑛– 1 Divide. Break up polynomial into even and odd powers. • 𝐴 𝑒𝑣𝑒𝑛 𝑥 = 𝑎0 + 𝑎2 𝑥 + 𝑎2 𝑥2 + ⋯ + 𝑎 𝑛−1 𝑥 𝑛 2 −1 • 𝐴 𝑜𝑑𝑑 𝑥 = 𝑎1 + 𝑎3 𝑥 + 𝑎5 𝑥2 + ⋯ + 𝑎 𝑛−1 𝑥 𝑛 2 −1 • 𝐴 𝑥 = 𝐴 𝑒𝑣𝑒𝑛 𝑥2 + 𝑥𝐴 𝑜𝑑𝑑(𝑥2 ) Conquer. Evaluate 𝐴 𝑒𝑣𝑒𝑛(𝑥) and 𝐴 𝑜𝑑𝑑(𝑥) at the ½𝑛𝑡ℎ roots of unity: 𝜈0, 𝜈1, … , 𝜈 𝑛 2 – 1. • 𝐴 𝜔 𝑘 = 𝐴 𝑒𝑣𝑒𝑛 𝑣 𝑘 + 𝜔 𝑘 𝐴 𝑜𝑑𝑑(𝑣 𝑘 ) • 𝐴 𝜔 𝑘+1/2𝑛 = 𝐴 𝑒𝑣𝑒𝑛 𝑣 𝑘 − 𝜔 𝑘 𝐴 𝑜𝑑𝑑(𝑣 𝑘 ) Fast Fourier transform 𝒗 𝒌 = 𝝎 𝒌+ 𝟏 𝟐 𝒏 𝟐 𝝎 𝒌+ 𝟏 𝟐 𝒏 = −𝝎 𝒌 13 May 2016 By:Muhammad Umer Javed 14
- 15. 𝑋 𝑘 = 𝑛=0 𝑁−1 𝑥[𝑛]𝑒− 𝑖2𝜋𝑘𝑛 𝑛 ; 𝑘 = 0, ⋯ , 𝑁 − 1 For each k 𝑁 𝑐𝑜𝑚𝑝𝑒𝑙𝑒𝑡𝑒 𝑚𝑢𝑙𝑡𝑠 𝑁 − 1 𝑐𝑜𝑚𝑝𝑙𝑒𝑥 𝑎𝑑𝑑𝑠 • 𝑂(𝑁2) for DFT • 𝑂(𝑁𝑙𝑜𝑔𝑁) for FFT ……. How! lets see As 𝜔 𝑁 = 𝑒− 𝑖2𝜋 𝑁 where 𝜔 𝑁 𝑘𝑁 = 𝑒−𝑖2𝜋𝑘 = 1 Fast Fourier Transform 13 May 2016 By:Muhammad Umer Javed 15
- 16. Complex multiplies require 4 real multiplies and 2 real additions, whereas complex additions require just 2 real additions. So the 𝑁2 complex multiplies are the primary concern. 𝑁2 increases rapidly with N, By exploiting the following properties of W: • Symmetry property: 𝑊𝑁 𝑘+ 𝑁 2 = −𝑊𝑁 𝑘 = 𝑒 𝑖𝜋 𝑊𝑁 𝑘 • Periodicity property: 𝑊𝑁 𝑘+𝑁 = 𝑊𝑁 𝑘 • Recursion property: 𝑊𝑁 2 = 𝑊 𝑁 2 FFT So how can we reduce the amount of computation? 13 May 2016 By:Muhammad Umer Javed 16
- 17. As we take an example of DFT of N=4 so it becomes • 𝑋0 = 𝑥0 𝑒− 𝑖2𝜋 0 0 4 + 𝑥1 𝑒− 𝑖2𝜋 1 0 4 + 𝑥2 𝑒− 𝑖2𝜋 2 0 4 + 𝑥3 𝑒− 𝑖2𝜋 3 0 4 • 𝑋0 = 𝑥0 + 𝒙 𝟏 + 𝑥2 + 𝒙 𝟑 • 𝑋1 = 𝑥0 + 𝑥1 𝑒− 𝑖2𝜋 4 − 𝑥2 + 𝑥3 𝑒 −𝑖2𝜋 3 4 • 𝑋1 = 𝑥0 + 𝒙 𝟏 𝒆− 𝒊𝟐𝝅 𝟒 − 𝑥2 − 𝒙 𝟑 𝒆− 𝒊𝟐𝝅 𝟒 • 𝑋2 = 𝑥0 − 𝒙 𝟏 + 𝑥2 − 𝒙 𝟑 • 𝑋3 = 𝑥0 + 𝒙 𝟏 𝒆 −𝒊𝟐𝝅 𝟑 𝟒 − 𝑥2 − 𝒙 𝟑 𝒆 −𝒊𝟐𝝅 𝟑 𝟒 FFT: Symmetric Property 𝑒 −𝑖2𝜋 3 4 = −𝑒− 𝑖2𝜋 4 is symmetric in unit circle but in opposite direction so having a –ve sign 13 May 2016 By:Muhammad Umer Javed 17
- 18. When 𝑁 = 4 equation becomes • 𝑋0 = (𝑥0 + 𝑒 −𝑖2𝜋 0 2 𝑥2) + 𝑒 −𝑖2𝜋 0 4 (𝑥1 + 𝑒 −𝑖2𝜋 0 2 𝑥3) • 𝑋1 = (𝑥0 − 𝑒 −𝑖2𝜋 0 2 𝑥2) + 𝑒 −𝑖2𝜋 0 4 (𝑥1 − 𝑒 −𝑖2𝜋 0 2 𝑥3) • 𝑋2 = 𝑥0 + 𝑒 −𝑖2𝜋 0 2 𝑥2 − 𝑒 −𝑖2𝜋 0 4 (𝑥1 + 𝑒 −𝑖2𝜋 0 2 𝑥3) • 𝑋3 = 𝑥0 − 𝑒 −𝑖2𝜋 0 2 𝑥2 − 𝑒 −𝑖2𝜋 0 4 (𝑥1 − 𝑒 −𝑖2𝜋 0 2 𝑥3) FFT: Example 13 May 2016 By:Muhammad Umer Javed 18
- 19. FFT For indices 𝑒𝑣𝑒𝑛 𝑛 = 2𝑟 𝑜𝑑𝑑 𝑛 = 2𝑟 + 1 where 𝑟 = 0, 1, ⋯ , 𝑁 2 − 1 𝑋 𝑘 = 𝑟=0 𝑁 2 −1 𝑥 2𝑟 𝑊𝑁 2𝑟𝑘 + 𝑟=0 𝑁 2 −1 𝑥 2𝑟 + 1 𝑊𝑁 (2𝑟+1)𝑘 𝑋 𝑘 = 𝑟=0 𝑁 2 −1 𝑥 2𝑟 𝑊𝑁 2 𝑘𝑟 + 𝑊𝑁 𝑘 𝑟=0 𝑁 2 −1 𝑥 2𝑟 + 1 𝑊𝑁 2 𝑘𝑟 𝑋 𝑘 = 𝑟=0 𝑁 2 −1 𝑥 2𝑟 𝑊𝑁 2 𝑘𝑟 + 𝑊𝑁 𝑘 𝑟=0 𝑁 2 −1 𝑥 2𝑟 + 1 𝑊𝑁 2 𝑘𝑟 So we get 𝑋 𝑘 = 𝑋 𝑒 𝑘 + 𝑊𝑁 𝑘 𝑋 𝑜[𝑘] also 𝑋[𝑘 + 𝑁/2] = 𝑋 𝑒[𝑘] − 𝑊𝑁 𝑘 𝑋 𝑜[𝑘], 𝑵 𝟐 DFT of even samples 𝑿 𝒆[𝒌] 𝑵 𝟐 DFT of odd samples 𝑿 𝒐[𝒌] But 𝑾 𝒏 𝟐 = 𝒆− 𝒊𝟐𝝅 𝑵 𝟐 = 𝒆 − 𝒊𝟐𝝅 𝑵 𝟐 = 𝐖𝐍 𝟐 13 May 2016 By:Muhammad Umer Javed 19
- 20. FFT: Algorithem 13 May 2016 By:Muhammad Umer Javed 20
- 21. FFT : Algorithem 13 May 2016 By:Muhammad Umer Javed 21
- 22. Theorem. The FFT algorithm evaluates a degree n – 1 polynomial at each of the nth roots of unity in O(n log n) steps and O(n) extra space 𝑇(𝑛) = 2𝑇(𝑛 /2) + Θ(𝑛) ⇒ 𝑇(𝑛) = Θ(𝑛 𝑙𝑜𝑔𝑛) FFT : Time Complexty Assume 𝒏 is in power of 2 13 May 2016 By:Muhammad Umer Javed 22
- 23. Theorem. The inverse FFT algorithm interpolates a degree n – 1 polynomial given values at each of the nth roots of unity in O(n log n) steps. IFFT : Summery and Algorithem 13 May 2016 By:Muhammad Umer Javed 23
- 24. Polynomial : Summery Runs 2 FFT for two polynomials with time 2Ο(𝑛𝑙𝑜𝑔𝑛) Runs 1 IFFTfor one polynomials with time Ο(𝑛𝑙𝑜𝑔𝑛) Runs 𝑛 multiplecations Ordanery multiplecation 𝑛2 13 May 2016 By:Muhammad Umer Javed 24