![Defining convolution
∑ −
−
=
∗
l
k
l
k
g
l
n
k
m
f
n
m
g
f
,
]
,
[
]
,
[
]
,
)[
(
f
• Let f be the image and g be the kernel. The
output of convolving f with g is denoted f * g.
Source: F. Durand
• Convention: kernel is “flipped”
• MATLAB: conv2 vs. filter2 (also imfilter)](https://image.slidesharecdn.com/lec05filter-210326054034/85/Lec05-filter-5-320.jpg)
![Properties in more detail
• Commutative: a * b = b * a
• Conceptually no difference between filter and signal
• Associative: a * (b * c) = (a * b) * c
• Often apply several filters one after another: (((a * b1) * b2) * b3)
• This is equivalent to applying one filter: a * (b1 * b2 * b3)
• Distributes over addition: a * (b + c) = (a * b) + (a * c)
• Scalars factor out: ka * b = a * kb = k (a * b)
• Identity: unit impulse e = […, 0, 0, 1, 0, 0, …],
a * e = a](https://image.slidesharecdn.com/lec05filter-210326054034/85/Lec05-filter-7-320.jpg)
![Median filter
Salt-and-pepper noise Median filtered
Source: M. Hebert
MATLAB: medfilt2(image, [h w])](https://image.slidesharecdn.com/lec05filter-210326054034/85/Lec05-filter-37-320.jpg)
Lec05 filter
- 2. Motivation: Noise reduction Given a camera and a still scene, how can you reduce noise? Take lots of images and average them! What’s the next best thing? Source: S. Seitz
- 3. • Let’s replace each pixel with a weighted average of its neighborhood • The weights are called the filter kernel • What are the weights for a 3x3 moving average? Moving average 1 1 1 1 1 1 1 1 1 “box filter” Source: D. Lowe
- 4. Review: Color • What are some linear color spaces? • What are some non-linear color spaces? • What is a perceptually uniform color space? • What is color constancy? • What are some applications of color in computer vision?
- 5. Defining convolution ∑ − − = ∗ l k l k g l n k m f n m g f , ] , [ ] , [ ] , )[ ( f • Let f be the image and g be the kernel. The output of convolving f with g is denoted f * g. Source: F. Durand • Convention: kernel is “flipped” • MATLAB: conv2 vs. filter2 (also imfilter)
- 6. Key properties • Linearity: filter(f1 + f2 ) = filter(f1) + filter(f2) • Shift invariance: same behavior regardless of pixel location: filter(shift(f)) = shift(filter(f)) • Theoretical result: any linear shift-invariant operator can be represented as a convolution
- 7. Properties in more detail • Commutative: a * b = b * a • Conceptually no difference between filter and signal • Associative: a * (b * c) = (a * b) * c • Often apply several filters one after another: (((a * b1) * b2) * b3) • This is equivalent to applying one filter: a * (b1 * b2 * b3) • Distributes over addition: a * (b + c) = (a * b) + (a * c) • Scalars factor out: ka * b = a * kb = k (a * b) • Identity: unit impulse e = […, 0, 0, 1, 0, 0, …], a * e = a
- 8. Annoying details What is the size of the output? • MATLAB: filter2(g, f, shape) • shape = ‘full’: output size is sum of sizes of f and g • shape = ‘same’: output size is same as f • shape = ‘valid’: output size is difference of sizes of f and g f g g g g f g g g g f g g g g full same valid
- 9. Annoying details What about near the edge? • the filter window falls off the edge of the image • need to extrapolate • methods: – clip filter (black) – wrap around – copy edge – reflect across edge Source: S. Marschner
- 10. Annoying details What about near the edge? • the filter window falls off the edge of the image • need to extrapolate • methods (MATLAB): – clip filter (black): imfilter(f, g, 0) – wrap around: imfilter(f, g, ‘circular’) – copy edge: imfilter(f, g, ‘replicate’) – reflect across edge: imfilter(f, g, ‘symmetric’) Source: S. Marschner
- 11. Practice with linear filters 0 0 0 0 1 0 0 0 0 Original ? Source: D. Lowe
- 12. Practice with linear filters 0 0 0 0 1 0 0 0 0 Original Filtered (no change) Source: D. Lowe
- 13. Practice with linear filters 0 0 0 1 0 0 0 0 0 Original ? Source: D. Lowe
- 14. Practice with linear filters 0 0 0 1 0 0 0 0 0 Original Shifted left By 1 pixel Source: D. Lowe
- 15. Practice with linear filters Original ? 1 1 1 1 1 1 1 1 1 Source: D. Lowe
- 16. Practice with linear filters Original 1 1 1 1 1 1 1 1 1 Blur (with a box filter) Source: D. Lowe
- 17. Practice with linear filters Original 1 1 1 1 1 1 1 1 1 0 0 0 0 2 0 0 0 0 - ? (Note that filter sums to 1) Source: D. Lowe
- 18. Practice with linear filters Original 1 1 1 1 1 1 1 1 1 0 0 0 0 2 0 0 0 0 - Sharpening filter - Accentuates differences with local average Source: D. Lowe
- 20. Smoothing with box filter revisited • Smoothing with an average actually doesn’t compare at all well with a defocused lens • Most obvious difference is that a single point of light viewed in a defocused lens looks like a fuzzy blob; but the averaging process would give a little square Source: D. Forsyth
- 21. Smoothing with box filter revisited • Smoothing with an average actually doesn’t compare at all well with a defocused lens • Most obvious difference is that a single point of light viewed in a defocused lens looks like a fuzzy blob; but the averaging process would give a little square • Better idea: to eliminate edge effects, weight contribution of neighborhood pixels according to their closeness to the center, like so: “fuzzy blob”
- 22. Gaussian Kernel • Constant factor at front makes volume sum to 1 (can be ignored, as we should re-normalize weights to sum to 1 in any case) 0.003 0.013 0.022 0.013 0.003 0.013 0.059 0.097 0.059 0.013 0.022 0.097 0.159 0.097 0.022 0.013 0.059 0.097 0.059 0.013 0.003 0.013 0.022 0.013 0.003 5 x 5, σ = 1 Source: C. Rasmussen
- 23. Choosing kernel width • Gaussian filters have infinite support, but discrete filters use finite kernels Source: K. Grauman
- 24. Choosing kernel width • Rule of thumb: set filter half-width to about 3 σ
- 25. Example: Smoothing with a Gaussian
- 26. Mean vs. Gaussian filtering
- 27. Gaussian filters • Remove “high-frequency” components from the image (low-pass filter) • Convolution with self is another Gaussian • So can smooth with small-width kernel, repeat, and get same result as larger-width kernel would have • Convolving two times with Gaussian kernel of width σ is same as convolving once with kernel of width σ√2 • Separable kernel • Factors into product of two 1D Gaussians Source: K. Grauman
- 28. Separability of the Gaussian filter Source: D. Lowe
- 29. Separability example * * = = 2D convolution (center location only) Source: K. Grauman The filter factors into a product of 1D filters: Perform convolution along rows: Followed by convolution along the remaining column:
- 30. Separability • Why is separability useful in practice?
- 31. Noise • Salt and pepper noise: contains random occurrences of black and white pixels • Impulse noise: contains random occurrences of white pixels • Gaussian noise: variations in intensity drawn from a Gaussian normal distribution Original Gaussian noise Salt and pepper noise Impulse noise Source: S. Seitz
- 32. Gaussian noise • Mathematical model: sum of many independent factors • Good for small standard deviations • Assumption: independent, zero-mean noise Source: M. Hebert
- 33. Smoothing with larger standard deviations suppresses noise, but also blurs the image Reducing Gaussian noise
- 34. Reducing salt-and-pepper noise What’s wrong with the results? 3x3 5x5 7x7
- 35. Alternative idea: Median filtering • A median filter operates over a window by selecting the median intensity in the window • Is median filtering linear? Source: K. Grauman
- 36. Median filter • What advantage does median filtering have over Gaussian filtering? • Robustness to outliers Source: K. Grauman
- 37. Median filter Salt-and-pepper noise Median filtered Source: M. Hebert MATLAB: medfilt2(image, [h w])
- 38. Median vs. Gaussian filtering 3x3 5x5 7x7 Gaussian Median
- 39. Sharpening revisited What does blurring take away? original smoothed (5x5) – detail = sharpened = Let’s add it back: original detail + α
- 40. Unsharp mask filter Gaussian unit impulse Laplacian of Gaussian ) ) 1 (( ) 1 ( ) ( g e f g f f g f f f − + ∗ = ∗ − + = ∗ − + α α α α image blurred image unit impulse (identity)
- 41. Application: Hybrid Images A. Oliva, A. Torralba, P.G. Schyns, “Hybrid Images,” SIGGRAPH 2006
- 42. Application: Hybrid Images A. Oliva, A. Torralba, P.G. Schyns, “Hybrid Images,” SIGGRAPH 2006