Oral presentation on Asymmetric recursive Gaussian filtering for space-variant artificial bokeh
1 like139 views
1) The document describes an asymmetric recursive Gaussian filter for space-variant artificial bokeh. 2) It proposes using different directional sigma values (σx+, σx-, σy+, σy-) at each pixel to allow for discontinuous blur, minimizing intensity leakage across blur boundaries. 3) The approach constrains the rate of change of the directional sigma values to taper increases and decreases, producing good defocus blur for scenes with depth discontinuities like portraits.
![Recursive Gaussian filter
4
Second-order IIR filter [1] uses up to 2 previous taps
Anti-causal
filter
x + y
Causal
filter y+
y̶
𝑦 𝑛
+
= 𝑛0 𝑥 𝑛 + 𝑛1 𝑥 𝑛−1 − 𝑑1 𝑦 𝑛−1
+
− 𝑑2 𝑦 𝑛−2
+
𝑦 𝑛
−
= 𝑚1 𝑥 𝑛+1 + 𝑚2 𝑥 𝑛+2 − 𝑑1 𝑦 𝑛+1
−
− 𝑑2 𝑦 𝑛+2
−
𝑦 𝑛 = 𝑦 𝑛
+ + 𝑦 𝑛
−
Parallel IIR implementation of
Gaussian filter
4th-order filter is more accurate
𝑦 𝑛
+
= 𝑛0 𝑥 𝑛 + 𝑛1 𝑥 𝑛−1 + 𝑛2 𝑥 𝑛−2 + 𝑛3 𝑥 𝑛−3
− 𝑑1 𝑦 𝑛−1
+
− 𝑑2 𝑦 𝑛−2
+
− 𝑑3 𝑦 𝑛−3
+
− 𝑑4 𝑦 𝑛−4
+
𝑦 𝑛
−
= 𝑚1 𝑥 𝑛+1 + 𝑚2 𝑥 𝑛+2 + 𝑚3 𝑥 𝑛+3 + 𝑚4 𝑥 𝑛+4
− 𝑑1 𝑦 𝑛+1
−
− 𝑑2 𝑦 𝑛+2
−
− 𝑑3 𝑦 𝑛+3
−
− 𝑑4 𝑦 𝑛+4
−
𝑦 𝑛 = 𝑦 𝑛
+ + 𝑦 𝑛
−
-40 -20 0 20 40
10
-6
10
-5
10
-4
10
-3
10
-2
10
-1
t
h(t)
Gaussian =10
2nd
order, RMSE=2×10-3
4nd
order, RMSE=4×10-5
-40 -20 0 20 40
-0.01
0
0.01
0.02
0.03
0.04
0.05
t
h(t)
Gaussian
2nd
order
4nd
order
Log
space
Forth-order IIR filter [2]: use up to 4 previous taps
→ more accurate
where the filter coefficients ni, mi, and di (i=1...4) are
functions of the Gaussian smoothing scale σ
[2] Farnebäck and Westin, Improving Deriche-style recursive Gaussian filters, JMIV, 2006.
[1] Deriche, Fast algorithms for low-level vision, PAMI, 1990.](https://image.slidesharecdn.com/dicta18oral-190219044345/85/Oral-presentation-on-Asymmetric-recursive-Gaussian-filtering-for-space-variant-artificial-bokeh-4-320.jpg)
![Space-variant IIR filter: the straightforward extension [3]
5
Vary the filter coefficients ni, mi, and di according to the varying σ
This isotropic filter works OK for smoothly varying σ
not OK for discontinuous blur: colour bleeds across blur discontinuities
foveation blur: 0 ≤ σ ≤ 20 discontinuous blur σ ∈ {0, 10}
[3] Tan, et al., Performance of three recursive algorithms for fast space-variant Gaussian filtering, 2003.
y+
Σ
0
1
1
0
0
1 Σ
σ
x σ=0 σ=1 σ=1
leakage
causal filters
y̶
Σ
0
1
1
0
0
1
σ=0 σ=0 σ=1
σ
x
anti-causal filters
causal pass: bleeding anti-causal pass: no bleeding
input
output
Colour bleeds to adjacent pixels if the blur difference is large enough](https://image.slidesharecdn.com/dicta18oral-190219044345/85/Oral-presentation-on-Asymmetric-recursive-Gaussian-filtering-for-space-variant-artificial-bokeh-5-320.jpg)
Oral presentation on Asymmetric recursive Gaussian filtering for space-variant artificial bokeh
- 1. C A N O N C O N F I D E N T I A L Asymmetric recursive Gaussian filtering for space-variant artificial bokeh Tuan Pham Canon Information Systems Research Australia Oral Session 5 15:45-16:00 on Thursday 13 December 2018
- 2. Overview 2 1. Recursive Gaussian filter 2. Space-variant recursive filter 3. Artificial bokeh application
- 3. FIR filter versus IIR (recursive) filter 3 Finite Impulse Response (FIR) filter Output is a weighted average of input samples: O(N), where N is filter size Xn-2 Xn-1 Xn Xn+1 Xn+2 ynyn-1yn-2 yn+1 yn+2 Impulse input Finite impulse response 0 0 w1 w2 w3 w4 w5 w1 w2 w3 w4 w5 Infinite Impulse Response (IIR) filter Output is a weighted average of input and previous output samples O(1) computation regardless of filter size Xn-2 Xn-1 Xn yn-2 yn-1 yn yn+1 yn+2 xn+1 xn+2 Infinite impulse response 0 Causal direction Xn-2 Xn-1Xn yn-2 yn-1ynyn+1 yn+2 xn+1 xn+2 Anti-causal direction 0 ∑wi = 1 for a normalised filter
- 4. Recursive Gaussian filter 4 Second-order IIR filter [1] uses up to 2 previous taps Anti-causal filter x + y Causal filter y+ y̶ 𝑦 𝑛 + = 𝑛0 𝑥 𝑛 + 𝑛1 𝑥 𝑛−1 − 𝑑1 𝑦 𝑛−1 + − 𝑑2 𝑦 𝑛−2 + 𝑦 𝑛 − = 𝑚1 𝑥 𝑛+1 + 𝑚2 𝑥 𝑛+2 − 𝑑1 𝑦 𝑛+1 − − 𝑑2 𝑦 𝑛+2 − 𝑦 𝑛 = 𝑦 𝑛 + + 𝑦 𝑛 − Parallel IIR implementation of Gaussian filter 4th-order filter is more accurate 𝑦 𝑛 + = 𝑛0 𝑥 𝑛 + 𝑛1 𝑥 𝑛−1 + 𝑛2 𝑥 𝑛−2 + 𝑛3 𝑥 𝑛−3 − 𝑑1 𝑦 𝑛−1 + − 𝑑2 𝑦 𝑛−2 + − 𝑑3 𝑦 𝑛−3 + − 𝑑4 𝑦 𝑛−4 + 𝑦 𝑛 − = 𝑚1 𝑥 𝑛+1 + 𝑚2 𝑥 𝑛+2 + 𝑚3 𝑥 𝑛+3 + 𝑚4 𝑥 𝑛+4 − 𝑑1 𝑦 𝑛+1 − − 𝑑2 𝑦 𝑛+2 − − 𝑑3 𝑦 𝑛+3 − − 𝑑4 𝑦 𝑛+4 − 𝑦 𝑛 = 𝑦 𝑛 + + 𝑦 𝑛 − -40 -20 0 20 40 10 -6 10 -5 10 -4 10 -3 10 -2 10 -1 t h(t) Gaussian =10 2nd order, RMSE=2×10-3 4nd order, RMSE=4×10-5 -40 -20 0 20 40 -0.01 0 0.01 0.02 0.03 0.04 0.05 t h(t) Gaussian 2nd order 4nd order Log space Forth-order IIR filter [2]: use up to 4 previous taps → more accurate where the filter coefficients ni, mi, and di (i=1...4) are functions of the Gaussian smoothing scale σ [2] Farnebäck and Westin, Improving Deriche-style recursive Gaussian filters, JMIV, 2006. [1] Deriche, Fast algorithms for low-level vision, PAMI, 1990.
- 5. Space-variant IIR filter: the straightforward extension [3] 5 Vary the filter coefficients ni, mi, and di according to the varying σ This isotropic filter works OK for smoothly varying σ not OK for discontinuous blur: colour bleeds across blur discontinuities foveation blur: 0 ≤ σ ≤ 20 discontinuous blur σ ∈ {0, 10} [3] Tan, et al., Performance of three recursive algorithms for fast space-variant Gaussian filtering, 2003. y+ Σ 0 1 1 0 0 1 Σ σ x σ=0 σ=1 σ=1 leakage causal filters y̶ Σ 0 1 1 0 0 1 σ=0 σ=0 σ=1 σ x anti-causal filters causal pass: bleeding anti-causal pass: no bleeding input output Colour bleeds to adjacent pixels if the blur difference is large enough
- 6. Our solution: constraining the rate of change of directional σ’s 6 xy-separable filtering, each with 2 directions -> 4 directional sigmas: 𝜎𝑥 + , 𝜎𝑥 − , 𝜎𝑦 + , 𝜎𝑦 − , which can be different at a pixel Σ 0 1 1 0 0 1 σ ̶ x y Σ anti-causal filterscausal filters 0 1 1 0 1 σ+ 0 0 1 σ 0 σ+ =0 σ̶ =0 σ ̶ =1 σ+ = 1 σ̶ =.5σ+ =.1 σ+ =0 σ̶ =0 x + x - y + y - 0 10 5 To minimise leakage, the rate of increase of 𝜎+ and the rate of decrease of 𝜎− is tapered to at most 1/3 per pixel. input output
- 7. 7 Bokeh = Japanese for visually pleasing out-of-focus blur Application: artificial bokeh Depth map Large Depth-of-Field input
- 8. Deriche's 2-tap symmetric IIR Our 2-tap asymmetric IIR Farneback's 4-tap symmetric IIR Our 4-tap asymmetric IIR 8 Application: artificial bokeh Bokeh = Japanese for visually pleasing out-of-focus blur Depth map
- 9. Summary 9 IIR filter is more efficient than FIR filter for large filter size Naïve extension of existing IIR filters to space-varying Gaussian filters produces artefacts at sudden blur discontinuities We proposed asymmetric IIR filters that minimise intensity leakage across blur discontinuities Our asymmetric IIR filters produce good defocus blur for scenes with depth discontinuities (e.g. portrait shot)
- 11. Details: filter normalisation 11 0 0 0 0 0 0 0 Anti-causal filter σ̶ x + y Causal filter σ+ y+ y̶ + + w+ δ w̶ δ × × 2σ+ σ+ + σ̶ 2σ̶ σ+ + σ̶ The asymmetric left & right blur weights may not add up to 1 Need to add a small delta response to normalise the total filter
- 12. Application: speeded-up anisotropic filtering 12 20 iterations of Perona-Malik anisotropic diffusion with λ=0.25, 𝑔 𝛻𝐼 = exp (− 𝛻𝐼 /5) Our edge-stopping blur with 𝜎 = 16 × exp − 𝛻𝐼 in the inset