Influence of Signal-to-Noise Ratio and Point Spread Function on Limits of Super-Resolution
2 likes846 views
The document discusses the limits of super-resolution imaging, focusing on the influence of signal-to-noise ratio and point spread function. It presents methodologies for estimating the super-resolution factor based on noise variance and registration error, with experimental validation showing consistent results. Key findings indicate that the achievable super-resolution factor is heavily dependent on the sampling factors and levels of aliasing.
Influence of Signal-to-Noise Ratio and Point Spread Function on Limits of Super-Resolution
- 1. Conf. 5672: Image Processing Algorithms and Systems IV Influence of Signal-to-Noise Ratio and Point Spread Function on Limits of Super-Resolution Tuan Pham Quantitative Imaging Group Delft University of Technology The Netherlands
- 2. Super-Resolution: an example 128x128x100 infra-red sequence 4x super-resolution © 2004 Tuan Pham 2
- 3. Super-Resolution: an example Low resolution 4x super-resolution © 2004 Tuan Pham 3
- 4. Overview and Goal System inputs No. of inputs SNR PSF Positioning SNR Resolving limit limit limit GOAL: Derive the Limits of Super-Resolution given system inputs © 2004 Tuan Pham 4
- 5. Limit of registration • Cramer-Rao Lower Bound for 2D shift: I2(x, y) = I1(x+vx, y+vy) : var(v x ) ≥ F111 = σ n ∑ I2 Det (F) − 2 y S var(v y ) ≥ F221 = σ n ∑ I2 Det (F) − 2 x S where I x = ∂I / ∂x , I y = ∂I / ∂y , σ n is noise variance, and F is the 2 Fisher Information Matrix: ⎡ ∑ I2 ∑I I ⎤ 1 ⎢ S x x y ⎥ F( v ) = 2 ⎢ S σ n ∑ IxIy ⎢S ⎣ ∑I ⎥ S ⎥ ⎦ 2 y • Optimal registration is achievable by iterative optimization • CRLB also exists for more complicated motion models: - 2D projective - optic flow © 2004 Tuan Pham 5
- 6. Noise of HR image after fusion • Total noise = Intensity noise + Noise due to registration error μ : zoom factor μ2 2 N : # of LR images σ 2 n = σ I 2 + ∇I σ 2 reg 2 ∇I : gradient energy N position error distribution Intensity error distribution σreg I σI x local signal ∂I σI = σ ∂x re g Blurred & mis-registered Noise due to 5x5 box blur, σ reg = 0.2 pixel mis-registration mis-registration → noise © 2004 Tuan Pham 6
- 7. The need for deconvolution • After fusion, the High-Resolution image is still blurry due to: – Sensor integration blur (severe if high fill-factor) – Optical blur (severe if high sampling factor) On-chip microlens of Sony Super HAD CCD © 2004 Tuan Pham 7
- 8. The necessity of aliasing • Spectrum is cut off beyond fc due to optics → data forever lost 1 1 OTF (sampling factor = 0.25) OTF (sampling factor = 1) frequency spectra / transfer functions frequency spectra / transfer functions STF (fill factor = 1) STF (fill factor = 1) 0.8 Original scene spectrum 0.8 Original scene spectrum Band−limited spectrum Band−limited spectrum Aliased image spectrum Sampled image spectrum 0.6 0.6 0.4 0.4 0.2 0.2 0 0 −0.2 −0.2 −0.4 −0.4 0 0.5 1 1.5 0 0.5 1 1.5 2 frequency in unit of sampling frequency (f/fs) frequency in unit of sampling frequency (f/fs) Aliasing due to No aliasing at under-sampling (fs < 2fc) critical sampling (fs = 2fc) © 2004 Tuan Pham 8
- 9. Limit of deconvolution • Blur = attenuation of HF spectrum recoverable • Deconvolution = amplify HF spectrum: – noise is also amplified → limit the deconvolution PS>PN • Deconvolution can only recover: Not – Spectrum whose signal power > noise power recoverable resolution factor = 0.44 fusion result © 2004 Tuan Pham after deconvolution simulated at resolution = 0.44 9
- 10. SR reconstruction experiment • Aim: show that the attainable SR factor agrees with the prediction • Experiment: – Inputs: sufficient shifted LR images of the Pentagon – Output: SR image and a measure of SR factor from edge width 64x64 LR input 4xHR after fusion 4xSR after deconvolution sampling=1/4, fill = 100% BSNR = 20 dB SR factor = 3.4 © 2004 Tuan Pham 10
- 11. SR reconstruction experiment • Aim: show that the attainable SR factor agrees with the prediction • Experiment: – Inputs: sufficient shifted LR images of the Pentagon – Output: SR image and a measure of SR factor from edge width 64x64 LR input 4xHR after fusion 4xSR after deconvolution sampling=1/4, fill = 100% BSNR = 20 dB SR factor = 3.4 © 2004 Tuan Pham 11
- 12. SR factor at BSNR=20dB • Consistent results between prediction and measurement: – Attainable SR factor depends mainly on sampling factor (i.e. level of aliasing) 6 3.2 6 3.0 3.4 2.5 4 SR limit SR factor 4 1.7 1.9 2 2 1.0 1.0 0 0.6 0 0 0 0 0 0.6 0.5 1 0.5 1 sampling factor sampling factor fill factor 1 2 (f /2f ) fill factor 1 2 (fs/2fc) s c Measured SR factor Predicted SR factor © 2004 Tuan Pham 12
- 13. Summary • Limit of Super-Resolution depends on: – input Signal-to-Noise Ratio – System’s Point Spread Function and how well it can be estimated • Procedure for estimating SR factor directly from inputs: – Measure noise variance from LR images σ I2 – Derive registration error σ reg 2 – Determine SR factor from the Power Spectrum Density (PS > PN) © 2004 Tuan Pham 13