Lecture 12
2 likes1,473 views
BM3D is a denoising algorithm that uses block matching and collaborative filtering in 3D transform domains. It finds similar 2D image fragments, stacks them into 3D groups, applies a 3D transformation, shrinks the transform coefficients to attenuate noise, and inverse transforms to estimate denoised fragments. BM3D improves on previous methods by exploiting inter-fragment correlation both within and between similar image blocks grouped together. It provides multiple estimates for each pixel by processing overlapping blocks, then fuses the estimates to produce the final denoised image.
Lecture 12
- 1. BM3D FILTERING CS-467 Digital Image Processing 1
- 2. BM3D â Block Matching 3D Filtering ⢠K. Dabov, A. Foi, V. Katkovnik, and K. Egiazarian, âImage denoising by sparse 3D transform-domain collaborative filtering,â IEEE Transactions on Image Processing, vol. 16, No. 8, pp. 2080-2095, August 2007 ⢠http://www.cs.tut.fi/~foi/GCF-BM3D/ 2
- 3. BM3D â Block Matching 3D Filtering ⢠An enhanced sparse representation in transform domain. ďThe enhancement of the sparsity is achieved by similar 2-D image fragments(e.g.,blocks) into 3-D data arrays which we call âgroups.â ⢠Collaborative filtering is a special procedure developed to deal with these 3-D groups. ďUsing the three successive steps: 3-D transformation of a group of blocks Shrinkage of the 3-D spectrum Inverse 3-D transformation
- 4. Grouping ⢠Grouping is the concept of collecting similar n-dimensional fragments of a given signal into an n+1-dimentisonal data structure based on some similarity criterion (different criteria can be used) ⢠The importance of grouping is to enable the use of a higher dimensional filtering of each group , which exploits the potential similarity between grouped fragments in order to estimate the true signal in each of them. ďThis approach is referred to as collaborative filtering
- 5. Grouping by matching ⢠Matching is a method for finding signal fragments similar to a given reference one ⢠Achieving by pairwise testing ⢠Distance (for example, if SSD â sum of pixel-wise squared differences between blocks B1 and B2 is less than a pre-determined threshold ⢠Block-matching (BM) is a particular matching approach that has been extensively used for motion estimation in video compression (MPEG 1, 2, and 4, and H.26x). As a particular way of grouping, it is used to find similar blocks, which are then stacked together in a 3-D array (i.e., a group). ( ) ( )( ) 2 1 2 , , , x y B B x y B x y â ââ
- 6. Grouping by matching 6 R is a âtargetingâ block, other blocks are groped with this block by matching
- 8. Collaborative Filtering by Shrinkage in Transform Domain ⢠Assuming 2D groups of similar signal fragments are already formed, the collaborative shrinkage comprises of the following steps. ď Apply a 3D linear transform to the group ď Shrink (e.g., by Wiener filtering) the transform coefficients to attenuate the noise ď Invert the linear transform to produce estimates of all grouped fragments ⢠These groups are characterized by both: â interfragment correlation which appears between the pixels of each grouped fragmentâa peculiarity of natural images â interfragment correlation which appears between the corresponding pixels of different fragmentsâa result of the similarity between grouped fragments
- 9. BM3D Filtering R Block matching 3D grouping 3D transform Filter / thresholding Inverse 3D transform Denoised 3D group
- 10. Collaborative Filtering ⢠Use hard thresholding or Wiener filter ⢠Each patch in the group gets a denoised estimate ⢠Unlike spatial domain filtering â where only central pixel in reference patch got an estimate Filter / thr Denoised Patches Noisy patches
- 11. Multiple BM3D Estimates R thr thr R R R Collaborative filtering Collaborative filtering
- 13. The benefit of the collaborative shrinkage ⢠A 2-D transform is applied separately to each individual block in a given group of n fragments. ⢠Since these grouped blocks are very similar, for any of them we should get approximately the same number of significant transform coefficients.
- 14. Fusion ⢠Each pixel gets multiple estimates from different groups ⢠Naive approach Average all estimates of each pixel âŚ. not all estimates are as good ⢠Suggestion Give higher weight to more reliable estimates
- 15. BM3D - Fusion ⢠Give each estimate a weight according to denoising quality of its group ⢠Quality = Sparsity induced by the denoising
- 16. BM3D in Practice ⢠Noise may result in poor matching â Degrades de-noising performance ⢠Improvements: 1. Match using a smoothed version of the image 2. Perform BM3D in 2 phases: a. Basic BM3D estimate â improved 3D groups b. Final BM3D
- 18. Two Step BM3D Filtering R t t R R R R t t R R R (a) Basic denoising: Hard thresholding (b) Final denoising: Wiener filtering
- 19. Algorithm ⢠The input noisy image is processed by successively extracting reference blocks from it and for each such block: ď find blocks that are similar to the reference one (block matching) and stack them together to form a 3-D array (group) ď perform collaborative filtering of the group and return the obtained 2-D estimates of all grouped blocks to their original locations ⢠After processing all reference blocks, the obtained block estimates can overlap, and, thus, there are multiple estimates for each pixel. We aggregate these estimates to form an estimate of the whole image.
- 20. BM3D Filtering - Examples
- 21. BM3D Filtering - Examples 21
- 22. BM3D Filtering - Examples 22