N42018588

Rajul Suthar et al Int. Journal of Engineering Research and Applications
ISSN : 2248-9622, Vol. 4, Issue 2( Version 1), February 2014, pp.85-88

RESEARCH ARTICLE

www.ijera.com

OPEN ACCESS

A Survey on Various Image Inpainting Techniques to Restore
Image
Rajul Suthar, Mr. Krunal R. Patel
P.G. Student Computer Science and Engineering Saffrony Institute of technology Mehsana Gujarat.
Assistant. Prof. CSE Dept. Saffrony Institute of technology Mehsana Gujarat.

Abstract
Image Inpainting or Image Restore is technique which is used to recover the damaged image and to fill the
regions which are missing in original image in visually plausible way. Inpainting, the technique of modifying an
image in an invisible form, it is art which is used from the early year. Applications of this technique include
rebuilding of damaged photographs& films, removal of superimposed text, removal/replacement of unwanted
objects, red eye correction, image coding. The main goal of the Inpainting is to change the damaged region in
an image. In this paper we provide a review of different techniques used for image Inpainting. We discuss
different inpainting techniques like Exemplar based image inpainting, PDE based image inpainting, texture
synthesis based image inpainting, structural inpainting and textural inpainting.
Keywords: Image inpainting, Image Restore, Exemplar, Object Removal, wavelet transformation

I.

Introduction

Inpainting is the art of restoring lost parts of
an image and reconstructing them based on the
background information. This has to be done in an
undetectable way. The term Inpainting is derived
from the ancient art of restoring image by
professional image restorers in museums etc. Digital
Image Inpainting tries to imitate this process and
perform the Inpainting automatically .The filling of
lost information is essential in image processing, with
applications as well as image coding and wireless
image transmission, special effects and image
restoration. The basic idea at the back of the
algorithms that have been proposed in the literature is
to fill-in these regions with available information
from their environment[1]
Image
Inpainting

Input image
Output image
Fig 1 Image Inpainting Method

Figure.2. Ice Drill
Figure 2(1) Original Ice Drill Image.2(2)
Image with mask for guiding inpainting 2(3) Result
of simple Exemplar Base Inpainting 2(4) Adaptive
Inpainting with patch size 10 X 10 pixels. [2]
The following groups of Various Image
Inpainting Techniques
A. Partial Differential Equation (PDE) based
B. Texturesynthesis based
C. Exemplar and search based
D. Wavelet Transformbased
E. Semi-automatic and Fast Inpainting.

II.

(1)

www.ijera.com

(2)

Partial Differential Equation(PDE)
based algorithm.

Partial Differential Equation (PDE) based
algorithm is proposed by Marcelo Bertalmioet.al
[1]
.This algorithmis the iterative algorithm. The
algorithm is to continue geometric and photometric
information that arrives at the border of the occluded
area into area itself. This is done by propagating
theinformation in the direction of minimal change
using is ophotelines. This algorithm will produce
good resultsif missed regions are small one. But
when the missed regions are large this algorithm will
take so long time andit will not produce good results.
85 | P a g e

Then inspired by this work, Chan and Shen [3]
proposed the Total Variational (TV) Inpainting
model. This algorithm is good due to Isophote driven
Approach we find the line of equal gray scale values
which contains the more promising information and
this used to complete the image with less time. This
algorithm also provide some problem, The main
difficulty with this algorithm is imitation of large
texture regions. This algorithm also unable to recover
Partially Degraded Image.

III.

Texturesynthesis based Image
Inpainting

The Texture synthesis is a field of study
independent from, butrelated to inpainting.In the
general definition of this problem,an input sample of
a texture is given, and the goal is to producemore of
that texture.The simplest solution is to tile the texture
sample on arectangular grid of desired size. However,
even if the sample can be tiled seamlessly, the
resulting larger grid structureis easily noticeable and
it distorts the perception of theactual texture. More
sophisticated techniques are required forreproducing
the actual texture with all its features and
nothingmore.
A regular (also called deterministic,
structured, periodic) textureis characterized by a
primitive element (texton or texel) that isregularly
placed on a grid or a lattice. For example, floor
tiles,brick walls are regular textures, sand, smoke are
non-regular.Contrarily, in non-regular (stochastic,
random) textures, thereis no apparent repeating
pattern or local structure, but globalstatistical
properties.The texture synthesis based Inpainting
perform well inapproximating textures.
These algorithms have difficulty in handling
natural images as they are composed ofstructures in
form of edges.
Hencewhile appreciating the use of texture
synthesis techniques in Inpainting, it is important to
understand that thesemethods address only a small
subset of Inpainting issues and these methods are not
suitable for a large objects.

IV.

ExemplarbasedImage Inpainting

The exemplar based consists of two basic
steps1.priority assignment is done and the 2.the
selection of the best matching patch. The exemplar
based approach samples the bestmatching patches
from the known region, whose similarity is measured
by certain metrics, and pastes into thetarget patches
in the missing region.Exemplar- based Inpainting
iteratively synthesizes the unknown region i. e. target
region, by the mostsimilar patch in the source region.
The method fills structures in the missing
regions using spatial information of neighboring
regions. This method is an efficient approach
www.ijera.com

www.ijera.com

forreconstructing big target regions.Normally, an
exemplar-based Inpainting algorithm includes the
following four main steps:
I.
Initializing the Target Region:, in which the
initial missing areas are extracted and
represented with appropriate data structures.
II.
Computing Filling Priorities: in this a
predefined priority function is used to
compute the filling order for all unfilled
pixels p∈δΩ in the beginning of each filling
iteration.
III.
Searching Example and Compositing: in
which the most similar example is searched
from the sourceregion Φ to compose the
given patch, Ψ (of size N × N pixels) that
centered on the given pixel p.
IV.
Updating Image Information: in which the
boundary δΩ of the target region Ω and the
requiredinformation for computing filling
priorities are updated.

Figure 3: Structure propagation by exemplar-based
texture synthesis.
(a) Original image, with the target region Ω, its
contour δΩ and the source region Φ clearly
marked. (b) We want to synthesize the area
delimited by the patch ψpcentred on the point p
€δΩ
The Exemplar-based algorithms adopt the
greedy strategy, so these algorithms suffer fromthe
common problems of the greedy algorithm, being the
filling order is very critical.Exemplar based
Inpainting will produce good results only if the
missing region consists of simple structure
andtexture. And if there are not sufficient samples in
image then it is impossible to synthesize the desired
image.

V.

Wavelet Transform based

The algorithm [4] presented the technique
with the help of the wavelet transform. Here we
expect the best global structure estimation of
damaged regions in addition to shape and texture
properties. If we consider the fact of multi-resolution
analysis, data separation, compaction along with the
86 | P a g e

statistical properties then we have to consider the
wavelet transform due to its good image
representation quality. Wavelet transform try to
satisfy the human visual system (HVS). The
algorithm decomposition of incomplete image is
done with the help of wavelet and after that wavelet
and scaling coefficients is found. The image
inpainting process is applied in the wavelet domain
by considering both scaling and wavelet coefficient
from coarse to fine scales in the target region. Using
this algorithm one benefit is This utilizes inter and
intra scale dependency to maintain image structure
and texture quality using Wavelet Transform. But
difficulties In this algorithm mask for regions are
defined manually.

Figure.4. Wavelet transform decomposition of image
[2]

Figure.5. Result of Wavelet Transform
[2]

VI.

Semi-automatic and Fast In painting.

This image in painting requires user
assistance the in theform of guide lines to help in
structure completion has found favour with
researchers. The method by Jian et.al [5] proposed
inpainting with Structure propagation. this perform
two-step process. First A user manually specifies
important missing information in the hole by
www.ijera.com

www.ijera.com

sketching object boundaries from theknown to the
unknown region and then a patch based texture
synthesis is used to generate the texture. Themissing
image patches are synthesized along the user
specified curves by formulating the problem as a
global optimization problem under various structural
and consistency constraints. Simple dynamic
programming can beused to derive the optimal
answer if only a single curve is present. For multiple
objects, the optimization is greatdeal more difficult
and the proposes approximated the answer by using
belief propagation. All the methods discussed above
take minutes to hours to complete depending on the
size of the Inpainting area and hencemaking it
unacceptable for interactive user applications. To
speed
up
the
conventional
image
Inpaintingalgorithms, new classes of fast Inpainting
techniques are being developed. Oliviera et.al [6]
proposed a fast digital In painting technique based on
an isotropic diffusion model which performs
Inpainting by repeatedlyconvolving the Inpainting
region with a diffusion kernel. A new method which
treats the missing regions aslevel sets and uses Fast
Marching Method (FMM) to propagate image
information has been proposed by Teleain [7]. These
fast techniques are not suitable in filling large hole
regions as they lack explicit methods to in paintedge
regions. This technique results in blur effect in
image.

VII.

Conclusion

In this paper a variety of image Inpainting
techniques such as texture synthesis based Inpainting,
PDE
based
Inpainting,
Exemplar
based
Inpainting,wavelet
transformation
and
semiautomatic and fast Inpainting techniques are
studied.Image inpainting is recently very important
research area in the field of image processing.
Theperformance of different techniques is compared
based on the area to be inpainted. Most of the
algorithms work wellfor small scratch regions or
small regions to be inpainted. In future we would like
to implement algorithms reviewedin this paper would
like to compare their performances. We would like to
improve those algorithms and would liketo propose a
new inpainting algorithm for inpainting large regions.
And includes growth of efficient algorithm to
decrease the time required for Inpainting and reduce
computational cost.

References
[1]

Marcelo
Bertalmio,
Luminita
Vese,
Guillermo Sapiro (2003), “Simultaneous
Structure and Texture Image In painting”,
IEEE transactions on image processing, vol.
12

87 | P a g e

[2]

[3]

[4]

[5]

[6]

[7]

[8]

[9].

[10].

[11].

[12].

[13].

[14].

Bhimaraju
Swati,
Naveen
Malviya,
Shrikant Lade “Analysis of Exemplar Base
In painting for Adaptive Patch
Propagation using Wavelet Transform”.
IJETAE Volume 3,May 2013
T. Chan and J. Shen, “Local in painting
models and TV in painting,” SIAM Journal
on Applied Mathematics, Vol. 62, 2001, pp.
1019-1043
Dong wookcho and Tien D. Bui “Image In
painting Using Wavelet-Based Inter and
Intra-Scale Depedency” IEEE Transactions
on Image Processing, 2008.
Z. Xu and S. Jian, “Image in painting by
patch propagation using patch sparsity,”
IEEE Transactions on Image Processing,
Vol. 19, 2010, pp. 1153-1165
M. Oliviera, B. Bowen, R. Mckenna, and
Y.-S. Chang. Fast Digital Image Inpainting.
In Proc. Of Intl. Conf. On Visualization,
Imaging And Image Processing (VIIP), Page
261266, 2001.
Telea,”An Image In painting Technique
Based On The Fast Marching Method”,
Journal Of Graphics Tools, Vol.9, No. 1,
ACMPress 2004.
Naser Jawas and Nanik Suciati“ Image In
painting using Erosion and Dilation
Operation”, International Journal of
Advanced Science and Technology Vol.51,
February,2013
Usha Kiran, Om Prakash Yadav” Digital
Image In painting Using Cellular Neural
Network and Contour Tracking Using Run
Length Coding”, International Journal of
Advanced Research in Computer and
Communication Engineering Vol. 2, Issue 1,
January 2013
Firas A. Jassim“ Image In painting by
Kriging Interpolation Technique”, ISSN:
2221- 0741 Vol. 3, No. 5, 91-96, 2013.
H. Noori, S. Saryazdi, H. Nezamabadipour,” A Convolution Based Image In
painting”
Pranali Dhabekar, Geeta Salunke “The
Examplar-based Image
In painting
algorithm through Patch Propagation”,
International Journal of Recent Technology
and Engineering (IJRTE) ISSN: 2277-3878,
Volume-1, Issue-4, October 2012.
A. Criminisi*, P. P´erez and K. Toyama
“Region Filling and Object Removal by
Exemplar-Based Image In painting”, IEEE
TRANSACTIONS
ON
IMAGE
PROCESSING, VOL. 13, NO. 9, SEP 2004.
Shivali Tyagi, Sachin Singh, “ Image In
painting By Optimized Exemplar Region

www.ijera.com

[15].

[16].

[17].

[18.]

[19].

[20].

[21].

[22].

www.ijera.com

Filling Algorithm”, International Journal of
Soft Computing and Engineering (IJSCE)
ISSN: 2231-2307, Volume-2, Issue-6,
January 2013.
Xiaobao Lu, Weilan Wang, Duojie Zhuoma,
“A Fast Image In painting Algorithm Based
on TV Model”, IMECS 2010, March 17-19,
2010 Hong kong.
S. Esedoglu and J. Shen, “Digital In painting
Based on the Mumford-Shah-Euler Image
Model”, European Journal of Applied
Mathematics, 2002.
A. Bertozzi, S. Esedoglu, and A. Gillette,
“Analysis of a Two-Scale Cahn-Hilliard
Model for Image In painting”, Submitted for
publication in IEEE Trans. Image Proc.,
2006.
CELIA A. ZORZO BARCELOS, MARCOS
AUR´ELIO BATISTA, “Image Inpainting
and DE noising by Nonlinear Partial
Differential Equations”, IEEE Computer
Society Conference on Computer Graphics
and Image Processing, 2003.
A. Criminisi, P. Perez and K. Toyama,
“Region Filling and Object Removal by
Exemplar- Based Image Inpainting”, in
IEEE Transactions on Image Processing,
Vol. 13, No. 9, September 2004
S. Masnou, “Disocclusion: A Variational
Approach Using Level Lines”, IEEE
Transactions on Signal Processing, 11(2),
p.68–76, February 2002.
M. Oliveira, B. Bowen, R. McKenna, Y.
Chang,
“Fast
Digital
Inpainting”,
Proceedings of the International Conference
on Visualization, Imaging and Image
Processing, Marbella, Spain, pp. 261–266,
2001.
C. Ballester, M. Bertalmio, V. Caselles, G.
Sapiro, J. Verdera, “Filling–In by Joint
Interpolation of Vector Fields and Gray
Levels”, IEEE Transactions on Signal
Processing, 10(8) , p.1200–1211, August
2001.

88 | P a g e

N42018588

More Related Content

What's hot (19)

Viewers also liked (20)

Similar to N42018588 (20)

Recently uploaded (20)

N42018588