AUTOMATED IMAGE MOSAICING SYSTEM WITH ANALYSIS OVER VARIOUS IMAGE NOISE

International Journal on Computational Science & Applications (IJCSA) Vol.6, No. 3, June 2016
DOI:10.5121/ijcsa.2016.6202 13
AUTOMATED IMAGE MOSAICING SYSTEM WITH
ANALYSIS OVER VARIOUS IMAGE NOISE
Smriti Ayushi, Tejashwi Kalp Taru and Rajiv Kumar
Department of Computer Science and Engineering, Sharda University,
Greater Noida, Uttar Pradesh, India
ABSTRACT
Mosaicing is blending together of several arbitrarily shaped images to form one large balanced image such
that boundaries between the original images are not seen. Image mosaicing creates a large field of view
using of scene and the result image can be used for texture mapping of a 3D environment too. Blended
image has become a wide necessity in images captured from real time sensor devices, bio-medical
equipment, satellite images from space, aerospace, security systems, brain mapping, genetics etc. Idea
behind this work is to automate the Image Mosaicing System so that blending may be fast, easy and
efficient even if large number of images are considered. This work also provides an analysis of blending
over images containing different kinds of distortion and noise which further enhances the quality of the
system and make the system more reliable and robust.
KEYWORDS
RANSAC, SNR, Blending, Feature Extraction, Entropy
1. INTRODUCTION
Image mosaicing is the process of combining multiple overlapping images of same scene into a
larger image. A mosaiced image is the composition of sequence of images, it can be obtained by
finding out the coordinate relationship between the overlapping images. [1] Coordinate
relationship refers to the relationship between the pixels of one part of image with another part of
image which later on represents entire scene. The relationship depends on several parameters like
matching corners and matching pixel values. Hence to form an entire large mosaic image of a
scene, the pixel relationship must be transformed in such a way that can fit with other coordinates
of other part of the scene. This process is known as homography estimation. [2] Automated image
mosaicing system uses various procedures such as feature extraction, homography estimation and
blending. Feature extraction includes the process to find out the corners in the image, then these
corners must be translated in such a way that can fit with coordinates of another image they can
be transformed, compared and analyzed in a common reference frame. Finally using this
coordinate relationship the images are blended and final mosaiced image is obtained.
Mosaicing is the process of stitching several images together, hence the border and the matching
features of image must be handled properly in such a way that border between the images are
hidden and blended correctly. [3]There are lots of techniques which are used for the same purpose
such as “Projected Mask”. It is used to overcome the issues with cumulated errors between the

14
borders of images. In this technique coordinate transformation is used between successive pairs of
image patches. Hence this technique is highly resource and computation intensive. The basic
challenge behind stitching images is to detect the common feature between two successive
images like corners, edges and objects. After detection of such feature stitching is done by
projecting one image onto another. This technique is useful for a wide range of scope like [4]
microscopic image stitching to thermal image stitching but the core idea remains the same i.e.
figuring out the features in image and then projecting one image onto another image. One of the
novel idea that is also introduced in this process is recovering an image from noise, there are
some techniques used to remove noises like smoothing the image using filters, but the main
challenge remains same that there is a wide range of noise and system cannot remove every noise
by just using a single filter, also sometimes it is possible that in the process of noise removal,
image itself loses its features. This case is commonly caused due to the use of multiple filters or
wrong filter with wrong type of noise.Hence an automated system which blends images
irrespective of the quality and quantity of noise content has always been a requirement in this
field.
Image mosaicing [6-10] is a major research area from the starting era of digital image processing.
Anna Eivazi and Alexandar Kolesnikov in year 2015 have worked on Variance-preserving
mosaicing of multiple satellite images for forest parameter estimation: Radiometric
normalization. They have proposed a variance-preserving mosaic (VPM) algorithm that considers
all images at the same time, minimizes overall error of the normalization and aims to preserve
average variance of input images. Their proposed algorithm allows avoiding iterative pair-wise
normalization, results in visually uniform mosaics while maintaining also the original image
variance. Another remarkable work in this area is of B.Rousso and S. Peleg who have worked
over video mosaicing to increase the visual field of view by pasting together many video frames.
In year 2000, Shmuel Peleg, Benny Rousso presented a new methodology to allow image
mosaicing in more general cases of camera motion. They performed mosaicing by projecting thin
strips from the images onto manifolds which are adapted to the camera motion. Richa Singh, in
2007 provided A Mosaicing Scheme for Pose-Invariant Face Recognition. She described a face
mosaicing scheme that generates a composite face image during enrolment based on the evidence
provided by frontal and semiprofile face images of an individual. Some researchers have also
worked on Real-time Image Mosaicing. Nikhil Ranjan and Braj Bihari Soni chose RANSAC
(Random Sample Consensus Algorithm) as their major technique in achieving image mosaicing
and presented their work in year 2015.
Our system has been designed to use only one filter i.e. median filter as an optional step. It works
by replacing the current pixel with the cumulative sum of neighbouring pixels. It also preserves
the edge and features of an image in some cases. Hence it is possible to remove few noises from
the image and then we can detect features and blend images even in the presence of certain
quantity of noise.

15
2. METHODOLOGY
Figure1: Proposed model for image mosaicing system.
In the proposed model (as shown in Fig.1) the system will be using two images of a same scene to
create a mosaic image. For this purpose, it will be using few algorithms including Harris Corner
detection, Bidirectional cross correlation, Random Sample Consensus algorithm and alpha
blending.
2.1 Feature Extraction
For feature extraction, the system needs to find matching points between different images of an
environment. Due to the fact that if we know how two images relate to each other, we can
use both images to extract information of them. Matching points refers to marking
characteristics in the scene that can be recognized easily. [5] These are termed as
characteristics features. Feature defines both corners and edges but here we are concerned of
determining corners only. Corner detection is frequently used in motion detection, image
registration, video tracking, image mosaicing, panorama stitching, 3D modelling and object
recognition. Harris Corner Algorithm is used to detect corners in the input images. A corner can
be defined as the intersection of two edges.
Figure2: Flat region, edge and corner defined
Harris corner detector gives a mathematical approach to determine which of the above case (Fig
2) holds. Consider a grayscale image I. We are going to sweep a window w(x, y) with
displacement u in the x direction and v in the right direction and will calculate the variation of
intensity:
E(u,v) = ∑ x,y w(x,y) [ I(x+u , y+v) – I(x,y)]2
(Equation 1)

16
Where w(x,y) is the window at position (x,y), I(x,y) is the intensity at (x,y) and I(x+u, y+v) is the
intensity at the moved window (x+u, y+v). Shifting the window in any direction will yield a large
change in appearance. Since we are looking for windows with corners, we are looking for
windows with a large variation in intensity. Hence, we have to maximize the equation above,
specifically the term:
∑ x,y [I(x+u , y+v) – I(x,y)]2
(Equation 2)
2.2 Bidirectional Cross Correlation
Correlation Algorithm is used to relate the corners found in the input images after Harris corner
detection. Maximum bidirectional cross correlation rule has been used to determine matches
between two images. The cross-correlation works by analyzing a window of pixels around every
point in the first image and correlating them with a window of pixels around every other point in
the second image. Points which have maximum bidirectional correlation will be taken as
corresponding pairs. By using bidirectional cross correlation the designed system minimized the
chances of having fake correlated points. When one point from the first image gives a positive
correlation with another point of the second image, the system cross correlates that point with
perspective of image two to image one. Then system selects those points which scored maximum
cross correlation values.
Formulae for correlation ‘r’ can be expressed as:
r = (1/(n-1)) . [ (∑x ∑y ( x – x’) ( y –y’)) / (Sx .Sy) ] (Equation 3)
Where n is the number of pairs of data. x’ and y’ are the means of all the x-values and y-values
and Sx and Sy are the standard deviation of all x and y values. To find the standard deviation,
following equation can be used:
Sx = ( (∑ (x – x’)2
) / (n-1) ) 1/2
(Equation 4)
2.3 Random Sample Consensus Algorithm
Random sample consensus algorithm or RANSAC is a type of mathematical parameters model
estimation method from a set of data which contains outliers, hence it is also known as outliers’
detection method. [6] It is a non-deterministic algorithm because it produces a result only with
certain probability and with this the probability is increased if more iteration on the data set is
allowed. Data have inliers and outliers. Inliers are those data, whose distribution can be explained
by some set of model parameters, but it may be subject to noise. Outliers are data that does not fit
into any model, it may come from extreme values of noise. If fitting of a line is considered,
assuming two dimensions to a set of observations which contains both inliers that is points and
they have to be fitted in line. Outliers being points which cannot be fitted to this line. In this case
RANSAC can produce a model which is computed from the inliers, provided that the probability
of choosing only inliers in the selection of data is high. Due to probabilistic nature of RANSAC
there is no guarantee for this situation but there are number of algorithm parameters which can be
chosen to keep level of probability high.

17
Figure3: RANSAC selected best fit Probabilistic Model
2.4 Linear Gradient Alpha Blending
It is the process of combining an image with a background to create the appearance of partial or
full transparency. It is often useful to render image elements in separate passes, and then combine
the resulting multiple 2D images into a single, final image paragraphs must be indented. [7] The
alpha channel is a colour component that represents the degree of transparency (or opacity) of a
colour (i.e., the red, green and blue channels). It is used to determine how a pixel is rendered
when blended with another. The alpha channel controls the transparency or opacity of a colour.
Its value are 0 for full transparency whereas 1 for full opacity. When a colour (source) is blended
with another colour (background), e.g., when an image is overlaid onto another image, the alpha
value of the source colour is used to determine the resulting colour. If the alpha value is opaque,
the source colour overwrites the destination colour. If transparent, the source colour is invisible,
allowing the background colour to show through. If the value is in between, the resulting colour
has a varying degree of transparency/opacity, which creates a translucent effect. It is primarily
used in Alpha Blending. Our system uses linear gradient alpha blending from the centre of one
image to the other. The gradient blending works by simulating a gradual change in one image's
alpha channel over the line which connects the centres of the two images.
2.5 Algorithm
For Image (X, Y)
Step 1: Extract feature using equation 1 and 2.
Step 2: Feature matching done using correlation equation 3 and 4.
Step 3: Homography estimation using RANSAC rule.
Step 4: Blending done using linear gradient alpha blending rule.
3. EXPERIMENTS WITH NUMERICAL AND GRAPHICAL RESULTS
Dataset used for the implementation of algorithm and output of the algorithm is presented in
Table 1. Also the histogram for the data analysis of the input and output are presented in Table 2.

18
3.1 Observations and Results
Mosaicing System is successfully merging or combining two images which are having some
overlapped regions. (Refer Image set 1, 9, 10, and 11). Mosaicing System is taking care of proper
blending even if one of the images is distorted. (Refer Image set 2). System is taking care of
proper blending even is one of the image is subset of second image. (Refer Image set 2, 3, 6, 7,
and 8). System is taking care of proper blending even when intensity variation in closely
associated pixels is very high (Refer Image set 6). Example in this image, considering even a
small image matrix, intensity value of each pixel is highly different from other pixels in same
image matrix. System is taking care of proper blending even when second image is highly
zoomed-In version of first image. (Refer Image set 7). System is taking care of proper blending
even when one of the images is highly blurred subset of other image. (Refer Image set 8).
Mosaicing System is taking care of proper blending even when one of the images is highly
blurred subset of other image. (Refer Image set 11).
Signal to Noise ratio of the image input and output is tabulated in Table 3. [11] It can be observed
from the analysis that the output image has approximately the same SNR value as of the two
input images or if there is any decline in SNR value, it is up to a very smaller extent. Our
mosaicing tool does not impact the noise quality of images to a noticeable extent. The quality of
the input image that is provided to the system, the output image is given accordingly.
Entropy is the value to figure out the uncertainty i.e. nature of the image. Entropy value of input
and output image is tabulated in Table 4. Low entropy image will have very little contrast and
large number of pixels with similar pixel values, while high entropy images shows that image
contains high contrast and lots of difference in pixel values.
Blur metric is a measure to find out the level of blur in image. Blur metric measure is tabulated in
Table 5. This analysis is used to find out if the mosaicing process, especially the alpha blending
which is changing the alpha values of images is making any changes to the blur level of output
image or not.
Table 1. Output of Algorithm over Data Set
Sl.
No.
First Input image Second Input image Mosaiced output image
1
2
3

19
4
5
6
7
8
9
10
11
Table 2. Histogram of Input Data Set and Mosaiced Output
Sl.
No.
Input image 1 Input image 2 Mosaiced image
1

20
2
3
4
5
6
7
8
9

21
10
11
Table 3. Signal To Noise Ratio Analysis of Data
S. No SNR of Image One SNR of Image Two SNR of output image
1 12.8108 11.6365 12.8300
2 10.7148 10.7846 10.7207
3 11.6088 12.7368 11.6068
4 11.2705 12.6914 11.2692
5 10.3590 17.8004 10.7730
6 11.7764 9.6033 11.7975
7 10.6909 10.8503 10.6918
8 12.0465 12.0140 11.6369
9 10.4062 10.0790 10.5424
10 8.1255 8.5420 9.8013
11 10.2548 11.9724 11.1100
Table 4. Entropy Analysis of Data
S. No Entropy of Image One Entropy of Image Two Entropy of output image
1 7.3991 7.2723 7.3971
2 7.6945 7.5342 7.6961
3 7.2594 7.4733 7.2609
4 7.5594 7.3433 7.5591
5 6.5805 5.3791 6.5497
6 7.7342 6.8419 7.7305
7 7.1153 7.3418 7.1267
8 7.5890 7.6470 7.5521
9 6.8608 6.8492 6.9584
10 6.9543 7.1694 6.9584
11 7.1278 6.5489 7.1551
Table 5. Blur Metric Analysis of Data
S.
No
Blur Metric of Image One Blue Metric of Image Two Blur Metric of output
image
1 11.3451 11.4349 10.2372
2 15.1143 15.4973 14.8458
3 12.0988 11.3454 10.8457
4 10.9457 11.0396 10.3945
5 13.1377 13.1102 12.8236

22
6 9.4343 10.2329 9.1212
7 12.4397 11.9326 11.2737
8 11.2323 11.9993 11.1110
9 10.4931 11.0033 10.0901
10 14.2483 13.9878 13.1974
11 11.1216 11.9363 10.5785
Figure 4: Graphical analysis of snr data, entropy data and blur metric data as tabulated above.

23
4. CONCLUSIONS
All results clearly demonstrate that our system provides efficient mosaicing irrespective of
various types of distortion present in image. Three image quality assessment techniques, namely
SNR, Entropy and Blur Metric have been used. All of the result shows one final and concrete
point that the quality of output image depends on the quality of input image supplied to the
system. Hence all the algorithms that is used in this project does not effect in a high ratio the
quality of result. Value of SNR is controlled in the output image if the value of SNR in input
image is high, but this feature does not mean that the system will control a very high amount of
SNR in the output image. Image Histogram comparison is done which stated that contrast is
almost preserved as the input images and output image have nearly same histogram patterns.
Similarly with the entropy and blur metric case their values come up to be approximately same or
little declined when input and output are compared statistically and graphically. Hence perfect
blending is achieved with single seamless mosaiced output for any kind of arbitrarily shaped
input images (which can be taken from different angle too). A future work can be performed
under this which can compare the results of the already existing mosaicing systems with the
mosaicing tool built under this project. Quality assessment can be done from the mosaiced output
of each system and their accuracy can be compared and significantly judged. Further the time and
space complexity of the techniques and algorithms that are used in the project at each step for the
processing of image can be found out and that time and space complexity can be tried to reduce
as well. This Image Mosaicing that has been discussed can also be applied on 3D images in
future.
REFERENCES
[1] Anna Eivazi And Alexander Kolesnikov, “Variance-Preserving Mosaicing Of Multiple Satellite
Images For Forest Parameter Estimation: Radiometric Normalization”, ISPRS Journal Of
Photogrammetry And Remote Sensing, ELSEVIER, 2015.
[2] Dr. Ramesh B Inampudi, “Image Mosaicing”, IEEE, 1998, 0-7803-4403-0/98.
[3] Shunichi Takeuchi, “Image Mosaicing Without Distortion Using Projected Mask For Image
Digitization”, IEEE, 0-7803-5467-2/99, 1999.
[4] Amar K. Bendale And Kantilal P. Rane, “Μ-Mosaicing And Thermo-Mosaicing Using Image In-
Painting”, International Journal Of Computer Applications (0975 – 8887) Volume 95– No.21, June
2014.
[5] Darya Frolova, Denis Simakov, “Harris Corner Detector: Matching With Invariant Features”,The
Weizmann Institute Of Science, March 2004.
[6] Tsiombikas John, “An Efficient Technique For Image Mosaicing Using Random Sample Consensus
Algorithm”, International Journal Of Computer Applications (0975 – 8887) Volume 118 – No.16,
May 2015.
[7] Shmuel Peleg, Benny Rousso, Alex “Mosaicing On Adaptive Manifolds”, IEEE Transactions On
Pattern Analysis And Machine Intelligence, Vol. 22, No. 10, October 2000.
[8] Richa Singh, Student Member, IEEE, Mayank Vatsa, Student Member, IEEE, Arun Ross, Member,
IEEE, And Afzel Noore, Member, IEEE, “A Mosaicing Scheme For Pose-Invariant Face
Recognition”. IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICS—PART B:
CYBERNETICS, VOL. 37, And NO. 5, OCTOBER 2007.

24
[9] Marc Vivet, Brais Martinez, Student Member, IEEE, And Xavier Binefa, “DLIG: Direct Local
Indirect Global Alignment For Video Mosaicing”, IEEE TRANSACTIONS ON CIRCUITS AND
SYSTEMS FOR VIDEO TECHNOLOGY, VOL. 21, NO. 12, DECEMBER 2011.
[10] B. Russo, S. Peleg, “Universal Mosaicing Using Pipe Projection”, Institute Of Computer Science, The
Hebrew University Of Jerusalem 91904 Jerusalem, ISRAEL.
[11] Rajiv Kumar, Amresh Kumar, Pervez Ahmed, “A Benchmark Dataset For Devnagari Document
Recognition Research”, 6th International Conference On Visualization, Imaging And Simulation
(VIS’13) Pages 258-63

AUTOMATED IMAGE MOSAICING SYSTEM WITH ANALYSIS OVER VARIOUS IMAGE NOISE

More Related Content

What's hot (20)

Similar to AUTOMATED IMAGE MOSAICING SYSTEM WITH ANALYSIS OVER VARIOUS IMAGE NOISE (20)

Recently uploaded (20)

AUTOMATED IMAGE MOSAICING SYSTEM WITH ANALYSIS OVER VARIOUS IMAGE NOISE