Detection of Fruits Defects Using Colour Segmentation Technique

Detection of Fruits Defects Using Colour Segmentation
Technique
Md. Imran Hosen
Department of CSE
Jahangirnagar University
Dhaka, Bangladesh
imranjucse21@gmail.com
Tahsina Tabassum
Department of CSE
Dhaka, Bangladesh
tabassumankhi1993@gmail.com
Jesmin Akhter
Department of IIT
Dhaka, Bangladesh
togorcse@gmail.com
Md. Imdadul Islam
Department of CSE
Dhaka, Bangladesh
imdad@juniv.edu
Abstract—In image processing colour segmentation is used to
extract features of an object in both special and frequency
domains. The objective of this paper is to use colour segmentation
technique to identify the defected region of fruits and
corresponding percentage of frequency components from its
Spectrogram. Here we separate the defective portion of fruit
using colour segmentation technique taking four images from
four directions to get the appropriate result of 3D images. The
percentage of the defective portion is determined using
scatterplot of the colours of the image. Next, we apply the similar
concept to spectrogram of an image (even applicable in speech
signal) to extract the percentages of frequency components of the
signal.
Keywords-Scatter plot, mean Euclidian distance, L*a*b image,
Spectrogram of image and spectral components of speech signal.
I. INTRODUCTION
The value of fruits depends on its quality hence retailers
categorize various fruits according to their quality. For
example in the European Community, apples have three
categories in fresh apple market. Fruits with no defects are in a
class called 'extra'. Fruits with little defects are placed in class
I. If the defects are too large, these fruits are considered as a
Class II category and these types of fruits are rejected as given
in [1]. To meet the increasing demand of high-quality
products, fruits are graded before being sent to market or for
further processing analyzed in [2]. However, manually
identification of fruit‟s defects requires more time as well as
cost. This type of process could be done automatically with
the help of computer vision systems. To identify the defected
region, the first step is to segment the defect.
Image segmentation means separation of an image in
different region based on its properties. The goal of
segmentation is to simplify and change the description of an
image into something that is more suggestive and easier to
analyze as explained in [3-4]. The colour is one of the
properties which bears information of the image and colour
based image segmentation has wide applications. It is
convenient to identify different colours in an image on L*a*b
colour space instead of conventional RGB component.
For automating detecting of fruits defects, many types of
research have been done using the computer vision system, but
it is still challenging task due to various types of defects,
shape, the presence of stem and so on described in [5].
In this paper, we proposed a very efficient and quick
segmentation technique based on colour segmentation. Our
work carried out several stages; the steps of algorithm is show
in next section. In the frequency domain, each colour
indicates the different frequency components. Therefore,
getting the spectrogram of an image we can separate the
sequences of colour components which resembles to different
frequency components. That‟s why it is easy to identify the
defected part at a glance as given in [6].
Histogram based image segmentation technique is
computationally very efficient when compared to other image
segmentation techniques because they usually require only a
single pass through the image pixels inspected in [1]. Many
colour models are used to represent the colour like RGB,
CMY, HSV, HSL; an effort is made to defeat the problems
encountered while segmenting an onset by using the colour
properties of the image explored in [3]. A region growing
algorithm typically starts with some seed pixels in an image
and from these, it grows regions by iteratively adding
unassigned neighboring pixels that satisfy some homogeneity
criterion with the existing region of the seed pixel found in[7].
In paper [8] the authors proposed a new quantization method
for HSV colour space to create a colour histogram and a gray
histogram for K-means clustering which operates across
different dimension in HSV colour space. Image acquisition is
the process of acquiring an image from some hardware-based
sauces in which the output image can be used for further
processing that is analyzed in[9]. The choice of colour space
representation could be taken to enhance the performance of
processes such as segmentation because of the increment in
demand for the colour-driven images as compared to grayscale
images inspected in [10-11]. A hybrid method for colour
segmentation based on seeded region growing in which the
initial seeds are provided by a conservative threshold colour
segmentation found in [12]. Creating code elements on the
description hexagonal hierarchical structure each island has
one or more so-called code elements as explored in[13].
Identifying of fruit defects based on the selection of image
region and object offering has been proposed in [14]. A
International Journal of Computer Science and Information Security (IJCSIS),
Vol. 16, No. 6, June 2018
215 https://sites.google.com/site/ijcsis/
ISSN 1947-5500

technique has been proposed which can isolate the healthy
parts of olive fruits as well as the actual defected region
analyzed in[15]. A computer vision method has been used to
identify the grade quality of agricultural products described in
[16].
The entire paper is organized as: section II deals with
theoretical analysis and algorithm of colour segmentation of
fruits, section III provides result based on analysis of section II
and chapter IV concludes entire analysis with some future
work plan.
II. BASIC THEORY OF L*A*B COLOUR SPACE
It is three axis colour system; where the first axis is L
channel or lightness, goes up and down the three-dimensional
model and consists of white and black. When L* = 0, it
indicates the darkest black and L* = 100 indicates the brightest
white found in [10]. The axis a* indicates where the colour
falls along the red to green axis, the negative value of a*
indicates green, and the positive value of a* indicates
magenta. Along b* axis the colour runs between the blue to
yellow. Positive values of b* represents Yellow and the
negative values of b* indicate Blue. When the channels a*=0
and b* = 0, these represent the true neutral gray. As L*a*b
model is a three-dimensional model, so it can only be
represented accurately in a three-dimensional space. The
formula for converting digital images from RGB space to the
L*a*bare given below.
𝐿 ∗= 116𝑓
𝑌
𝑌𝑛
− 16
𝑎 ∗= 500[𝑓
𝑋
𝑋 𝑛
− 𝑓(
𝑌
𝑌𝑛
)
𝑏 ∗= 200[𝑓
𝑌
𝑌𝑛
− 𝑓(
𝑍
𝑍 𝑛
)
(1)
𝑓 𝑥 =
x
1
3 ; if x > 0.620
7.787x +
16
116
; Others
;Where X, Xn, Y, Yn, Z, Zn are the coordinates of CIEXYZ
colour space.
For many digital image manipulation, L*a*bcolour space is
more suitable than the RGB colour space since it is device
independent.
A. Colour segmentation of image
Generally colour is the most important and influential
attribute of fruits quality. Numerous defects of fruits appear as
discolouration on surface as mentioned in [2]. Colour based
image segmentation means the image will be separated
according to colour. The primary aim is to identify different
colours in an image by analyzing the L*a*b colour space.
Segmentation of an image is referred to separate the image
into a non-overlapping region based on some feature as given
in[17]. Colour image segmentation simplifies the vision
problem by assuming that objects are coloured distinctively;
where the gross colour differences matters. In our paper, we
concentrated on the colour since it is easy to find out defected
region according to colour variation.
B. Algorithm
Algorithm for the proposed work is given below.
Step 1. Read the RGB image.
Step 2. Convert the image into L*a*b.
Step 3. Select a region of a particular colour of L*a*b image.
Step4.Take the average value of the pixels excluding
luminance component. Let the average value is (ua, va) = za.
Step 5. If the magnitude of ith
pixel of the image is (ui, vi) = zi.
Evaluate the Euclidian distance,||za – zi || =Di
Step 6. If Di≤ τ; (τ is a threshold value of Di), Then select the
pixel, otherwise, ignore it.
Step 7. Repeat step 5 and 6 for all pixels of the image.
Step 8. Now show the image for pixels satisfies steps 5 to 7.
Step 9. Repeat steps 3 to 8 for all the required colours.
Step 10. Draw the scatter plot of all colours on a – b axis.
Step 11. Repeat steps 1 to 10 for another image of same class.
Step 12. Determine to mean Euclidian distance between the
pixels of a particular colour (for example purple) on scatter
plot. If it is less than the threshold, then the 2nd image is
identical with the 1st one.
C. Scatter plot of segmented image
Sometimes instead of x-y, two orthogonal basis functions:
φ1(x) and φ2(x) are used along x and y direction; where the
cross-correlation between them are zero i.e. < φ1(x) ,φ2(x)> =
0. The signal component correlated with φ1(x) gives abscissa
and that of with φ2(x) gives ordinate. In colour segmentation,
we separate the 'a*' and 'b*' components of each pixel then
plot a* level values along horizontal axis and b* level along
the vertical axis as found in [18-19]. By counting different
colour points from the scatter plot, defected portion is
detected.
D. Spectrogram of a signal
The spectrogram is usually represented on a two
dimensional plane where the horizontal and vertical axis
represents time and frequency; a third dimension indicating
the value of a particular frequency at a given time is depicted
by the intensity or colour of each point in the image [20-21].
The defected region is usually darker than the original colour
of fruits hence the spectrogram will provide distinct region.
Vol. 16, No. 6, June 2018
ISSN 1947-5500

Figure 1. Spectrogram of audio signal
Figure 2. Segmented image of different frequency
Figure 3. Scatter plot of speech signal
For example a sound wave in time domain varies with
amplitude and each segment of the wave possesses different
frequency components. Applying simple Fourier transform
will provide frequency components but time information will
be lost. Applying short time Fourier transform on each
segment will preserve both time and frequency components.
The graphical presentation of such phenomenon is actually the
spectrogram as shown in fig.1. The colour segmentation and
scatter plot of the spectrogram are shown in fig. 2 and 3.
III. RESULT AND DISCUSSION
In this section, we detect the faulty portion of a fruit using
colour segmentation technique along with the scatter plot of
the different portion of a fruit. In the second technique, we did
the similar job using spectrogram of the image. In Fig.4 four
images are shown for a particular region of a defective banana.
The first image of the figure shows the original image, the
second image provides a healthy portion of the image, the
third image shows the partially defected portion and the fourth
image reveals the fully rotten portion of the image. The Fig.5
shows the scatter plot of the corresponding region of the
image. Here we show three colours (green colour represents
the faultless portion, the colour of partially defected portion is
represented by yellow and red colour represents the fully
rotten portion) to acquire the percentage of partially or fully
defected portion of the defective fruits.
Next, we take a similar image from four sides of 3-D fruits
then measure the percentage of the defective portion from
individual scatter plot. Then four scatter plots are summed up
to get the illusion of scatter plot of the 3-D image of a defected
fruit. The percentage of individual colour are measured from
the combined data of four scatter plot. The corresponding
figures are shown in Fig. 6-11. Next, we apply similar
operation on oranges and apples shown in Fig. 12-27. The
entire result of above analysis is shown in table 1.
Next part of the result section, we consider different colours
of an image using its spectrogram. In this paper we only
provide the guideline of colour segmentation of a spectrogram
but detail analysis will be done in future. For simplicity of
analysis we show the spectrogram of grayscale image of
defective fruit in fig. 28 (a) and (b). In real life situation, we
have to separate a and b components of the image first then
spectrogram of both component will be taken. The scatter plot
of each spectrogram will be count to get the real scenario of
the rotten fruit. However, in spectrogram different colour or
frequency of an image are separated along frequency axes,
hence separation of colour is a little bit easier to compare to
the original image. Therefore, we expected to get more
accurate result from the spectrogram of an image. This
analysis will be performed in details in future and expect to
make compares in future. The concept is applicable in
biometric identification.
0 0.5 1 1.5 2 2.5 3
x 10
5
-2
-1
0
1
2
t
x(t)
Speech Signal
Time
Frequency
Spectrogram of Speech Signal
5 10 15 20 25 30
0
1000
2000
3000
4000
image Blue
Red Yellow
110 120 130 140 150 160 170 180 190 200 210
40
60
80
100
120
140
160
180
200
220
Scatterplot of the segmented pixels in 'a*b' space
'a' values
'b'values
Vol. 16, No. 6, June 2018
ISSN 1947-5500

Figure 4. Segmented image of banana of side 1
Figure 5: Scatter plot diagram of banana of side 1
Figure 7. Scatter plot diagram of banana of side 2
Vol. 16, No. 6, June 2018
ISSN 1947-5500

Figure 12. Segmented image of orange of side 1
Figure 13. Scatter plot diagram of orange of side 1
Vol. 16, No. 6, June 2018
ISSN 1947-5500

Figure 20. Segmented image of apple of side 1
Figure 21. Scatter plot diagram of apple of side 1
Vol. 16, No. 6, June 2018
ISSN 1947-5500

Vol. 16, No. 6, June 2018
ISSN 1947-5500

Table 1
Percentage of slightly defected and rotten portion of fruits
Fruits Name Slightly Defected Portion Rotten Portion
Banana 22% 14%
Orange 42% 20%
Apple 13% 45%
(a) Banana (b) Orange
Figure 28. Spectrogram of fruit
IV. CONCLUSION
We took four images from four directions to get the illusion
of 3-D image but due to few overlapping region, there is some
error in the numerical result. To overcome the situation 3-D
image can be used directly but the system model will be very
complicated. The concept of the paper can be applied to
identify the disease of crop fields and percentage of
contamination of field, the condition of the flower of mango,
percentage of damage of an object, the condition of soil from
satellite map etc. Another application of the paper will be
separation of background from the foreground of video
frames. Several consecutive frames of a video file can be
analyzed, based on the concept of the paper to model the
background of the video in identification of a moving object
from the foreground.
REFERENCES
[1] V. Leemans, H. Magin and M. F. Destain, “Defects segmentation on
„Golden Delicious‟ apples by using colour machine vision,” Computers
and Electronics in Agriculture, vol. 20, no. 2, pp. 117–130, July1998.
[2] S. Mittal,Computerized control systems in the food Industry, image
processing and its applications in food process control, CRC press, New
York, 1996.
[3] D. Napoleon, A. Shameena and R. Santhosh, “Colour image
segmentation using OTSU method and colour space,” International
Journal of Computer Applications,vol. ICICIC 2013, no. 1, pp. 5-8,
Mar.2013.
[4] M. Pereyra and S. McLaughlin, “Fast unsupervised bayesian image
segmentation with adaptive spatial regulation,” IEEE Transaction on
Image Processing, vol. 26, no 6, pp. 2577-2587, 2017.
[5] S. R. Dubey and A. S. Jalal, “Adapted approach for fruit disease
identification using images,” International Journal of Computer Vision
and Image Processing, vol. 5, 2002.
[6] A. Stann, C. V. Botinhao and B. Orza, “Blind speech segmentation using
spectrogram image-based feature and mel cepstral coefficients,” In:
Proc. of Spoken Language Technology Workshop (SLT), IEEE, San
Diego, CA, USA, Dec.13-16, 2016.
[7] N. Ikonomakis, K. N. Plataniotis and A. N. Venetsanopoulos , “Colour
image segmentation for multimedia application,” Journal of Intelligent
and Robotics System, vol. 28, no. 1-2, pp.5-20, June 2000.
[8] D. J. Bora, A. K. Gupta and F. A. Khan, “Comparing the performance of
L*A*B and HSV colour space on colour image segmentation,”
International Journal of Emerging Technology and Advanced
Engineering, vol. 2, no 2, Feb. 2015.
[9] L.J.Rozario, T. Rahman and M. S. Uddin, “Segmentation of the region
of defects in fruits and vegetables,” International Journal of Computer
Science and Information Security, vol. 14,no. 5, pp. 398-406,2016.
[10] V.S. Rathore, M. S. Kumar and A. Verma, “Colour based image
segmentation using L*A*B colour space based on genetic algorithm,”
International Journal of Emerging Technology and Advanced
Engineering, vol. 2, no 6, pp. 152-162, 2012.
[11] D. J. Bora and A. K. Gupta “A Novel Approach towards clustering
based image segmentation,” International Journal of Emerging Science
and Engineering, vol. 2, no. 11, pp. 6-10, 2014.
[12] Z. Wasik and A. Saffiotti, “Robust colour segmentation for the robocup
domain,” In: Proc. of the Int. Conf. on Pattern Recognition (ICPR),
August 11-15, 2002,vol. 2, pp.651-654.
[13] Gy. Dorko, D. Paulus and U. Ahlrichs, “Colour segmentation for scene
exploration,” In: Proc. of Workshop Farbbildverarbeitung, Berlin,
Germany, October 2000.
[14] H. Kuang, C. Liu, L. L. H. Chan and H. Yan,” Multi-class fruit detection
based on image region selection and improved object proposals,”
Neurocomputing , col. 283, pp. 241-255, Mar. 2018.
[15] N.M. Hussain and A.A. Nashat.(2018, March).” New effective
techniques for automatic detection and classification of external olive
fruits defects based on image processing techniques”. Multidimensional
Systems and Signal Processing. [Online] pp. 1-19. Available:
https://link.-springer.com/article/10.1007%2Fs11045-018-0573-5.
[16] K. Zhang, X. Chen ,H. Wang.(2018, March). “Research on External
Quality Inspection Technology of Tropical Fruits Based on Computer
Vision”. Recent Developments in Data Science and Business
Analytics.[Online] pp 165-174 Available: https://link.springer.-
com/chapter/10.1007%2F978-3-319-72745-5_18.
[17] M. Zand, S. Doraisamy, A. A. Halin and M. R. Mustafa, “Ontology
based segmentation image segmentation using mixture models and
multiple CRFs,” IEEE Transaction on Image Processing, vol. 25, no
7,pp. 3233-3248, 2016.
[18] H. Janetzko, M. C. Hao, S. Mittelstadt, U. Dayal and D. Keim,
“Enhancing Scatter Plots Using Ellipsoid Pixel Placement and
Shadding,” In: Proc. of 46th Hawaii International Conference on System
Science, Wailea, Maui, HI, USA, January 7-10, 2013, pp. 1522-1531.
[19] A. Mayorga and M. Gleicher, “Splatterplots: overcoming overdraw in
scatter plots,” IEEE Transactions on Visualization and Computer
Graphics, vol. 19, no 9, pp.1526-1538, 2013.
[20] R. Decorsiere, P. L. Sondergaard and E. N. MacDonald, “Inversion of
auditory spectrograms, traditional spectrograms, and other envelope
representations,” IEEE/ ACM Transaction on Audio, Speech and Lang.
Processing, vol. 23, no. 1,pp. 46-56, 2015.
[21] A. Rizal, R. Hidayat and H. A. Nugroho, “Lung sounds classification
using spectrogram‟s first order statistics features,” In: Proc. of 6th
International Annual Engineering Seminar (InAES), Yogyakarta,
Indonesia, August 1-3, 2016, pp. 96-100.
Md. Imran Hosen has completed his B.Sc. in Computer Science
and Engineering from Jahangirnagar University, Savar, Dhaka,
Bangladesh in 2017. He is doing his M.Sc. in the same
University. His research interests include Image Processing and
Machine Learning, Artificial Intelligence, Wireless
Communication, and Computer Networking.
Tabassum Ankhi has completed her B.Sc. in Computer Science
and Engineering from Jahangirnagar University, Savar, Dhaka,
Bangladesh in 2017. She is doing his M.Sc. in the same
University. Her research interests include Image Processing and
Machine Learning, Artificial Intelligence, Wireless Communication, and
Computer Networking.
Original RGB Image Original Grayscale Image
Time
Frequency
0.5 1 1.5
0
500
1000
1500
2000
2500
3000
3500
4000
Original RGB Image Original Grayscale Image
Time
Frequency
0.5 1 1.5
0
500
1000
1500
2000
2500
3000
3500
4000
Vol. 16, No. 6, June 2018
ISSN 1947-5500

Jesmin Akhter received her B.Sc. Engineering degree in
Computer Science and Engineering from Jahangirnagar
University, Savar, Dhaka, Bangladesh in 2004 and M.Sc
Engineering degree in Computer Science and Engineering from
Jahangirnagar University, Savar, Dhaka, Bangladesh in 2012.
Since 2008, she is a faculty member having current designation "Associate
Professor" at the Institute of Information Technology in Jahangirnagar
University, Savar, Dhaka, Bangladesh. Her research areas are on network
traffic, complexity and algorithms and software engineering. Now she is
pursuing PhD at the Department of Computer Science and Engineering,
Jahangirnagar University, Dhaka, Bangladesh in the field of 4G wireless
networks.
Md. Imdadul Islam has completed his B.Sc. and M.Sc
Engineering in Electrical and Electronic Engineering from
Bangladesh University of Engineering and Technology, Dhaka,
Bangladesh in 1993 and 1998 respectively and has completed his
PhD degree from the Department of Computer Science and
Engineering, Jahangirnagar University, Dhaka, Bangladesh in the field of
network traffic in 2010. He is now working as a Professor at the Department
of Computer Science and Engineering, Jahangirnagar University, Savar,
Dhaka, Bangladesh. Previously, he worked as an Assistant Engineer in Sheba
Telecom (Pvt.) LTD, a joint venture company between Bangladesh and
Malaysia from September 1994 to July 1996. His main duty was to design and
planning of Mobile Cellular Network and Wireless Local Loop for southern
part of Bangladesh. Md. Imdadul Islam has good field experience in
installation Radio Base Station and configuration of Switching Centers for
both mobile and WLL. His research field is network traffic, wireless
communications, cognitive radio, wavelet transform, OFDMA, adaptive filter
theory, algorithms, ANFIS and array antenna systems. He has more than
hundred and seventy research papers in national and international journals and
conference proceedings.
Vol. 16, No. 6, June 2018
ISSN 1947-5500

Detection of Fruits Defects Using Colour Segmentation Technique

More Related Content

What's hot (20)

Similar to Detection of Fruits Defects Using Colour Segmentation Technique (20)

Recently uploaded (20)

Detection of Fruits Defects Using Colour Segmentation Technique