Content-Based Image Retrieval Using Modified Human Colour Perception Histogram

Natarajan Meghanathan, et al. (Eds): ITCS, SIP, JSE-2012, CS & IT 04, pp. 229–237, 2012.
© CS & IT-CSCP 2012 DOI : 10.5121/csit.2012.2121
Content-Based Image Retrieval Using
Modified Human Colour Perception
Histogram
Shaila S.G and A.Vadivel
Department of Computer Applications, National Institute of Technology,
Tiruchirappalli 620 015, India, Fax: +91-431-2500133
shaila@nitt.edu vadi@nitt.edu
Abstract
This paper proposes Modified Human Colour Perception (MHCPH) based on human visual
perception. The colour and gray weights are distributed to neighbouring bins smoothly with
respect to pixel information. The amount of weight distributed to the neighbouring bins is
estimated using NBS distance, which is for human visual perception of colour. This distribution
makes it possible to extract the background colour information effectively along with the
foreground information. The low-level feature of all the database images are extracted and
stored in feature database. The relevant images are retrieved for a query based on the similarity
ranking between the query and database images. In this work, Manhattan distance is used as
distance metric. The experimental results are promising and show that the proposed approach
identifies relevant images based on the level of smooth distribution even for an image with
complex background colour.
Keywords
Content based image retrieval, Human colour perception, HSV colour space, Smooth
distribution, MHCPH histogram, NBS distance.
1. Introduction
Various techniques have been proposed in academia and industry for image retrieval applications.
These approaches can roughly be classified into three categories such as text-based retrieval,
content-based retrieval and semantic-based retrieval. In the text-based retrieval technique, each
image has a number of keywords for describing the image and the keyword based matching is
performed to retrieve relevant images. In content-based image retrieval applications, various
well-known low-level features like colour, texture and shape are extracted for describing the
image semantics. In the semantic-based retrieval technique, semantic is used to retrieve relevant
images. In recent years, attention is focused by researchers on Content-Based Image Retrieval
(CBIR), which is a sub-problem of content-based retrieval. It is noticed that the size of the image
database used in image retrieval is increasing exponentially and hence it is necessary to propose
and use effective tools for retrieving images. Well-known and most popular image retrieval
systems are QBIC[13], NeTra[11], PicToSeek[6], Blobworld[1], etc.

230 Computer Science & Information Technology (CS & IT)
In this paper, our goal is to retrieve set of images that are similar to a given query image. In image
retrieval applications, it has been observed that the colour histogram based approach is well
suited, since colour matching generates the strongest perception of similarity to the human eye. It
is often represented in the form of a histogram, which is a first-order statistical measure that
captures the global distribution of colour in a given image. This can be represented in various
forms such as colour histogram [15], colour moments and cumulative colour histogram [16]. A
colour histogram may be generated in the RGB colour space, HSV colour space, YCbCr colour
space, etc. Since, the RGB colour space is having some drawbacks such as, it does not explicitly
distinguish between colour and intensity components, other colour space like HSV, YCbCr,
which separate saturation and intensity components are frequently used. RGB values by suitable
conversions may be computed to get the values for each component in HSV colour space. Some
of the recently proposed colour representatives are colour saliency histogram [23], which
supports human visual attention principle and based on the features of colour, orientation and
intensity and followed by the difference of Gaussians and normalization processing, the
comprehensive saliency map of an image is generated. Recently, circular ring histogram[22] has
been proposed, which has the spatial information. The image is segmented initially using group of
circular rings and then the histogram is constructed using the segmented rings. The statistical
feature of image blocks has been extracted for representing the colour of blocks, which is named
as Order Based Block Colour Feature[19].In this approach image is divided into 48 blocks and
the feature is extracted. In our proposed work, we modify the HCPH histogram [21] to capture the
colour information in high-dense background of images for retrieval applications. The weight
calculated is distributed to the neighbouring bins based on NBS distance and the performance of
the modified HCPH is encouraging while retrieving high-dense background images.
The outline of this paper is as follows. Section 2 presents the related works and proposed
technique is explained in Section 3. In Section 4, the experimental results are presented and we
conclude the paper in the last section
.
2. Related Works
In colour-based image retrieval, there are primarily two methods: one based on colour layout[18]
and the other based on colour histogram[3]. In the colour layout approach, two images are
matched by their exact colour distribution. This means that two images are considered close if
they not only have similar colour content, but also if they have similar colour in approximately
the same positions. In the second approach, each image is represented by its colour histogram. A
histogram is a vector, whose components represent a count of the number of pixels having
similar colours in the image. Thus, a colour histogram may be considered to be a signature
extracted from a complete image. Colour histograms extracted from different images are indexed
and stored in a database. During retrieval, the histogram of a query image is compared with the
histogram of each database image using a standard distance metric like Euclidean distance or
Manhattan distance. Since, colour histogram is a global feature of an image, the approaches
based on colour histogram are invariant to translation, rotation and scale.
Various techniques has been proposed to represent the colour of an image [2][7][9][10][14][17].
It is found that robust histogram construction scheme using the HSV colour space in which a
perceptually smooth transition is captured based on the Human Visual Perception of colour
(HCPH) can represent semantic information up to a certain degree, due to the complexity in
background [21]. In colour saliency histogram [24], extraction of salient regions is based on the
bottom-up visual attention model. Although it has introduced prior knowledge in the model, the
bottom-up attention is only suitable for the primary stage of visual perception, it has limitations.
Order-based Block Colour Feature [19] is one type of image’s colour feature. It has an advantage
of the local colour statistical information but it has some bionic traits that this colour feature

Computer Science & Information Technology (CS & IT) 231
cannot alone suffice for CBIR. It has to be combined with other features such as shape, texture,
and so on. Hence, Modified HCPH (MHCPH), which is an improved version of HCPH is
developed. In HCPH, each pixel is divided into true colour component and grey colour
component, in a single iteration. But in MHCPH, we distribute the true colour components and
grey colour components into many levels thus distributing the weights to neighbouring bins based
depending on the weight values and NBS distance.
3. Modified HCPH
Pixels in a image can be represented with the combination of Hue, Saturation and Intensity Value
in the HSV colour space. It is a three dimensional hexacone representation and the central vertical
axis is intensity, I. Hue is an angle in the range [0,2π] and is relative to the red axis with red at
angle 0, green at 2π/3, blue at 4π/3 and red again at 2π[23] respectively. Saturation, S, is
measured as a radial distance from the central axis with value between 0 at the centre to 1 at the
outer surface. This is represented in Fig.1.
Fig 1. HSV Colour Model
The pixel colour is approximated by its intensity or by its hue, say for example while the intensity
is low and saturation is high, a pixel colour is very much close to the grey colour. Similarly for
other combination of intensity and saturation, we can approximate the pixel value the other way.
Therefore, the saturation and intensity values of a pixel are used to determine whether it can be
treated as a true colour pixel or a gray colour pixel. It is emphasized that this approach treats the
pixels as a distribution of “colours” in an image where a pixel may be of a “gray colour” (i.e.,
somewhere between black and white, both inclusive) or of a “true colour” (i.e., somewhere in the
red, green, blue, red spectrum). The reason is that, for an observer, this is what an image
represents is a collection of points having colours – red, yellow, green, blue, black, gray, white,
etc.
In our proposed work, colour information of each pixel is converted to HSV colour space. An
image pixel contains true colour components, in which the dominant factor is hue. Intensity is the
dominant factor for grey colour components. In the first iteration, we calculate true colour
components and grey colour components and they are smoothly (i.e., iteratively) distributed,
based on hue and intensity values of neighbouring pixel. This is shown in Fig.2.

Fig 2. Distribution of true colour and grey colour components
As noticed form Fig. 2, the weights are distributed to the neighbouring bins. The colour
information obtained in first level is taken as reference and is distributed smoothly into more
iteration. In this paper, we have considered five iteration levels. NBS distance [23] is calculated
using true colour and grey colour for distributing the pixel weight to the neighbouring bins. The
distance between true colour weight of reference pixel and its iterative distribution with adjacent
bins is calculated. Since, soft decision is done for a single pixel, saturation appears to be very
small and hence ∆S is considered as 0. The ∆I is considered as the distance calculation and is
only for true colour which is shown in Eq. (1).
( )yxd , = 1.2* ( )22
22 4
100
2
cos12 IS
H
yx ∆+∆+










 ∆
−
π (1)
In Table.1, we present the NBS distance value, which gives the colour difference with respect to
the human visual perception based on the distance range. Considering the colour difference and
distance, the weight of true colour is distributed. Since, true colour is smoothly distributed and
true colour weight also should be smoothly distributed. As per Eq. (1), for true colour, ∆S and ∆I
remains zero. This is due to the fact that the saturation and intensity distributed difference is
minimum with respect to a single pixel.
Table 1. NBS distance Table
NBS Value Human Perception
0~1.5 Almost the same
1.5~3.0 Slightly different
3.0~6.0 Remarkably different
6.0~12.0 Very different
12.0~ Different colour

For grey colour, ∆S and ∆H remains zero, since saturation and hue difference appears to be very
minimum. Based on these conditions, true colour weight and grey colour weights are distributed.
NBS distance is calculated to the adjacent bins of true colour, it is observed that the immediate
adjacent bin close to the reference bin will lie in the distance ranging from 0~1.5. As per the NBS
table it is “almost the same”. Thus 100% of true colour weight will be distributed to the
immediate adjacent bin. As smooth distribution of hue goes on, the NBS distance is calculated for
all the adjacent bins iteratively. We observe that while distance starts increasing, say in the range
of 1.6~3.0, there is a “slight colour difference”. We divide it into 3 groups with 15 iterations.
Now, the weight is distributed from 99 to 85. When the distance is measured with respect to the
adjacent bins in the range of 3.1~6.0, there is a “remarkable colour difference” range. Hence, we
have divided it into 6 groups with 30 iterations and weight distribution starts from 84 to 55. As
distance increases, the colour difference will be more, which may lie in the range of 6.1~12.0 and
is represented as “very different colour”. We divide it into 12 groups with 60 iterations and
weight distribution starts from 54 to 00, if distance is greater than 12.0, then it is totally different
colour and weight distribution is 0%.
The percentage of weight distribution to the adjacent bins is calculated using Eq. (2) and the same
is represented in the form of tree in Fig.3.
( )[ ] (2)10*5.1100),( −−= NBSSW ISH
Where NBS - True colour distance
During grey colour weight distribution, intensity is distributed iteratively or smoothly. However,
we need not apply NBS distance, since the colour difference is very minimum. Thus, we
distribute our grey colour weight using Eq. (3).
),(),( 1 ISHISI SWSW −=
(3)
Fig 3. Smooth distribution of true colour weight

4. Experimental Results
In CBIR application, Precision and Recall are used as parameters for evaluating the performance.
Precision is the ratio of the number of the relevant images retrieved to the total number of the
irrelevant and relevant images retrieved. Precision is defined as:
Precision ====
r
r
T
R
(4)
Where rR is number of relevant images retrieved. rT is the total images retrieved.
Recall is defined as the ratio of the number of relevant images retrieved to the total number of the
relevant images in the database. Recall is a measure of completeness.
Recall =
T
Rr
(5)
Where rR is number of relevant images retrieved. T is the total relevant images retrieved.
In order to evaluate the performance of the proposed scheme, coral benchmark database images
are used, which consists of 10 classes such as people, vehicle, building, flower, horses, etc. In
experiments, the HSV colour space is chosen for chromatic image processing. We have selected
large number of query images from different categories and retrieved the result for each query.
Average Precision and Recall is calculated and is shown in the Fig.4. From the figure ,It is
observed that for lower values of recall, the precision is getting higher, which is greater than 65%.
Similarly, for higher value of recall, the precision is comparable and the performance of the
proposed method is encouraging.
Fig 4. Average Precision versus Recall
In addition, we have compared the performance of the proposed method with DCT histogram
quantization algorithm [12] and are shown in Table.2. In the DCT Histogram Quantization, due to
the complex background with respect to a object, the efficiency in the retrieval rate of relevant
images has gone down to 40% in the sample dataset of elephant. However, for Dinosaurs
category, the precision of has increased to 99%. Here, the retrieval rate is highly dependent on the
content of background. However, the precision rate is not completely influenced by the
background content of the images on the proposed approach.

Table 2. Performance comparison between DCT Quantization and MHCPH
Class Average precision
MHCPH DCT
Dinosaur 0.86 0.99
Elephant 0.60 0.40
Since, weight is smoothly distributed to neighbouring bins based on the true colour and gray
colour weight, the entire information of the background as well as the object is captured and thus
the precision retrieval is 60%. Fig.5 and Fig.6 depicts sample retrieval result for both the
proposed and DCT based histogram. The image on the top centre is the query and the rest of the
image below is the retrieved image set.
Query Image
Fig 5. Sample Retrieval set using DCT Quantization.
Query Image
Fig 6. Sample Retrieval set using MHCPH
5.Conclusion
In this paper, we have proposed content based image retrieval technique using MHCPH. By
extracting the colour features of each pixel of the image, the smooth distribution is done with

respect to true colour and grey colour. Manhattan distance is used to measure the similarity
distance in order to retrieve the similar images from the database. In our work, we took only
colour feature and smoothly distributed its information for effective retrieval. However, further
increase in the iterations, will certainly improve the retrieval results with high precision and
recall. Thus, further care will be taken to balance both of these parameters. For future work, we
plan to improve the rate of retrieval by adding more iterations and adding extra features such as
texture and shape with the colour to improve the relevance.
Acknowledgment
The work done by Dr. A.Vadivel and is supported by research grant from the Department of
Science and Technology, India, under Grant DST/TSG/ICT/2009/27 dated 3rd September 2010.
References
[1] Carson, C., Thomas, M., Belongie, S., Hellerstein, JM, & Malik, J, (1999) ”Blobworld: A system for
region-based image indexing and retrieval”, Third Int. Conf. on Visual Information Systems, pp 217-
225.
[2] Deng, Y., Manjunath, B.S., Kenney, C., Moore, M.S. & Shin, H. (2001) “An efficient colour
representation for image retrieval”, IEEE Transactions on Image Processing, Vol. 10, pp140-147.
[3] Deb & Y. Zhang, (2004) “An overview of content-based Image retrieval techniques,” Proc. on 18th
Int. Conf. on Advanced Information Networking and Applications, Vol. 1, pp59-64.
[4] Fuxiang Lu, Xiaokang Yang, Rui Zhang, & Songyu Yu , (2009) "Image classification based on
pyramid histogram of topics," Multimedia and Expo, ICME 2009. IEEE International Conference on ,
pp398-401.
[5] Gevers, T. & Smeulders, A.W.M., (1999) “Colour-based object recognition”, Pattern Recognition,
Vol. 32, pp453-464.
[6] Gevers,T & Smeulders, A.W.M., (2000) “PicToSeek: Combining colour and shape invariant features
for image retrieval”, IEEE Trans. on Image Processing, Vol. 9 pp102-119.
[7] Han, J. & Ma, K-K., (2002) “Fuzzy colour histogram and its use in colour image retrieval”, IEEE
Transactions on Image Processing, Vol. 2, pp944-952.
[8] Jain, A & Vailaya,A ,(1996) ” Image retrieval using colour and shape”, Pattern Recognition, Vol. 29
pp1233-1244.
[9] Kender, J.R., (1976) “Saturation, Hue and Normalised Colour: Calculation, Digitisation and Use”,
Computer Science Technical Report, Carnegie-Mellon University, Pittsburg, USA.
[10] Lei, Z., Fuzong, L. & Bo, Z., (1999) “A CBIR method based colour-spatial feature”, Proc. IEEE
Region 10 Annual International Conference on TENCON 99, Proceedings ,Cheju Island, South
Korea, pp166-169.
[11] Ma W.Y. & Manjunath, B.S., (1997) “NeTra: A toolbox for navigating large image databases”, IEEE
Int. Conf. on Image Processing pp568-571.
[12] Mohamed, A., Khellfi, F., Ying Weng, Jianmin Jiang & Ipson, S., (2009) "An efficient Image
Retrieval through DCT Histogram Quantization" , International Conference on CyberWorlds,,
pp237-240.
[13] Niblack,W, Barber,R Equitz,W, Flickner,M, Glasman,E,.Petkovic,D Yanker,P,.Faloutsos,C &
Taubin,G., (1993) “The QBIC project: Querying images by content using colour, texture and shape”,
SPIE Int. Soc. Opt. Eng., in Storage and Retrieval for Image and Video Databases.Vol. 1908, pp173-
187.

[14] Nezamabadi-pour, H. & Kabir, E., (2004) “Image retrieval using histograms of uni-colour and bi-
colour blocks and directional changes in intensity gradient”, Pattern Recognition Letters, Vol. 25,
pp1547-1557.
[15] Swain, M.J & allard, D.H., (1991) “Colour Indexing”, Computer Vision. Vol. 7, pp11-32.
[16] Stricker, M A. & Orengo, M., (1995) “Similarity of colour images”, SPIE, Vol. 2420, pp381-392.
[17] Shih, J.L. & Chen, L.H., (2002) “Colour image retrieval based on primitives of colour moments”,
Proc. IEEE Vision, Image and Signal Processing, pp88-94.
[18] Smith, J. R., & Chang, S.-F., (1996) ”VisualSEEk: A Fully Automated Content-Based Image Query
System”, ACM Multimedia, pp87-98
[19] Shoujue Wang & Hong Qin , (2009) "A Study of Order-Based Block Colour Feature Image Retrieval
Compared with Cumulative Colour Histogram Method", Sixth International Conference on Fuzzy
Systems and Knowledge Discovery , Vol. 1, pp81-84.
[20] Vadivel A. ,Majumdar A. K. & Sural Shamik, (2003) "Perceptually Smooth Histogram Generation
from the HSV Colour Space for Content Based Image Retrieval", International Conference on
Advances in Pattern Recognition, Kolkata, pp 248-251.
[21] Vadivel.A, Shamik Sural & A.K.Majumdar, (2008) “Robust Histogram Generation from the HSV
Space based on Visual Colour Perception", International Journal of Signal and Imaging Systems
Engineering, InderScience, Vol. 1(3/4), pp245-254.
[22] Wang & Xiaoling, (2009) "A Novel Circular Ring Histogram for Content-Based Image Retrieval",
First International Workshop on Education Technology and Computer Science, Vol. 2, pp785-788.
[23] Yihong Gong, Proietti, G &, Faloutsos, C., (1998) "Image indexing and retrieval based on human
perceptual color clustering,". Proc. IEEE Computer Society Conference on Computer Vision and
Pattern Recognition, pp.578-583.
[24] Yunqi Lei, Zhenxiang Shi, Xutuan Jiang, Qingmin Li & Dongjie Chen , (2009) "Image Retrieval
Based on Colour Saliency Histogram", International Symposium on Computer Network and
Multimedia Technology , pp1-4.
Authors
Short Biography
Shaila S.G received her M-Tech in Computer Science and Engineering in 2005
from M.V.Jayaraman College of Engineering, Vishweshwarayya Technological
University, Bangalore, India. Currently she is a research scholar in the
Department of Computer Applications, National Institute of Technology,
Tiruchirappalli, India. Her research interest is Multimedia Information retrieval
from distributed environment.
A. Vadivel is an Associate Professor at the National Institute of Technology,
Tiruchirappalli, India. He received his MTech and PhD from the Indian Institute
of Technology (IIT), Kharagpur, India, in 2000 and 2006 respectively. His
research interest includes image and video processing, medical image analysis,
object tracking, multimedia information retrieval from web and frequent pattern
mining. He was awarded an Indo-US Research Fellow Award in 2008 by the
Indo-US Science and Technology Forum, India and young scientist award by
DST Government of India in 2007.

Content-Based Image Retrieval Using Modified Human Colour Perception Histogram

More Related Content

What's hot (19)

Similar to Content-Based Image Retrieval Using Modified Human Colour Perception Histogram (20)

More from cscpconf (20)

Recently uploaded (20)

Content-Based Image Retrieval Using Modified Human Colour Perception Histogram