A SURVEY ON CONTENT BASED IMAGE RETRIEVAL USING MACHINE LEARNING

International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395-0056
Volume: 09 Issue: 06 | June 2022 www.irjet.net p-ISSN: 2395-0072
© 2022, IRJET | Impact Factor value: 7.529 | ISO 9001:2008 Certified Journal | Page 153
A SURVEY ON CONTENT BASED IMAGE RETRIEVAL USING MACHINE
LEARNING
Poorna Chandra D 1, Poornesh H 2, Shobitha B 3, Spoorthy N 4, Prof. Gowrishankar B S 5
1,2,3,4 Student, Information Science and Engineering, Vidyavardhaka College of Engineering,
Karnataka, India
5 Assistant Professor, Information Science and Engineering, Vidyavardhaka College of Engineering,
Karnataka, India
---------------------------------------------------------------------***--------------------------------------------------------------------
Abstract - The Content Based Image Retrieval aims to find the similar images from a large-scale dataset against a query
image. Generally, the similarity between the representative features of the query image and dataset images is used to rank the
images for reclamation. Also, the cornucopia of online networks for product and distribution, as well as the number of images
accessible to consumers, continues to expand. Thus, in numerous areas, endless as well as wide digital image processing takes
place. Thus, the quick access to these large image databases as well as the extraction of identical images from this large set of
images from a given image (Query) pose significant challenges as well as involves effective ways. A CBIR system's effectiveness
depends basically on the computation of point representation as well as similarity. For this purpose, we present a introductory
but important Machine learning algorithm like Convolutional Neural Networks (CNN) or DCNN which has further delicacy
and we can train further and further images, which has comparatively bigger database. CBIR systems allow another image
dataset to detect affiliated images to such a query image. The search per picture function of Google search has to be the most
popular CBIR method.
Key Words: CBIR, Content-based Image Retrieval, CNN, Convolutional Neural Networks, DCNN, Image processing, Query
image.
1. INTRODUCTION
In the recent past the advancement in computer and multimedia technologies has led to the product of digital images and
cheap large image depositories. The size of image collections has increased fleetly due to this, including digital libraries,
medical images etc. To attack this rapid-fire growth, it's needed to develop image reclamation systems which operates on a
large scale. The primary end is to make a robust system that creates, manages and query image databases in an accurate
manner. CBIR is the following advancement to the stride of keyword- grounded systems in which images are mended
grounded on the data of their contents. The reclamation prosecution of a CBIR frame for the utmost part relies upon the
two variables; 1) feature representation 2) similarity estimation. The fundamental substance of CBIR is the point birth
process. CNNs made out of a class of learnable models which can make use in operations like Image Retrieval, Image
Bracket, Image Annotation, Image Recognition and so forth. With the provocation of the extraordinary success of deep
learning algorithms to the invention in this paper, they've used for the reclamation the images. CBIR is substantially used
for looking through grounded on the content as opposed to the image reflections. It incorporates the system of
representing and sorting out images grounded on the word query image.
Content based image retrieval (CBIR) is a computer vision fashion that gives a way for searching applicable images in large
databases. This hunt is grounded on the image features like color, texture and shape or any other features being deduced
from the image itself. The performance of a CBIR system substantially depends on the named features. The images are first
represented in terms of features in a high dimensional point space. Also, the similarity among images stored in the
database and that of a query image is measured in the point space by using a distance metric e.g., Euclidean distance.
Hence, for CBIR systems, representation of image data in terms of features and opting a similarity meassimilure, are the
most critical factors. In CBIR, certain image features incorporate shading, face, and shape that can be resolved from the
images. CBIR can be acted in two different ways. for illustration the primary fashion is ordering and second are looking.
Exercising the strategy ordering, the birth of the features from the image and can be used to store this feature in point
database as point vectors. In the alternate strategy for illustration in looking, the birth of the point vectors from the
information images and these separated features are taken for correlation with point vectors accessible in the database.
And this outgrowth is used for recovering most matched images from the database to the query image. Unnaturally, there
are two kinds of images recovery live, (1) exact image retrieval and (2) applicable image retrieval. For exact image
reclamation, the coordinating of 100% with the query is done and in applicable image reclamation, the reclamation relies

upon the contents or features of image. In this we probe, Machine learning algorithm like Convolutional Neural Networks
(CNN) or DCNN which has further delicacy and we can train further and further images, which has comparatively bigger
database.
2. LITERATURE SURVEY
The reviews based on the methodology used by various authors in their research works for Content Based Image Retrieval
(CBIR), following are:
2.1 Content based Image Retrieval using Deep Learning Technique with Distance Measures
The exploration work carried out in [1], Sirisha Kopparthi, Dr. N. K. Kameswara Rao, used Convolution neural networks
(CNN) with deep learning performed an excellent performance in numerous operations of image processing. The use of
CNN based techniques to extract image features from the final layer and the use of a single CNN structure may be used for
identifying matching images. Learning feature extraction and effective similarity comparison comprises the Content-Based
Image Retrieval (CBIR). In CBIR feature extraction, as well as similarity measures, play a vital role. The experiments are
carried out in two datasets such as UC Merced Land Use Dataset By using a pre-trained model that is trained on millions of
images and is fine-tuned for the reclamation task. Pre-trained CNN models are used for generating feature descriptors of
images for the retrieval process. This method deals with the attribute extraction from the two fully connected layers,
which is present in the VGG-16 and VGG19 network by using transfer learning and retrieval of feature vectors using
various similarity measures. The proposed architecture demonstrates an outstanding performance in extracting the
features as well as learning features without a prior knowledge about the images. By using various performance metrics.
2.2 Image Retrieval based on the Combination of Color Histogram and Color Moment
The research work carried out in [2], S. Mangijao Singh, K. Hemachandran, used a novel technique for Content based image
retrieval (CBIR) that employs color histogram and color moment of images is proposed. The color histogram has the
advantages of rotation and translation invariance and it has the disadvantages of lack of spatial information. In this paper,
to improve the retrieval accuracy, a content-based image retrieval method is proposed in which color histogram and color
moment feature vectors are combined. For color moment, to improve the discriminating power of color indexing
techniques, a minimal amount of spatial information is encoded in the color index by dividing the image horizontally into
three equal nonoverlapping regions. The three moments (mean, variance and skewness) are extracted from each region
(in this case three regions), for all the color channels. Thus, for a HSV color space, 27 floating point numbers are used for
indexing. The HSV (16, 4, 4) quantization scheme has been adopted for color histogram and an image is represented by a
vector of 256-dimension. Weights are assigned to each feature respectively and calculate the similarity with combined
features of color histogram and color moment using Histogram intersection distance and Euclidean distance as similarity
measures.
2.3 Content-Based Image Retrieval Using Deep Learning
The exploration work carried out in [3], Anshuman Vikram Singh, used deep learning approaches especially Convolutional
Neural Networks (CNN) in working computer vision operations which has inspired author to work on this thesis so as to
break the problem of CBIR using a dataset of annotated images. He worked with only 3000 images from 41 orders and 8
classes. In future to make the system more generalized and effective the dataset can be increased and further number of
classes similar as man, person, aeroplane etc can be added. The neural network was trained on the dataset for each
marker. It can be observed that the confirmation error rate and testing error rate for each marker was relatively low. The
stylish test and confirmation error rates are achieved on replication 3, 6 or 9. Training runs for 500 duplications just to
validate that there's no change in the error rate after every 100 duplications and once it reaches 500 with a constant rate it
stops and returns the stylish error rates. The images in our dataset contain reflections of different regions in the form of
XML lines. The Extensible Markup Language (XML) reflections give the annotated image description of each image in the
dataset. A semantic gap exists between low- position image pixels captured by machines and the high- position semantics
perceived by humans. The thesis shows that deep learning will produce better results for annotated images and which will
affect in more accurate image reclamation.

2.4 Fast content-based image retrieval using Convolutional Neural Network and hash function
The exploration work carried out in [4], Domonkos Varga and Tamas Sziranyi, habituated success ways of deep learning
similar as Convolution Neural Network (CNN), it has motivated them to explore its operation in their own environment.
Due to the explosive increase of online images, content- based image retrieval has gained a lot of attention. The main
donation of their work is a new end-to- end supervised learning frame that learns probability- grounded semantic-
position similarity and point- position similarity contemporaneously. The main advantage of new mincing scheme that it's
suitable to reduce the computational cost of reclamation significantly at the state-of-the- art effectiveness position. They
report on comprehensive trials using publicly available datasets similar as Oxford, Leaves and ImageNet 2012 retrieval
datasets.
2.5 Medical Image Retrieval using Deep Convolutional Neural Network
The exploration work carried out in [5], Adnan Qayyum, Syed Muhammad Anwar, Muhammad Awais, Muhammad Majid,
used a frame of deep learning for CBMIR system by using deep Convolutional Neural Network (CNN) that's trained for
bracket of medical images. A major challenge in CBMIR systems is the semantic gap that exists between the low position
visual information captured by imaging bias and high position semantic information perceived by mortal. The efficacity of
similar systems is more pivotal in terms of point representations that can characterize the high- position information fully.
The learned features and the bracket results are used to recoup medical images. For reclamation, stylish results are
achieved when class grounded prognostications are used. An average bracket delicacy of 99.77 and a mean average
perfection of 0.69 is achieved for reclamation task. The proposed system is best suited to recoup multimodal medical
images for different body organs.
2.6 Content based image retrieval using deep learning process
The exploration work carried out in [6], R. Rani Saritha, Varghese Paul, P. Ganesh Kumar, used the deep belief network
(DBN) system of deep learning which is used to prize the features and bracket and is an arising exploration area, because
of the generation of large volume of data. A multi-feature image reclamation system is introduced by combining the
features of color histogram, edge, edge directions, edge histogram and texture features, etc. In this model, the content
grounded image will be uprooted from a collection of intended image groups. After performing some pre-processing way
like selection junking, its below features are uprooted and are stored as small hand lines. CBIR uses image content features
to search and recoup digital images from a large database. A variety of visual point birth ways have been employed to
apply the searching purpose. Due to the calculation time demand, some good algorithms aren't been used. The reclamation
performance of a content- grounded image reclamation system crucially depends on the point representation and
similarity measures. The ultimate end of the proposed system is to give an effective algorithm. The proposed system is
tested through simulation in comparison and the results show a huge positive divagation towards its performance.
2.7 Content-based Image Retrieval from Videos using CBIR and ABIR algorithm
The exploration work carried out in [7], Vrushali A. Wankhede, used Annotation-based image retrieval (ABIR) and Content
based image retrieval (CBIR). Videotape reclamation can be used for videotape hunt and browsing which are useful in web
operations. Selection of uprooted features play an important part in content-based videotape reclamation. There are two
types of point birth, low position point birth and high position point birth. Low position point birth grounded on color,
shape, texture, spatial relationship. The main thing of this paper is that, stoner can give the two different types of input in
the form of image query and the textbook query. The ABIR is more practical in some other sphere. In that, they will get the
set of frames or set of images from the below step, also give the labelling of one image from every videotape. The
information store in the XML train information contains their path, images assign with the word or textbook. Reflection
means what's in the image, what's it about, what does it bring? Give the detail description to that image. Prepare annotated
data. Take any one word or textbook as an input from the annotated data. Find the keyword from pre-processed data if
they match take that image as an affair image. Display the result means retrieves the image by using the textbook query
hunt. With the help of labelling, they got the image from the particular path. The main thing of this paper is to apply the
multi query image retrieval.

2.8 Context-aware Recommendation System using Content Based Image Retrieval with Dynamic
Context Considered
The exploration work carried out in [8], Yuta Miyazawa, Yukiko Yamamoto, Takashi Kawabe, proposed paper conceptually
proposes a environment-apprehensive recommendation system that gives optimal information for druggies grounded on
1) a content- based image retrieval (CBIR) medium to search the similar images aiming to prize the detailed information to
the textbook-indescribable images 2) the contextual information of similar analogous images searched from the Web, and
3) stoner’s dynamic environment or situation considering time-variant factors as well as space factors. It's anticipated to
increase the perfection or optimality of recommendation by matching and fusing the environment of analogous images
attained by CBIR with textual and star information about stoner’s situation or dynamic environment. is described just at a
abstract position; thus, as a coming step for the exploration, the prototype system will be developed grounded on more
detailed perpetration design. Also, the utility and effectiveness of the proposed idea and its perpetration will be validated
through the evaluation trial using the prototype system.
2.9 Content-based image retrieval: A review of recent trends
The exploration work carried out in [9], Ibtihaal M. Hameed, Sadiq H. Abdulhussain & Basheera M. Mahmmod, says that
there are adding exploration in this field, this paper checks, analyses and compares the current state-of-the- art
methodologies over the last six times in the CBIR field. This paper also provides an overview of CBIR frame, recent low-
position point birth styles, machine literacy algorithms, similarity measures, and a performance evaluation to inspire
farther exploration sweats. To negotiate an effective CBIR frame, the frame’s factors must be chosen in a balanced way; this
study helps in probing these factors. To add up, an algorithm that elevates the semantic gap is largely demanded. The
design of the algorithm should consider the following first, the algorithm needs to consider the point birth as well as
similarity measure as they impact the performance of the CBIR. Second, further features can be uprooted to enhance the
delicacy of the CBIR and maintain the computational cost as it's considered important factor in the real- time operations.
Third, incorporating original and global features will lead to a balanced design because original features are more robust
against scale, restatement and gyration changes than global features; and global features are briskly in point birth and
similarity measures. Fourth, machine literacy algorithms can be used in different stages of CBIR to increase system
delicacy but need further attention to be paid to their calculation cost. Eventually, there's a dicker between system’s
delicacy and computational cost.
2.10 A Decade Survey of Content Based Image Retrieval using Deep Learning
The exploration work carried out in [10], Shiv Ram Dubey, this check also presents a performance analysis for the state-of-
the-art deep literacy grounded image reclamation approaches. The Mean Average Precision (chart) reported for the
different image reclamation approaches is epitomized. ThemAP@5000 (i.e., 5000 recaptured images) using colorful being
deep learning approaches is epitomized over CIFAR-10, NUS-WIDE and MS COCO datasets. The results over CIFAR-10,
ImageNet and MNIST datasets using different state-of-the- art deep literacy grounded image reclamation styles are
collected in terms of themAP@1000. ThemAP@54000 using many styles is reported over the CIFAR-10 dataset. The
standard chart is also depicted by considering all the recaptured images for CIFAR-10 dataset using some of the available
literature. In general, information reclamation algorithms in recent times attained the benefits of using different machine
learning algorithms similar as deep literacy, SVM, and k- means. Thus, they're prognosticated to admit further attention in
the forthcoming times.
Table -1: Summary of papers reviewed
Sl.No Title Year of
Publication
Result
1 Content Based Image Retrieval using Deep
Learning Technique with Distance Measures
2020 Euclidean distance and Cosine similarity measured separately.
Precision rate of 0.86 obtained for vgg16 and 0.89 for vgg19.
2 Image Retrieval based on the Combination
of Color Histogram and Color Moment
2012 Two features are introduced color histogram and color
moment. The retrieval efficiency of the proposed method is
tested for both features separately and combined.
Increase in retrieval efficiency with both combined is
observed.

3 Content-Based Image Retrieval Using Deep
Learning
2015 Used CNN and Deep learning methods.
Number of iterations were performed in which lowest test
and validation errors were obtained in 3rd,6th and 9th
iterations.
Training period runs for 500 iterations to return best error
rates.
4 Fast Content-Based image retrieval using
Convolutional Neural Network and Hash
Function
2016 Results obtained based on query image according to the
datasets used.
Semantics are present which makes target images similar to
query image.
5 Medical image Retrieval using Deep
Convolutional Neural Network
Torch7 is used as a tool for development and training.
System is evaluated based on classification and retrieval.
6 Content based Image Retrieval using Deep
Learning Process
2021 Accuracy calculated based on precision rates and maps
obtained.
mAP = 85.23%
mAR = 88.53%
7 Content Based Image Retrieval from Videos
using CBIR and ABIR algorithms
2015 CBIR algorithm is used to try and extract image for multiple
queries.
Multi-query image retrieval is introduced.
8 Context-aware Recommendation System
using Content Based Image Retrieval with
Dynamic Context Considered
2013 CBIR mechanism is used to search similar images.
Contextual information of images searched from web.
Precision rate is obtained through fusing the information.
9 Content-based Image Retrieval: A review of
recent trends
2021 Studies in CBIR domain is present.
General CBIR framework stages also discussed.
Factors affecting CBIR performance is highlighted.
10 A Decade Survey of Content Based Image
Retrieval using Deep Learning
2021 Different supervision types, different networks used are
explained in detail.
Evolution of deep learning models is also shown.
Summary of large-scale datasets commonly used in CBIR is
also present.
3. DATASET DESCRIPTION
The following are the datasets and the attributes used by various authors from different repositories in their research
works.
Table -2: Dataset sample used by various researchers to predict CBIR status
Sl.No Author Names No. of Attributes Dataset Type
1 Sirisha Kopparthi
Dr. N. K. Kameswara Rao
21 UC Merced Land use.
2 S. Mangijao Singh
K. Hemachandran
10 Corel image.
3 Anshuman Vikram Singh 8 Chosen from subset of SUN database.
4 Domonkos Varga
Tamas Sziranyi
12
500
1000
Oxford.
Holidays.
ImageNet 2012.

5 Adnan Qayyum
Syed Muhammad Anwar
Muhammad Awais
Muhammad Majid
24 Publicly available medical databases.
6 R. Rani Saritha
Varghese Paul
P. Ganesh Kumar
6 Image dataset.
7 Vrushali A, Wankhede
Prakash S. Mohod
8 Image Database.
8 Yuta Miyazawa
Yukiko Yamamoto
Takashi Kawabe
5 Image and Text.
9 Ibtihaal M. Hameed
Sadiq H. Abdulhussian
Basheera M. Mahmmod
10
10
10
26
Corel.
CIFAR.
WANG.
Olivia.
10 Shiv Ram Dubey 10
10
10
80
397
CIFAR-10.
MNIST.
SVHN.
MS COCO.
SUN397.
4. CONCLUSIONS
The need to find an effective image reclamation medium grounded on image content is motivated by the large quantum of
image databases and the absence of an effective text- based image retrieval system. This paper presents a comprehensive
check of deep learning styles for content- based image retrieval. As utmost of the deep learning grounded developments
are recent, this check majorly focuses over the image reclamation styles using deep learning in a decade. The exploration
trend in image reclamation suggests that the deep learning grounded models are driving the progress. The lately
developed models similar as generative inimical networks, autoencoder networks and reinforcement learning networks
have shown the superior performance for image reclamation. The discovery of better objective functions has been also the
trend in order to constrain the literacy of the hash code for discriminative, robust and effective image reclamation. The
semantic conserving class-specific point literacy using different networks and different quantization ways is also the
recent trend for image reclamation. This paper also bandied the general CBIR frame stages and the most recent ways used
to reduce the semantic gap. To negotiate an effective CBIR frame, the frame’s factors must be chosen in a balanced way;
this study helps in probing these factors. Similar effective CBIR architecture will contribute to numerous real-world
operations, similar as medical operations, web quests, and social media.
REFERENCES
[1] Sirisha Kopparthi, Dr. N. K. Kameswara Rao, “Content based Image Retrieval using Deep Learning Technique with
Distance Measures”, Science, Technology & Human Values 9(12):251-261, 2020.
[2] S. Mangijao Singh, K. Hemachandran, “Image Retrieval based on the Combination of Color Histogram and Color
Moment”, International Journal of Computer Applications, Volume 58– No.3, November 2012.
[3] Anshuman Vikram Singh, “Content-Based Image Retrieval Using Deep Learning”, Thesis. Rochester Institute of
Technology, RIT Scholar Works, 2015.
[4] Domonkos Varga, Tamas Sziranyi, “Fast Content-Based image retrieval using Convolutional Neural Network and Hash
Function”, IEEE International Conference on Systems, Man, and Cybernetics (SMC), 2016.

[5] Adnan Qayyum, Syed Muhammad Anwar, Muhammad Awais, Muhammad Majid, “Medical Image Retrieval using Deep
Convolutional Neural Network”, Neurocomputing, 2017 – Elsevier.
[6] R. Rani Saritha, Varghese Paul, P. Ganesh Kumar, “Content based image retrieval using deep learning process”, 2018.
[7] Vrushali A. Wankhede , and Prakash S. Mohod, “Content-based Image Retrieval from Videos using CBIR and ABIR
algorithm”, Global Conference on Communication Technologies, 2015.
[8] Yuta Miyazawa, Yukiko Yamamoto, Takashi Kawabe , “Context-aware Recommendation System using Content Based
Image Retrieval with Dynamic Context Considered”, International Conference on Signal-Image Technology & Internet-
Based Systems, 2013.
[9] Ibtihaal M. Hameed, Sadiq H. Abdulhussain & Basheera M. Mahmmod , “Content-based image retrieval: A review of
recent trends”, ISSN, 2021.
[10] Shiv Ram Dubey, “A Decade Survey of Content Based Image Retrieval using Deep Learning”, IEEE Transactions on
Circuits and Systems for Video Technology, May 2021.

A SURVEY ON CONTENT BASED IMAGE RETRIEVAL USING MACHINE LEARNING

More Related Content

Similar to A SURVEY ON CONTENT BASED IMAGE RETRIEVAL USING MACHINE LEARNING (20)

More from IRJET Journal (20)

Recently uploaded (20)

A SURVEY ON CONTENT BASED IMAGE RETRIEVAL USING MACHINE LEARNING